Superimposed image display device

Abstract

To provide a superimposed image display device capable of easily performing guidance of a merging point of merging roads and a branching point of branching roads visually in the case where the merging point and the branching point continuously appear in front of a travelling direction of a vehicle.SOLUTION: A superimposed image display device is configured such that a merging point of merging roads and a branching point which appear in front of a travelling direction of a vehicle are acquired, and in the case where the merging point and the branching point continuously appear in this order in front of a travelling direction of the vehicle, guidance objects performing guidance for the merging point and guidance for the branching point in combination is displayed, and on the other hand, the shapes of the guidance objects are changed and displayed on the basis of positional relation of the merging point and the branching point particularly in displaying the guidance objects.SELECTED DRAWING: Figure 12

Description

[Technical field]

[0001] The present invention relates to a superimposed image display device that performs driving support for a vehicle. [Background technology]

[0002] Conventionally, various means have been used as information providing means for providing various information for vehicle driving support such as route guidance and warning of obstacles to vehicle occupants. For example, the information is displayed on a liquid crystal display installed in the vehicle, or sound is output from a speaker. In recent years, one such information providing means is a device that provides information by displaying an image superimposed on the surrounding environment (landscape, real scene) of the occupant. For example, a head-up display, a windshield display, or a method of displaying an image superimposed on a captured image of the vehicle surroundings displayed on a liquid crystal display corresponds to the information providing means.

[0003] Here, useful information for a vehicle when it is traveling includes the road conditions of the road on which the vehicle is about to travel, i.e., the road ahead in the direction of travel. For example, JP 2021-128769 A proposes a technology for providing information on road conditions using the above-mentioned information providing means, which identifies the head and tail of a traffic jam queue when a traffic jam queue is detected on a road on which a vehicle is traveling, and displays the positions of the head and tail vehicles of the traffic jam queue on a display as guidance information for the traffic jam queue. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2021-128769 A (paragraphs 0060-0066) Summary of the Invention [Problem to be solved by the invention]

[0005] Here, the technology of the above-mentioned Patent Document 1 can guide the lane and the range where the congestion queue occurs, but even in a lane where the congestion queue does not occur, a situation may occur where the flow of traffic suddenly deteriorates in a specific section. For example, as shown in FIG. 15, it is known that in a section where a merging point where roads merge and a branching point where roads branch off are consecutively present, the flow deteriorates in that section due to vehicles moving between the lanes. In particular, as shown in FIG. 15, when a merging lane or a branching lane 201 is congested, the flow may deteriorate significantly even in the main lane 202, which is not congested, due to vehicles attempting to enter the congested lane. In addition, even if the merging lane or the branching lane 201 is not congested, if the distance from the merging point to the branching point is short (for example, less than 1 km), the flow of traffic in the main lane 202 may also deteriorate significantly due to vehicles making unreasonable lane changes.

[0006] The technology of the above-mentioned Patent Document 1 is capable of providing guidance on traffic jams, but is unable to provide advance guidance on the poor flow of the main lane 202 if no traffic jams have occurred. As a result, there is a problem that the flow of traffic suddenly and unexpectedly worsens when approaching the relevant section, making it necessary to apply the brakes, which places a burden on the driver. In view of this problem, it is desirable to provide special guidance for merging points and branching points when there are consecutive merging points and branching points ahead in the vehicle's traveling direction.

[0007] The present invention has been made to solve the above-mentioned problems in the conventional art, and has an object to provide an overlay image display device that makes it possible to provide visually easy-to-understand guidance of junctions and branching points, particularly when there are successive junctions and branching points ahead of the vehicle in the direction of travel. [Means for solving the problem]

[0008] In order to achieve the above-mentioned object, the superimposed image display device of the present invention is an overlay image display device that allows a guide object that provides information to a vehicle occupant in a vehicle to be visually recognized by being superimposed on the scenery around the vehicle, and has: a junction point acquisition means for acquiring a junction point where roads merge ahead of the vehicle in the direction of travel; a branch point acquisition means for acquiring a branch point where roads branch off ahead of the vehicle in the direction of travel; and an object display means for displaying the guide object that provides both guidance to the junction point and guidance to the branch point when the junction point and the branch point are located in succession ahead of the vehicle in the direction of travel, and the object display means changes and displays the shape of the guide object based on the positional relationship between the junction point and the branch point. It should be noted that the "scenery" includes not only the scenery actually viewed from the vehicle (actual scenery), but also an image of the scenery captured, an image of the scenery reproduced, and the like. In addition, "contiguous" does not include a point between the junction and the branch point where at least another road is connected. Effect of the Invention

[0009] According to the superimposed image display device of the present invention having the above-mentioned configuration, particularly when there are successive junctions where roads merge and junctions where roads diverge ahead in the vehicle's direction of travel, guidance of the junctions and junctions can be provided together with guide objects displayed superimposed on the scenery around the vehicle, making it possible to provide visually easy-to-understand guidance that not only notifies the vehicle of the existence of the junctions and junctions, but also specifies the positional relationship between the junctions and junctions. [Brief description of the drawings]

[0010] [Figure 1] 1 is a block diagram showing a navigation device according to a first embodiment. [Diagram 2] FIG. 1 is a diagram illustrating a probe car system for generating traffic congestion information. [Diagram 3] 4 is a flowchart of a driving support processing program according to the first embodiment. [Figure 4]FIG. 13 is a diagram illustrating a method for calculating the distance of a merging section. [Diagram 5] 13 is a flowchart of a sub-processing program of a first guidance object display process. [Figure 6] 1A to 1C are diagrams illustrating a method for detecting a lane included in a captured image. [Figure 7] FIG. 1 is a diagram showing an example of a road shape in which a junction and a branch point are successively provided; [Figure 8] FIG. 13 is a diagram showing an example of a first guide object displayed on a liquid crystal display. [Figure 9] 13 is a diagram showing a display example of a first guide object. FIG. [Figure 10] 13 is a flowchart of a sub-processing program of a second guidance object display process. [Figure 11] 13 is a diagram showing an example of a second guide object displayed on the liquid crystal display. FIG. [Figure 12] 13 is a diagram showing a display example of a second guidance object. FIG. [Figure 13] FIG. 11 is a schematic configuration diagram of a superimposed image display device according to a second embodiment. [Figure 14] 13A and 13B are diagrams showing a display example of a guide object in a superimposed image display device according to a second embodiment. [Figure 15] FIG. 1 is a diagram illustrating problems with the conventional technology. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] Hereinafter, a first embodiment and a second embodiment in which the superimposed image display device according to the present invention is embodied in a navigation device, which is one of the vehicle-mounted devices, will be described in detail with reference to the drawings. Note that, in the following embodiments, an example in which the navigation device functions as the superimposed image display device will be described, but a vehicle-mounted device other than the navigation device or an information terminal connected to the vehicle may function as the superimposed image display device, and further, a combination of multiple vehicle-mounted devices or a vehicle-mounted device and an external server may function as the superimposed image display device.

[0012] [First embodiment] First, a schematic configuration of a navigation device 1 according to the first embodiment will be described with reference to Fig. 1. Fig. 1 is a block diagram showing the navigation device 1 according to the first embodiment.

[0013] 1, the navigation device 1 according to the first embodiment includes a current position detection unit 11 for detecting the current position of the vehicle on which the navigation device 1 is mounted, a data recording unit 12 for recording various data, a navigation ECU 13 for performing various arithmetic processing based on input information, an operation unit 14 for receiving operations from a user, a liquid crystal display 15 for displaying to the user an actual scene image captured in the forward direction of travel, a speaker 16 for outputting voice guidance regarding route guidance, a DVD drive 17 for reading a DVD as a storage medium, and a communication module 18 for communicating with an information center such as a probe center or a VICS (registered trademark: Vehicle Information and Communication System) center. In addition, the navigation device 1 is connected to a front camera 19 and various sensors installed in the vehicle on which the navigation device 1 is mounted via an in-vehicle network such as a CAN.

[0014] Each of the components of the navigation device 1 will be described below in order. The current position detection unit 11 is made up of a GPS 21, a vehicle speed sensor 22, a steering sensor 23, a gyro sensor 24, etc., and is capable of detecting the current vehicle position, direction, vehicle running speed, current time, etc. Here, the vehicle speed sensor 22 in particular is a sensor for detecting the travel distance and vehicle speed of the vehicle, and generates pulses in response to the rotation of the drive wheels of the vehicle and outputs the pulse signal to the navigation ECU 13. The navigation ECU 13 then calculates the rotation speed and travel distance of the drive wheels by counting the generated pulses. Note that the navigation device 1 does not need to be equipped with all of the above four types of sensors, and the navigation device 1 may be configured to be equipped with only one or more types of sensors among these.

[0015] The data recording unit 12 also includes a hard disk (not shown) as an external storage device and recording medium, and a recording head (not shown) which is a driver for reading out the map information DB 31 and predetermined programs recorded on the hard disk and writing predetermined data to the hard disk. The data recording unit 12 may be configured with a flash memory, a memory card, or an optical disk such as a CD or a DVD instead of a hard disk. The map information DB 31 may also be stored in an external server and acquired by the navigation device 1 through communication.

[0016] Here, the map information DB 31 is a storage means that stores, for example, link data 32 relating to roads (links), node data 33 relating to node points, branch point data 34 relating to branch points, point data relating to points such as facilities, map display data for displaying a map, search data for searching for routes, search data for searching for points, etc.

[0017] The link data 32 includes, for each link constituting the road network, data representing the width, gradient, cant, bank, road surface condition, number of lanes on the road, lane width, traffic division in the direction of travel for each lane, presence or absence of oncoming lanes (whether it is a two-way traffic section or not), presence or absence of a median strip or center line, points where the number of lanes decreases, points where the road width narrows, railroad crossings, etc.; for corners, data representing the radius of curvature, intersections, T-junctions, corner entrances and exits, etc.; for road attributes, data representing downhill roads, uphill roads, etc.; and for road types, data representing expressways and general roads (national highways, prefectural roads, narrow streets, etc.).

[0018] The node data 33 also includes data regarding the coordinates (positions) of actual road branching points (including intersections, T-junctions, etc.) and node points set at predetermined distances on each road depending on the radius of curvature, etc., node attributes indicating whether the node corresponds to an intersection, etc., a connecting link number list which is a list of link numbers of links connecting to the node, an adjacent node number list which is a list of node numbers of nodes adjacent to the node via links, and the height (altitude) of each node point, etc.

[0019] The branch point data 34 includes the intersection name of the branch point, node information for identifying the node forming the branch point, connecting link information for identifying the link connected to the branch point, the destination name corresponding to the link connected to the branch point, information for identifying the shape of the branch point, etc. In the first embodiment, the branch points include "a point where another road merges (enters) a main road (hereinafter referred to as a merge point)" and "a branch point where another road branches (exits) a main road (hereinafter referred to as a branch point)" that exist at interchanges, junctions, etc.

[0020] On the other hand, the navigation ECU (electronic control unit) 13 is an electronic control unit that controls the entire navigation device 1, and includes a CPU 41 as an arithmetic device and control device, a RAM 42 that is used as a working memory when the CPU 41 performs various arithmetic processing and stores route data and the like when a route is searched, a ROM 43 that stores a driving support processing program (FIG. 3) and the like described below as well as control programs, and a flash memory 44 that stores programs read from the ROM 43. The navigation ECU 13 has various means as processing algorithms. For example, the junction point acquisition means acquires a junction point where roads merge ahead of the vehicle in the traveling direction. The branch point acquisition means acquires a branch point where roads branch off ahead of the vehicle in the traveling direction. The object display means displays a guide object that provides both guidance to the junction point and guidance to the branch point when a junction point and a branch point are consecutively located ahead of the vehicle in the traveling direction.

[0021] The operation unit 14 is operated when inputting a departure point as a travel start point and a destination point as a travel end point, and has a plurality of operation switches (not shown) such as various keys and buttons. The navigation ECU 13 performs control to execute various corresponding operations based on switch signals output by pressing each switch. The operation unit 14 may be configured to have a touch panel provided on the front surface of the liquid crystal display 15. It may also be configured to have a microphone and a voice recognition device.

[0022] The liquid crystal display 15 also displays map images including roads, traffic information, operation guidance, operation menus, key guidance, guided route from the departure point to the destination, guidance information along the guided route, news, weather forecasts, time, emails, television programs, etc. In particular, in the first embodiment, when a guided route is set in the navigation device 1 and the vehicle approaches a guided branch point that is the subject of guidance, or when a guided route is not set and the vehicle approaches a section where a branch point and a junction point are successive, the liquid crystal display 15 displays an image captured by the front camera 19, i.e., a landscape (actual landscape image) around the vehicle at the current time (particularly in front of the vehicle), and further displays a guide object superimposed on the landscape as necessary.

[0023] Here, the guidance objects displayed superimposed on the scenery include information related to the vehicle and various information used to support the driver. For example, there are warnings for objects (other vehicles, pedestrians, guide signs) that should be warned to the driver, guidance information based on the guidance route set by the navigation device 1 and the guidance route (arrows indicating the direction of right and left turns, icons indicating the markers of the guidance branch point, distance to the guidance branch point, the position and direction of the recommended lane in which the vehicle should travel, guidance to encourage the driver to change lanes to the recommended lane, etc.), warnings displayed on the road surface (collision warning, speed limit, etc.), lane markings on which the vehicle travels, current vehicle speed, shift position, remaining energy, advertisement images, facility information, guide signs, map images, traffic information, news, weather forecasts, time, the screen of a connected smartphone, etc. In the first embodiment described below, the guidance objects are guidance information for providing guidance to junctions and branch points that exist consecutively ahead in the traveling direction of the vehicle. More specifically, the images may include guidance images that show the number of lanes at a merging or branching point, the condition of each lane (traffic jam or congestion), the distance from the vehicle to the merging point, the distance from the vehicle to the branching point, and the distance between the merging point and the branching point.

[0024] The speaker 16 also outputs voice guidance for guiding the vehicle along a guide route based on instructions from the navigation ECU 13, and traffic information guidance.

[0025] The DVD drive 17 is a drive capable of reading data recorded on a recording medium such as a DVD or a CD. Based on the read data, music and video are played, and the map information DB 31 is updated. Instead of the DVD drive 17, a card slot for reading and writing data into a memory card may be provided.

[0026] The communication module 18 is a communication device for receiving traffic information consisting of various information such as congestion information, regulation information, and traffic accident information transmitted from a traffic information center, such as a VICS center or a probe center, and is, for example, a mobile phone or a DCM.

[0027] Here, in the first embodiment, the navigation device 1 is capable of acquiring congestion information indicating the congestion state for each lane, which is generated in an external server device 51 provided in the probe center 50, via the communication module 18, as shown in FIG. 2. The server device 51 and a plurality of vehicles 52 communicably connected to the server device 51 constitute a so-called probe car system 53. Here, the probe car system 53 is a system that collects information using the vehicles 52 as sensors. Specifically, the vehicle 52 transmits speed data and the operating status of each system, such as steering operation and shift position, together with GPS position information, to the probe center 50 via a communication device previously mounted on the vehicle 52, and the collected data is reused on the center side as various information.

[0028] The server device 51 provided in the probe center 50 is an information management server that appropriately collects and accumulates probe information (material information) including the current time and driving information from each vehicle 52 traveling throughout the country, generates various support information related to roads (for example, road closure information, accident information, traffic congestion information, travel time, etc.) from the accumulated probe information, distributes the generated support information to the navigation device 1, and performs various processes using the support information. In particular, in the first embodiment, the server device 51 collects from each vehicle 52 the current position coordinates and vehicle speed of the vehicle 52, and captured images of the surroundings captured by the front camera 19 provided in the vehicle 52, and generates data indicating the traffic congestion situation for each lane by statistically or analytically analyzing the collected information, and is capable of distributing the data to the vehicle 52.

[0029] However, the traffic congestion information acquired by the navigation device 1 does not necessarily have to be traffic congestion information generated by the above-mentioned probe car system 53, and may be traffic congestion information generated by other means as long as the information enables identification of the traffic congestion situation for each lane.

[0030] The front camera 19 is an imaging device having a camera using a solid-state imaging element such as a CCD, and is installed, for example, on the back side of the rearview mirror or on the front bumper with the optical axis direction facing forward in the traveling direction of the vehicle. The image captured by the front camera 19 is displayed on the liquid crystal display 15 as the scenery (actual scene image) around the vehicle (particularly in front of the vehicle) as described above. The image captured by the front camera 19 is also used, for example, to identify the lane in which the vehicle is currently traveling and to detect the traffic congestion state by the server device 51 described above.

[0031] Next, a driving support processing program executed by the navigation ECU 13 in the navigation device 1 having the above configuration will be described with reference to Fig. 3. Fig. 3 is a flowchart of the driving support processing program according to the first embodiment. Here, the driving support processing program is executed after the ACC power supply (accessory power supply) of the vehicle is turned on, and is a program that assists driving of the vehicle by allowing the driver to visually recognize a guide object superimposed on the scenery around the vehicle displayed on the liquid crystal display 15. The programs shown in the flowcharts of Figs. 3, 5 and 10 below are stored in the RAM 42 and ROM 43 of the navigation device 1, and are executed by the CPU 41.

[0032] In the following description, an example of vehicle driving assistance using a guide object will be described in which a vehicle driving guide is provided when a guide route is not set in the navigation device 1. The guide object to be displayed is guide information for providing guidance to a junction and a branch point that are consecutively present ahead in the traveling direction of the vehicle, and a process will be described in which a guide image showing the number of lanes at the junction or branch point, the state of each lane (congestion or congestion), the distance from the vehicle to the junction, the distance from the vehicle to the branch point, and the distance between the junction and the branch point is displayed as a guide object. However, the navigation device 1 can also provide guidance and information other than the above-mentioned driving assistance using a guide object. The guide object to be displayed can also include information other than the above-mentioned guide image. For example, guidance objects can be used to display warnings to occupants about objects that should be warned about (other vehicles, pedestrians, guide signs), warnings to be displayed on the road surface (watch out for rear-end collisions, speed limits, etc.), current vehicle speed, shift position, remaining energy, advertising images, facility information, guide signs, map images, traffic information, news, weather forecasts, time, the screen of a connected smartphone, etc.

[0033] Further, details of vehicle driving guidance when a guide route is set in the navigation device 1 are omitted, but in that case, the vehicle is guided to drive along the set guide route using a guide object. The guide object to be displayed is guidance information for providing guidance at a guide branch point ahead of the vehicle in the traveling direction, and in particular, an arrow indicating the exit direction of the guide branch point, a guide image indicating the position and direction of a recommended lane, and a guide image encouraging movement to the recommended lane are displayed as guide objects. However, even when a guide route is set in the navigation device 1, guidance may be provided for a junction and a branch point that are consecutively present ahead of the vehicle in the traveling direction using a guide object, in the same way as when a guide route is not set.

[0034] First, in the driving support processing program, in step (hereinafter abbreviated as S) 1, the CPU 41 identifies the current position of the vehicle based on the detection result of the current position detection unit 11 and map information. When identifying the current position of the vehicle, a map matching process is also performed to match the current position of the vehicle with the map information. Then, road information within a predetermined distance (e.g., 3 km) ahead in the traveling direction of the vehicle from the identified current position of the vehicle is acquired from the map information. The road information acquired in S1 includes information for identifying the positions of junctions and branching points in particular.

[0035] If there is a branch point ahead in the travel direction, road information is acquired on the assumption that the vehicle will basically travel along the road. "Follow the road" basically specifies the relationship between links. Also, which links have a "follow the road" relationship is basically specified in map information, and for example, links of the same road type (which may have the same road standard such as expressways or national roads, or may have the same route number) and connected within a certain angle are considered to have a "follow the road" relationship. However, it is also possible to specify the "follow the road" relationship under conditions other than those mentioned above.

[0036] Next, in S2, the CPU 41 judges whether or not a junction and a branch point are consecutively located within a predetermined guidance start distance ahead in the traveling direction of the vehicle. However, it is not necessary that both the junction and the branch point are included within the guidance start distance, but it is sufficient that at least the junction point is included. Furthermore, "consecutively located" means that at least a point (such as an intersection) where another road is connected is not included between the junction and the branch point, and the distance between the junction and the branch point is basically not important. However, if the two are too far apart (for example, 5 km or more), it is preferable not to consider them as consecutively located. The guidance start distance is determined by the road type of the road on which the vehicle is traveling, and for example, an expressway is 1 km, and an ordinary road is 300 m, which is shorter than an expressway. However, the guidance start distance may be a variable value rather than a fixed value.

[0037] If it is determined that a junction and a branch point are consecutively located within a predetermined guidance start distance ahead in the vehicle's traveling direction (S2: YES), the process proceeds to S3. On the other hand, if it is determined that a junction and a branch point are not consecutively located within a predetermined guidance start distance ahead in the vehicle's traveling direction (S2: NO), the process ends without providing guidance.

[0038] In S3, the CPU 41 calculates the distance between the merging point and the branching point (hereinafter referred to as the merging section) that are determined to be located ahead in the traveling direction of the vehicle from the map information, and determines whether the distance of the merging section is 1 km or more. Note that the criteria for calculating the distance of the merging section can be set appropriately, but for example, as shown in Fig. 4, the position S of the point (node) where the merging link and the main line link are connected is regarded as the merging point, and the position E of the point (node) where the branching link and the main line link are connected is regarded as the branching point, and the distance L along the link from the position S to the position E is calculated as the distance of the merging section. The distance used as the criterion for determination in S3 can also be changed appropriately, and may be 700 m or 500 m.

[0039] If it is determined that the distance of the merging section ahead in the vehicle's traveling direction is 1 km or more (S3: YES), the process proceeds to S4. On the other hand, if it is determined that the distance of the merging section ahead in the vehicle's traveling direction is less than 1 km (S3: NO), the process proceeds to S5.

[0040] In S4, the CPU 41 performs a first guidance object display process (FIG. 5) described later. The first guidance object display process is a process in which a guidance image for providing guidance to a junction and a branch point that exist consecutively ahead in the traveling direction of the vehicle is displayed as a guidance object (hereinafter referred to as a first guidance object) on the liquid crystal display 15. Note that an image captured by the front camera 19 before the distance from the vehicle to the junction becomes less than the guidance start distance, that is, a scene (actual scene image) around the vehicle (particularly in front of the vehicle) at the current time, is displayed on the liquid crystal display 15. As a result, it becomes possible for the vehicle occupants to visually recognize the first guidance object superimposed on the scene.

[0041] Meanwhile, in S5, the CPU 41 performs a second guidance object display process (FIG. 10) described later. The second guidance object display process is a process in which a guidance image for providing guidance to a junction and a branch point that are consecutively present ahead in the traveling direction of the vehicle is displayed as a guidance object (hereinafter referred to as a second guidance object) on the liquid crystal display 15. However, unlike the first guidance object, the second guidance object includes a combination of two guidance objects: a junction guidance object that shows the lane conditions at the junction by dividing the area into lanes, and a branch guidance object that shows the lane conditions at the branch point by dividing the area into lanes. Note that the liquid crystal display 15 displays an image captured by the front camera 19 before the distance from the vehicle to the junction becomes less than the guidance start distance, that is, a scenery (actual scenery image) around the vehicle (particularly in front of the vehicle) at the current time. As a result, it is possible for the vehicle occupant to visually recognize the second guidance object superimposed on the scenery.

[0042] After that, in S6, the CPU 41 judges whether or not the vehicle has passed a branch point. For example, this is judged based on the current position of the vehicle detected by the current position detection unit 11 and map information. Note that it may be judged whether or not the vehicle has passed a merging point before the branch point, instead of the branch point.

[0043] Then, when it is determined that the vehicle has passed the branch point (S6: YES), a control signal is sent to the liquid crystal display 15, and the guidance object displayed on the liquid crystal display 15 is made invisible (S7). When the guidance object is made invisible, the transparency of the image of the displayed guidance object may be increased stepwise according to the distance to the branch point, and the transparency may be set to 100% when the vehicle finally reaches the branch point. Also, the image captured by the front camera 19, i.e., the scenery (actual scenery image) around the vehicle at the current time (particularly in front of the vehicle), continues to be displayed for a certain period of time even after the guidance object is made invisible, and then the display is switched to the map image.

[0044] On the other hand, if it is determined that the vehicle has not passed the branch point (S6: NO), the process returns to S3 and the display of the guide object continues.

[0045] Next, the sub-processing of the first guidance object display processing executed in S4 will be described with reference to Fig. 5. Fig. 5 is a flowchart of the sub-processing program of the first guidance object display processing.

[0046] First, in S11, the CPU 41 recognizes (detects) features located around the vehicle by performing image processing on the image captured by the front camera 19. Specifically, in the first embodiment, the objects to be recognized (detected) are dividing lines drawn on the road surface (including the road surface of the lane in which the vehicle is traveling as well as the road surface of lanes other than the lane in which the vehicle is traveling) and road edges (specifically, the edges of the roadway, which are the boundaries between the roadway and the sidewalk if there is one).

[0047] In addition, the features to be recognized (detected) in S11 do not basically include features on the oncoming lane side, but for example, in a general road where there is no median between the lane corresponding to the travel direction of the vehicle and the oncoming lane and it is divided by a center line, or in a case where there is no center line, features on the oncoming lane are also included in the recognition (detection) targets. In addition, it is desirable to detect the color and type (solid line, dashed line, etc.) of the dividing line. In addition, regarding the road edge, basically, structures such as blocks, guard rails, and median strips provided at the road edge are detected as the road edge, but for roads without such structures at the road edge, a break in the asphalt or the outermost dividing line (when the lane is divided from the oncoming lane by a center line, the center line also becomes the road edge) may be detected as the road edge.

[0048] In the above-mentioned marking detection process, the CPU 41 performs brightness correction on the road surface and marking based on the brightness difference in order to detect markings in the captured image captured by the front camera 19. After that, binarization processing to separate the markings from the image, geometric processing to correct distortion, smoothing processing to remove noise from the image, and other processing are performed to detect the boundary between the road surface and the markings. The existence and type of the markings are identified by the detected boundary. In addition, the color of the markings can also be detected by extracting the image portion of the range where the markings are detected and performing color recognition (RGB value detection).

[0049] On the other hand, in the above road edge detection process, the CPU 41 performs luminance correction based on the luminance difference between the road surface and structures (blocks, guard rails, median strips, etc.) provided on the road edge in the captured image captured by the front camera 19. After that, binarization processing to separate the structures from the image, geometric processing to correct distortion, smoothing processing to remove noise from the image, etc. are performed to detect the boundary between the road surface and the structures. The existence of the road edge is specified by the detected boundary. Note that pattern matching processing using feature points and templates may be performed to detect the division lines and road edges. These image recognition processes are already known, so details will be omitted. In addition to the division lines and road edges, other vehicles and the like located on the road may also be detected.

[0050] After that, in S12, the CPU 41 identifies the range of each lane (excluding the oncoming lane) included in the road on which the vehicle is traveling within the scenery around the vehicle based on the detection results of the dividing line and road edge acquired in the image recognition process of S11. Specifically, the CPU 41 identifies the range of each lane within the captured image captured by the front camera 19 (i.e., the captured image displayed on the liquid crystal display 15).

[0051] For example, in the case where the vehicle is traveling on a road with two lanes on each side, in S11, when the captured image 61 shown in Fig. 6 is captured by the front camera 19, the demarcation lines 62 to 64 are detected. However, for the demarcation line 64, the width of the lane may be calculated from the detection results of the demarcation lines 62 and 63, and the position of the demarcation line 64 may be predicted assuming that it exists to the right of the demarcation line 63, the calculated lane width away. Then, the range 65 surrounded by the demarcation lines 62 and 63, and the range 66 surrounded by the demarcation lines 63 and 64 can be identified as the ranges in which the lanes included in the road on which the vehicle is traveling exist.

[0052] In addition, in S12, it is desirable to specify the range in which lanes exist by taking map information into consideration. For example, by using map information, it is possible to easily determine the presence or absence of oncoming lanes and the presence or absence of a median strip, which are difficult to determine by image recognition alone, and therefore it is possible to more easily and accurately specify the range in which lanes exist on the road on which the vehicle is traveling. Furthermore, since the map information also includes information on the road shape (such as the radius of a curve), road width, and lane width, it is possible to estimate the position of the lane line using the road shape and lane width even in a range where it is difficult to accurately detect the lane line away from the vehicle, and to more accurately specify the range in which lanes exist. In addition, when other vehicles traveling around the vehicle can be detected, it is desirable to specify the range in which lanes exist by using the positions of the detected other vehicles.

[0053] Next, in S13, the CPU 41 determines whether or not a lane of a road merging at a merging point determined to be ahead in the traveling direction of the vehicle (hereinafter referred to as a merging lane) and a lane of a road diverging at a diverging point (hereinafter referred to as a diverging lane) are congested. It may be determined whether both the merging lane and the diverging lane are congested, or it may be determined whether or not at least one of them is congested. It is assumed in the first embodiment that the merging lane 71 and the diverging lane 72 are continuous (connected as one lane) as shown in FIG. 7 in particular, and in that case, the lane between the merging point and the diverging point (the leftmost lane of the three-lane section shown in FIG. 7) is also included in the merging lane 71 or the diverging lane 72.

[0054] As described above, the navigation device 1 of the first embodiment can acquire congestion information indicating the congestion state for each lane generated in the external server device 51 provided in the probe center 50 via the communication module 18, and judges whether the merging lane and the branching lane are congested based on the acquired congestion information. The judgment of whether a lane is congested or not is made based on, for example, the average vehicle speed of the vehicles traveling in that lane, and if the average vehicle speed is equal to or lower than a threshold value (for example, 10 km / h), the lane is deemed to be congested. Alternatively, it may be judged that the lane is congested when a line of congestion is detected in that lane from an image captured by the front camera 19 of the probe car.

[0055] If it is determined that the merging lane and diverging lane ahead in the direction of travel of the vehicle are congested (S13: YES), the display color of the merging lane and diverging lane in the first guidance object to be displayed in S19 described below is set to red (S14).On the other hand, if it is determined that the merging lane and diverging lane ahead in the direction of travel of the vehicle are not congested (S13: NO), the process proceeds to S15.

[0056] Next, in S15, the CPU 41 judges whether or not the main lane (hereinafter referred to as the main lane) to which the merging lane or the branching lane is connected is congested, particularly around the merging point and the branching point ahead in the traveling direction of the vehicle (or between the merging point and the branching point). If there are multiple main lanes, it judges whether or not the multiple lanes are congested. Note that in the first embodiment, it is assumed that the merging lane 71 and the branching lane 72 are continuous (connected as one lane) as shown in FIG. 7, and in that case, among the lanes between the merging point and the branching point, the lanes 73 and 74, excluding the leftmost lane, are regarded as the main lane.

[0057] As described above, the navigation device 1 of the first embodiment can acquire congestion information indicating the congestion state for each lane generated in the external server device 51 provided in the probe center 50 via the communication module 18, and judges whether the main lane is congested based on the acquired congestion information. In addition, the judgment of whether the lane is congested or not is, for example, determined as being congested if the average vehicle speed of the vehicles traveling on the main lane is slower than the current vehicle speed of the vehicle (i.e., the vehicle speed of the vehicle traveling at a position away from the merging point or branching point) by a threshold value (e.g., 20 km / h) or more. Alternatively, the main lane may be deemed to be congested if the average vehicle speed of the vehicles traveling on the main lane is equal to or lower than a threshold value (e.g., 20 km / h). Furthermore, it may be determined that the main lane is congested when the density of vehicles on the main lane is equal to or higher than a threshold value from the image captured by the front camera 19 of the probe car. Basically, the congestion judgment in S15 has a looser judgment criterion than the congestion judgment in S13.

[0058] Then, if it is determined that the main lane ahead in the vehicle's travel direction is congested (S15: YES), the display color of the main lane determined to be congested among the first guidance objects to be displayed in S19 described below is set to gray (S16). Note that if there are multiple main lanes, the display color of only the main lane determined to be congested is set to gray. On the other hand, if it is determined that the main lane ahead in the vehicle's travel direction is not congested (S15: NO), the process proceeds to S17.

[0059] In S17, the CPU 41 sets the display color of the first guidance object to be displayed in S19 described below, except for the lane whose display color is set in S14 and S16, to white. However, the display colors set in S14, S16, and S17 are merely examples, and any combination of colors may be set as long as they are different from each other.

[0060] Next, in S18, the CPU 41 generates a first guide object 75 to be displayed on the liquid crystal display 15. Here, the first guide object 75 generated in S18 has a plurality of rectangular objects 76-78 divided into areas in the left-right direction as shown in FIG. 8. The number of the objects 76-78 (the number of divided areas) corresponds to the number of lanes at the merging point and branching point, and further, the state of each lane (particularly the state of congestion or congestion in the first embodiment) is also indicated by the display color. For example, the example shown in FIG. 8 is a case where a merging / branching lane of one lane is made from the left side of a main line with two lanes on each side, and three objects 76-78 corresponding to the two lanes of the main line and the one lane where the merging / branching is made are generated. Specifically, the leftmost object 76 corresponds to the merging lane and branching lane, the central object 77 corresponds to the lane on the left side of the main line, and the rightmost object 78 corresponds to the lane on the right side of the main line. In addition, an arrow indicating the direction of travel is drawn in each of the objects 76-78, and each of the objects 76-78 is displayed in the display color set in S14, S16, and S17. Therefore, as shown in FIG. 9, when it is determined that the merging lane and the branching lane are not congested and the main lane is not congested, the display color of each of the objects 76-78 is white. When it is determined that the merging lane and the branching lane are congested but the main lane is not congested, the display color of the object 76 is red, and the display colors of the objects 77 and 78 are white. When it is determined that the merging lane and the branching lane are congested and the lane to the left of the main lane is also congested, the display color of the object 76 is red, the display color of the object 77 is gray, and the display color of the object 78 is white. The display color of each of the objects 76-78 may change over time. For example, if the traffic congestion situation changes while the first guidance object 75 is being displayed, the display color may change in the middle of the display.

[0061] The first guidance object 75 generated in S18 also includes an object 79 indicating the distance from the vehicle to the junction (or the distance to the branch point). The number on the object 79 decreases in stages as the vehicle travels.

[0062] Furthermore, in S18, the CPU 41 refers to the lane positions identified in S12 and calculates the display positions of the first guide object 75 required to superimpose each of the objects 76 to 78 above the corresponding lane on the liquid crystal display 15. However, for objects corresponding to merging lanes and branching lanes, the display positions are determined on the assumption that the merging lane and branching lane are located adjacent to the left side of the main lane (the right side if the merging / branching is performed on the right side). For example, in the example shown in FIG. 8, the display positions are determined so that the central object 77 is located above the lane 65 on the left side of the main line displayed on the liquid crystal display 15, and the right object 78 is located above the lane 66 on the right side of the main line. In addition, the display position of the left object 76 is determined to be adjacent to the left of the object 77 according to the above assumption, regardless of the actual positions of the merging lane and branching lane. The display positions of the above objects 76 to 78 will change if the positions of the lanes displayed on the liquid crystal display 15 change. Furthermore, the display position of the object 79 is, for example, to the right of the objects 76 to 78, but it may be above or below.

[0063] Next, in S19, the CPU 41 transmits a control signal to the liquid crystal display 15, and draws the first guidance object 75 generated in S18 in the position (range) also determined in S18. Note that the liquid crystal display 15 displays an image captured by the front camera 19 before the distance from the vehicle to the junction becomes less than the guidance start distance, that is, the scenery (actual scenery image) around the vehicle (particularly in front of the vehicle) at the current time. As a result, it becomes possible for the vehicle occupant to visually recognize the first guidance object 75 superimposed on the scenery, as shown in FIG. 8.

[0064] By visually checking the first guidance object 75 shown in FIG. 8, vehicle occupants can easily grasp that there are successive merging points and branching points ahead in the direction of travel, the distance to the merging point, and further, the traffic congestion and congestion conditions of each lane at the merging point and branching point based on the display color of the areas divided by lane.

[0065] The processes of S11 to S19 are repeatedly executed until the vehicle passes the branch point.

[0066] Next, the sub-processing of the second guide object display processing executed in S5 will be described with reference to Fig. 10. Fig. 10 is a flowchart of the sub-processing program of the second guide object display processing.

[0067] First, in S21, the CPU 41 recognizes (detects) features located around the vehicle by performing image processing on the image captured by the front camera 19. Specifically, in the first embodiment, the objects to be recognized (detected) are dividing lines drawn on the road surface (including the road surface of the lane in which the vehicle is traveling, as well as the road surface of lanes other than the lane in which the vehicle is traveling) and road edges (specifically, the edges of the roadway, which are the boundaries between the roadway and the sidewalk if there is one). The details are the same as in S11 of the first guidance object display processing, so a description thereof will be omitted.

[0068] Thereafter, in S22, the CPU 41 identifies the range of each lane (excluding the oncoming lane) included in the road on which the vehicle is traveling within the scenery around the vehicle based on the detection results of the lane markings and road edges acquired in the image recognition process of S21. Specifically, the range of each lane is identified within the captured image captured by the front camera 19 (i.e., the captured image displayed on the liquid crystal display 15). The details are the same as those of S12 in the first guidance object display process, and therefore a description thereof will be omitted.

[0069] Next, in S23, the CPU 41 determines whether or not at least one of the merging lane and the diverging lane is congested. In the first embodiment, it is assumed that the merging lane 71 and the diverging lane 72 are continuous (connected as one lane) as shown in Fig. 7. In this case, the lane between the merging point and the diverging point (the leftmost lane of the three-lane section shown in Fig. 7) may be included in the merging lane 71 or the diverging lane 72.

[0070] As described above, the navigation device 1 of the first embodiment can acquire congestion information indicating the congestion state for each lane generated in the external server device 51 provided in the probe center 50 via the communication module 18, and judges whether the merging lane and the branching lane are congested based on the acquired congestion information. The judgment of whether a lane is congested or not is made based on, for example, the average vehicle speed of the vehicles traveling in that lane, and if the average vehicle speed is equal to or lower than a threshold value (for example, 10 km / h), the lane is considered to be congested. Alternatively, it may be judged that a lane is congested when a line of congestion is detected in that lane from an image captured by the front camera 19 of the probe car.

[0071] Then, if it is determined that at least one of the merging lane and diverging lane ahead in the direction of travel of the vehicle is congested (S23: YES), the display color of the merging lane and diverging lane determined to be congested among the second guidance objects to be displayed in S31 described below is set to red (S24). Note that the display colors are set separately for the merging lane and the diverging lane. On the other hand, if it is determined that neither the merging lane nor the diverging lane ahead in the direction of travel of the vehicle is congested (S23: NO), the process proceeds to S25.

[0072] Next, in S25, the CPU 41 determines whether or not the main lanes (hereinafter referred to as main lanes) to which the merging lanes and branching lanes are connected are congested, particularly around the merging point and branching point ahead in the traveling direction of the vehicle. If there are multiple main lanes, it determines whether or not the multiple lanes are congested. Note that in the first embodiment, it is assumed that the merging lane 71 and the branching lane 72 are continuous (connected as one lane) as shown in FIG. 7, and in that case, among the lanes between the merging point and the branching point, lanes 73 and 74, excluding the leftmost lane, are regarded as the main lanes.

[0073] As described above, the navigation device 1 of the first embodiment can acquire congestion information indicating the congestion state for each lane generated in the external server device 51 provided in the probe center 50 via the communication module 18, and judges whether the main lane is congested based on the acquired congestion information. In addition, the judgment of whether the lane is congested or not is, for example, determined as being congested if the average vehicle speed of the vehicles traveling on the main lane is slower than the current vehicle speed of the vehicle (i.e., the vehicle speed of the vehicle traveling at a position away from the merging point or branching point) by a threshold value (e.g., 20 km / h) or more. Alternatively, the main lane may be deemed to be congested if the average vehicle speed of the vehicles traveling on the main lane is equal to or lower than a threshold value (e.g., 20 km / h). Furthermore, it may be determined that the main lane is congested when the density of vehicles on the main lane is equal to or higher than a threshold value from the image captured by the front camera 19 of the probe car. Basically, the congestion judgment in S25 has a looser judgment criterion than the congestion judgment in S23.

[0074] Then, if it is determined that the main lane ahead in the vehicle's traveling direction is congested around the junction and branch point (S25: YES), the display color of the main lane determined to be congested among the second guidance objects to be displayed in S31 described below is set to gray (S26). Note that if there are multiple main lanes, the display color of only the main lane determined to be congested is set to gray. Note that separate display colors are set for the main lane around the junction and the main lane around the branch point. On the other hand, if it is determined that the main lane ahead in the vehicle's traveling direction is not congested (S25: NO), the process proceeds to S27.

[0075] In S27, the CPU 41 sets the display color of the second guidance object to be displayed in S31 described below, except for the lane whose display color is set in S24 and S26, to white. However, the display colors set in S24, S26, and S27 are merely examples, and any combination of colors may be set as long as they are different from each other.

[0076] Next, in S28, the CPU 41 acquires the distance between the junction point and the branch point (the junction section) calculated in S3.

[0077] Next, in S29, the CPU 41 determines the shape of a second guide object 80 to be displayed in S31, which will be described later, based on the distance of the merging section acquired in S28. Here, as shown in Fig. 11 and Fig. 12, the second guide object 80 to be displayed has a merging guide object 81 showing the lane conditions at the merging point by dividing areas by lane, and a branch guide object 82 showing the lane conditions at the branching point by dividing areas by lane, and further the merging guide object 81 and the branch guide object 82 are displayed side by side in the vertical direction. Also, the merging guide object 81 and the branch guide object 82 are divided into a plurality of areas, and notches 83 are formed in the corners of the areas.

[0078] Therefore, in S29, the CPU 41 first determines the arrangement interval D between the merging guide object 81 and the branch guide object 82 based on the distance of the merging section. Basically, the longer the distance of the merging section, the wider the arrangement interval D between the merging guide object 81 and the branch guide object 82 is set; for example, if the distance of the merging section is less than 300 m, D=1 mm, if the distance of the merging section is 300 m or more but less than 500 m, D=3 mm, and if the distance of the merging section is 500 m or more, D=5 mm. Furthermore, in S29, the CPU 41 secondly determines the angle θ of the notch 83 formed based on the distance of the merging section. Basically, the longer the distance of the merging section, the larger the angle θ of the notch 83 is set (as a notch with an obtuse angle), and the angle is changed in a stepwise manner, for example, θ=30 degrees if the distance of the merging section is less than 300 m, θ=45 degrees if the distance of the merging section is 300 m or more but less than 500 m, and θ=60 degrees if the distance of the merging section is 500 m or more. In other words, in the first embodiment, the shape of the second guide object 80 to be displayed (specifically, the angle of the cutout 83 and the spacing between the merging guide object 81 and the branching guide object 82) will change depending on the positional relationship between the merging point and the branching point.

[0079] Next, in S30, the CPU 41 generates a second guide object 80 to be displayed on the liquid crystal display 15 based on the arrangement interval D and the cutout angle θ set in S29. Here, the second guide object 80 generated in S30 has a merging guide object 81 showing the lane conditions at the merging point divided into areas for each lane, and a branch guide object 82 showing the lane conditions at the branching point divided into areas for each lane, as shown in Fig. 11, and further the merging guide object 81 has a plurality of rectangular objects 84-86 whose areas are divided in the left-right direction. Similarly, the branch guide object 82 has a plurality of rectangular objects 87-89 whose areas are divided in the left-right direction.

[0080] First, the merging guidance object 81 will be described. The number of objects 84-86 of the merging guidance object 81 (the number of divided areas) corresponds to the number of lanes at the merging point, and the state of each lane at the merging point (particularly the state of congestion or congestion in the first embodiment) is also indicated by the display color. For example, the example shown in FIG. 11 is a case where one lane merges from the left side of a main line with two lanes on each side, and three objects 84-86 corresponding to the two lanes of the main line and the one lane where the merging occurs are generated. Specifically, the leftmost object 84 corresponds to the merging lane, the central object 85 corresponds to the lane on the left side of the main line, and the rightmost object 86 corresponds to the lane on the right side of the main line. Also, an arrow indicating the traveling direction is drawn in each of the objects 84-86, and each of the objects 84-86 is displayed in the display color set in the above S24, S26, and S27. The display colors of the objects are the same as those of the first guidance object 75 already described. When it is determined that the merging lane is not congested and the main lane is not congested, the display colors of the objects 84 to 86 are white. When it is determined that the merging lane is congested but the main lane is not congested, the display color of the object 84 is red, and the display colors of the objects 85 and 86 are white. When it is determined that the merging lane is congested and the lane to the left of the main lane is also congested, the display color of the object 84 is red, the display color of the object 85 is gray, and the display color of the object 86 is white. The display colors of the objects 84 to 86 may change over time. For example, if the traffic congestion situation changes while the second guidance object 80 is being displayed, the display color may change accordingly. In addition, a notch 83 is formed at the corner of the object between the left object 84 corresponding to the merging lane and the objects 85 and 86 corresponding to the main lane. The angle θ of the notch 83 is set in S29 above, and is determined based on the distance of the merging section as described above.

[0081] Next, the branch guide object 82 will be described. The number of objects 87-89 of the branch guide object 82 (the number of divided areas) corresponds to the number of lanes at the branch point, and the state of each lane at the branch point (particularly the state of congestion or congestion in the first embodiment) is also indicated by the display color. For example, the example shown in FIG. 11 is a case where a branch of one lane is made from the left side of a main line with two lanes on each side, and three objects 87-89 corresponding to the two lanes of the main line and the one lane where the branch is made are generated. Specifically, the leftmost object 87 corresponds to the branch lane, the central object 88 corresponds to the lane on the left side of the main line, and the rightmost object 89 corresponds to the lane on the right side of the main line. Also, an arrow indicating the traveling direction is drawn in each of the objects 87-89, and further, each of the objects 87-89 is displayed in the display color set in the above S24, S26, and S27. The display colors of the objects are the same as those of the first guidance object 75 already described. When it is determined that the branch lane is not congested and the main lane is not congested, the display colors of the objects 87 to 89 are white. When it is determined that the branch lane is congested but the main lane is not congested, the display color of the object 87 is red, and the display colors of the objects 88 and 89 are white. When it is determined that the branch lane is congested and the lane to the left of the main lane is also congested, the display color of the object 87 is red, the display color of the object 88 is gray, and the display color of the object 89 is white. The display colors of the objects 87 to 89 may change over time. For example, if the traffic congestion situation changes while the second guidance object 80 is being displayed, the display color may change accordingly. In addition, a notch 83 is formed at the corner of the object between the left object 87 corresponding to the branch lane and the objects 88 and 89 corresponding to the main lane. The angle θ of the notch 83 is set in S29 above, and is determined based on the distance of the merging section as described above.

[0082] The second guidance object 80 generated in S30 also includes an object 90 indicating the distance from the vehicle to the junction, and an object 91 indicating the distance from the vehicle to the branch point. The numbers on the objects 90 and 91 decrease gradually as the vehicle travels.

[0083] Furthermore, in S30, the CPU 41 refers to the lane positions identified in S22 and calculates the display positions of the second guide object 80 required to superimpose each of the objects 84 to 89 above the corresponding lane on the liquid crystal display 15. However, for objects corresponding to merging lanes and branching lanes, the display positions are determined on the assumption that the merging lane and branching lane are located adjacent to the left side of the main lane (the right side if the merging / branching is performed on the right side). For example, in the example shown in FIG. 11, the display positions are determined so that the central object 85 and object 88 are located above the lane 65 on the left side of the main line displayed on the liquid crystal display 15, and the right object 86 and object 89 are located above the lane 66 on the right side of the main line. In addition, for the left object 84 and object 87, the display positions are determined to be adjacent to the left of the objects 85 and 88 according to the above assumption, regardless of the actual positions of the merging lane and branching lane.

[0084] The merging guidance object 81 and the branch guidance object 82 are displayed in this order from the bottom up so as to be superimposed above the road on which the vehicle is traveling, but the arrangement interval D between the merging guidance object 81 and the branch guidance object 82 is set in S29 and is determined based on the distance of the merging section as described above. The display positions of the above objects 84 to 89 change when the position of the lane displayed on the liquid crystal display 15 changes. The display positions of the objects 90 and 91 are, for example, to the right of the objects 84 to 89, but may be above or below them.

[0085] Next, in S31, the CPU 41 transmits a control signal to the liquid crystal display 15, and draws the second guide object 80 generated in S30 in the position (range) determined in S30. The liquid crystal display 15 displays an image captured by the front camera 19 before the distance from the vehicle to the junction becomes less than the guidance start distance, that is, the scenery (actual scenery image) around the vehicle (particularly in front of the vehicle) at the current time. As a result, it becomes possible for the vehicle occupant to visually recognize the second guide object 80 superimposed on the scenery, as shown in FIG. 11.

[0086] By visually checking the second guidance object 80 shown in Fig. 11, the vehicle occupant can easily grasp that there are consecutive merging points and branching points ahead in the traveling direction, the distance to the merging point, the distance to the branching point, and further the traffic congestion and congestion of each lane at the merging point and the branching point. In particular, unlike the first guidance object 75, the second guidance object 80 separates a merging guidance object 81 that provides guidance to the merging point and a branching guidance object 82 that provides guidance to the branching point, and displays them on the screen at the same time, that is, the guidance objects provide guidance to the merging point and the branching point together (in parallel), so that it is easy to grasp the relative relationship between the merging point and the branching point. Furthermore, the length of the merging section (the distance from the merging point to the branching point) can be intuitively identified by the angle of the cutout 83 and the arrangement interval of the merging guidance object 81 and the branching guidance object 82. As a result, for example, if a vehicle occupant plans to exit the main lane at a junction ahead in the direction of travel, the distance of the merging section (the distance it takes to change lanes) and the congestion status of the lanes can be known in advance, making it possible to take measures such as planning an early lane change while taking into account the congestion status of the main lane and the junction lanes.

[0087] The processes of S21 to S31 are repeatedly executed until the vehicle passes the branch point.

[0088] As described above in detail, the navigation device 1 and the computer program executed by the navigation device 1 according to the first embodiment acquire the junction and branch points where roads merge ahead of the vehicle in the direction of travel (S1), and if there are a junction and a branch point in succession ahead of the vehicle in the direction of travel, a guidance object that provides guidance to both the junction and the branch point is displayed (S5). In particular, when displaying the guidance object, the shape of the guidance object is changed based on the positional relationship between the junction and the branch point (S29), so that specific guidance can be provided in a visually easy-to-understand manner, not only regarding the existence of the junction and branch point, but also regarding the positional relationship between the junction and the branch point. In addition, the guide objects include a merging guide object that shows the lane conditions at a merging point by dividing areas by lane, and a branching guide object that shows the lane conditions at a branching point by dividing areas by lane, so that vehicle occupants can simultaneously grasp the lane conditions at a merging point and the lane conditions at a branching point separately, and also easily grasp the relative relationship between the merging point and the branching point. In addition, the distance between the merging point and the branching point is obtained (S28), a merging guide object and a branching guide object with cutouts formed in the corners of the area are displayed, and the angle of the cutout formed in the merging guide object is gradually changed depending on the distance between the merging point and the branching point (S29), so that the vehicle occupants can intuitively identify the length of the merging section based on the angle of the cutout formed in the displayed guide object. In addition, the distance between the merging point and the branching point is obtained (S28), and the merging guidance object and the branching guidance object are displayed in that order from bottom up, superimposed on the road on which the vehicle is traveling, in the scenery around the vehicle. The longer the distance between the merging point and the branching point, the greater the spacing between the merging guidance object and the branching guidance object (S29), so that the vehicle occupants can intuitively determine the length of the merging section from the spacing between the displayed guidance objects.

[0089] [Second embodiment] Next, a superimposed image display device according to a second embodiment will be described with reference to Fig. 13 and Fig. 14. In the following description, the same reference numerals as those in the configuration of the superimposed image display device according to the first embodiment in Fig. 1 to Fig. 12 indicate the same or corresponding parts as those in the configuration of the superimposed image display device according to the first embodiment.

[0090] The schematic configuration of the superimposed image display device according to the second embodiment is almost the same as that of the superimposed image display device according to the first embodiment. In addition, various control processes are also almost the same as those of the superimposed image display device according to the first embodiment. However, the superimposed image display device of the first embodiment differs in that it displays an image captured by the front camera 19 on the LCD display 15 of the navigation device 1, and further displays a guide object on the LCD display 15, thereby superimposing the guide object on the scenery around the vehicle, whereas the superimposed image display device of the second embodiment uses a head-up display system as a means for displaying an image to be superimposed on the scenery around the vehicle.

[0091] The schematic configuration of the superimposed image display device according to the second embodiment will be described below with reference to Fig. 13. Fig. 13 is a schematic configuration diagram of a superimposed image display device 101 according to the second embodiment. 13, the superimposed image display device 101 basically includes a navigation device 103 mounted on a vehicle 102, and a front display 104 also mounted on the vehicle 102 and connected to the navigation device 103. The front display 104 functions as a head-up display together with a windshield 105 of the vehicle 102, and serves as an information providing means for providing various information to an occupant 106 of the vehicle 102.

[0092] Here, the front display 104 is a liquid crystal display that is installed inside the dashboard 107 of the vehicle 102 and has a function of displaying an image on an image display surface provided on the front. For example, a CCFL (cold cathode fluorescent lamp) or a white LED is used as a backlight. Note that, as the front display 104, in addition to a liquid crystal display, an organic EL display or a combination of a liquid crystal projector and a screen may also be used.

[0093] The front display 104 functions as a head-up display together with a windshield 105 of the vehicle 102, and is configured to reflect an image output from the front display 104 onto the windshield 105 in front of the driver's seat so that the image can be visually recognized by an occupant 106 of the vehicle 102. Note that a guide object is displayed on the front display 104 as necessary. Note that in the second embodiment described below, the guide object is a guide image for providing guidance to a junction and a branch point that are consecutively present ahead in the traveling direction of the vehicle, similar to the first embodiment.

[0094] Also, when the occupant 106 visually recognizes the image displayed on the front display 104 reflected by the windshield 105, the image displayed on the front display 104 at a distant position beyond the windshield 105 is visually recognized by the occupant 106 as a virtual image 110, not at the position of the windshield 105. Also, the virtual image 110 is displayed superimposed on the surrounding environment (landscape, real scene) in front of the vehicle, and can be displayed superimposed on any object (road surface, building, object to be warned, etc.) located in front of the vehicle.

[0095] Here, the position where the virtual image 110 is generated, more specifically, the distance L from the occupant 106 to the virtual image 110 (hereinafter referred to as the imaging distance), is determined by the position of the front display 104. For example, the imaging distance L is determined by the distance (optical path length) along the optical path from the position where the image is displayed on the front display 104 to the windshield 105. For example, the optical path length is set so that the imaging distance L is 1.5 m.

[0096] A front camera 111 is installed above the front bumper of the vehicle or on the back side of the rearview mirror. The front camera 111 is an imaging device having a camera using a solid-state imaging element such as a CCD, and is installed with its optical axis direction facing forward in the traveling direction of the vehicle. Then, image processing is performed on the image captured by the front camera 111, so that the situation of the front environment (i.e., the environment on which the virtual image 110 is superimposed) visually recognized by the occupant 106 through the windshield is detected. Note that a sensor such as a millimeter wave radar may be used instead of the front camera 111.

[0097] An in-vehicle camera 112 is installed on the top surface of the vehicle's instrument panel. The in-vehicle camera 112 is an imaging device having a camera using a solid-state imaging element such as a CCD, and is installed with its optical axis direction facing the driver's seat. A range in the vehicle where the face of an occupant is generally expected to be located is set as a detection range (imaging range of the in-vehicle camera 112), and an image of the face of the occupant 106 sitting in the driver's seat is captured. Then, image processing is performed on the captured image captured by the in-vehicle camera 112, thereby detecting the eye position (starting point of gaze) and gaze direction of the occupant 106.

[0098] Then, in S4 and S5 of the driving support processing program (FIG. 3) described above, the superimposed image display device according to the second embodiment displays an image 120 of a guide object on the front display 104 as shown in Fig. 14. As a result, when a vehicle occupant visually recognizes the image 120 of the guide object displayed on the front display 104 as shown in Fig. 14, the vehicle occupant visually recognizes a virtual image 121 of the image 120 of the guide object superimposed on the scenery seen through the windshield 105.

[0099] This makes it possible to easily grasp the number of lanes at a junction or branching point, the state of each lane (congestion), the distance from the vehicle to the junction, the distance from the vehicle to the branching point, the distance of the junction section, etc. In the superimposed image display device according to the second embodiment, the size, shape, and position (range) of the guide object to be displayed on the front display 104 are determined in the guide object display process of S4 and S5.

[0100] Incidentally, the present invention is not limited to the above-described embodiment, and it is needless to say that various improvements and modifications are possible without departing from the spirit and scope of the present invention. For example, in the first embodiment, the liquid crystal display 15 displaying the actual scene image is used as a means for displaying an image to be superimposed on the scenery around the vehicle, and in the second embodiment, a head-up display system is used, but an image may be displayed directly on the windshield. Specifically, the windshield may be used as a screen to display an image from a projector, or the windshield may be a transparent liquid crystal display. The image displayed on the windshield is the image to be superimposed on the scenery around the vehicle.

[0101] In the first and second embodiments, guidance is provided by the first guidance object when the distance of the merging section is 1 km or more, and guidance is provided by the second guidance object when the distance of the merging section is less than 1 km, but guidance may be provided by the second guidance object regardless of the distance of the merging section. Also, the first and second guidance objects do not necessarily have to be rectangular as shown in Fig. 8 and Fig. 11, and may be other shapes as long as they can indicate the lane conditions (congestion and traffic congestion) at the merging point or branching point.

[0102] In addition, in the first embodiment, the real scene image captured by the front camera 19 and the guide objects are displayed on the liquid crystal display 15 of the navigation device 1, but the display for displaying the real scene image and the guide objects may be a display other than the liquid crystal display 15 as long as it is a display arranged inside the vehicle.

[0103] In the second embodiment, the front display 104 is configured to generate a virtual image in front of the windshield 105 of the vehicle 102, but the virtual image may be generated in front of a window other than the windshield 105. In addition, the target onto which the image is reflected by the front display 104 may be a visor (combiner) installed around the windshield 105, rather than the windshield 105 itself.

[0104] In the first and second embodiments, the navigation ECU 13 of the navigation device 1 executes the driving support processing program (FIG. 3), but the execution entity can be changed as appropriate. For example, the control unit of the liquid crystal display 15, the vehicle control ECU, or other in-vehicle device may execute the program. [Explanation of symbols]

[0105] 1...navigation device, 15...liquid crystal display, 19...front camera, 41...CPU, 42...RAM, 43...ROM, 61...captured image, 75...first guidance object, 80...second guidance object, 81...merging guidance object, 82...junction guidance object, 83...notch

Claims

1. 1. A superimposed image display device for displaying information to a vehicle occupant in a vehicle by superimposing a guide object on a landscape around the vehicle, the superimposed image display device comprising: A junction point acquisition means for acquiring a junction point where roads merge, the junction point being located ahead in a traveling direction of the vehicle; a branch point acquisition means for acquiring a branch point at which a road branches ahead in a traveling direction of the vehicle; an object display means for displaying the guide object that provides guidance to both the junction and the branch point when the junction and the branch point are located consecutively in this order ahead in a traveling direction of the vehicle, The object display means is a superimposed image display device that changes and displays a shape of the guide object based on a positional relationship between the junction and the branching point.

2. 2. The superimposed image display device according to claim 1, wherein the guidance objects displayed by the object display means include a merging guidance object that shows the lane conditions at the merging point by dividing areas for each lane, and a branching guidance object that shows the lane conditions at the branching point by dividing areas for each lane.

3. a distance acquisition means for acquiring a distance between the junction and the branch point, The object display means includes: displaying the junction guide object and the branch guide object with notches formed at corners of the area; The superimposed image display device according to claim 2 , wherein an angle of the cutout formed in the merging guide object is changed stepwise depending on a distance between the merging point and the branching point.

4. a distance acquisition means for acquiring a distance between the junction and the branch point, The object display means includes: displaying the junction guidance object and the branch guidance object in this order from bottom to top so as to be superimposed above a road on which the vehicle is traveling in the scenery around the vehicle; The superimposed image display device according to claim 2 , wherein the longer the distance between the junction point and the branch point, the wider the interval between the junction guide object and the branch guide object that are displayed.

Images