Superimposed image display device

Abstract

To provide a superimposed image display device for preventing the occupants of a vehicle from being disadvantaged by displaying guidance objects in a display mode according to the reliability of the specific result of a driving lane even when the reliability is low.SOLUTION: A superimposed image display device includes the steps of: acquiring a recommended lane which is recommended for driving on a road on which a vehicle currently drives; identifying a driving lane which is a traffic lane on which the vehicle currently drives on the road on which the vehicle currently drives; furthermore, acquiring the confidence indicating how reliable the identified result is, for the identified traveling lane; and displaying the guidance objects for performing the guiding on the recommended lane in a display mode according to the reliability.SELECTED DRAWING: Figure 3

Description

Technical Field

[0001] The present invention relates to a superimposed image display device for performing driving support of a vehicle.

Background Art

[0002] Conventionally, various means have been used as information providing means for providing various types of information for performing driving support of a vehicle, such as route guidance and warning of obstacles, to a vehicle occupant. For example, display on a liquid crystal display installed in a vehicle, voice output from a speaker, and the like. In recent years, as one of such information providing means, there is a device that provides information by displaying an image superimposed on the surrounding environment (scenery, real scene) of the occupant. For example, a head-up display, a windshield display, and a method of displaying an image superimposed on an imaging image around the vehicle displayed on a liquid crystal display are applicable.

[0003] Here, as a technique for guiding particularly the recommended lane on which the vehicle should travel by displaying an image superimposed on the surrounding environment, for example, in Japanese Patent Application Laid-Open No. 2014-48146, an image obtained by imaging the front scenery of the vehicle is displayed on a display unit, and when the remaining distance to an intersection is greater than a reference value and the vehicle is not traveling on the recommended lane for passing through the intersection, an arrow image is displayed superimposed on the recommended lane included in the front scenery in order to notify the occupant of the position of the recommended lane.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] However, in order to provide guidance for the recommended lane by superimposing an image as disclosed in Patent Document 1 on the road surface of the recommended lane, it is necessary to be able to specify the position of the recommended lane as seen from the vehicle (that is, on which side and at what distance from the lane in which the vehicle is currently traveling the recommended lane is located). And, in order to specify the position of the recommended lane as seen from the vehicle, it is necessary to accurately specify the lane in which the vehicle is currently traveling (hereinafter referred to as the traveling lane) as a prerequisite.

[0006] In the technology of Patent Document 1, as a means for specifying the traveling lane, image recognition is performed on a camera image that captures the front of the vehicle to detect lane lines, and the traveling lane is specified based on the result. However, for example, in the case of poor weather or when the paint of the lane lines is thin, it is considered that the specified result of the traveling lane may not be reliable. In Patent Document 1, even when the specified result of such a traveling lane is not reliable, guidance by superimposing an image on the road surface of the recommended lane is performed, so there is a risk of incorrect guidance, which may cause disadvantages to the passengers of the vehicle.

[0007] The present invention has been made to solve the above-mentioned conventional problems. When guiding the recommended lane by a guidance object displayed superimposed on the scenery around the vehicle, by displaying the guidance object in a display mode according to the reliability of the specified result of the traveling lane, even when the reliability is low, it is an object of the present invention to provide a superimposed image display device that prevents disadvantages to the passengers of the vehicle.

Means for Solving the Problem

[0008] In order to achieve the above object, the superimposed image display device according to the present invention is mounted on a vehicle and superimposes a guiding object for guiding information to an occupant of the vehicle on a landscape around the vehicle for visual recognition, and includes a recommended lane acquisition means for acquiring a recommended lane recommended for traveling on a road on which the vehicle is currently traveling, a traveling lane identification means for identifying a traveling lane, which is the lane on which the vehicle is currently traveling, on the road on which the vehicle is currently traveling, a reliability acquisition means for acquiring a reliability indicating how reliable the identified result is for the traveling lane identified by the traveling lane identification means, and an object display means for displaying the guiding object for guiding the recommended lane based on the traveling lane and the recommended lane, wherein the object display means displays the guiding object in a display mode according to the reliability. Note that the "landscape" includes not only the actual landscape visually recognized from the vehicle (actual scene), but also an image obtained by imaging the landscape, an image reproducing the landscape, and the like. In addition, the "guidance for the recommended lane" may be guidance for allowing the vehicle to recognize the position and direction of the recommended lane without the purpose of guiding the vehicle to the recommended lane, or may be guidance for guiding the vehicle to the recommended lane.

Effect of the Invention

[0009] According to the superimposed image display device according to the present invention having the above configuration, when guiding the recommended lane by a guiding object displayed superimposed on the landscape around the vehicle, by displaying the guiding object in a display mode according to the reliability of the identification result of the traveling lane, it is possible to provide accurate guidance when the reliability is high and to prevent causing disadvantages to the occupants of the vehicle even when the reliability is low.

Brief Description of the Drawings

[0010]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Figure 16

Figure 17

Figure 18

Figure 19

Figure 20

Embodiments for Carrying Out the Invention

[0011] Hereinafter, the first embodiment and the second embodiment in which the superimposed image display device according to the present invention is embodied in a navigation device will be described in detail with reference to the drawings.

[0012] [First Embodiment] First, the schematic configuration of the navigation device 1 according to the first embodiment will be described with reference to FIG. 1. FIG. 1 is a block diagram showing the navigation device 1 according to the first embodiment.

[0013] As shown in FIG. 1, the navigation device 1 according to the first embodiment includes a current position detection unit 11 that detects the current position of the vehicle on which the navigation device 1 is mounted, a data recording unit 12 in which various data are recorded, a navigation ECU 13 that performs various arithmetic processes based on the input information, an operation unit 14 that receives operations from the user, a liquid crystal display 15 that displays a real-scene image obtained by imaging the front in the traveling direction for the user, a speaker 16 that outputs voice guidance regarding route guidance, a DVD drive 17 that reads a DVD which is a storage medium, and a communication module 18 that communicates with an information center such as a probe center or a VICS (registered trademark: Vehicle Information and Communication System) center. Further, the navigation device 1 is connected to a front camera 19 and various sensors installed in the vehicle on which the navigation device 1 is mounted via an in-vehicle network such as CAN.

[0014] Hereinafter, each component of the navigation device 1 will be described in order. The current position detection unit 11 consists of a GPS 21, a vehicle speed sensor 22, a steering sensor 23, a gyro sensor 24, etc., and is capable of detecting the current position, orientation, traveling speed of the vehicle, the current time, etc. Here, in particular, the vehicle speed sensor 22 is a sensor for detecting the moving distance and vehicle speed of the vehicle, generates a pulse according to the rotation of the driving wheels of the vehicle, and outputs a pulse signal to the navigation ECU 13. Then, the navigation ECU 13 calculates the rotational speed and moving distance of the driving wheels by counting the generated pulses. Note that it is not necessary for the navigation device 1 to be equipped with all of the above four types of sensors, and the navigation device 1 may be configured to be equipped with only one or more types of these sensors.

[0015] Also, the data recording unit 12 includes an external storage device and a hard disk (not shown) as a recording medium, and a recording head (not shown) which is a driver for reading map information DB 31 and a predetermined program etc. recorded on the hard disk and writing predetermined data to the hard disk. Note that the data recording unit 12 may be configured by a flash memory, a memory card, an optical disk such as a CD or a DVD instead of the hard disk. Also, the map information DB 31 may be stored in an external server and the navigation device 1 may be configured to acquire it by communication.

[0016] Here, the map information DB 31 is, for example, a storage means in which link data 32 regarding roads (links), node data 33 regarding node points, branch point data 34 regarding branch points, point data regarding points such as facilities, map display data for displaying a map, search data for searching for a route, search data for searching for a point, etc. are stored.

[0017] In addition, as the link data 32, for each link constituting the road, data representing the width, gradient, cant, bank, road surface condition, number of lanes of the road, traffic control for the traveling direction of each lane, presence or absence of oncoming lanes (whether it is a oncoming traffic section), locations where the number of lanes decreases, locations where the width narrows, level crossings, etc. are recorded. For corners, data representing the radius of curvature, intersections, T-junctions, entrances and exits of corners, etc. are recorded. For road attributes, data representing downhill roads, uphill roads, etc. are recorded. For road types, data representing expressways and general roads (national roads, prefectural roads, narrow streets, etc.) are recorded respectively.

[0018] In addition, as the node data 33, coordinates (positions) of node points set at predetermined distances according to the radius of curvature, etc. at the branch points (including intersections, T-junctions, etc.) of the actual road, node attributes representing whether the node is a node corresponding to an intersection, a list of link numbers of links connected to the node (connection link number list), a list of node numbers of adjacent nodes adjacent to the node via a link (adjacent node number list), data regarding the height (altitude) of each node point, etc. are recorded.

[0019] In addition, as the branch point data 34, the intersection name of the branch point, corresponding node information identifying the nodes forming the branch point, connection link information identifying the links connected to the branch point, aspect names corresponding to the links connected to the branch point, information identifying the shape of the branch point, etc. are stored. Also, structures that can serve as landmarks when providing right and left turn guidance at the branch point are also stored.

[0020] On the one hand, the navigation ECU (Electronic Control Unit) 13 is an electronic control unit that controls the entire navigation device 1, and includes a CPU 41 as an arithmetic unit and a control unit, and a RAM 42 that is used as a working memory when the CPU 41 performs various arithmetic processes and stores route data and the like when a route is searched. In addition to the control program, an internal storage device such as a ROM 43 in which a driving support processing program (Fig. 2) described later and the like are recorded, and a flash memory 44 that stores the program read from the ROM 43. The navigation ECU 13 has various means as processing algorithms. For example, the recommended lane acquisition means acquires the recommended lane for driving on the road on which the vehicle is currently traveling. The driving lane identification means identifies the driving lane in which the vehicle is currently traveling on the road on which the vehicle is currently traveling. The reliability acquisition means acquires the reliability indicating the degree of reliability of the identified result for the driving lane identified by the driving lane identification means. The object display means displays a guidance object for guiding the vehicle with respect to the recommended lane based on the driving lane and the recommended lane.

[0021] The operation unit 14 is operated when inputting the departure point as the driving start point and the destination as the driving end point, and has a plurality of operation switches (not shown) such as various keys and buttons. Then, the navigation ECU 13 performs control to execute corresponding various operations based on the switch signals output by pressing each switch. Incidentally, the operation unit 14 may be configured to have a touch panel provided on the front surface of the liquid crystal display 15. Further, it may be configured to have a microphone and a voice recognition device.

[0022] In addition, on the liquid crystal display 15, map images including roads, traffic information, operation guidance, operation menus, key guidance, guidance routes from the departure point to the destination, guidance information along the guidance routes, news, weather forecasts, time, emails, TV programs, etc. are displayed. In particular, in the first embodiment, at the stage when the vehicle approaches a guidance branch point, the liquid crystal display 15 displays the captured image taken by the front camera 19, that is, the scenery (real scene image) around the vehicle at the current time (especially in front of the vehicle), and further, if necessary, guidance objects are superimposed and displayed on the scenery.

[0023] Here, the guidance objects superimposed and displayed on the scenery include various information used for information about the vehicle and the support of the driver's driving. For example, warnings for objects to be warned against by the occupant (other vehicles, pedestrians, guidance signs), guidance routes set by the navigation device 1 and guidance information based on the guidance routes (arrows indicating the right / left turn directions, icons indicating landmarks of the guidance branch points, distances to the guidance branch points, positions and directions of the recommended lanes on which the vehicle should travel, guidance prompting lane changes to the recommended lanes, etc.), warnings displayed on the road surface (collision warning, speed limit, etc.), lane dividing lines of the lane on which the vehicle is traveling, current vehicle speed, shift position, remaining energy, advertisement images, facility information, guidance signs, map images, traffic information, news, weather forecasts, time, screens of connected smartphones, etc. In the first embodiment described below, the guidance objects are guidance information for guiding at the guidance branch point in front of the traveling direction of the vehicle. More specifically, arrows indicating the exit directions of the guidance branch points along the guidance routes, guidance images indicating the positions and directions of the recommended lanes on the road on which the vehicle is currently traveling for passing through the guidance branch points along the guidance routes, guidance images prompting movement to the recommended lanes, etc.

[0024] In addition, the speaker 16 outputs voice guidance for guiding travel along the guidance route and guidance of traffic information based on instructions from the navigation ECU 13.

[0025] In addition, the DVD drive 17 is a drive capable of reading data recorded on a recording medium such as a DVD or a CD. Based on the read data, music and video are played back, and the map information DB 31 is updated, etc. Note that instead of the DVD drive 17, a card slot for reading and writing a memory card may be provided.

[0026] In addition, the communication module 18 is a communication device for receiving traffic information composed of various information such as traffic jam information, regulation information, and traffic accident information transmitted from a traffic information center, for example, a VICS center or a probe center. For example, a mobile phone or a DCM corresponds to this.

[0027] In addition, the front camera 19 is an imaging device having a camera using a solid-state imaging device such as a CCD, and is installed, for example, on the back side of the rearview mirror or the front bumper with the optical axis direction facing forward in the traveling direction of the vehicle. Then, the captured image captured by the front camera 19 is displayed on the liquid crystal display 15 as a landscape (real scene image) around the vehicle (especially in front of the vehicle) as described above. In addition, the captured image captured by the front camera 19 is also used when identifying the lane in which the vehicle is currently traveling as described later.

[0028] Subsequently, the driving support processing program executed by the navigation ECU 13 in the navigation device 1 having the above configuration will be described with reference to FIG. 2. FIG. 2 is a flowchart of the driving support processing program according to the first embodiment. Here, the driving support processing program is executed after the ACC power supply (accessory power supply) of the vehicle is turned on, and is a program for providing driving support for the vehicle by allowing the user to visually recognize a guidance object superimposed on the landscape around the vehicle displayed on the liquid crystal display 15. Note that the programs shown as flowcharts in FIGS. 2, 5, and 16 below are stored in the RAM 42 and the ROM 43 provided in the navigation device 1 and are executed by the CPU 41.

[0029] In the following description, as an example of vehicle driving support using a guidance object, an example of performing vehicle driving guidance along a guidance route set in the navigation device 1 will be described. In addition, the guidance object to be displayed is guidance information for performing guidance at a guidance branch point in front of the vehicle's traveling direction. In particular, the process of displaying, as a guidance object, an arrow indicating the exit direction of the guidance branch point, a guidance image indicating the position and direction of the recommended lane, and a guidance image prompting movement to the recommended lane will be described as an example. However, in the navigation device 1, it is also possible to perform guidance and information provision other than the above driving support using the guidance object. In addition, the guidance object to be displayed can also be information other than the above arrow and guidance image. For example, as a guidance object, a warning to an object (other vehicle, pedestrian, guidance sign) that is a warning target for the occupant, a warning displayed on the road surface (collision warning, speed limit, etc.), the distance to the next guidance branch point, the current vehicle speed, the shift position, the remaining energy, an advertisement image, facility information, a guidance sign, a map image, traffic information, news, weather forecast, time, the screen of the connected smartphone, etc. can also be displayed.

[0030] First, in the driving support processing program, in step (hereinafter abbreviated as S) 1, the CPU 41 specifies the current position of the vehicle based on the detection result of the current position detection unit 11 and the map information. In addition, when specifying the current position of the vehicle, map matching processing for matching the current position of the vehicle with the map information is also performed. Then, the guidance route set in the navigation device 1 is read out, and the distance from the specified current position of the vehicle to the next guidance branch point along the guidance route is calculated. In addition, the guidance branch point is a branch point (intersection) that is the target of guidance such as a right / left turn instruction when the navigation device 1 performs driving guidance according to the guidance route set in the navigation device 1. In addition, a branch point (complicated intersection) that does not make a right / left turn but has a special shape also corresponds to the guidance branch point.

[0031] Next, in S2, the CPU 41 determines whether the distance to the next guidance branch point calculated in S1 is less than a predetermined guidance start distance. Note that the guidance start distance is determined according to the road type of the road on which the vehicle travels. For example, for a highway, it is 1 km, and for a general road, it is 300 m, which is shorter than that of a highway. However, the guidance start distance may be a variable value rather than a fixed value. For example, when there is another branch point within 300 m before the guidance branch point on a general road, it may be the distance from the guidance branch point to the other branch point.

[0032] And when it is determined that the distance to the next guidance branch point calculated in S1 is less than the guidance start distance (S2: YES), the process proceeds to S3. On the contrary, when it is determined that the distance to the next guidance branch point calculated in S1 is not less than the guidance start distance (S2: NO), the process returns to S1.

[0033] In S3, the CPU 41 determines whether the distance to the next guidance branch point is less than a predetermined exit direction guidance start distance. Note that the exit direction guidance start distance is a distance shorter than the guidance start distance that is the determination criterion in S2, and is determined according to the road type of the road on which the vehicle travels. For example, for a highway, it is 300 m, and for a general road, it is 50 m, which is shorter than that of a highway.

[0034] And when it is determined that the distance to the next guidance branch point is less than the exit direction guidance start distance (S3: YES), the process proceeds to S6. On the contrary, when it is determined that the distance to the next guidance branch point is not less than the exit direction guidance start distance (S3: NO), the process proceeds to S4.

[0035] Next, in S4, the CPU 41 performs the first guidance object display position determination process (Fig. 5) described later. The first guidance object display position determination process designates a guidance image indicating the position and direction of the recommended lane or a guidance image prompting movement to the recommended lane as a guidance object (hereinafter referred to as the first guidance object), and specifically determines the size, shape, and position (range) of the first guidance object to be displayed on the liquid crystal display 15. Note that the size, shape, and position of the first guidance object determined in S4 are conditions for superimposing the guidance object on, for example, the recommended lane in the scenery, the driving lane of the host vehicle, or the lane between them for the occupant to visually recognize.

[0036] Subsequently, in S5, the CPU 41 generates an image of the first guidance object having the size and shape determined in S4, and further transmits a control signal to the liquid crystal display 15 to draw the generated image of the first guidance object on the liquid crystal display 15 at the position (range) determined in S4. Note that an imaging image captured in advance by the front camera 19 before the distance from the vehicle to the guidance branch point becomes less than the guidance start distance, that is, a scenery (real scene image) around the vehicle at the current time (particularly in front of the vehicle) is displayed on the liquid crystal display 15. As a result, the occupant of the vehicle can visually recognize the first guidance object superimposed on the scenery.

[0037] Fig. 3 is a diagram showing an example of a driving guidance screen 51 displayed on the liquid crystal display 15 in S5. As shown in Fig. 3, a scenery 52 in front of the vehicle at the current time captured by the front camera 19 is displayed on the liquid crystal display 15. Then, an image 53 of the first guidance object is displayed superimposed on the scenery 52 in front of the vehicle.

[0038] Here, in the first embodiment, there are multiple types of guiding objects used for guiding, and one or more types of guiding objects selected according to the guiding content and the current situation are displayed. Also, there may be cases where multiple types of guiding objects are simultaneously displayed as display targets. The example shown in FIG. 3 is an example of a driving guidance screen 51 displayed in a state where the current position of the vehicle is closer to the guiding branch point than the guiding start distance and equal to or more than the exit direction guiding start distance (for example, 1 km to 300 m from the guiding branch point). A guiding image that discriminates and guides a recommended lane 54 where driving is recommended on the road on which the vehicle is currently traveling in order to pass through the guiding branch point along the guiding route from other lanes is displayed as an image 53 of the first guiding object, a first display mode, a guiding image that discriminates and guides the recommended lane 54 from other lanes and promotes movement to the recommended lane 54 is displayed as an image 53 of the first guiding object, a second display mode, and a guiding image that guides the direction of the recommended lane (which also corresponds to the exit direction of the guiding branch point) is displayed as an image 53 of the first guiding object, a third display mode, are shown.

[0039] Regarding the image 53 of the first guidance object, as shown in FIG. 3, it is basically superimposed and displayed in the recommended lane 54 within the scenery 52, or in the area straddling from the lane in which the vehicle is currently traveling (hereinafter referred to as the own vehicle traveling lane) 55 to the recommended lane 54. Also, regarding which of the above-described first display mode, second display mode, and third display mode the first guidance object is displayed in, it is selected based on the reliability of the result of identifying the lane in which the vehicle is currently traveling, as will be described later. Specifically, when the reliability of the result of identifying the lane in which the vehicle is currently traveling is high (when the lane identification result is reliable), the first guidance object is displayed in the first display mode or the second display mode with high guidance for guiding the vehicle to the recommended lane. On the other hand, when the reliability of the result of identifying the lane in which the vehicle is currently traveling is low (when the lane identification result is unreliable), the first guidance object is displayed in the third display mode with low guidance for guiding the vehicle to the recommended lane. Also, regarding which of the first display mode and the second display mode to display in the case where the above reliability is high, it may be switched based on the distance from the own vehicle to the guidance branch point, or it may be switched based on the positional relationship between the own vehicle traveling lane and the recommended lane, or it may be displayed only in any one of the pre-determined display modes.

[0040] Incidentally, the distance to the guidance branch point may be displayed for the image 53 of the first guidance object. Then, return to S3, and continue to display the image 53 of the first guidance object until the distance to the guidance branch point becomes less than the exit direction guidance start distance. The first guidance object will be described in more detail later.

[0041] On the other hand, in S6, the CPU 41 performs the second guide object display position determination process (FIG. 16) described later. The second guide object display position determination process designates an arrow indicating the exit direction of the guide branch point as a display target as a guide object (hereinafter referred to as the second guide object), and specifically determines the size, shape, and position (range) where the second guide object is to be displayed on the liquid crystal display 15. Note that the size, shape, and position where the second guide object is determined in S6 are conditions for superimposing the guide object above the road at a predetermined distance from the vehicle in the scenery or above the road at the guide branch point so that the occupant can visually recognize it.

[0042] Subsequently, in S7, the CPU 41 generates an image of the second guide object having the size and shape determined in S6, and further transmits a control signal to the liquid crystal display 15 to draw the generated image of the second guide object on the liquid crystal display 15 at the position (range) determined in S6. Note that on the liquid crystal display 15, an imaging image captured in advance by the front camera 19 before the distance from the vehicle to the guide branch point becomes less than the guide start distance, that is, a scenery (real scene image) around the vehicle at the current time (particularly in front of the vehicle) is displayed. As a result, the occupant of the vehicle can visually recognize the second guide object superimposed on the scenery.

[0043] FIG. 4 is a diagram showing an example of the travel guide screen 51 displayed on the liquid crystal display 15 in S7. As shown in FIG. 4, a scenery 52 in front of the vehicle at the current time captured by the front camera 19 is displayed on the liquid crystal display 15. Then, an image 57 of the second guide object is displayed superimposed on the scenery 52 in front of the vehicle.

[0044] Here, in the first embodiment, there are multiple types of guiding objects used for guiding, and one or more types of guiding objects selected according to the guiding content and the current situation are displayed. Also, there may be cases where multiple types of guiding objects are simultaneously the display targets. The example shown in FIG. 4 is an example of a driving guidance screen 51 displayed when the current position of the vehicle approaches within a distance less than the exit direction guiding start distance with respect to the guiding branch point (for example, within 300 m from the guiding branch point). Above the road on which the vehicle is currently traveling, at a position along the future route of the vehicle, a plurality of arrows indicating the exit direction of the guiding branch point are displayed as an image 57 of the second guiding object.

[0045] Regarding the image 57 of the second guiding object, it includes images of a plurality of arrow-shaped objects, and the images of the plurality of objects are displayed at predetermined intervals along the future route of the vehicle above the road on which the vehicle is currently traveling. The direction of each arrow indicates the exit direction of the guiding branch point of the vehicle. Also, as will be described later, the image 57 of the second guiding object is displayed in a mode where the relative position with the vehicle is fixed when the vehicle is in a state of being away from the guiding branch point (hereinafter referred to as the first mode), and when the vehicle approaches the guiding branch point to a certain extent, it is switched to and displayed in a mode where the relative position with the guiding branch point within the scenery 52 is fixed (hereinafter referred to as the second mode). In particular, in the second mode, a part of the image 57 of the second guiding object is fixed in a state of overlapping the guiding branch point. Therefore, when the driver of the vehicle views the driving guidance screen 51, the driver can accurately grasp the route of the vehicle, the position of the guiding branch point to be turned right or left, and the exit direction at the guiding intersection. After that, the image 57 of the second guiding object is continuously displayed until the vehicle passes through the guiding branch point. The second guiding object will be described in more detail later.

[0046] After that, in S8, the CPU 41 determines whether the vehicle has passed through the guiding branch point. For example, it is determined based on the current position of the vehicle detected by the current position detection unit 11 and the map information.

[0047] When it is determined that the vehicle has passed the guiding intersection (S8: YES), a control signal is sent to the liquid crystal display 15 to make the guiding object displayed on the liquid crystal display 15 non-displayed (S9). Incidentally, when making the guiding object non-displayed, it is desirable to gradually increase the transmittance of the image of the displayed guiding object according to the distance to the guiding intersection, and finally make the transmittance 100% at the timing when the vehicle reaches the guiding intersection. Also, with respect to the captured image captured by the front camera 19, that is, the scenery (real scene image) around the vehicle at the current time (especially in front of the vehicle), after making the guiding object non-displayed, it continues to be displayed for a certain period and then switches to the display of the map image.

[0048] On the other hand, when it is determined that the vehicle has not passed the guiding intersection (S8: NO), the process returns to S6 and the display of the guiding object continues.

[0049] Next, the sub-process of the first guiding object display position determination process executed in S4 will be described with reference to FIG. 5. FIG. 5 is a flowchart of the sub-process program of the first guiding object display position determination process.

[0050] First, in S11, the CPU 41 generates a three-dimensional space corresponding to the vicinity of the current position of the vehicle (especially in front of the vehicle in the traveling direction of the vehicle). Incidentally, in the three-dimensional space, in addition to the road, buildings, road signs, etc. may be modeled, or only the road may be modeled. Or it may be a simple blank three-dimensional space with only the ground without modeling the road. Also, the three-dimensional space may be stored in the map information DB 31 in advance as three-dimensional map information, and in S11, the corresponding three-dimensional map information around the vehicle position may be read from the map information DB 31. Also, a three-dimensional space may be generated based on the image captured by the front camera 19. For example, by performing point cloud matching on the captured image captured by the front camera 19, roads and structures around the road can be detected and a three-dimensional space can be generated.

[0051] Also, in S11, the CPU 41 specifies the current position and orientation of the host vehicle in the generated three-dimensional space based on the parameters detected by the current position detection unit 11. In particular, the position of the front camera 19 installed in the vehicle is set as the current position of the host vehicle, and the optical axis direction of the front camera 19 is set as the orientation of the host vehicle. Note that the position of the front camera 19 also corresponds to the position of the vehicle occupant, and the optical axis direction of the front camera 19 also corresponds to the line of sight direction of the vehicle occupant. Also, in S11, the position of a guiding branch point ahead in the traveling direction of the vehicle in the generated three-dimensional space is also specified.

[0052] Next, in S12, the CPU 41 performs image processing on the image captured by the front camera 19 to recognize (detect) ground objects located around the vehicle. Specifically, in the present embodiment, the lane lines drawn on the road surface (including the road surface of the lane in which the host vehicle travels and the road surface of lanes other than the lane in which the host vehicle travels) and the road edge (specifically, the edge of the lane, and when there is a sidewalk, the boundary between the lane and the sidewalk) are set as recognition (detection) targets. Note that it is desirable to also detect the color and type (solid line, broken line, etc.) of the lane lines. Also, for the road edge, basically, structures such as blocks, guardrails, and median strips provided at the road edge are detected as the road edge, but for roads without such structures at the road edge, the cut-off of asphalt or the outermost lane line may be detected as the road edge.

[0053] The lane line detection process in S12 is briefly described below. First, the CPU 41 performs luminance correction on the road surface and the lane lines based on the luminance difference in order to detect the lane lines in the captured image captured by the front camera 19. Thereafter, a binarization process for separating the lane lines from the image, a geometric process for correcting distortion, a smoothing process for removing noise from the image, etc. are performed to detect the boundary line between the road surface and the lane lines. The presence and type of the lane lines are specified by the detected boundary line. Also, by extracting the image portion of the detected range of the lane lines and performing color recognition (RGB value detection), the color of the lane lines can also be detected.

[0054] Next, the detection process of the road edge in S12 will be briefly described. First, in order to detect the road edge in the captured image captured by the front camera 19, the CPU 41 performs brightness correction on the road surface and the structures (blocks, guardrails, median strips, etc.) provided on the road edge based on the brightness difference. Then, it performs a binarization process to separate these structures from the image, a geometric process to correct distortion, a smoothing process to remove image noise, etc., and detects the boundary line between the road surface and the structures. The presence of the road edge is specified by the detected boundary line. Note that for the detection of lane lines and road edges, a pattern matching process using feature points or templates may be performed. Since these image recognition processes are already known, the details are omitted. Also, other vehicles located on the road other than lane lines and road edges may also be set as detection targets.

[0055] Next, in S13, the CPU 41 specifies the "own vehicle driving lane", which is the driving lane on which the own vehicle is currently traveling, based on the detection result of the ground object recognition process performed in S12. Specifically, the own vehicle driving lane is specified by the following process. In S13, the position of the "own vehicle driving lane" in the three-dimensional space generated in S11 is also specified.

[0056] Examples of methods for specifying the own vehicle driving lane include a method using the type and number of lane lines and a method using the distance from the road edge and the lane width. [Specification of Own Vehicle Driving Lane (Pattern 1)] Regarding the method of using the types and numbers of lane lines, for example, when driving on a three-lane highway on one side as shown in FIGS. 6 to 8, first, the CPU 41 acquires the recognition results of the road edge and lane lines on the left side of the host vehicle. Regarding the recognition results of the lane lines, the colors and types of the lane lines are also acquired. As a result, for example, the road edge 61 is detected, the outside lane line 62 which is a white solid line adjacent to the road edge 61 is detected, and if one lane boundary line 63 which is a white broken line or solid line is detected between the outside lane line 62 and the vehicle, it can be specified that the host vehicle is driving in the second lane from the left as shown in FIG. 6. Similarly, if two lane boundary lines 63 are detected between the outside lane line 62 and the vehicle, it can be specified that the host vehicle is driving in the third lane from the left as shown in FIG. 7. If no lane boundary line 63 is detected between the outside lane line 62 and the vehicle, it can be specified that the host vehicle is driving in the leftmost lane as shown in FIG. 8. Furthermore, the CPU 41 similarly acquires the recognition results of the road edge and lane lines on the right side. Regarding the recognition results of the lane lines, the colors and types of the lane lines are also acquired. As a result, for example, the road edge 64 is detected, the outside lane line 65 which is a white solid line adjacent to the road edge 64 is detected, and if one lane boundary line 66 which is a white broken line or solid line is detected between the outside lane line 65 and the vehicle, it can be specified that the host vehicle is driving in the second lane from the right as shown in FIG. 6. Similarly, if two lane boundary lines 66 are detected between the outside lane line 65 and the vehicle, it can be specified that the host vehicle is driving in the third lane from the right as shown in FIG. 8. If no lane boundary line 66 is detected between the outside lane line 65 and the vehicle, it can be specified that the host vehicle is driving in the rightmost lane as shown in FIG. 7. The CPU 41 finally combines the position of the host vehicle driving lane specified based on the left end of the road (the number of lanes from the left end of the road to the lane in which the vehicle is currently driving) and the position of the host vehicle driving lane specified based on the right end of the road (the number of lanes from the right end of the road to the lane in which the vehicle is currently driving) to specify the host vehicle driving lane. For example, in the example shown in FIG. 6, the host vehicle driving lane is specified as the second lane from the left and the second lane from the right. In the example shown in FIG. 7, the host vehicle driving lane is specified as the third lane from the left and the first lane from the right. In the example shown in FIG. 8, the host vehicle driving lane is specified as the first lane from the left and the third lane from the right. In addition to the recognition results of the road edge and the lane line, it is desirable to specify the own vehicle's driving lane in consideration of the map information. Note that the link data 32 included in the map information stores the number of lanes per road and the presence or absence of oncoming lanes (whether it is a two-way traffic section). In image recognition by the front camera 19, for example, when a solid or broken white line is detected near the center of a general road, it is difficult to determine whether the section line is a center line or a lane boundary line. However, by using the map information, it becomes possible to easily make such a determination. As a result, even in a section where there are oncoming lanes, it is possible to accurately specify the own vehicle's driving lane considering the presence of oncoming lanes. For example, when a vehicle is traveling on a four-lane road with two lanes on each side without a median strip and it is specified that the own vehicle is traveling in the third lane from the right, it can be determined from the map information that the rightmost and the second lane from the right are oncoming lanes, so it can be specified that the vehicle is traveling in the right lane of a two-lane road on one side. [Specification of Own Vehicle's Driving Lane (Pattern 2)] Next, regarding the method using the distance from the road edge and the lane width, taking as an example the case of traveling on a three-lane highway on one side shown in FIGS. 6 to 8, first, the CPU 41 detects the lane lines on both sides of the own vehicle (in the example shown in FIG. 6, the lane boundary line 63 and the lane boundary line 66, in the example shown in FIG. 7, the lane boundary line 66 and the outside lane line 65, in the example shown in FIG. 8, the outside lane line 62 and the lane boundary line 63), and specifies the width of the lane of the road on which the vehicle is traveling from the distance between the lane lines. Then, the CPU 41 calculates the distance along the road width direction from the road edge 61 or the outside lane line 62 on the left side of the own vehicle to the own vehicle, and calculates how many lane widths the calculated distance is. As a result, when the distance from the road edge 61 or the outside lane line 62 to the own vehicle is the width of one lane, it can be specified that the own vehicle is traveling in the second lane from the left as shown in FIG. 6. Similarly, when the distance from the road edge 61 or the outside lane line 62 to the own vehicle is the width of two lanes, it can be specified that the own vehicle is traveling in the third lane from the left as shown in FIG. 7. When the distance from the road edge 61 or the outside lane line 62 to the own vehicle is less than the width of one lane, it can be specified that the own vehicle is traveling in the leftmost lane as shown in FIG. 8. Furthermore, the CPU 41 similarly calculates the distance from the road edge 64 or the outside lane line 65 on the right side. As a result, when the distance from the road edge 64 or the outside lane line 65 to the host vehicle is equal to the distance of one lane, the host vehicle can be identified as traveling in the second lane from the right as shown in FIG. 6. Similarly, when the distance from the road edge 64 or the outside lane line 65 to the host vehicle is equal to the distance of two lanes, the host vehicle can be identified as traveling in the third lane from the right as shown in FIG. 8. When the distance from the road edge 64 or the outside lane line 65 to the host vehicle is less than the distance of one lane, the host vehicle can be identified as traveling in the rightmost lane as shown in FIG. 7. The CPU 41 finally combines the position of the host vehicle's traveling lane specified based on the left end of the road (the number of lanes from the left end of the road to the lane in which the vehicle is currently traveling) and the position of the host vehicle's traveling lane specified based on the right end of the road (the number of lanes from the right end of the road to the lane in which the vehicle is currently traveling) to specify the host vehicle's traveling lane. For example, in the example shown in FIG. 6, the host vehicle's traveling lane is specified as the second lane from the left and the second lane from the right. In the example shown in FIG. 7, the host vehicle's traveling lane is specified as the third lane from the left and the first lane from the right. In the example shown in FIG. 8, the host vehicle's traveling lane is specified as the first lane from the left and the third lane from the right. In addition to the recognition results of the road edge and the road width, it is desirable to specify the host vehicle's traveling lane in consideration of the map information. Note that the link data 32 included in the map information stores the number of lanes for each road and the presence or absence of oncoming lanes (whether it is a two-way traffic section). As a result, even in a section where oncoming lanes exist, it is possible to accurately specify the host vehicle's traveling lane in consideration of the presence of oncoming lanes. For example, when a vehicle is traveling on a four-lane road with two lanes on each side without a median strip and the host vehicle is identified as traveling in the third lane from the right, it can be determined from the map information that the rightmost and the second lane from the right are oncoming lanes, so that the vehicle can be identified as traveling in the right lane of the two-lane road on one side.

[0057] Currently, for identifying the own vehicle's driving lane, in addition to the lane lines and the road edge, the detection results of other vehicles driving on the road may also be used. For example, even when the lane lines cannot be clearly detected, it is possible to estimate the existence of the lane from the positions of other vehicles.

[0058] Subsequently, in S14, the CPU 41 calculates a reliability indicating how reliable the identification result of the own vehicle's driving lane in S13 is. In particular, in this embodiment, (a) the right-side reliability which is the reliability when identifying the position of the own vehicle's driving lane based on the road right edge based on the detection results of the lane lines and the road edge on the right side with respect to the vehicle's traveling direction, (b) the left-side reliability which is the reliability when identifying the position of the own vehicle's driving lane based on the road left edge based on the detection results of the lane lines and the road edge on the left side with respect to the vehicle's traveling direction, and (c) the overall reliability which is the reliability obtained by summarizing the right-side reliability and the left-side reliability (total reliability) are calculated.

[0059] Examples of the calculation methods for each of the right-side reliability, left-side reliability, and overall reliability are given and described below. [Right-side reliability] First, for each lane line estimated to exist on the right side of the own vehicle's driving lane identified in S13, the reliability for each lane line (hereinafter referred to as lane line reliability C) is calculated by the following formula (1). For example, in the case where the own vehicle's driving lane is identified as the central lane on a highway with three lanes on one side as shown in FIG. 6, there should be a lane boundary line 66 and an outside lane line 65 on the right side of the own vehicle's driving lane, and the lane line reliability C is calculated for each of these two lane lines. Also, in the case where the own vehicle's driving lane is identified as the rightmost lane on a highway with three lanes on one side as shown in FIG. 7, there should be an outside lane line 65 on the right side of the own vehicle's driving lane, and the lane line reliability C is calculated for that one lane line. Further, in the case where the own vehicle's driving lane is identified as the leftmost lane on a highway with three lanes on one side as shown in FIG. 8, there should be a lane boundary line 63, a lane boundary line 66, and an outside lane line 65 on the right side of the own vehicle's driving lane, and the lane line reliability C is calculated for each of these three lane lines. C = (10×V1 + V2 + V3 + V4) / 13 ····(1) Here, V1 indicates whether the lane line to be used for calculating the lane line reliability C could be detected in the image recognition process of S12 (whether the presence of the lane line could be recognized). If it can be detected, it is set to "1"; if not, it is set to "0". Also, V2 indicates whether the color of the lane line to be used for calculating the lane line reliability C could be detected in the image recognition process of S12. If it can be detected, it is set to "1"; if not, it is set to "0". However, even if the color of the lane line can be detected, if it does not match the color estimated using map information or the like, it may be set to "0". Also, V3 indicates whether the type of the lane line to be used for calculating the lane line reliability C (for example, solid line, dashed line) could be detected in the image recognition process of S12. If it can be detected, it is set to "1"; if not, it is set to "0". However, even if the type of the lane line can be detected, if it does not match the type estimated using map information or the like, it may be set to "0". Also, V4 detects the inclination of the lane line to be used for calculating the lane line reliability C, particularly with respect to the traveling direction of the road, by the image recognition process of S12. If the detected inclination is less than the threshold value (for example, 10 degrees), it is set to "1"; if it is equal to or greater than the threshold value, it is set to "0". After that, the lane line reliability C calculated for each lane line estimated to exist on the right side with respect to the own vehicle's traveling lane is added to calculate the right - hand side reliability CR. When calculating the right - hand side reliability CR, the lane line reliability C of the lane line at a position farther from the vehicle is emphasized more than the lane line reliability C of the lane line at a position closer to the vehicle. For example, as shown in FIG. 6, in the case where the own vehicle's traveling lane is identified as the central lane on a highway with three lanes on one side, if the lane line reliability C calculated for the lane boundary line 66 at the position closest to the own vehicle's traveling lane is C1, and the lane line reliability C calculated for the outer lane line 65 at the second - closest position to the own vehicle's traveling lane is C2, the right - hand side reliability CR is calculated by the following formula (2). CR = (C1 + 2×C2) / 3 ····(2) Also, for the case where the own vehicle driving lane is identified as the rightmost lane on a highway with three lanes on one side as shown in Fig. 7, when the lane line reliability C calculated for the outside lane line 65 that is the only one on the right side of the own vehicle driving lane is set as C1, the right-side reliability CR is calculated by the following formula (3). CR = C1 ···· (3) Also, for the case where the own vehicle driving lane is identified as the leftmost lane on a highway with three lanes on one side as shown in Fig. 8, when the lane line reliability C calculated for the lane boundary line 63 at the position closest to the own vehicle driving lane is C1, the lane line reliability C calculated for the lane boundary line 66 at the second-closest position to the own vehicle driving lane is C2, and the lane line reliability C calculated for the outside lane line 65 at the third-closest position to the own vehicle driving lane is C3, the right-side reliability CR is calculated by the following formula (4). CR = (C1 + 2 × C2 + 4 × C3) / 7 ···· (4) Note that the coefficients multiplied by each lane line reliability C in the above formulas (2) to (4) can be changed as appropriate, but it is desirable to set them such that a larger coefficient is multiplied by the lane line reliability C calculated for the lane line at a position farther from the own vehicle driving lane. The right-side reliability CR finally calculated by the above formulas (2) to (4) is a value between 0 and 1, and the closer it is to 1, the higher the reliability of the position of the own vehicle driving lane specified with reference to the right end of the road. [Left-side reliability] Next, for each lane line presumed to exist on the left side of the host vehicle's driving lane specified in S13 above, the lane line reliability C for each lane line is similarly calculated by the above formula (1). For example, in the case where the host vehicle's driving lane is specified as the central lane on a highway with three lanes on one side as shown in FIG. 6, there should be a road outer side line 62 and a lane boundary line 63 on the left side of the host vehicle's driving lane, and the lane line reliability C is calculated for each of these two lane lines. Also, in the case where the host vehicle's driving lane is specified as the rightmost lane on a highway with three lanes on one side as shown in FIG. 7, there should be a road outer side line 62, a lane boundary line 63, and a lane boundary line 66 on the left side of the host vehicle's driving lane, and the lane line reliability C is calculated for each of these three lane lines. Further, in the case where the host vehicle's driving lane is specified as the leftmost lane on a highway with three lanes on one side as shown in FIG. 8, there should be a road outer side line 62 on the left side of the host vehicle's driving lane, and the lane line reliability C is calculated for this one lane line. Note that the method for calculating the lane line reliability C is the same as the right side reliability described above, so it is omitted. Thereafter, the left side reliability CL is calculated by adding the lane line reliability C calculated for each lane line presumed to exist on the left side of the host vehicle's driving lane. Note that when calculating the left side reliability CL, the lane line reliability C of the lane line at a position farther from the vehicle is emphasized more than the lane line reliability C of the lane line at a position closer to the vehicle. For example, in the case where the host vehicle's driving lane is specified as the central lane on a highway with three lanes on one side as shown in FIG. 6, if the lane line reliability C calculated for the lane boundary line 63 at the position closest to the host vehicle's driving lane is C1, and the lane line reliability C calculated for the road outer side line 62 at the second closest position to the host vehicle's driving lane is C2, the left side reliability CL is calculated by the following formula (5). CL=(C1 + 2×C2) / 3····(5) Also, for the case where the own vehicle's driving lane is identified as the rightmost lane on a highway with three lanes on one side as shown in FIG. 7, when the lane line reliability C calculated for the lane boundary line 66 at the position closest to the own vehicle's driving lane is C1, the lane line reliability C calculated for the lane boundary line 63 at the second closest position to the own vehicle's driving lane is C2, and the lane line reliability C calculated for the outside lane line 62 at the third closest position to the own vehicle's driving lane is C3, the left-side reliability CL is calculated by the following formula (6). CL=(C1 + 2×C2 + 4×C3) / 7····(6) Also, for the case where the own vehicle's driving lane is identified as the leftmost lane on a highway with three lanes on one side as shown in FIG. 8, when the lane line reliability C calculated for the outside lane line 62 that is the only one on the left side of the own vehicle's driving lane is C1, the left-side reliability CL is calculated by the following formula (7). CL = C1····(7) Note that the coefficients multiplied by each lane line reliability C in the above formulas (5) to (7) can be changed as appropriate, but it is desirable to set them so that a larger coefficient is multiplied by the lane line reliability C calculated for the lane line at a position farther from the own vehicle's driving lane. The left-side reliability CL finally calculated by the above formulas (5) to (7) is a value between 0 and 1, and the closer it is to 1, the higher the reliability of the position of the own vehicle's driving lane specified with reference to the left end of the road. [Overall Reliability] Finally, the overall reliability CT is calculated by adding the right-side reliability CR and the left-side reliability CL calculated as above. Note that when calculating the overall reliability CT, the right-side reliability CR and the left-side reliability CL may be simply added, or one of them may be weighted and then added. For example, the overall reliability CT is calculated by the following formula (8). CT=(CR + CL) / 2····(8) Note that the overall reliability CT finally calculated by the above formula (8) is a value between 0 and 1, and the closer it is to 1, the higher the total reliability of the position of the own vehicle's driving lane specified with reference to the right end and the left end of the road.

[0060] Next, in S15, the CPU 41 identifies the "recommended lane", which is the lane recommended for driving on the road where the vehicle is currently traveling. Specifically, the recommended lane is identified through the following process. In S15, the position of the "recommended lane" in the three-dimensional space generated in S11 is also identified. [Identification of Recommended Lane] Based on the map information and the guidance route, the CPU 41 identifies, as the recommended lane, the lane that the vehicle needs to drive on to pass through the upcoming guidance bifurcation point in the exit direction (guidance direction) along the guidance route. More specifically, the lane corresponding to the exit direction of the vehicle at the guidance bifurcation point is acquired as the recommended lane. Note that the link data 32 included in the map information stores the traffic control for the traveling direction of each lane, and the bifurcation point data 34 stores the shape of the bifurcation point. The recommended lane is identified using this information. For example, as shown in FIG. 9, when the exit direction (guidance direction) at the upcoming guidance bifurcation point 70 in front of the traveling direction of a vehicle driving on a highway is diagonally left where there is an access road, the leftmost lane among the three lanes of the road on which the own vehicle is traveling becomes the recommended lane. On the other hand, as shown in FIG. 10, when the exit direction (guidance direction) at the upcoming guidance bifurcation point 70 in front of the traveling direction of a vehicle driving on an ordinary road is right, the rightmost lane corresponding to the traffic control for a right turn among the four lanes of the road on which the own vehicle is traveling becomes the recommended lane. Note that the recommended lane is not necessarily only one lane and may be two or more lanes.

[0061] Subsequently, in S16, the CPU 41 determines whether the overall reliability CT calculated in S14 is equal to or greater than the first threshold value. Note that the first threshold value can be set as appropriate, and for example, it is set to 0.8.

[0062] When it is determined that the overall reliability CT calculated in S14 is equal to or greater than the first threshold value (S16: YES), the own vehicle driving lane identified in S13 is considered to be sufficiently reliable, and the process proceeds to S17. In contrast, when it is determined that the overall reliability CT calculated in S14 is less than the first threshold value (S16: NO), the process proceeds to S20.

[0063] In S17, the CPU 41 generates a first guidance object 71 to be displayed. Here, when the reliability of the result of identifying the lane on which the vehicle is currently traveling is high (when the lane identification result is reliable), as described above, the first guidance object is displayed in a first display mode or a second display mode with high guidance property for guiding the vehicle to the recommended lane. Here, as shown in FIG. 3, in the first display mode, a guidance image for guiding and identifying the recommended lane from other lanes is displayed as the first guidance object, and in the second display mode, a guidance image for guiding and identifying the recommended lane from other lanes and prompting movement to the recommended lane is displayed as the first guidance object. Regarding which of the first display mode and the second display mode to use for display in the case of high reliability, it may be switched based on the distance from the own vehicle to the guidance bifurcation point, or may be switched based on the positional relationship between the own vehicle traveling lane and the recommended lane, or the display may be performed only in any one of the pre-determined display modes. Hereinafter, an example of the case where the first guidance object is displayed in the first display mode will be described.

[0064] As shown in FIG. 11, the first guidance object 71 has a rectangular shape, and the horizontal length is set to the lane width of the recommended lane, and the vertical length is set to, for example, 90 m. Further, the distance to the guidance bifurcation point may be drawn inside. Also, the first guidance object 71 is a two-dimensional polygon and basically has no thickness. However, it may be a three-dimensional polygon with thickness. Further, the shape of the first guidance object 71 generated in S17 can be appropriately changed, and may be a shape other than a rectangle as long as it can distinguish the recommended lane from other lanes.

[0065] Furthermore, in S17, the CPU 41 arranges the generated first guidance object 71 in the three-dimensional space generated in S11. Incidentally, the position where the first guidance object 71 is arranged with respect to the three-dimensional space is on the road surface of the recommended lane as shown in FIG. 11, and is arranged in the range up to 90 m ahead from the current position of the host vehicle. Incidentally, although FIG. 11 shows an example where the recommended lane and the host vehicle traveling lane are different, when the recommended lane and the host vehicle traveling lane coincide, it will be arranged in the host vehicle traveling lane. Also, the height at which it is arranged is the same as the road surface. Thereafter, the process proceeds to S18.

[0066] Next, in S18, the CPU 41 stores, as the size and shape of the guidance object to be displayed on the liquid crystal display 15, the size and shape of the first guidance object 71 that can be visually recognized in the traveling direction of the vehicle from the current position of the vehicle and the position at the height of the front camera 19 in the three-dimensional space where the first guidance object 71 was arranged in S17. Here, the size and shape of the guidance object stored in S18 are the size and shape of the first guidance object 71 that can be visually recognized when the first guidance object 71 arranged in the three-dimensional space is visually recognized from the viewpoint of the current vehicle (more precisely, the front camera 19).

[0067] Thereafter, in S19, the CPU 41 estimates the position of the recommended lane in the scenery 52 displayed on the liquid crystal display 15 based on the current position of the vehicle and the position of the guidance branch point in the three-dimensional space generated in S11 and the positions of the recommended lane and the host vehicle traveling lane specified in S13 and S15, and determines the range up to 90 m ahead from the current position of the host vehicle on the road surface of the estimated recommended lane as the position where the first guidance object 71 is to be displayed on the liquid crystal display 15. Incidentally, when the recommended lane and the host vehicle traveling lane coincide, the road surface of the host vehicle traveling lane becomes the position where the first guidance object 71 is to be displayed.

[0068] Thereafter, before shifting to S5, an image of the first guide object 71 having the size and shape determined in S18 is generated, and a control signal is further transmitted to the liquid crystal display 15, and the image of the first guide object 71 generated for the liquid crystal display 15 is drawn at the position (range) determined in S19.

[0069] Next, in S20 which is executed when it is determined that the overall reliability CT is less than the first threshold value, the CPU 41 determines whether the reliability corresponding to the exit direction of the vehicle at the guide branch point among the right reliability CR and the left reliability CL calculated in S14 is greater than or equal to the second threshold value. The second threshold value can be set as appropriate, for example, 0.8. Further, the reliability corresponding to the exit direction of the vehicle means that, for example, if the vehicle exits to the right at the guide branch point, the reliability corresponding to the exit direction of the vehicle is the right reliability CR, and if the vehicle exits to the left at the guide branch point, the reliability corresponding to the exit direction of the vehicle is the left reliability CL. Here, as shown in FIGS. 9 and 10, the recommended lane basically exists in the exit direction of the guide branch point. Therefore, for example, when the exit direction of the guide branch point is to the right, if the right reliability CR, which is the reliability of the position of the own vehicle traveling lane specified based on the right end of the road, is high, at least the positional relationship between the own vehicle traveling lane and the recommended lane can be trusted. That is, when performing the virtual image superposition, the position where the recommended lane exists in the scenery can be accurately estimated. On the other hand, when the exit direction of the guide branch point is to the left, if the left reliability CL, which is the reliability of the position of the own vehicle traveling lane specified based on the left end of the road, is high, at least the positional relationship between the own vehicle traveling lane and the recommended lane can be trusted. That is, when performing the virtual image superposition, the position where the recommended lane exists in the scenery can be accurately estimated.

[0070] Then, when it is determined that the reliability corresponding to the exit direction of the vehicle at the guide bifurcation point among the right reliability CR and the left reliability CL calculated in S14 is equal to or higher than the second threshold value (S20: YES), at least the positional relationship between the own vehicle driving lane specified in S13 and the recommended lane is regarded as reliable, and the process proceeds to S17. In S17, as described above, the first guide object is displayed in the first display mode or the second display mode with high guiding property for guiding the vehicle to the recommended lane.

[0071] On the contrary, when it is determined that the reliability corresponding to the exit direction of the vehicle at the guide bifurcation point among the right reliability CR and the left reliability CL calculated in S14 is less than the second threshold value (S16: NO), the own vehicle driving lane specified in S13 is regarded as unreliable, and the process proceeds to S21.

[0072] In S21, the CPU 41 generates the first guide objects 72 to 74 to be displayed. Here, when the reliability of the result of specifying the lane in which the vehicle is currently traveling is low (when the lane specification result is unreliable), as described above, the first guide object is displayed in the third display mode with low guiding property for guiding the vehicle to the recommended lane. Here, as shown in FIG. 3, the third display mode displays a guide image for guiding the direction of the recommended lane (which also corresponds to the exit direction of the guide bifurcation point) as the first guide object. Note that the direction of the recommended lane is the direction in which the recommended lane exists in the road, not the direction with respect to the current position of the own vehicle. Therefore, for example, as shown in FIG. 9, even if the leftmost lane is the recommended lane and the own vehicle is traveling in the recommended lane, a guide image for guiding the left direction is displayed as the first guide object.

[0073] The first guiding objects 72 to 74 are arrows indicating the direction of the recommended lane (which also corresponds to the exit direction of the guiding bifurcation point), and in particular, generate three arrows having an isosceles triangle shape as shown in FIGS. 12 and 13. When the recommended lane is in the right direction (i.e., the exit direction of the guiding bifurcation point is in the right direction), it becomes an arrow indicating the right direction. When the recommended lane is in the left direction (i.e., the exit direction of the guiding bifurcation point is in the left direction), it becomes an arrow indicating the left direction. Also, the first guiding objects 72 to 74 are two-dimensional polygons and basically have no thickness. However, they may be three-dimensional polygons with thickness. Further, the shape of the first guiding objects 72 to 74 generated in S21 can be appropriately changed, and may be a shape other than an arrow as long as it can indicate the direction of the recommended lane. Also, it is not necessary to have three arrows, and only one arrow may be used.

[0074] Furthermore, in S21, the CPU 41 arranges the generated first guiding objects 72 to 74 with respect to the three-dimensional space generated in S11. Note that the positions where the first guiding objects 72 to 74 are arranged with respect to the three-dimensional space are positions where three arrows having a first guiding object 72 to 74 are arranged at predetermined intervals above the road on which the vehicle is currently traveling in the three-dimensional space and at a predetermined distance (for example, 10 m) in front of the current position of the vehicle as shown in FIG. 12. In particular, among the three first guiding objects 72 to 74, they are arranged at positions where the distance from the current position of the vehicle to the first guiding object 72 closest to the vehicle side is a fixed predetermined distance (for example, 10 m). Here, the first guiding objects 72 to 74 each have an isosceles triangle shape and are arranged so as to be parallel to the direction intersecting the traveling direction (that is, facing the front toward the vehicle side). Further, the first guiding objects 72 to 74 indicate the direction of the recommended lane depending on the direction of the apex angle with respect to the base. When the recommended lane is located on the left side of the road, it has an isosceles triangle shape with the apex angle facing left as shown in FIG. 12. Also, the arrangement intervals of the three first guiding objects 72 to 74 can be changed as appropriate, but for example, they are set at 10 m intervals. Also, the three first guiding objects 72 to 74 are not arranged parallel to the vehicle's path (that is, the traveling direction of the road), but are gradually inclined and arranged so that the arrow visually recognized at a position closer to the guiding branch point 70 is located on the recommended lane direction side (left side in FIG. 12) with respect to the vehicle's path. In particular, when viewed from the vehicle, it is also arranged as a condition that a plurality of the first guiding objects 72 to 74 do not overlap with each other by a predetermined ratio (for example, 20%) or more. However, they may be arranged parallel to the traveling direction. On the other hand, in the vertical direction, as shown in FIG. 13, the lower ends of the first guiding objects 72 to 74 are arranged at positions separated from the road surface by a predetermined distance (for example, 1 m).

[0075] Next, in S22, the CPU 41 stores, as the size and shape of the guidance object to be displayed on the liquid crystal display 15, the size and shape of the first guidance objects 72 to 74 that can be visually recognized when visually recognizing from the current position of the vehicle and the position of the height of the front camera 19 in the traveling direction in the three-dimensional space where the first guidance objects 72 to 74 were arranged in S21. Here, the size and shape of the guidance object stored in S22 are the size and shape of the first guidance objects 72 to 74 that can be visually recognized when visually recognizing the first guidance objects 72 to 74 arranged in the three-dimensional space from the viewpoint of the current vehicle (more precisely, the front camera 19).

[0076] After that, in S23, the CPU 41 estimates the future route of the own vehicle in the scenery 52 displayed on the liquid crystal display 15 based on the current position of the vehicle and the position of the guidance branch point in the three-dimensional space generated in S11, and determines, on the liquid crystal display 15, the range along the estimated route as the position where the first guidance objects 72 to 74 are to be displayed.

[0077] After that, it proceeds to S5, generates images of the first guidance objects 72 to 74 having the size and shape determined in S22, and further transmits a control signal to the liquid crystal display 15 to draw the generated images of the first guidance objects 72 to 74 at the position (range) determined in S23 on the liquid crystal display 15.

[0078] As a result of the above processing, when the distance from the vehicle to the guidance branch point is less than the guidance start distance and equal to or more than the exit direction guidance start distance, the driving guidance screen 51 displayed on the liquid crystal display 15 as the vehicle travels will differ depending on the reliability of the result of specifying the lane in which the vehicle is currently traveling, and will be a screen like FIG. 14 or FIG. 15.

[0079] For example, when the reliability of the result of specifying the lane in which the vehicle is currently traveling is high (when the lane specification result is reliable), the driving guidance screen 51 displayed on the liquid crystal display 15 will be a screen like FIG. 14. Specifically, “the image 53 of the first guiding object that differentiates the recommended lane from other lanes” is displayed. As shown in FIG. 14, the image 53 of the first guiding object is displayed superimposed on the road surface of the recommended lane 54 (S17 - S19). As a result, it becomes possible for the passenger to clearly recognize the existence and position of the recommended lane 54. Also, by clearly identifying which lane the recommended lane 54 is in the scenery 52, it becomes a highly guiding guidance for guiding the vehicle into the recommended lane.

[0080] On the other hand, when the reliability of the result of specifying the lane in which the vehicle is currently traveling is low (when the lane specification result is not reliable), the driving guidance screen 51 displayed on the liquid crystal display 15 becomes a screen as shown in FIG. 15. Specifically, “the image 53 of the first guiding object that indicates the direction of the recommended lane” is displayed. As shown in FIG. 15, the image 53 of the first guiding object is not superimposed on the road surface of the recommended lane 54, but is displayed superimposed above the road surface in the traveling direction of the vehicle (S21 - S23). It only indicates the direction in which the recommended lane exists without clearly showing the position of the recommended lane 54 to the passenger, resulting in a less guiding guidance for guiding the vehicle into the recommended lane. Therefore, even in a situation where the position of the recommended lane in the scenery 52 cannot be accurately specified, it is possible to prevent incorrect guidance from being given.

[0081] When the distance from the vehicle to the guidance branch point is less than the guidance start distance and greater than or equal to the exit direction guidance start distance, which of the driving guidance screens 51 in FIGS. 14 and 15 is displayed is determined by the reliability of the specific result of the own vehicle driving lane at the current time as described above. Therefore, for example, when the vehicle enters a section with a thin painted dividing line while the driving guidance screen 51 in FIG. 14 is displayed on the liquid crystal display 15 and the reliability of the specific result of the own vehicle driving lane decreases, it will switch to the driving guidance screen 51 in FIG. 15. Similarly, when the reliability of the specific result of the own vehicle driving lane increases while the driving guidance screen 51 in FIG. 15 is displayed on the liquid crystal display 15, it will switch to the driving guidance screen 51 in FIG. 14. However, it may be possible not to perform the display switching even if the reliability of the specific result of the own vehicle driving lane changes midway.

[0082] Furthermore, when the distance from the vehicle to the guidance branch point subsequently becomes less than the exit direction guidance start distance (300 m on a highway), instead of the image 53 of the first guidance object, an image 57 of a second guidance object, which is an arrow indicating the exit direction of the guidance branch point as described later, will be newly displayed (see FIG. 18).

[0083] Next, the sub-process of the second guidance object display position determination process executed in S6 will be described with reference to FIG. 16. FIG. 16 is a flowchart of the sub-process program of the second guidance object display position determination process.

[0084] First, in S31, the CPU 41 acquires the distance from the current position of the vehicle to the coordinates X of the next guidance branch point along the guidance route. For example, on a highway, as shown in FIG. 17, the front apex of the zebra strip at the branch is set as the coordinates X of the guidance branch point.

[0085] Next, in S32, the CPU 41 determines whether the distance to the coordinate X of the guidance branch point acquired in S31 is less than a predetermined threshold value. The timing at which the distance to the coordinate X of the guidance branch point becomes the threshold value is the timing at which at least a part of the image of the guidance object displayed (displayed in the first mode) on the liquid crystal display 15 with the relative position to the vehicle fixed overlaps the coordinate X of the guidance branch point as described in S33 below, that is, the timing at which at least a part of the guidance object overlaps the guidance branch point and is visually recognized by the vehicle occupants. Details will be described later.

[0086] And when it is determined that the distance to the coordinate X of the guidance branch point acquired in S31 is less than the threshold value (S32: YES), the process proceeds to S34. On the other hand, when it is determined that the distance to the coordinate X of the guidance branch point acquired in S31 is equal to or greater than the threshold value (S32: NO), the process proceeds to S33.

[0087] In S33, the CPU 41 sets the "position where the second guidance object overlaps the scenery" to the position where the second guidance object overlaps and is visually recognized in a state where the relative position to the vehicle is fixed to the occupant (the position for display in the first mode). Specifically, a position a predetermined distance ahead (for example, 10 m ahead) of the current position of the vehicle is set as the overlapping (arrangement) position of the second guidance object.

[0088] On the other hand, in S34, the CPU 41 sets the "position where the second guidance object overlaps the scenery" to the position where the second guidance object overlaps and is visually recognized in a state where the relative position to the guidance branch point is fixed to the occupant (the position for display in the second mode). Specifically, it is set to the position where at least a part of the guidance object overlaps and is visually recognized with respect to the guidance branch point (including the periphery thereof).

[0089] Next, in S35, the CPU 41 generates a three-dimensional space corresponding to the vicinity of the current position of the vehicle (particularly the front in the vehicle traveling direction). Also, the current position and orientation of the host vehicle and the position of the guidance branch point in the generated three-dimensional space are specified. Details are the same as in S11, so they are omitted.

[0090] Subsequently, in S36, the CPU 41 generates second guide objects 81 to 83 to be displayed on the liquid crystal display 15. The shapes of the second guide objects 81 to 83 are the same as those of the first guide objects 72 to 74 displayed in the above-described third display mode, and are arrows indicating the exit directions of the guide branch points ahead in the traveling direction of the vehicle. As shown in FIGS. 12 and 13, three arrows having an isosceles triangle shape are generated. In addition, when the guide route is a route that turns right at a guide intersection ahead in the traveling direction, it becomes an arrow indicating the right direction. When the guide route is a route that turns left at a guide intersection ahead in the traveling direction, it becomes an arrow indicating the left direction. Further, the second guide objects 81 to 83 are two-dimensional polygons and basically have no thickness. However, they may be three-dimensional polygons with thickness. In addition, the shapes of the second guide objects 81 to 83 generated in S36 can be appropriately changed, and any shape other than an arrow may be used as long as it can indicate the exit direction at the guide branch point. Also, it is not necessary to have three arrows, and only one arrow may be used.

[0091] Furthermore, in S36, the CPU 41 arranges the generated second guide objects 81 to 83 with respect to the three-dimensional space generated in S35. Note that the positions where the second guide objects 81 to 83 are arranged with respect to the three-dimensional space are determined based on the "position where the second guide object is superimposed on the scenery" set in S33 or S34.

[0092] For example, in S33, when setting the position where the "second guiding object is superimposed on the scenery" to a position a predetermined distance ahead (for example, 10 m ahead) with respect to the current position of the vehicle, it is made the same as the first guiding objects 72 to 74 (FIG. 12) displayed in the third display mode. That is, the second guiding objects 81 to 83 having three arrows are arranged at predetermined intervals at a position above the road on which the vehicle is currently traveling in the three-dimensional space and a predetermined distance ahead with respect to the current position of the vehicle. Further, the three second guiding objects 81 to 83 are arranged at positions along the future route of the vehicle. Details are the same as those in S21, and thus are omitted.

[0093] On the other hand, in the case where in S34, the "position where the second guiding object is superimposed on the scenery" is set to a position where at least a part of the second guiding object is superimposed and visible with respect to the guiding bifurcation point, as shown in FIG. 17, the second guiding objects 81 to 83 having three arrows are arranged at predetermined intervals above the road on which the vehicle is currently traveling in the three-dimensional space and at the position of the guiding bifurcation point. Further, the three second guiding objects 81 to 83 are arranged at positions along the future route of the vehicle. Here, FIG. 17 is a diagram showing an example of the arrangement of the second guiding object in the case where a guiding route for exiting the highway guiding bifurcation point 70 to the left is set. Specifically, as shown in FIG. 17, three second guiding objects 81 to 83 are arranged at equal intervals along the route of the vehicle. In particular, among the three second guiding objects 81 to 83, the left end portion (right end portion in the case of an arrow in the right direction) of the second guiding object 83 on the most forward side coincides with the coordinate X of the guiding bifurcation point (coincides in the horizontal direction). The coordinate X of the guiding bifurcation point shall be specified from the map information possessed by the navigation device 1, but it may also be specified by performing image recognition processing on the image captured by the front camera 19. In that case, the apex of the zebra stripe at the bifurcation is set as the coordinate X of the guiding bifurcation point. Also, the arrangement interval of the three second guiding objects 81 to 83 can be appropriately changed, but for example, it is set to an interval of 10 m. Also, the three second guiding objects 81 to 83 are not arranged in parallel with respect to the route of the vehicle, but are gradually inclined and arranged so that the arrow visible at a position closer to the guiding bifurcation point 70 is located on the traveling direction side (left side in FIG. 17) of the guiding bifurcation point 70 with respect to the route of the vehicle. In particular, when viewed from the vehicle, it is also arranged as a condition that the plurality of second guiding objects 81 to 83 do not overlap each other by a predetermined ratio (for example, 20%) or more. However, they may be arranged in parallel with respect to the traveling direction. On the other hand, in the vertical direction, as shown in FIG. 13, the lower ends of the second guiding objects 81 to 83 are arranged at a position separated from the road surface by a predetermined distance (for example, 1 m).

[0094] Next, in S37, the CPU 41 stores, as the size and shape of the guide object to be displayed on the liquid crystal display 15, the size and shape of the second guide objects 81 to 83 that can be visually recognized when looking in the traveling direction of the vehicle from the current position of the vehicle and the position of the height of the front camera 19 in the three-dimensional space where the second guide objects 81 to 83 were arranged in S36. Here, the size and shape of the guide object stored in S37 are the size and shape of the second guide objects 81 to 83 that can be visually recognized when the second guide objects 81 to 83 arranged in the three-dimensional space are visually recognized from the viewpoint of the current vehicle (more precisely, the front camera 19).

[0095] Thereafter, in S38, the CPU 41 estimates the future route of the own vehicle in the scenery 52 displayed on the liquid crystal display 15 based on the current position of the vehicle and the position of the guide branch point in the three-dimensional space generated in S31, and determines the range along the estimated route as the position where the second guide objects 81 to 83 are to be displayed on the liquid crystal display 15.

[0096] Thereafter, the process proceeds to S7, where images of the second guide objects 81 to 83 having the size and shape determined in S37 are generated, and a control signal is further transmitted to the liquid crystal display 15 to draw the generated images of the second guide objects 81 to 83 on the position (range) determined in S38 on the liquid crystal display 15.

[0097] As a result, the driving guidance screen 51 displayed on the liquid crystal display 15 as the vehicle travels becomes a screen as shown in FIG. 18. First, when the distance from the vehicle to the guiding branch point becomes less than the exit direction guiding start distance (for example, 300 m), the image 53 of the first guiding object shown in FIGS. 14 and 15 becomes invisible, and instead, the image 57 of the second guiding object is superimposed on the scenery 52 in front of the traveling direction of the vehicle imaged by the front camera 19 in the first mode and displayed. Incidentally, the position where the image 57 of the second guiding object is first superimposed is a position relatively fixed with respect to the current position of the vehicle, for example, 10 m in front of the vehicle. In the first mode, the image 57 of the second guiding object is visually recognized by the passenger in a state where the relative position with the vehicle is fixed and superimposed. After that, for a while, the image 57 of the second guiding object is superimposed and displayed in a state relatively fixed with respect to the current position of the vehicle. Since the relative position with respect to the vehicle is fixed, the display size of the image 57 of the second guiding object to be displayed is also fixed.

[0098] Thereafter, when the vehicle approaches the guiding fork and the image 57 of the second guiding object superimposed on a position relatively fixed with respect to the current position of the vehicle overlaps (reaches) the guiding fork (i.e., is first determined to be YES in S32), the mode switches from the first mode to the second mode. Thereafter, the relative position of the image 57 of the second guiding object with respect to the guiding fork within the scenery 52 is fixed. That is, the image 57 of the second guiding object is fixedly displayed at a position overlapping the guiding fork. Note that the switch from the first mode to the second mode does not change the guiding object to be the display target. The guiding object to be the display target is the same before and after the switch. That is, the guiding object to be the display target is the same, and only the display mode (the mode visually recognized by the occupant) switches from the first mode to the second mode. In the second mode, the image 57 of the second guiding object is visually recognized while being superimposed with the relative position to the guiding fork fixed for the occupant. Thereafter, as the vehicle approaches the guiding fork, the image 57 of the second guiding object gradually enlarges. Also, the transmittance of the image 57 of the second guiding object gradually increases, and the image 57 of the second guiding object disappears from the liquid crystal display 15 at the timing when the vehicle passes the guiding fork (S9).

[0099] As described in detail above, according to the navigation device 1 and the computer program executed by the navigation device 1 according to the first embodiment, the recommended lane recommended for driving on the road on which the vehicle is currently traveling is acquired (S15), the driving lane that is the lane on which the vehicle is currently traveling on the road on which the vehicle is currently traveling is specified (S13), and for the further specified driving lane, the reliability indicating how reliable the specified result is is acquired (S14), and a guiding object for guiding the recommended lane in a display mode according to the reliability is displayed (S5). Therefore, when the reliability of the driving lane is high, accurate guidance can be provided, and even when the reliability is low, it is possible to prevent causing a disadvantage to the occupant of the vehicle. In addition, since the guiding object is displayed in a more guiding display mode that guides the vehicle to the recommended lane as the reliability increases, when the reliability of the driving lane is high, accurate guidance to the recommended lane can be provided, while when the reliability is low, it is possible to prevent guiding the vehicle occupants to a lane different from the recommended lane. In addition, when the reliability is lower than the threshold value, an image indicating the direction of the recommended lane is displayed as the guiding object, and when the reliability is higher than the threshold value, an image that differentiates the recommended lane from other lanes is displayed as the guiding object. Thus, when the reliability of the driving lane is high, it is possible to enable the vehicle occupants to accurately recognize the recommended lane, while when the reliability is low, it is possible to prevent incorrect guidance by not providing guidance that specifically differentiates the recommended lane from other lanes. In addition, since an image that overlaps the road surface of the recommended lane is displayed as the image that differentiates the recommended lane from other lanes, when the reliability of the driving lane is high, it is possible to enable the vehicle occupants to accurately recognize the recommended lane based on the position where the image is overlapped. In addition, based on the captured image of the periphery of the vehicle, the lane lines around the vehicle and the road edge of the road on which the vehicle is currently traveling are respectively detected, the number of lanes from the road edge to the lane on which the vehicle is currently traveling is estimated based on the detection results of the lane lines and the road edge, and the driving lane is specified based on the estimation result (S12, S13). Therefore, it is possible to accurately specify the driving lane based on the detection results of the lane lines and the road edge. In addition, the reliability includes a right reliability that is the reliability when specifying the position of the driving lane based on the road right edge based on the detection results of the lane line and the road edge on the right side with respect to the traveling direction of the vehicle, and a left reliability that is the reliability when specifying the position of the driving lane based on the road left edge based on the detection results of the lane line and the road edge on the left side with respect to the traveling direction of the vehicle. Since the guiding object is displayed in a display mode corresponding to the overall reliability calculated based on the right reliability and the left reliability, it is possible to calculate the total reliability indicating whether the specification result of the driving lane can be trusted without distinction between the left and right based on the reliabilities based on the road edges on the left and right, and it is possible to provide appropriate guidance according to the total reliability. In addition, when there is a guiding branch point within a predetermined guiding start distance ahead in the traveling direction of the vehicle, a lane corresponding to the exiting direction of the vehicle at the guiding branch point is acquired as a recommended lane (S15). The reliability includes a right-side reliability which is the reliability when identifying the position of the traveling lane based on the detection result of the dividing line on the right side and the road edge with respect to the traveling direction of the vehicle, and a left-side reliability which is the reliability when identifying the position of the traveling lane based on the detection result of the dividing line on the left side and the road edge with respect to the traveling direction of the vehicle. Among the right-side reliability and the left-side reliability, the guiding object is displayed in a display mode corresponding to the reliability corresponding to the exiting direction of the vehicle at the guiding branch point. Therefore, even when there is uncertainty in identifying the position of the traveling lane, accurate guidance can be provided at least when the positional relationship between the traveling lane and the recommended lane can be trusted. In addition, the right-side reliability is calculated using at least one or more of the presence or absence of detection of the dividing line, the color of the detected dividing line, the type of the detected dividing line, and the inclination of the detected dividing line with respect to the traveling direction of the road, which are the results of detecting the dividing line on the right side with respect to the traveling direction of the vehicle by image recognition of the imaging image that images the periphery of the vehicle. The left-side reliability is calculated using at least one or more of the presence or absence of detection of the dividing line, the color of the detected dividing line, the type of the detected dividing line, and the inclination of the detected dividing line with respect to the traveling direction of the road, which are the results of detecting the dividing line on the left side with respect to the traveling direction of the vehicle by image recognition of the imaging image that images the periphery of the vehicle. Therefore, based on the result of the image recognition of the dividing line, it is possible to accurately calculate the reliability based on the left and right road edges. In addition, the right-side reliability and the left-side reliability are calculated by emphasizing the detection result of the dividing line at a position farther from the vehicle than the detection result of the dividing line at a position closer to the vehicle. Therefore, by calculating the reliability with the determination condition of whether the object that is more difficult to recognize than the easily recognizable object can be recognized, it is possible to calculate the reliability more accurately.

[0100] [Second Embodiment] Next, the superimposed image display device according to the second embodiment will be described with reference to FIGS. 19 and 20. In the following description, the same reference numerals as those in the configuration of the superimposed image display device according to the first embodiment in FIGS. 1 to 18 indicate the same or corresponding parts as the configuration of the superimposed image display device and the like according to the first embodiment.

[0101] The schematic configuration of the superimposed image display device according to the second embodiment is substantially the same as that of the superimposed image display device according to the first embodiment. Also, various control processes are substantially the same as those of the superimposed image display device according to the first embodiment. However, the superimposed image display device according to the first embodiment displays the captured image captured by the front camera 19 on the liquid crystal display 15 of the navigation device 1, and further displays a guide object on the liquid crystal display 15, thereby superimposing and displaying the guide object on the scenery around the vehicle. In contrast, the superimposed image display device according to the second embodiment is different in that it uses a head-up display system as a means for displaying an image to be superimposed on the scenery around the vehicle.

[0102] The schematic configuration of the superimposed image display device according to the second embodiment will be described below with reference to FIG. 19. FIG. 19 is a schematic configuration diagram of the superimposed image display device 101 according to the second embodiment. As shown in FIG. 19, the superimposed image display device 101 basically includes a navigation device 103 mounted on the vehicle 102 and a front display 104 that is also mounted on the vehicle 102 and connected to the navigation device 103. Note that the front display 104 functions as a head-up display together with the front glass 105 of the vehicle 102 and serves as an information providing means for providing various information to the passenger 106 of the vehicle 102.

[0103] Here, the front display 104 is a liquid crystal display installed inside the dashboard 107 of the vehicle 102 and having a function of displaying an image on an image display surface provided on the front surface. As the backlight, for example, a CCFL (cold cathode tube) or a white LED is used. Note that, as the front display 104, in addition to a liquid crystal display, an organic EL display or a combination of a liquid crystal projector and a screen may be used.

[0104] Then, the front display 104 functions as a head-up display together with the front glass 105 of the vehicle 102, and is configured to reflect the image output from the front display 104 on the front glass 105 in front of the driver's seat so that the passenger 106 of the vehicle 102 can visually recognize it. Note that the front display 104 displays a guidance object as necessary. In the second embodiment described below, the guidance object is an arrow indicating the exit direction of a guidance branch point along the guidance route, a guidance image indicating the position and direction of a recommended lane on the road on which the vehicle is currently traveling in order to pass through the guidance branch point along the guidance route, and a guidance image prompting movement to the recommended lane, as in the first embodiment.

[0105] Further, when the passenger 106 visually recognizes the video displayed on the front display 104 by reflecting off the front glass 105, the video displayed on the front display 104 is visually recognized as a virtual image 110 at a position far ahead of the front glass 105 instead of at the position of the front glass 105 for the passenger 106. The virtual image 110 is to be displayed superimposed on the surrounding environment (scenery, real scene) in front of the vehicle, and can be displayed superimposed on an arbitrary object (road surface, building, object to be warned, etc.) located in front of the vehicle, for example.

[0106] Here, the position where the virtual image 110 is generated, more specifically, the distance L from the occupant 106 to the virtual image 110 (hereinafter referred to as the imaging distance) is determined by the position of the front display 104. For example, the imaging distance L is determined by the distance (optical path length) along the optical path from the position where the video is displayed on the front display 104 to the front glass 105. For example, the optical path length is set so that the imaging distance L is 1.5 m.

[0107] In addition, a front camera 111 is installed above the front bumper of the vehicle or behind the rearview mirror. The front camera 111 is an imaging device having a camera using a solid-state imaging device such as a CCD, and is installed with the optical axis direction facing forward in the traveling direction of the vehicle. Then, by performing image processing on the captured image captured by the front camera 111, the situation of the front environment (that is, the environment where the virtual image 110 is superimposed) visually recognized by the occupant 106 through the front glass is detected. Note that a sensor such as a millimeter-wave radar may be used instead of the front camera 111.

[0108] Also, an in-vehicle camera 112 is installed on the upper surface of the instrument panel of the vehicle. The in-vehicle camera 112 is an imaging device having a camera using a solid-state imaging device such as a CCD, and is installed with the optical axis direction facing the driver's seat. A range where the face of the occupant is generally expected to be located in the vehicle is set as the detection range (imaging range of the in-vehicle camera 112), and the face of the occupant 106 sitting in the driver's seat is imaged. Then, by performing image processing on the captured image captured by the in-vehicle camera 112, the position (line-of-sight starting point) and the line-of-sight direction of the eyes of the occupant 106 are detected.

[0109] Then, in S5 and S7 of the driving support processing program (FIG. 2) according to the second embodiment, the superimposed image display device displays an image 120 of a guiding object on the front display 104 as shown in FIG. 20. As a result, when the vehicle occupant visually recognizes the image 120 of the guiding object displayed on the front display 104 as shown in FIG. 20, a virtual image 121 of the image 120 of the guiding object is visually recognized superimposed on the scenery through the front glass 105.

[0110] Thereby, similar to the superimposed image display device according to the first embodiment, the position and orientation of the recommended lane and the exit direction at the guiding branch point can be accurately grasped. In addition, in the superimposed image display device according to the second embodiment, in the guiding object display position determination process of S4 and S6, the size, shape of the guiding object to be displayed on the front display 104, and the position (range) where the guiding object is displayed are determined. Further, it is desirable that the current position and orientation of the host vehicle specified in the three-dimensional space in S11 and S35 be the position of the vehicle occupant and the line-of-sight direction of the occupant detected using the in-vehicle camera 112.

[0111] Note that the present invention is not limited to the above embodiments, and it goes without saying that various improvements and modifications can be made without departing from the gist of the present invention. For example, as a means for displaying an image superimposed on the scenery around the vehicle, in the first embodiment, the liquid crystal display 15 on which the real-scene image is displayed is used, and in the second embodiment, the head-up display system is used. However, a windshield display (WSD) that displays an image on the front glass may also be used. In the WSD, an image may be displayed from a projector using the front glass as a screen, or the front glass may be used as a transmissive liquid crystal display. The image displayed on the front glass by the WSD becomes an image superimposed on the scenery around the vehicle.

[0112] Also, in the first and second embodiments, when the reliability regarding the identification result of the host vehicle's traveling lane is higher than the threshold value, an image of a guiding object that discriminates the recommended lane from other lanes is displayed as highly guiding information for guiding the vehicle to the recommended lane (S17 - S19). When the reliability is lower than the threshold value, an image of a guiding object indicating the direction of the recommended lane is displayed as guiding information with low guiding ability for guiding the vehicle to the recommended lane (S21 - S23). However, how to change the guiding mode based on the reliability can be appropriately changed. For example, the guiding object may be displayed with a transmittance or color corresponding to the reliability. Specifically, the higher the reliability, the guiding object is displayed with a transmittance or color with higher visibility for the user. For example, when the reliability regarding the identification result of the host vehicle's traveling lane is higher than the threshold value, the display color of the guiding object is set to eye-catching yellow or red, or the transmittance is lowered (e.g., 10%). On the other hand, when the reliability regarding the identification result of the host vehicle's traveling lane is lower than the threshold value, the display color is set to inconspicuous green or white, or the transmittance is increased (e.g., 80%).

[0113] Also, in the first and second embodiments, as the reliability regarding the identification result of the host vehicle's traveling lane, the right - hand reliability CR, the left - hand reliability CL, and the overall reliability CT are calculated respectively. However, it is not necessary to calculate all of the right - hand reliability CR, the left - hand reliability CL, and the overall reliability CT. It is also possible to calculate only any one or two of them. Also, as the calculation method of the right - hand reliability and the left - hand reliability, it is calculated based on whether the lane lines that should exist can be correctly recognized (Equations (1) - (7)). However, it may be calculated using the recognition result of the road edge instead of the recognition result of the lane lines. Also, in Equations (1) - (7) related to the calculation of the right - hand reliability CR and the left - hand reliability CL, all of the presence or absence of lane line detection, the color of the detected lane lines, the type of the detected lane lines, and the inclination of the detected lane lines with respect to the traveling direction of the road are used to calculate each reliability. However, it is not necessary to use all of them, and it is also possible to calculate the reliability using only a part of them.

[0114] In the first and second embodiments, the guiding objects are the first guiding objects 71 to 74 that guide the recommended lanes of the vehicle and the second guiding objects 81 to 83 that are arrows indicating the exiting directions of the vehicle at the guiding branch points ahead in the traveling direction. However, only the first guiding objects 71 to 74 may be used. Further, the first guiding objects 71 to 74 do not necessarily have to be in a rectangular shape or a triangular shape that overlaps with the lane, and other shapes may be used as long as they can indicate the position and orientation of the recommended lane.

[0115] In the first and second embodiments, the guiding branch points are guided using the guiding objects. However, the points to be guided by the guiding objects are not limited to the guiding branch points. For example, when the vehicle passes through a lane reduction point or a merging section, the position and direction of the recommended lane may be guided using the guiding objects.

[0116] In the first embodiment, the real-scene image captured by the front camera 19 and the guiding objects are displayed on the liquid crystal display 15 of the navigation device 1. However, as long as it is a display arranged inside the vehicle, a display other than the liquid crystal display 15 may be used as the display for displaying the real-scene image and the guiding objects.

[0117] In the second embodiment, a virtual image is generated in front of the front windshield 105 of the vehicle 102 by the front display 104. However, a configuration in which a virtual image is generated in front of a window other than the front windshield 105 may also be used. Further, the object for reflecting the video by the front display 104 may be a visor (combiner) installed around the front windshield 105 instead of the front windshield 105 itself.

[0118] In the first and second embodiments, the navigation ECU 13 of the navigation device 1 executes the processing of the driving support processing program (FIG. 2). However, the execution entity can be appropriately changed. For example, a configuration in which the control unit of the liquid crystal display 15, the vehicle control ECU, or other in-vehicle devices execute the processing may also be used.

Description of Signs

[0119] 1... Navigation device, 15... Liquid crystal display, 19... Front camera, 41... CPU, 42... RAM, 43... ROM, 51... Driving guidance screen, 52... Landscape, 53... Image of the first guidance object, 54... Recommended lane, 55... Own vehicle driving lane, 57... Image of the second guidance object, 61, 64... Road edge, 62, 65... Outer lane line, 63, 66... Lane boundary line, 70... Guidance branch point, 71 - 74... First guidance object, 81 - 83... Second guidance object

Claims

1. A superimposed image display device mounted on a vehicle, which superimposes a guiding object for guiding information to an occupant of the vehicle on a scenery around the vehicle and makes it visible, comprising: A recommended lane acquisition means for acquiring a recommended lane recommended for driving on a road on which the vehicle is currently traveling; A traveling lane identification means for identifying a traveling lane, which is the lane on which the vehicle is currently traveling, on a road on which the vehicle is currently traveling; A reliability acquisition means for acquiring a reliability indicating to what extent the identified result is reliable for the traveling lane identified by the traveling lane identification means; Object display means for displaying the guiding object for guiding to the recommended lane based on the traveling lane and the recommended lane, and having: The object display means is a superimposed image display device that displays the guiding object in a display mode according to the reliability.

2. The superimposed image display device according to claim 1, wherein the object display means displays the guiding object in a display mode with higher guiding property for guiding the vehicle to the recommended lane as the reliability is higher.

3. The object display means: When the reliability is lower than a threshold value, displays an image indicating the direction of the recommended lane as the guiding object; The superimposed image display device according to claim 2, wherein when the reliability is higher than the threshold value, displays an image for distinguishing the recommended lane from other lanes as the guiding object.

4. The superimposed image display device according to claim 3, wherein the object display means displays an image superimposed on the road surface of the recommended lane as an image for distinguishing the recommended lane from other lanes.

5. The object display means: Displays the guiding object with a transmittance or color according to the reliability; The superimposed image display device according to claim 2, wherein the higher the reliability, the more the guiding object is displayed with a transmittance or color that is more visible to the user.

6. The driving lane specifying means Based on an imaging image obtained by imaging the periphery of the vehicle, respectively detects a lane line around the vehicle and a road edge of the road on which the vehicle is currently traveling, estimates the number of lanes from the road edge to the lane on which the vehicle is currently traveling based on the detection results of the lane line and the road edge, The superimposed image display device according to any one of claims 1 to 5, which specifies the driving lane based on the result of the estimation.

7. The reliability includes a right-side reliability that is the reliability when specifying the position of the driving lane based on the road right end based on the detection results of the lane line and the road edge on the right side with respect to the traveling direction of the vehicle, and a left-side reliability that is the reliability when specifying the position of the driving lane based on the road left end based on the detection results of the lane line and the road edge on the left side with respect to the traveling direction of the vehicle, The object display means displays the guiding object in a display mode corresponding to the overall reliability calculated based on the right-side reliability and the left-side reliability. The superimposed image display device according to any one of claims 1 to 6.

8. When there is a guiding branch point within a predetermined guiding start distance ahead in the traveling direction of the vehicle, the recommended lane acquisition means acquires the lane corresponding to the exiting direction of the vehicle at the guiding branch point as the recommended lane, The reliability includes a right-side reliability that is the reliability when specifying the position of the driving lane based on the road right end based on the detection results of the lane line and the road edge on the right side with respect to the traveling direction of the vehicle, and a left-side reliability that is the reliability when specifying the position of the driving lane based on the road left end based on the detection results of the lane line and the road edge on the left side with respect to the traveling direction of the vehicle, The object display means displays the guidance object in a display mode corresponding to the reliability corresponding to the exit direction of the vehicle at the guidance branch point among the right-side reliability and the left-side reliability, according to any one of claims 1 to 6 of the superimposed image display device.

9. The right-side reliability is a result calculated using at least one or more of the presence or absence of detection of a lane line, the color of the detected lane line, the type of the detected lane line, and the inclination of the detected lane line with respect to the traveling direction of the road, which is a result detected by image recognition of an imaging image obtained by imaging the periphery of the vehicle of a lane line on the right side with respect to the traveling direction of the vehicle. The left-side reliability is a result calculated using at least one or more of the presence or absence of detection of a lane line, the color of the detected lane line, the type of the detected lane line, and the inclination of the detected lane line with respect to the traveling direction of the road, which is a result detected by image recognition of an imaging image obtained by imaging the periphery of the vehicle of a lane line on the left side with respect to the traveling direction of the vehicle, according to the superimposed image display device of claim 7 or claim 8.

10. The right-side reliability and the left-side reliability of the superimposed image display device according to claim 9 are calculated while emphasizing the detection result of a lane line at a position farther than the detection result of a lane line at a position closer to the vehicle.

Images