Superimposed image display device

Abstract

To provide a superposed image display device that enables appropriate lane change guidance without significantly changing the content of an image visible to an occupant of a vehicle.SOLUTION: If, on the one hand, it is determined that a superposable area of scenery around a vehicle, which can be superposed with a guidance object and made visible to a vehicle occupant, does not include a parcel line to be crossed by the vehicle, a virtual parcel line imitating the parcel line is displayed in addition to the guidance object, and the guidance object is displayed based on the virtual parcel line. However, if, after the virtual parcel line is displayed, it is determined that the parcel line is included in the superposable area and that the parcel line and the virtual parcel line visible to the occupant are parallel, the virtual parcel line is not displayed and the guide object is then displayed based on the parcel line.SELECTED DRAWING: Figure 10

Description

Technical Field

[0001] The present invention relates to a superimposed image display device for performing driving support of a vehicle.

Background Art

[0002] Conventionally, various means have been used as information providing means for providing various information for performing driving support of a vehicle, such as route guidance and warning of obstacles, to an occupant of the vehicle. For example, display by a liquid crystal display installed in the vehicle, voice output from a speaker, and the like. In recent years, as one of such information providing means, there is a device that provides information by displaying an image superimposed on the surrounding environment (scenery, actual scene) of the occupant. For example, a head-up display, a windshield display, and a method of displaying an image superimposed on a captured image around the vehicle displayed on a liquid crystal display are applicable.

[0003] Among the technologies for displaying an image superimposed on the surrounding environment, in particular, a head-up display reflects an image displayed in a display area by a reflecting member such as a windshield or a combiner disposed in the vehicle and allows an occupant of the vehicle to visually recognize it, thereby superimposing a virtual image of the image on the actual scene around the vehicle. The head-up display has the merit that a guidance image can be superimposed on the actual scenery (actual scene) outside the vehicle that can be visually recognized by the occupant. However, on the other hand, there is a problem that the range in which the guidance image can be superimposed and visually recognized by the occupant of the vehicle (hereinafter referred to as the superimposable range) is limited. Therefore, for example, in Japanese Patent Laid-Open No. 2020-24561, when performing right / left turn guidance at a guidance branch point, if the guidance branch point is not included in the superimposable range, an enlarged view of the intersection periphery is displayed and an arrow image is displayed superimposed on the displayed enlarged view of the intersection. On the other hand, when the guidance branch point is included in the superimposable range, a technique of displaying a large arrow image superimposed on the actual guidance branch point is disclosed.

Prior Art Documents

Patent Documents

[0004] [Patent Document 1] Japanese Patent Publication No. 2020-24561 (Figures 8 and 9) [Overview of the project] [Problems that the invention aims to solve]

[0005] According to the technology described in Patent Document 1, even if the superimposable range is limited, guidance can be provided that aligns with the position within the superimposable range. However, with the technology described in Patent Document 1, the image to be displayed changes significantly at the moment when the guidance branching point switches from being included in the superimposable range to not being included while the vehicle is in motion. Such a sudden and large change in the image to be displayed attracts the attention of the vehicle's occupants and causes them to focus on areas other than the direction of travel, so it is desirable to avoid this.

[0006] The present invention was made to solve the aforementioned problems of the conventional invention, and aims to provide a superimposed image display device that enables appropriate guidance for lane changes without significantly altering the content of the image visible to the vehicle occupants, particularly when using images superimposed on the surrounding environment to guide lane changes. [Means for solving the problem]

[0007] To achieve the above objective, the superimposed image display device according to the present invention is mounted on a vehicle and allows the occupants of the vehicle to see a virtual image of the image superimposed on the scenery around the vehicle by reflecting the image displayed in the display area with a reflective member placed inside the vehicle, thereby allowing the occupants of the vehicle to see the image, and comprises an object display means for displaying guidance objects for guiding lane changes as the image when a lane change of the vehicle is recommended, and a lane line detection means for detecting the lane lines that the vehicle will cross when changing lanes, and the object display means is located around the vehicle If it is determined that the lane markings are not included in the superimposable range of the scenery in which the guide object can be superimposed and made visible to the vehicle occupants, then virtual lane markings that mimic the lane markings are displayed in addition to the guide object as part of the image, the guide object is displayed based on the virtual lane markings, and after the virtual lane markings are displayed, if it is determined that the lane markings are included in the superimposable range and that the lane markings and the virtual lane markings visible to the occupants are parallel, then the virtual lane markings are hidden, and thereafter the guide object is displayed based on the lane markings. [Effects of the Invention]

[0008] According to the superimposed image display device of the present invention having the above configuration, when providing guidance for lane changes using an image superimposed on the surrounding environment, it is possible to display guidance objects based on the lane markings whether or not the lane markings that the vehicle will cross during the lane change are included in the superimposed range. Furthermore, when transitioning from guidance based on virtual lane markings to guidance based on actual lane markings, the transition is performed while the lane markings are included in the superimposed range and the virtual lane markings visible to the vehicle occupants are parallel to the actual lane markings, thus enabling a seamless transition. In other words, at the timing of the transition from guidance based on virtual lane markings, which are virtual images, to guidance based on actual lane markings, the content of the image visible to the vehicle occupants does not change significantly, thus preventing the vehicle occupants from being distracted (by looking in directions other than the direction of travel). [Brief explanation of the drawing]

[0009] [Figure 1] This is a schematic diagram of the superimposed image display device according to this embodiment. [Figure 2] This diagram illustrates an example of a landscape seen by a vehicle occupant and a virtual image superimposed on that landscape. [Figure 3] This is a block diagram showing the navigation device according to this embodiment. [Figure 4] This is a flowchart of the driving support processing program according to this embodiment. [Figure 5] This diagram shows an example of a recommended lane when passing through a highway junction. [Figure 6] This diagram shows an example of a recommended lane when passing through a road junction with road guidance. [Figure 7] This diagram illustrates the method for detecting the vehicle's lane of travel. [Figure 8] This is a flowchart of the sub-processing program for displaying guidance objects. [Figure 9] This diagram illustrates the distance from a vehicle to the target lane marking along the road width. [Figure 10] This diagram shows examples of guide objects visible to vehicle occupants when the distance from the vehicle to the target lane marking along the road width direction is 1 meter or more. [Figure 11] This diagram illustrates the way the angle of the virtual lane lines changes. [Figure 12] This diagram shows examples of guide objects visible to vehicle occupants when the distance from the vehicle to the target lane marking along the road width direction is less than 1m but 0.5m or more. [Figure 13] This diagram shows an example of a guide object visible to a vehicle occupant when the distance from the vehicle to the target lane marking along the road width is less than 0.5m. [Figure 14] This diagram shows the changes in the guidance objects displayed on the front display as the vehicle is in motion. [Modes for carrying out the invention]

[0010] Hereinafter, an embodiment in which the superimposed image display device 1 according to the present invention is embodied will be described in detail with reference to the drawings. First, the schematic configuration of the superimposed image display device 1 according to the present embodiment will be described using FIG. 1. FIG. 1 is a schematic configuration diagram of the superimposed image display device 1 according to the present embodiment.

[0011] As shown in FIG. 1, the superimposed image display device 1 basically includes a navigation device 3 mounted on the vehicle 2 and a front display 4 that is also mounted on the vehicle 2 and connected to the navigation device 3. Note that the front display 4 functions as a head-up display together with the front glass 5 of the vehicle 2 as described later, and serves as an information providing means for providing various information to the occupant 6 of the vehicle 2.

[0012] Here, the navigation device 3 has functions such as searching for a recommended route to a destination, displaying a map image around the current position of the vehicle 2 based on map data acquired from a server or stored in a memory, and performing driving guidance along a set guidance route together with the front display 4. Note that it is not necessary for the navigation device 3 to have all of the above functions, and the present invention can be constituted as long as it has at least a function of performing driving guidance along a guidance route. The details of the structure of the navigation device 3 will be described later.

[0013] On the other hand, the front display 4 is a liquid crystal display installed inside the dashboard 7 of the vehicle 2 and having a function of displaying an image on an image display surface provided on the front surface. As the backlight, for example, a CCFL (cold cathode tube) or a white LED is used. Note that as the front display 4, in addition to a liquid crystal display, an organic EL display or a combination of a liquid crystal projector and a screen may be used.

[0014] [[ID=第十九]] The front display 4 functions as a head-up display together with the front windshield 5 of the vehicle 2, and is configured to reflect the image output from the front display 4 on the front windshield 5 in front of the driver's seat so that the passenger 6 of the vehicle 2 can visually recognize it.

[0015] Here, in the present embodiment, the images displayed on the front display 4 and visually recognized by the passenger 6 of the vehicle 2 include information about the vehicle 2 and various types of information used for assisting the driving of the passenger 6. For example, warnings for objects (other vehicles, pedestrians, guide signs) to be warned against for the passenger 6, the guide route set by the navigation device 3 and guide information based on the guide route (arrows indicating the right and left turn directions, the position of the recommended lane in which the vehicle should travel, guidance prompting lane change to the recommended lane, etc.), warnings displayed on the road surface (collision warning, speed limit, etc.), the lane dividing lines of the lane in which the vehicle is traveling, the current vehicle speed, the shift position, the remaining energy amount, advertisement images, facility information, map images, traffic information, news, weather forecasts, time, the screen of the connected smartphone, TV programs, etc. In the present embodiment described below, information for guiding particularly at a guide branch point in front of the traveling direction of the vehicle is displayed on the front display 4 as a guide object. More specifically, when the vehicle is not traveling in the recommended lane recommended for traveling on the road on which the vehicle is currently traveling to pass through the guide branch point, one or more arrows for prompting movement to the recommended lane are displayed.

[0016] Also, when the passenger 6 visually recognizes the video displayed on the front display 4 by reflecting off the front windshield 5, the video displayed on the front display 4 is visually recognized as a virtual image 10 at a position far ahead of the front windshield 5 rather than at the position of the front windshield 5 for the passenger 6. Further, the virtual image 10 is to be displayed superimposed on the surrounding environment (scenery, real scene) in front of the vehicle, and can be displayed superimposed on, for example, any object (road surface, intersection, building, object to be warned against, etc.) located in front of the vehicle.

[0017] [[ID=|11]] For example, Figure 2 shows an example of displaying a virtual image 10 superimposed on the surrounding environment (landscape, real scenery) in front of the vehicle. As shown in Figure 2, for example, an arrow image 9 is displayed on the front display 4. As a result, the arrow image 9 displayed on the front display 4 is reflected off the windshield 5 and viewed by the vehicle occupant 6, thereby allowing the virtual image 10 of the arrow image 9 to be seen superimposed on the scenery seen through the windshield 5. However, the virtual image 10 cannot be superimposed anywhere in the scenery seen through the windshield 5, and the range in which the virtual image 10 can be superimposed and seen by the vehicle occupant 6 is limited. Specifically, this range is determined by the size of the display screen of the front display 4 and its installation position relative to the dashboard 7. For example, in the example shown in Figure 2, the area R enclosed by the dashed line is the range in which the virtual image 10 can be superimposed and seen by the vehicle occupant 6 (superimpossible range).

[0018] On the other hand, the position in the depth direction where the virtual image 10 is generated, more specifically the distance L from the occupant 6 to the virtual image 10 (hereinafter referred to as the imaging distance), is determined by the position of the front display 4. For example, the imaging distance L is determined by the distance along the optical path from the position where the image is displayed on the front display 4 to the windshield 5 (optical path length). For example, the optical path length is set so that the imaging distance L is 1.5m.

[0019] Furthermore, in this embodiment, a front display 4 installed inside the dashboard 7 is used as a means to display an image superimposed on the scenery around the vehicle, but other means may be used. For example, a head-up display (HUD) device may be provided as an in-vehicle device, or the windshield 5 may be used as a transparent liquid crystal display to display an image directly on the windshield 5. Moreover, it is also possible to display the scenery around the vehicle captured by the front camera 11 (described later) on the in-vehicle liquid crystal display, and to display an image superimposed on the scenery displayed on the same liquid crystal display. In that case as well, the image displayed on the liquid crystal display will be an image superimposed on the scenery around the vehicle, similar to a HUD.

[0020] Furthermore, a front camera 11 is installed above the vehicle's front bumper or behind the rearview mirror. The front camera 11 is an imaging device that has a camera using a solid-state image sensor such as a CCD, and is installed with its optical axis facing forward in the direction of vehicle travel. Image processing is performed on the image captured by the front camera 11 to detect the conditions of the forward environment (i.e., the environment on which the virtual image 10 is superimposed) as seen by the occupant 6 through the windshield. In particular, the image captured by the front camera 11 is also used to detect lane markings that demarcate the road the vehicle is currently traveling on, as will be described later. Note that a sensor such as a millimeter-wave radar may be used instead of the front camera 11.

[0021] In addition, an in-vehicle camera 12 is installed on the top surface of the vehicle's instrument panel. The in-vehicle camera 12 is an imaging device that has a camera using a solid-state image sensor such as a CCD, and is installed with its optical axis facing the driver's seat. The detection range (imaging range of the in-vehicle camera 12) is set to the area in the vehicle where the occupant's face is generally expected to be located, and the face of the occupant 6 sitting in the driver's seat is captured. Then, image processing is performed on the image captured by the in-vehicle camera 12 to detect the position of the occupant 6's eyes (starting point of gaze) and the direction of their gaze.

[0022] Next, the schematic configuration of the navigation device 3 that constitutes the superimposed image display device 1 will be explained using Figure 3. Figure 3 is a block diagram showing the navigation device 3 according to this embodiment.

[0023] As shown in Figure 3, the navigation device 3 according to this embodiment includes a current location detection unit 13 that detects the current location of the vehicle 2 on which the navigation device 3 is installed, a data recording unit 14 on which various data are recorded, a navigation ECU 15 that performs various calculation processing based on the input information, an operation unit 16 that accepts operations from the user, a liquid crystal display 17 that displays a map of the area around the vehicle and facility information to the user, a speaker 18 that outputs voice guidance regarding route guidance, a DVD drive 19 that reads a DVD which is a storage medium, and a communication module 20 that communicates with information centers such as probe centers and VICS (registered trademark: Vehicle Information and Communication System) centers. Furthermore, the navigation device 3 is connected to the aforementioned front display 4, front camera 11, and in-vehicle camera 12, etc., via an in-vehicle network such as CAN.

[0024] The following describes each component of the navigation device 3 in order. The current position detection unit 13 consists of a GPS 21, a vehicle speed sensor 22, a steering sensor 23, a gyro sensor 24, etc., and is capable of detecting the current position, direction, vehicle speed, current time, etc. In particular, the vehicle speed sensor 22 is a sensor for detecting the distance traveled and vehicle speed of the vehicle, and generates pulses in accordance with the rotation of the vehicle's drive wheels and outputs the pulse signal to the navigation ECU 15. The navigation ECU 15 then calculates the rotation speed of the drive wheels and the distance traveled by counting the generated pulses. It should be noted that the navigation device 3 does not need to be equipped with all four types of sensors mentioned above, and the navigation device 3 may be configured to be equipped with only one or more of these types of sensors.

[0025] Furthermore, the data recording unit 14 includes a hard disk (not shown) as an external storage device and recording medium, and a recording head (not shown) which is a driver for reading map information DB31 and predetermined programs recorded on the hard disk and writing predetermined data to the hard disk. The data recording unit 14 may also have flash memory, a memory card, or an optical disc such as a CD or DVD instead of a hard disk. Additionally, the map information DB31 may be stored on an external server and acquired by the navigation device 3 via communication.

[0026] Here, the map information DB31 is a storage means that stores, for example, link data 32 related to roads (links), node data 33 related to node points, branching point data 34 related to branching points, point data related to facilities and other locations, map display data for displaying maps, search data for searching for routes, search data for searching for locations, and so on.

[0027] Furthermore, the link data 32 includes data for each link constituting the road network, such as the width, slope, cant, bank, road surface condition of the road to which the link belongs, shape interpolation point data to identify the link shape between nodes (for example, the shape of the curve in the case of a curved road), the number of lanes on the road, the direction of travel classification for each lane, the presence or absence of oncoming lanes (whether or not it is a two-way traffic section), the presence or absence of a median strip, sections where the width narrows, and level crossings. For corners, it includes data representing the radius of curvature, intersections, T-junctions, corner entrances and exits, etc. For roads, it includes data representing downhill roads, uphill roads, etc. For road types, it includes data representing expressways and general roads (national roads, prefectural roads, narrow streets, etc.).

[0028] Furthermore, the node data 33 records data such as the coordinates (positions) of node points set at predetermined distances according to the radius of curvature, etc., for actual road junctions (including intersections, T-junctions, etc.), node attributes indicating whether a node corresponds to an intersection, a list of connection link numbers which is a list of link numbers for links connected to a node, a list of adjacent node numbers which is a list of node numbers for nodes adjacent to a node via links, and data related to the height (altitude) of each node point.

[0029] Furthermore, the branching point data 34 stores information such as the name of the intersection at the branching point, the node information that identifies the node forming the branching point, the connection link information that identifies the link connected to the branching point, the name of the direction corresponding to the link connected to the branching point, and information that identifies the shape of the branching point (especially the direction of road connection).

[0030] On the other hand, the navigation ECU (Electronic Control Unit) 15 is an electronic control unit that controls the entire navigation device 3, and includes a CPU 41 as a processing unit and control device, a RAM 42 which is used as working memory when the CPU 41 performs various calculations and stores route data when a route is searched, a ROM 43 which stores control programs as well as driving support processing programs (Figure 4) described later, and a flash memory 44 which stores programs read from the ROM 43. The navigation ECU 15 also has various means as processing algorithms. For example, the object display means displays guidance objects as images to guide the vehicle to change lanes when a lane change is recommended. The lane marking detection means detects the lane markings that the vehicle will cross when changing lanes.

[0031] The operation unit 16 is operated when inputting the starting point (departure point) and the ending point (destination point), and has multiple operation switches (not shown), such as various keys and buttons. The navigation ECU 15 controls the system to perform various operations based on the switch signals output when each switch is pressed. The operation unit 16 may also have a touch panel located in front of the liquid crystal display 17. It may also have a microphone and a voice recognition device.

[0032] Furthermore, the liquid crystal display 17 displays map images including roads, traffic information, operation instructions, operation menus, key instructions, guided routes from the starting point to the destination, guidance information along the guided route, news, weather forecasts, time, emails, TV programs, etc. In this embodiment, a front display 4 is provided as a means of displaying information, so if the map images and the like are displayed on the front display 4, the liquid crystal display 17 may be omitted.

[0033] Furthermore, the speaker 18 outputs voice guidance that directs the driver along the designated route based on instructions from the navigation ECU 15, as well as traffic information.

[0034] Furthermore, the DVD drive 19 is a drive capable of reading data recorded on recording media such as DVDs and CDs. Based on the read data, it performs functions such as playing music and videos and updating the map information DB31. Alternatively, a card slot for reading and writing memory cards may be provided instead of the DVD drive 19.

[0035] Furthermore, the communication module 20 is a communication device for receiving traffic information consisting of various types of information such as congestion information, regulation information, and traffic accident information transmitted from traffic information centers, such as VICS centers and probe centers, and examples of such devices include mobile phones and DCMs.

[0036] Next, the driving support processing program executed by the navigation ECU 15 in the navigation device 3 having the above configuration will be explained with reference to Figure 4. Figure 4 is a flowchart of the driving support processing program according to this embodiment. Here, the driving support processing program is executed after the vehicle's ACC power supply (accessory power supply) is turned ON, and is a program that assists the vehicle's driving by superimposing the image displayed on the front display 4 onto the scenery (actual view) around the vehicle for visual confirmation. The programs shown in the flowcharts in Figures 4 and 8 below are stored in the RAM 42 and ROM 43 of the navigation device 3 and executed by the CPU 41.

[0037] The following explanation describes an example of using guidance objects to assist vehicle driving, specifically providing driving guidance to a vehicle along a route set by the navigation device 3. The guidance objects to be displayed are guidance information for guiding the vehicle at a guidance junction ahead in the direction of travel. In particular, the explanation will describe the process of displaying an arrow as a guidance object to encourage the vehicle to move to the recommended lane when the vehicle is not currently traveling in the recommended lane on the road it is currently traveling on in order to pass through a guidance junction. However, the superimposed image display device 1 can also provide guidance and information other than the above driving assistance using guidance objects. Furthermore, the guidance objects to be displayed can also be information other than the arrows mentioned above. For example, guidance objects can display warnings to occupants about objects that require warning (other vehicles, pedestrians, road signs), warnings to be displayed on the road surface (caution against rear-end collisions, speed limits, etc.), distance to the next guidance junction, current vehicle speed, shift position, remaining energy, advertising images, facility information, road signs, map images, traffic information, news, weather forecasts, time, the screen of a connected smartphone, etc.

[0038] First, in the driving support processing program, in step (hereinafter abbreviated as S) 1, the CPU 41 identifies the vehicle's current position based on the detection results of the current position detection unit 13 and map information. When identifying the vehicle's current position, a map matching process is also performed to match the vehicle's current position with the map information. After that, the guidance route set in the navigation device 3 is read, and the distance from the identified vehicle's current position to the next guidance junction (guidance target point) along the guidance route is calculated. A guidance junction is a branching point (intersection) where the navigation device 3 provides guidance such as right or left turn instructions (including instructions to enter left or right turn ramps) when providing driving guidance according to the guidance route set in the navigation device 3. Note that branching points with special shapes (difficult intersections) that do not involve right or left turns are also considered guidance junctions.

[0039] Next, in S2, the CPU 41 determines whether the distance to the next guidance branching point calculated in S1 is less than a predetermined guidance start distance. The guidance start distance is determined by the type of road the vehicle is traveling on; for example, it is 1 km for expressways and 700 m for general roads, which is shorter than expressways. However, the guidance start distance may be a variable value rather than a fixed value. For example, it may be set according to the vehicle's speed.

[0040] If it is determined that the distance to the next guidance junction calculated in S1 is less than the guidance start distance (S2: YES), the process proceeds to S3. Conversely, if it is determined that the distance to the next guidance junction calculated in S1 is not less than the guidance start distance (S2: NO), the process returns to S1.

[0041] In S3, the CPU 41 identifies the lane that the vehicle must travel in in order to pass through the guidance junction ahead of the vehicle in the direction of exit (guidance direction) along the guidance route, based on the map information and the guidance route, as the recommended lane. More specifically, it acquires the lane corresponding to the vehicle's exit direction at the guidance junction as the recommended lane. The link data 32 included in the map information stores the traffic classification for the direction of travel for each lane, and the junction data 34 stores the shape of the junction, and this information is used to identify the recommended lane. For example, as shown in Figure 5, if the exit direction (guidance direction) at the guidance junction 50 ahead of the vehicle traveling on the expressway is diagonally to the left where the access road is located, the leftmost lane of the three lanes on the road the vehicle is traveling on becomes the recommended lane. On the other hand, as shown in Figure 6, if the exit direction (guidance direction) at the guidance junction 50 ahead of the vehicle traveling on the general road is to the right, the rightmost lane of the four lanes on the road the vehicle is traveling on, corresponding to the right turn traffic classification, becomes the recommended lane. Please note that recommended lanes are not necessarily limited to one lane; there may be two or more lanes.

[0042] Next, in S4, CPU41 identifies the "vehicle's lane," which is the lane the vehicle is currently traveling in. For example, the vehicle's lane is identified by the following process. First, the CPU 41 performs image recognition processing on the image captured by the front camera 11, recognizing the lane markings drawn around the vehicle, the edge of the road, and surrounding vehicles. In addition to the recognition results, it also considers map information to identify the vehicle's lane. The link data 32 included in the map information stores the number of lanes and lane width for each road. For example, when driving on a three-lane highway as shown in Figure 7, if the edge of the road 55 on the left side of the vehicle is detected, and it is detected that there is one lane between the edge of the road 55 and the lane the vehicle is traveling in, the vehicle can be identified as being in the second lane from the left, i.e., the center lane. Similarly, if the edge of the road 55 on the left side of the vehicle is detected, and it is detected that there are two lanes between the edge of the road 55 and the lane the vehicle is traveling in, the vehicle can be identified as being in the third lane from the left, i.e., the right lane. The vehicle's lane can also be identified in the same way using the edge of the road 56 on the right side of the vehicle. Furthermore, the number of lanes between the road edge 55 and the lane in which the vehicle is traveling is determined based on the detection results of lane markings and other vehicles. For example, even if lane markings cannot be clearly detected, it is possible to estimate the existence of lanes from the positions of other vehicles. Other methods for identifying one's own lane include using the type and number of lane markings without relying on map information, or using the distance from the edge of the road and the lane width.

[0043] Next, in S5, the CPU 41 compares the recommended lane identified in S3 with the vehicle's own lane identified in S4 and determines whether the vehicle's own lane and the recommended lane are different.

[0044] Then, if it is determined that the vehicle's current lane and the recommended lane are different (S5: YES), that is, if the vehicle is not traveling in the recommended lane on the road it is currently on in order to pass the guidance junction, it is determined that a lane change is recommended. After that, the system proceeds to S6 and displays a guidance object to encourage the vehicle to change lanes to the recommended lane, as described below.

[0045] On the other hand, if it is determined that the vehicle's current lane and the recommended lane coincide (S5:NO), that is, if the vehicle is currently traveling in the recommended lane on the road to pass the guidance junction, it is determined that there is no situation in which a lane change is recommended. Therefore, the driving support processing program terminates without displaying a guidance object to encourage a lane change to the recommended lane. However, even if a guidance object to encourage a lane change is not displayed, if the vehicle is approaching within a predetermined distance from the guidance junction and the vehicle should be guided to exit the guidance junction, it is desirable to display a separate guidance object to guide the vehicle to exit the guidance junction.

[0046] In S6, CPU 41 performs the guidance object display processing (Figure 8), which will be described later. The guidance object display processing is the process of displaying guidance objects on the front display 4 to prompt the driver to change lanes to the recommended lane, based on the guidance route currently set in the navigation device 3 and the vehicle's current position. Details will be described later.

[0047] Subsequently, in S7, the CPU 41 determines whether the termination conditions for the guidance prompting a lane change to the recommended lane, which was performed in S6, have been met. The termination conditions for the guidance prompting a lane change to the recommended lane include, for example, the completion of the lane change to the recommended lane, or, if the vehicle does not change lanes to the recommended lane, approaching a predetermined distance from the guidance junction. Here, the predetermined distance is the distance necessary to properly complete the lane change before entering the guidance junction, for example, 50m.

[0048] If it is determined that the conditions for ending the guidance prompting a lane change to the recommended lane have not been met (S7: NO), the system returns to S6 and continues to display the guidance object through the guidance object display process. On the other hand, if it is determined that the conditions for ending the guidance prompting a lane change to the recommended lane have been met (S7: YES), a control signal is sent to the front display 4, and the guidance object displayed on the front display 4 is hidden (S8). However, even after the guidance object prompting a lane change has been hidden, if the vehicle then approaches within a predetermined distance from the guidance junction and the vehicle should be guided to exit the guidance junction, it is desirable to separately display a guidance object for guiding the vehicle to exit the guidance junction.

[0049] Next, the subprocessing of the guidance object display process executed in S6 will be explained with reference to Figure 8. Figure 8 is a flowchart of the subprocessing program for the guidance object display process.

[0050] First, in S11, the CPU 41 performs image processing on the image captured by the front camera 11 to recognize (detect) the lane markings on the road the vehicle is traveling on, particularly the lane markings that the vehicle will cross when changing lanes to the recommended lane (hereinafter referred to as the "target lane markings"). Specifically, if the recommended lane identified in S3 is to the right of the vehicle's own lane identified in S4, the lane to the right of the vehicle's own lane becomes the target lane markings. If the recommended lane identified in S3 is to the left of the vehicle's own lane identified in S4, the lane to the left of the vehicle's own lane becomes the target lane markings. If two or more lane changes are required to move to the recommended lane, the first lane marking that needs to be crossed is designated as the target lane markings. It is also desirable to detect the color and type of lane markings (solid line, dashed line, etc.).

[0051] In the lane marking detection process described in S11 above, the CPU 41 performs brightness correction based on the brightness difference between the road surface and the lane markings in order to detect the lane markings in the image captured by the front camera 11. Subsequently, it performs binarization to separate the lane markings from the image, geometric processing to correct distortion, and smoothing processing to remove noise from the image, thereby detecting the boundary line between the road surface and the lane markings. The presence and type of the lane markings are identified by the detected boundary line. Furthermore, by extracting the image portion of the lane markings and the detected area and performing color recognition (RGB value detection), the color of the lane markings can also be detected.

[0052] Subsequently, in S12, the CPU 41 obtains the distance from the vehicle to the target lane marking along the road width direction. Specifically, using the image recognition result from S11, it calculates the distance L from the center of the target lane marking 61 to the center of the vehicle 62, as shown in Figure 9. Although Figure 9 is an example where the recommended lane is to the left of the vehicle's lane, assuming that the vehicle 62 changes lanes to the recommended lane, as shown in Figure 9, the distance L from the center of the target lane marking 61 to the center of the vehicle 62 will gradually decrease as the vehicle 62 travels (and conversely gradually increase after passing the target lane marking 61). In the following explanation, it will be assumed that the vehicle is in the state before passing the target lane marking 61.

[0053] Next, in S13, the CPU 41 determines whether the distance from the vehicle to the target lane marking along the road width direction, acquired in S12, is less than 1m. Here, in the superimposed image display device 1 of this embodiment, as mentioned above, the virtual image of the video displayed on the front display 4 cannot be superimposed anywhere on the scenery seen through the windshield 5, and the superimposed range R in which the virtual image can be superimposed and made visible to the vehicle occupants 6 is limited (see Figure 2). The condition for the target lane marking to be included in this limited superimposed range R is that the distance from the vehicle to the target lane marking along the road width direction is at least less than 1.0m. Therefore, if the distance from the vehicle to the target lane marking along the road width direction is 1m or more, it means that the target lane marking is not included in the superimposed range R. Note that including the target lane marking in the superimposed range R may mean that at least some of the target lane markings are included in the superimposed range R, but in this embodiment, it means that the target lane markings are included to the extent that the display of guidance objects based on the target lane markings can be performed within the superimposed range R.

[0054] Then, if it is determined that the distance from the vehicle to the target lane line along the road width direction is 1m or more, that is, the target lane line is not included in the superimposition range R where a virtual image can be superimposed and made visible to the vehicle occupant 6 (S13:NO), the process proceeds to S14.

[0055] In S14, the CPU 41 sends a control signal to the front display 4 and displays virtual lane markings on the front display 4. As a result, as shown in Figure 10, it becomes possible for the vehicle occupants to see virtual lane markings 66 (more precisely, virtual images of the virtual lane markings) superimposed on the scenery. The position where the virtual lane markings 66 are displayed in S14 is basically fixed on the road surface (parallel to the road surface) (the relative position visible from the vehicle is fixed). Specifically, as shown in Figure 10, if the target lane marking 61 is on the left side, it is displayed so that it is visible in the upper left position of the superimpositionable range R, and if the target lane marking 61 is on the right side, it is displayed so that it is visible in the upper right position of the superimpositionable range R. In addition, when the virtual lane markings 66 are displayed, they are displayed at a different angle from the target lane markings 61. More specifically, the virtual lane markings 66 are at an angle that is more inclined inward towards the lane relative to the direction of travel of the vehicle than the target lane markings 61.

[0056] Furthermore, while the virtual lane line 66 is modeled after the target lane line 61, the line type and color of the virtual lane line 66 may be the same as the target lane line 61, or they may be deliberately made different. For example, if the target lane line 61 is a dashed white line, the virtual lane line 66 may also be a dashed white line, or it may be a solid yellow line, different from the target lane line 61, to indicate that it is a virtual lane line.

[0057] Next, in S15, the CPU 41 calculates the size, shape, and display position (display range) of the guidance object to be displayed on the front display 4. Here, the guidance object to be displayed in S15 is a guidance object based on the virtual lane markings 66 displayed in S14, more specifically, a set of arrows (for example, 6) arranged along the route of changing lanes across the virtual lane markings 66. The direction of the arrows indicates the direction of travel of the vehicle when changing lanes, and they are arranged on the road surface (parallel to the road surface). The spacing between the multiple arrows can be changed as appropriate, but for example, they should be spaced 1m apart when viewed from the perspective of the occupants. In addition, the guidance object is a 2D polygon and basically has no thickness. However, it may also be a 3D polygon with thickness.

[0058] Subsequently, in S16, the CPU 41 transmits a control signal to the front display 4, and displays an image of the guidance object of the size and shape calculated in S15 at the calculated display position on the front display 4. As a result, it becomes possible for the vehicle occupants to see the guidance object (more precisely, a virtual image of the guidance object) superimposed on the scenery. Specifically, as shown in Figure 10, the multiple arrows, which are virtual images of the guidance object 67, are superimposed on the route for changing lanes across the virtual lane markings 66 and are visible. The shape of the guidance object to be displayed in S15 and S16 can be changed as appropriate, and any shape other than arrows is acceptable as long as it can encourage lane changes. Also, it does not have to be six arrows; it could be a single long arrow that crosses the virtual lane markings 66.

[0059] Furthermore, the guidance object display process shown in Figure 8 will be repeatedly executed until the guidance termination condition in S7 is met. The display of the virtual lane markings 66 and guidance objects 67 shown in Figure 10 by S14 to S16 will continue with basically fixed display content (the content visible to the occupants will remain fixed) until the distance from the vehicle to the target lane markings along the road width direction becomes less than 1m.

[0060] On the other hand, if it is determined that the distance from the vehicle to the target lane marking along the road width direction is less than 1 meter (S13: YES), the process proceeds to S17.

[0061] In S17, the CPU 41 determines whether the target lane marking 61 is included in the superimposable range R where a virtual image can be superimposed and made visible to the vehicle occupant 6, and whether the virtual lane marking 66 visible to the vehicle occupant overlaps with and is parallel to the target lane marking 61 (whether the virtual lane marking 66 and the target lane marking 61 overlap and appear as a single lane marking to the vehicle occupant). Here, as mentioned above, when the virtual lane marking 66 is displayed in S14, the virtual lane marking 66 is displayed at a position away from the target lane marking 61 and at a different angle (see Figure 10). However, as will be described later, after the distance from the vehicle to the target lane marking along the road width direction becomes less than 1m, the angle of the virtual lane marking 66 is changed at a predetermined angular velocity to an angle parallel to the target lane marking 61 (S18). The angular velocity that changes the angle of the virtual lane marking 66 is set to the timing when the distance from the vehicle along the road width direction to the target lane marking 61 is a predetermined distance (a distance shorter than the distance used as the criterion in S13, for example, 0.5 m), or to the timing when a predetermined time (for example, 3 seconds) has elapsed since the start of the angle change. For example, in this embodiment, the timing is set to when the distance from the vehicle along the road width direction to the target lane marking 61 is 0.5 m. The closer the vehicle is to the target lane marking, the closer the virtual lane marking 66 and the target lane marking 61 will be as visible to the vehicle's occupants. In this embodiment, when the distance from the vehicle along the road width direction to the target lane marking 61 is less than 1.0 m, the target lane marking 61 is included in the superimpossible range R. Furthermore, when the distance reaches 0.5 m, it is estimated that the angle of the virtual lane marking 66 becomes parallel to the target lane marking 61, and that the virtual lane marking 66 visible to the vehicle occupant overlaps with the target lane marking 61 (i.e., to the vehicle occupant, the virtual lane marking 66 and the target lane marking 61 overlap and appear as a single lane marking).

[0062] Here, in S17, instead of calculating and comparing the position of the virtual lane marking 66 actually visible to the vehicle occupants with the position of the target lane marking 61, it is determined whether or not the distance from the vehicle along the road width direction to the target lane marking 61 is 0.5m. That is, if the distance from the vehicle along the road width direction to the target lane marking 61 is longer than 0.5m, it is determined that the target lane marking 61 is not included in the superimposable range R, or even if it is included, the virtual lane marking 66 visible to the vehicle occupants does not overlap with and is not parallel to the target lane marking (S17:NO). When the distance from the vehicle along the road width direction to the target lane marking 61 becomes 0.5m, it is determined that the target lane marking 61 is included in the superimposable range R, and the virtual lane marking 66 visible to the vehicle occupants overlaps with and is parallel to the target lane marking (S17:YES).

[0063] Here, the position of the virtual lane marking 66 (the position of the virtual image) and the position of the target lane marking 61 (the position of the actual lane marking) as seen by the vehicle occupant change depending on various factors such as the position of the occupant's head and line of sight, and the position where the virtual image is formed. Therefore, it is extremely difficult for the device to accurately determine whether the target lane marking 61 is included in the superimposition range R in which the virtual image can be superimposed and seen by the vehicle occupant 6, and whether the virtual lane marking 66 and the target lane marking 61 as seen by the vehicle occupant overlap and are parallel. In this embodiment, this problem can be solved by making the above determination using the distance from the vehicle to the target lane marking 61 along the road width direction.

[0064] Then, in S18, which is executed when it is determined that the virtual lane marking 66 visible to the vehicle occupants does not overlap with and is not parallel to the target lane marking (S17: NO), the CPU 41 sends a control signal to the front display 4 and changes the angle of the virtual lane marking 66 displayed on the front display 4. Specifically, the angle of the virtual lane marking 66 is gradually changed at a predetermined angular velocity to an angle parallel to the target lane marking 61. The angular velocity at which the angle of the virtual lane marking 66 is changed in S18 is set to the timing when the distance from the vehicle along the road width direction to the target lane marking 61 is a predetermined distance (a distance shorter than the distance used as the determination criterion in S13, for example, 0.5 m) or to the timing when a predetermined time (for example, 3 seconds) has elapsed since the start of the angle change. For example, in this embodiment, the timing is set to when the distance from the vehicle along the road width direction to the target lane marking 61 is 0.5 m. Furthermore, the timing at which the distance from the vehicle along the road width direction to the target lane marking 61 becomes 0.5m can be calculated from the image recognition results of S11 and the vehicle's current speed and steering angle.

[0065] Furthermore, as shown in Figure 11, the angle of the virtual boundary line 66 can be changed by rotating it around the center point P. However, it is also possible to rotate it around a point other than the center point P.

[0066] Next, in S19, the CPU 41 calculates the size, shape, and display position (display range) of the guidance object to be displayed on the front display 4. Here, the guidance object to be displayed in S19 is a guidance object based on the virtual lane markings 66 after their angle has been shifted in S18, more specifically, a set of arrows (for example, 6) arranged along the route of changing lanes across the virtual lane markings 66 after the angle shift. The direction of the arrows indicates the direction of travel of the vehicle when changing lanes, and they are arranged on the road surface (parallel to the road surface). The spacing between the multiple arrows can be changed as appropriate, but for example, they can be spaced 1m apart when viewed from the perspective of an occupant. The guidance object is a 2D polygon and basically has no thickness. However, it may also be a 3D polygon with thickness.

[0067] Subsequently, in S16, the CPU 41 transmits a control signal to the front display 4, and displays an image of the guidance object of the size and shape calculated in S19 at the calculated display position on the front display 4. As a result, it becomes possible for the vehicle occupants to see the guidance object (more precisely, a virtual image of the guidance object) superimposed on the scenery. Specifically, as shown in Figure 12, the multiple arrows, which are virtual images of the guidance object 67, are superimposed on the route of lane changes that cross the virtual lane markings 66 and are visible. In addition, the virtual lane markings 66 gradually move closer to the target lane markings 61 as the vehicle approaches the target lane markings 61, and the angle changes so that they become parallel. In accordance with the change in the angle of the virtual lane markings 66, the display position of the guidance object 67 that crosses the virtual lane markings 66 also changes. Then, at the moment when the distance from the vehicle to the target lane marking 61 becomes 0.5m, that is, when the virtual lane marking 66 visible to the vehicle occupants overlaps with and is parallel to the target lane marking, the virtual lane marking 66 is hidden as described later (S20). Note that the shape of the guidance object to be displayed in S19 and S16 can be changed as appropriate, and any shape other than an arrow is acceptable as long as it can encourage lane changes. Also, it does not have to be six arrows, and it may be a single long arrow that straddles the virtual lane marking 66.

[0068] Furthermore, the guidance object display process shown in Figure 8 will be repeatedly executed until the guidance termination condition of S7 is met. The display of the virtual lane markings 66 and guidance objects 67 shown in Figure 12 by S18, S19, and S16 will be continuously performed for a period of time until the distance from the vehicle to the target lane markings along the road width direction changes from 1.0m to 0.5m.

[0069] On the other hand, in S20, which is executed when it is determined that the target lane markings 61 are included in the superimposable range R and that the virtual lane markings 66 visible to the vehicle occupants overlap with and are parallel to the target lane markings 61 (S17:YES), the CPU 41 sends a control signal to the front display 4 and hides the virtual lane markings 66 displayed on the front display 4. It should be noted that the fact that the virtual lane markings 66 visible to the vehicle occupants overlap with and are parallel to the target lane markings 61 also means that the target lane markings 61 are included in the superimposable range R to the extent that the display of guidance objects based on the target lane markings 61 can be performed within the superimposable range R.

[0070] Next, in S21, the CPU 41 calculates the size, shape, and display position (display range) of the guidance object to be displayed on the front display 4. Here, the guidance object to be displayed in S21 is a guidance object based on the target lane markings 61 in the actual scenery, more specifically, a plurality of (e.g., 6) arrows arranged along the route of lane changes that cross the target lane markings 61. The direction of the arrows indicates the direction of travel of the vehicle when changing lanes, and they are arranged on the road surface (parallel to the road surface). The spacing between the multiple arrows can be changed as appropriate, but for example, they should be spaced 1m apart when viewed from the perspective of the occupants. In addition, the guidance object is a 2D polygon and basically has no thickness. However, it may also be a 3D polygon with thickness.

[0071] Subsequently, in S16, the CPU 41 sends a control signal to the front display 4, and displays an image of the guidance object of the size and shape calculated in S21 at the calculated display position on the front display 4. As a result, it becomes possible for the vehicle occupants to see the guidance object (more precisely, a virtual image of the guidance object) superimposed on the scenery. Specifically, as shown in Figure 13, the multiple arrows, which are virtual images of the guidance object 67, are superimposed on the route of lane change that crosses the target lane markings 61, which are the actual scenery. Also, the position of the target lane markings 61 as seen by the vehicle occupants gradually changes as the vehicle approaches and then passes over the target lane markings 61, so the display position of the guidance object 67 also changes accordingly. After the lane change to the recommended lane is completed, the guidance object is hidden (S8). Note that the shape of the guidance object to be displayed in S21 and S16 can be changed as appropriate, and any shape other than arrows is acceptable as long as it can encourage lane changes. Furthermore, it is not necessary to have six arrows; it could be a single long arrow that crosses the target lane line 61.

[0072] Furthermore, by hiding the virtual lane markings 66, the guidance transitions from the virtual lane markings 66, which are virtual images, to the actual target lane markings 61, as shown in Figure 13. Since the above transition occurs while the virtual lane markings 66, as seen by the vehicle occupants, overlap with and are parallel to the target lane markings 61, the transition can be performed seamlessly. In other words, at the timing of the transition from guidance based on the virtual lane markings 66, which are virtual images, to guidance based on the actual target lane markings 61, the content of the image seen by the vehicle occupants does not change significantly, thus preventing the vehicle occupants from being distracted (by looking in directions other than the direction of travel).

[0073] Furthermore, the guidance object display process shown in Figure 8 will be repeatedly executed until the guidance termination condition in S7 is met. The display of the guidance object 67 shown in Figure 13 by S20, S21, and S16 will be continuously performed from the time the distance from the vehicle to the target lane line along the road width direction becomes 0.5m until the lane change to the recommended lane is completed.

[0074] Then, as the guidance object display process (S6) described above is executed, in this embodiment, when the guidance branching point is less than the guidance start point distance, the guidance object (more precisely, a virtual image of the guidance object, the same applies hereafter) superimposed on the scenery visible to the vehicle occupant changes as shown in Figure 14 as the lane change is performed.

[0075] First, when the distance from the vehicle to the guidance branching point falls below the guidance start distance (1 km on expressways, 700 m on general roads), if the vehicle is traveling in a different lane than the recommended lane, a virtual lane marking 66, which mimics a lane marking, is displayed as shown in Figure 10. Furthermore, guidance objects 67 are displayed based on the virtual lane marking 66 to encourage a lane change. As a result, even if the target lane marking 61 (the lane marking to be crossed when changing lanes) is not included in the superimposed range R, it becomes possible to provide clear guidance encouraging a lane change using lane markings. Subsequently, when the vehicle begins to change lanes and the distance from the vehicle to the target lane marking 61 becomes less than 1m, the angle of the virtual lane marking 66 displayed as shown in Figure 12 is gradually changed to an angle parallel to the target lane marking 61. At the moment when the virtual lane marking 66 visible to the vehicle occupants overlaps with and is parallel to the target lane marking 61 (to the vehicle occupants, the virtual lane marking 66 and the target lane marking 61 overlap and appear as a single lane marking), the virtual lane marking 66 is hidden as shown in Figure 13. The determination of whether the virtual lane marking 66 visible to the vehicle occupants overlaps with and is parallel to the target lane marking 61 is not made by calculating and comparing the actual positions of the virtual lane marking 66 and the target lane marking 61 visible to the vehicle occupants, but rather by determining whether the distance from the vehicle to the target lane marking 61 along the road width direction is 0.5m. Subsequently, the target lane marking 61 is included in the superimposable range R, and guidance is provided to encourage lane changes by displaying the guidance object 67 based on the actual target lane marking 61. Since the transition of the guidance is performed while the virtual lane marking 66, which is visible to the vehicle occupant, overlaps with and is parallel to the target lane marking 61, the transition can be performed seamlessly. In other words, at the timing of the transition from guidance based on the virtual lane marking 66, which is a virtual image, to guidance based on the actual target lane marking 61, the content of the image visible to the vehicle occupant does not change significantly, thus preventing the vehicle occupant's gaze from being drawn to something other than the direction of travel.

[0076] Furthermore, in the embodiments described above, the vehicle's lane and target lane markings are detected based on the images captured by the front camera 11 in S4 and S11. However, if the map information is reliable, a 3D space corresponding to the vehicle's current position (especially in front of the vehicle's direction of travel) may be generated based on the map information, and the vehicle's lane and target lane markings may be identified within this 3D space. In addition to roads, buildings, road signs, etc., may also be modeled in the 3D space, or only roads may be modeled. Alternatively, it may be a simple blank 3D space with only ground and no roads modeled. An embodiment using the above 3D space will be described below.

[0077] First, the CPU 41 identifies the vehicle's current position (the vehicle's lane) and orientation in the generated three-dimensional space. Specifically, the position of the front camera 11 installed on the vehicle is defined as the vehicle's current position, and the direction of the optical axis of the front camera 11 is defined as the vehicle's orientation. The position of the front camera 11 also corresponds to the position of the vehicle's occupants, and the direction of the optical axis of the front camera 11 corresponds to the direction of the occupants' line of sight. The CPU 41 also identifies the position of the target lane markings that the vehicle will cross when changing lanes in the three-dimensional space.

[0078] Then, in S15, S19, and S21, the CPU 41 generates the guidance objects to be displayed, for example, as 2D polygons, and places the generated guidance objects in the generated 3D space. In particular, virtual lane markings are also placed in S15 and S19. The position where the virtual lane markings and guidance objects are placed in the 3D space is, for example, 10m ahead in the direction of travel of the vehicle. Next, the CPU 41 calculates the size and shape of the virtual lane markings and guidance objects that can be seen when viewed in the direction of travel of the vehicle from the current position of the vehicle and the height of the front camera 11 in the 3D space where the virtual lane markings and guidance objects are placed, and uses these as the size and shape of the virtual lane markings and guidance objects to be displayed on the front display 4. Here, the calculated size and shape of the virtual lane markings and guidance objects are the size and shape of the virtual lane markings and guidance objects that can be seen when the virtual lane markings and guidance objects placed in the 3D space are viewed from the viewpoint of the current vehicle (more precisely, the front camera 11).

[0079] As described in detail above, according to the superimposed image display device 1 and the computer program executed by the superimposed image display device 1 according to this embodiment, when a vehicle lane change is recommended, the system detects the lane markings that the vehicle will cross when changing lanes (S11), displays a guidance object for guiding the vehicle to change lanes on the front display 4, and provides guidance to the vehicle occupants by superimposing the virtual image onto the scenery around the vehicle for visual confirmation, if it is determined that the lane markings that the vehicle will cross are not included in the superimposed range of the scenery around the vehicle where the guidance object can be superimposed and made visible to the vehicle occupants, In addition to the guidance object, virtual lane lines that mimic the lane lines are displayed, and the guidance object is displayed based on the virtual lane lines (S14, S15, S18, S19). On the other hand, if it is determined that the lane lines are included in the overlapping range after the virtual lane lines are displayed and that the lane lines and the virtual lane lines visible to the occupants are parallel, the virtual lane lines are hidden (S20), and thereafter the guidance object is displayed based on the lane lines (S21). Therefore, whether or not the lane lines that the vehicle will cross when changing lanes are included in the overlapping range, the guidance object can be displayed based on the lane lines. Furthermore, when transitioning from guidance based on virtual lane lines to guidance based on actual lane lines, the above transition is performed when the lane lines are included in the overlapping range and the virtual lane lines visible to the vehicle occupants are parallel to the actual lane lines, so the above transition can be performed seamlessly. In other words, at the point of transition from guidance based on virtual lane markings (which are virtual images) to guidance based on actual lane markings (which are real images), the content of the image seen by the vehicle occupants does not change significantly, thus preventing the vehicle occupants from being distracted (by looking in directions other than the direction of travel). Furthermore, the distance from the vehicle to the lane marking along the road width direction is obtained (S12), and if the distance from the vehicle to the lane marking is greater than or equal to a threshold, it is determined that the lane marking is not included in the overlapping range, and if the distance from the vehicle to the lane marking is less than the threshold, it is determined that the lane marking is included in the overlapping range (S17). Thus, it becomes possible to easily determine whether or not the lane marking is included in the overlapping range, which is difficult to determine on the device side, by using the distance from the vehicle to the lane marking. Furthermore, when virtual lane lines are displayed, they are shown at a different angle than the actual lane lines. After the virtual lane lines are displayed, their angle is changed at a predetermined angular velocity to an angle parallel to the actual lane lines. This makes it possible to gradually bring the virtual lane lines closer to the actual lane lines after they have been displayed. Furthermore, the virtual lane markings are adjusted so that they become parallel to the actual lane markings as seen by the occupants, either when the distance from the vehicle to the lane markings along the road width reaches a predetermined distance, or when a predetermined time has elapsed since the change in the angle of the virtual lane markings began. This makes it possible to easily determine whether the movable lane markings as seen by the vehicle occupants and the actual lane markings are parallel, which is difficult for the device to determine, by using the distance from the vehicle to the lane markings or the elapsed time. Furthermore, if there is a target location within a predetermined guidance start distance ahead in the direction of travel of the vehicle, the system obtains the recommended lane to which the vehicle is advised to travel towards the target location (S3), identifies the driving lane in which the vehicle is currently traveling on the road (S4), and if the driving lane is different from the recommended lane, the system indicates a situation where a lane change is recommended and displays a guidance object to guide the vehicle to change lanes (S6). This enables the system to provide appropriate guidance for lane changes using the guidance object when a lane change to reach the target location is necessary.

[0080] It should be noted that the present invention is not limited to the embodiments described above, and various improvements and modifications are possible without departing from the spirit of the invention. For example, in this embodiment, a head-up display system is used as a means of displaying an image superimposed on the scenery around the vehicle. However, the windshield 5 may also be used as a transparent liquid crystal display to display the image directly on the windshield 5. Furthermore, it is also possible to display the scenery around the vehicle captured by the front camera 11 (described later) on an in-vehicle liquid crystal display, and then display an image superimposed on the scenery displayed on the same liquid crystal display. In that case as well, the image displayed on the liquid crystal display will be an image superimposed on the scenery around the vehicle, similar to a HUD.

[0081] Furthermore, in this embodiment, the front display 4 is configured to generate a virtual image in front of the windshield 5 of the vehicle 2, but it is also possible to configure it to generate a virtual image in front of a window other than the windshield 5. In addition, the object on which the image is reflected by the front display 4 may be a visor (combiner) installed around the windshield 5, rather than the windshield 5 itself.

[0082] Furthermore, in this embodiment, the condition for transitioning from guidance based on the virtual lane marking 66, which is a virtual image, to guidance based on the target lane marking 61, which is an actual scene, is that the virtual lane marking 66, as seen by the vehicle occupant, overlaps with and is parallel to the target lane marking 61 (so that the virtual lane marking 66 and the target lane marking 61 overlap and appear as a single lane marking to the vehicle occupant). However, the condition could also be that the virtual lane marking 66, as seen by the vehicle occupant, is parallel to the target lane marking 61. Even if the virtual lane marking 66 and the target lane marking 61 do not completely overlap, a seamless transition is possible as long as they are at least parallel.

[0083] Furthermore, in this embodiment, guidance objects are used to provide guidance for changing lanes before a junction. However, guidance for changing lanes is not limited to situations before a junction; it can also be provided in situations where it is recommended to drive in a specific lane, such as before a lane reduction point or a merging section.

[0084] Furthermore, in this embodiment, the navigation ECU 15 of the navigation device 3 executes the driving support processing program (Figure 4), but the execution entity can be changed as appropriate. For example, the control unit of the front display 4, the vehicle control ECU, or other in-vehicle devices may be used to execute the processing. [Explanation of Symbols]

[0085] 1…Superimposed image display device, 2…Vehicle, 3…Navigation device, 4…Front display, 5…Windshield, 6…Occupant, 15…Navigation ECU, 41…CPU, 42…RAM, 43…ROM, 50…Guidance junction, 61…Target lane marking, 62…Own vehicle, 66…Virtual lane marking, 67…Guidance object

Claims

1. A superimposed image display device mounted in a vehicle, which reflects an image displayed in a display area using reflective members placed inside the vehicle, allowing the vehicle occupants to see it, thereby superimposing the image onto the scenery around the vehicle and allowing them to see a virtual image of the image, In situations where a vehicle's lane change is recommended, an object display means displays a guidance object for guiding the vehicle to change lanes as part of the video; It includes a lane marking detection means for detecting the lane markings that the vehicle will cross when changing lanes, The object display means is If it is determined that the lane markings are not included in the superimposed area of ​​the surrounding scenery of the vehicle, where the guidance object can be superimposed and made visible to the vehicle occupants, then the video will display virtual lane markings that mimic the lane markings in addition to the guidance object, and the guidance object will be displayed based on the virtual lane markings. An image display device that, after the virtual lane markings are displayed, and it is determined that the lane markings are included in the superimposable range and that the lane markings and the virtual lane markings visible to the occupants are parallel, then hides the virtual lane markings and subsequently displays the guide objects based on the lane markings.

2. It has a distance acquisition means for acquiring the distance from the vehicle along the road width direction to the lane marking, The object display means is If the distance from the vehicle to the lane marking is greater than or equal to a threshold, it is determined that the lane marking is not included in the superimpossible range. The superimposed image display device according to claim 1, which determines that the lane mark is included in the superimposed range when the distance from the vehicle to the lane mark is less than a threshold.

3. The object display means is At the time the virtual lane lines are displayed, the virtual lane lines are displayed at a different angle from the actual lane lines. The superimposed image display device according to claim 1 or claim 2, wherein, after displaying the virtual dividing lines, the angle of the virtual dividing lines is changed at a predetermined angular velocity to an angle parallel to the dividing lines.

4. The object display means is The superimposed image display device according to claim 3, which changes the angle of the virtual lane line so that the lane line and the virtual lane line visible to the occupant become parallel when the distance from the vehicle along the road width direction to the lane line becomes a predetermined distance, or when a predetermined time has elapsed since the angle change of the virtual lane line began.

5. A recommended lane acquisition means that acquires a recommended lane to which the vehicle is recommended to travel when there is a target location within a predetermined guidance start distance in the direction of travel of the vehicle, It has a means for identifying the lane in which the vehicle is currently traveling on the road, The superimposed image display device according to claim 1, wherein the object display means displays a guidance object for guiding a vehicle to change lanes, with the case where the driving lane is different from the recommended lane being a situation in which a lane change is recommended.

Images