Display control device and display control program

Abstract

To provide a technique capable of suppressing the occurrence of the display position shift when an error occurs in estimation or calculation of the own vehicle position in a case where a virtual image display element for guiding the travel direction or the like is displayed so as to be superimposed on an intersection.SOLUTION: A display control device acquires a recognition result of a target (LL, TL) based on an image of a photographed foreground, acquires the own vehicle position, acquires a progress route of the own vehicle (1), and displays the travel direction display (TD) indicating the travel schedule direction of the own vehicle in a passage schedule intersection (IS) in a traveling road (RR) so as to be superimposed on the passage schedule intersection on the basis of the recognition result of the target, the own vehicle position and the progress route. At this time, the device estimates the position in the road extension direction orthogonal to the road width direction of at least the traveling road of the intersection center position (PC) being the center position of the passage schedule intersection on the basis of the recognition result of the target, and displays the travel direction display so as to be superimposed on the passage schedule intersection on the basis of the estimated intersection center position.SELECTED DRAWING: Figure 3

Description

Technical Field

[0001] The present invention relates to a display control device and a display control program for controlling a display operation in a display device that superimposes a virtual image on the foreground of a host vehicle.

Background Art

[0002] Patent Document 1 provides a superimposed image display device, a superimposed image drawing method, and a computer program that can clearly cause a passenger to recognize the content of a guidance object displayed superimposed on the scenery around the vehicle and enable an appropriate driving operation to be performed. Specifically, this technology displays a guidance object for guiding a guidance target point when there is a guidance target point to be guided ahead in the traveling direction of the vehicle. When displaying the guidance object, if the distance from the vehicle to the guidance target point is equal to or greater than a threshold value, the guidance object is displayed in a manner that is superimposed and visually recognized by the passenger in a state where the relative position with the vehicle is fixed. Thereafter, if the distance from the vehicle to the guidance target point becomes less than the threshold value, the guidance object is displayed in a manner that is superimposed and visually recognized by the passenger in a state where the relative position with the guidance target point is fixed.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] By the way, conventionally, there has been known a technique of using a head-up display to display guidance on a route or a traveling direction on a front windshield with a display element such as an arrow (see, for example, Japanese Patent Application Laid-Open No. 2020-196295, etc.). Further, in recent years, in this type of display technology, there has been a demand to recognize an object such as a white line using an image processing technology used in ADAS, utilize information on the posture of the vehicle and the position to an object, and present information without displacement in a display that helps recognition and judgment in the scene the driver is looking at. ADAS is an abbreviation for Advanced Driver Assistance System.

[0005] Here, with the position accuracy of conventional in-vehicle navigation, the front and rear positions of an intersection that is a target for guiding a route or a traveling direction (that is, the position in the current traveling direction of the host vehicle) may shift. The cause of this is, for example, a decrease in the accuracy of host vehicle position estimation when a positioning signal such as GPS cannot be received or is difficult to receive in a high-rise building, a tunnel, a multi-story parking lot, etc. Alternatively, the cause of this is, for example, an error in host vehicle position estimation due to a change in the actual road shape from the map when map matching. Thus, when an error occurs in the estimation or calculation of the host vehicle position, an error also occurs in the relative position relationship from the host vehicle position of an intersection that is a target for guiding a route or a traveling direction. Then, a display element for guiding a route or a traveling direction that should originally be displayed corresponding to the intersection is displayed at a position shifted from the intersection.

[0006] The present invention has been made in view of the circumstances exemplified above. That is, the present invention provides a technique capable of suppressing the occurrence of display position deviation even when an error occurs in the estimation or calculation of the host vehicle position, for example, when a virtual image display element for guiding a route, a traveling direction, etc. is superimposed and displayed at an intersection.

Means for Solving the Problems

[0007] The display control device (12) according to claim 1 is configured to control the display operation of a display device (11) mounted on the host vehicle (1) to superimpose a virtual image (IM) on the foreground of the host vehicle. This display control device includes a recognition result acquisition unit (121) that acquires a recognition result of a target based on an image of the foreground captured by an image capturing device (13), a host vehicle position acquisition unit (122) that acquires the host vehicle position, which is the position of the host vehicle, a route information acquisition unit (123) that acquires the traveling route of the host vehicle, and a display information generation unit (124) that generates display information for superimposing a traveling direction display (TD) indicating the traveling direction of the host vehicle at a planned intersection (IS), which is an intersection where passage is planned and exists in the traveling direction of the host vehicle on a traveling road (RR), which is the road on which the host vehicle is traveling, on the planned intersection based on the recognition result, the host vehicle position, and the traveling route. The display control device further includes: The display information generation unit includes a center position estimation unit (125) that estimates the position of the intersection center position (PC), which is the center position of the planned intersection, at least in the road extension direction orthogonal to the road width direction of the traveling road based on the recognition result. Based on the estimated intersection center position, the traveling direction display is superimposed on the planned intersection. The center position estimation unit estimates an intermediate position between the position of a stop line (LL) provided on the host vehicle side at the planned intersection in the road extension direction and the position of a traffic signal (TL) provided on the side opposite to the host vehicle side at the planned intersection in the road extension direction as the position of the intersection center position in the road extension direction. The display control device (12) according to claim 4 is configured to control the display operation of a display device (11) mounted on the host vehicle (1) to superimpose a virtual image (IM) on the foreground of the host vehicle. This display control device A recognition result acquisition unit (121) that acquires a recognition result of a target based on an image of the foreground imaged by an image capturing device (13); A host vehicle position acquisition unit (122) that acquires the host vehicle position, which is the position of the host vehicle; A route information acquisition unit (123) that acquires the traveling route of the host vehicle; Based on the recognition result, the host vehicle position, and the traveling route, display information for generating a traveling direction display (TD) indicating the planned traveling direction of the host vehicle at a planned intersection (IS), which is an intersection where passage is planned and exists in the traveling direction of the host vehicle on a traveling road (RR), which is the road on which the host vehicle is traveling, and superimposing the traveling direction display on the planned intersection is generated by a display information generation unit (124); Comprising; The display information generation unit: A center position estimation unit (125) that estimates, based on the recognition result, the position of the intersection center position (PC), which is the center position of the planned intersection, at least in the road extension direction orthogonal to the road width direction of the traveling road; Based on the estimated intersection center position, the traveling direction display is superimposed on the planned intersection; When the recognition result includes a traffic signal (TL) provided at the planned intersection, a first crosswalk (ZC1) provided at the planned intersection along the road width direction, and a second crosswalk (ZC2) provided at the planned intersection along the road width direction at a different position from the first crosswalk in the road extension direction, the center position estimation unit estimates the intermediate position in the road extension direction between the first crosswalk and the second crosswalk as the position of the intersection center position in the road extension direction. The display control device (12) according to claim 6 is configured to control a display operation of a display device (11) mounted on a host vehicle (1) to superimpose a virtual image (IM) on the foreground of the host vehicle. This display control device: A recognition result acquisition unit (121) that acquires a recognition result of a target based on an image of the foreground imaged by an image capturing device (13); A self-vehicle position acquisition unit (122) that acquires the self-vehicle position, which is the position of the own vehicle; A route information acquisition unit (123) that acquires the traveling route of the own vehicle; Based on the recognition result, the self-vehicle position, and the traveling route, display information for generating a traveling direction display (TD) indicating the planned traveling direction of the own vehicle at a planned intersection (IS), which is an intersection with a planned passage existing at the traveling destination of the own vehicle on a traveling road (RR), which is the road on which the own vehicle is traveling, and superimposing the traveling direction display on the planned intersection is generated by a display information generation unit (124); Comprising; The display information generation unit: Based on the recognition result, a center position estimation unit (125) that estimates the position of the intersection center position (PC), which is the center position of the planned intersection, at least in the road extension direction orthogonal to the road width direction of the traveling road is provided; Based on the estimated intersection center position, the traveling direction display is superimposed on the planned intersection; The center position estimation unit: Based on the recognized position of the road stud (RM), Estimates the center position in the road extension direction of a crosswalk (ZC) provided along the road extension direction as the position in the road extension direction of the intersection center position. The display control device (12) according to claim 8 is configured to control a display operation in a display device (11) that superimposes a virtual image (IM) on the foreground of the own vehicle when mounted on the own vehicle (1). This display control device: A recognition result acquisition unit (121) that acquires a recognition result of a target based on an image of the foreground captured by an image capturing device (13); A self-vehicle position acquisition unit (122) that acquires the self-vehicle position, which is the position of the own vehicle; A route information acquisition unit (123) that acquires the traveling route of the own vehicle; Based on the recognition result, the position of the host vehicle, and the travel route, a display information generation unit (124) generates display information for superimposing a travel direction display (TD) indicating the planned travel direction of the host vehicle at an intersection to be passed (IS), which is an intersection with a planned passage existing at the travel destination of the host vehicle on a running road (RR), which is the road on which the host vehicle is running, on the intersection to be passed. comprising the display information generation unit comprises a center position estimation unit (125) that estimates the position of the intersection center position (PC), which is the center position of the intersection to be passed, at least in the road extension direction orthogonal to the road width direction of the running road, based on the recognition result. Based on the estimated intersection center position, the travel direction display is superimposed and displayed on the intersection to be passed. The center position estimation unit estimates the intersection center position based on the position of the recognized road stud (RM). The display control device (12) according to claim 10 is configured to control the display operation of a display device (11) mounted on the host vehicle (1) to superimpose a virtual image (IM) on the foreground of the host vehicle. This display control device, A recognition result acquisition unit (121) that acquires a recognition result of a target based on an image of the foreground imaged by an image imaging device (13), A host vehicle position acquisition unit (122) that acquires the host vehicle position, which is the position of the host vehicle, A route information acquisition unit (123) that acquires the traveling route of the host vehicle, Based on the recognition result, the host vehicle position, and the traveling route, a traveling direction display (TD) indicating the planned traveling direction of the host vehicle at a planned intersection (IS), which is an intersection where passage is planned and exists in the traveling direction of the host vehicle on a traveling road (RR) that is the road on which the host vehicle is traveling, is superimposed on the planned intersection. A display information generation unit (124) that generates display information for this purpose, Comprising, The display information generation unit, Based on the recognition result, a center position estimation unit (125) that estimates the position of the intersection center position (PC), which is the center position of the planned intersection, at least in the road extension direction orthogonal to the road width direction of the traveling road, Based on the estimated intersection center position, the traveling direction display is superimposed on the planned intersection, The center position estimation unit, Estimates the center position in the road extension direction of a crosswalk (ZC) provided along the road extension direction as the position of the intersection center position in the road extension direction, Corrects the estimation result of the intersection center position based on the driving state of the host vehicle. The display control device (12) according to claim 12 is configured to control the display operation of a display device (11) mounted on the host vehicle (1) to superimpose a virtual image (IM) on the foreground of the host vehicle. This display control device, A recognition result acquisition unit (121) that acquires a recognition result of a target based on an image of the foreground imaged by an image imaging device (13), A host vehicle position acquisition unit (122) that acquires the host vehicle position, which is the position of the host vehicle, A route information acquisition unit (123) that acquires the traveling route of the host vehicle, Based on the recognition result, the position of the host vehicle, and the travel route, display information generation unit (124) generates display information for superimposing a travel direction display (TD) indicating the planned travel direction of the host vehicle at an intersection to be passed (IS), which is an intersection with a planned passage existing at the destination of the host vehicle on the running road (RR), which is the road on which the host vehicle is running, on the intersection to be passed. comprising The display information generation unit Based on the recognition result, it includes a center position estimation unit (125) that estimates the position of the intersection center position (PC), which is the center position of the intersection to be passed, at least in the road extension direction orthogonal to the road width direction of the running road. Based on the estimated intersection center position, the travel direction display is superimposed on the intersection to be passed. The center position estimation unit estimates the center position in the road extension direction of a crosswalk (ZC) provided along the road extension direction as the position of the intersection center position in the road extension direction. Based on the estimation result of the intersection center position in the past at an intersection with a traffic signal, the estimation result of the intersection center position at an intersection without a traffic signal is corrected. Claim 13 to The display control device (12) described is configured to control the display operation of a display device (11) mounted on the host vehicle (1) to superimpose a virtual image (IM) on the foreground of the host vehicle. This display control device includes a recognition result acquisition unit (121) that acquires a recognition result of a target based on an image of the foreground captured by an image capturing device (13), a host vehicle position acquisition unit (122) that acquires the host vehicle position, which is the position of the host vehicle, a route information acquisition unit (123) that acquires the traveling route of the host vehicle, and a display information generation unit (124) that generates display information for superimposing a traveling direction display (TD) indicating the traveling direction of the host vehicle at a planned intersection (IS), which is an intersection where passage is planned and exists ahead of the host vehicle in the traveling road (RR) on which the host vehicle is traveling, on the planned intersection based on the recognition result, the host vehicle position, and the traveling route. The display information generation unit includes a center position estimation unit (125) that estimates the position of the intersection center position (PC), which is the center position of the planned intersection, at least in the road extension direction orthogonal to the road width direction of the traveling road based on the recognition result, superimposes the traveling direction display on the planned intersection based on the estimated intersection center position, and corrects the estimation result of the intersection center position based on the driving state of the host vehicle. The center position estimation unit is before and Based on the estimation result of the intersection center position in the past at an intersection with a traffic signal, the estimation result of the intersection center position at an intersection without a traffic signal is corrected. Based on the estimation result of the intersection center position in the past at an intersection with a traffic signal, the estimation result of the intersection center position at an intersection without a traffic signal is corrected. . Claim 15 to The described display control device (12) is configured to control the display operation of a display device (11) mounted on the host vehicle (1) to superimpose a virtual image (IM) on the foreground of the host vehicle. This display control device includes a recognition result acquisition unit (121) that acquires a recognition result of a target based on an image of the foreground captured by an image capturing device (13), a host vehicle position acquisition unit (122) that acquires the position of the host vehicle, which is the position of the host vehicle, a route information acquisition unit (123) that acquires the traveling route of the host vehicle, and a display information generation unit (124) that generates display information for superimposing a traveling direction display (TD) indicating the planned traveling direction of the host vehicle at a planned intersection (IS), which is an intersection where passage is planned and exists in the traveling direction of the host vehicle on a traveling road (RR), which is the road on which the host vehicle is traveling, on the planned intersection based on the recognition result, the position of the host vehicle, and the traveling route. It is provided with The display information generation unit includes a center position estimation unit (125) that estimates the position of the intersection center position (PC), which is the center position of the planned intersection, at least in the road extension direction orthogonal to the road width direction of the traveling road based on the recognition result. Based on the estimated intersection center position, the traveling direction display is superimposed on the planned intersection. The center position estimation unit corrects the estimation result of the intersection center position at an intersection without a traffic signal based on the past estimation result of the intersection center position at an intersection with a traffic signal. 。 request Claim 18 to The described display control program is a program executed by a display control device (12) configured to control the display operation of a display device (11) mounted on the host vehicle (1) to superimpose a virtual image (IM) on the foreground of the host vehicle. This display control program, as a process executed by the display control device, An object recognition result acquisition process for acquiring an object recognition result based on the foreground image captured by the image capturing device (13), A host vehicle position acquisition process for acquiring the host vehicle position which is the position of the host vehicle, A route information acquisition process for acquiring the traveling route of the host vehicle, Based on the recognition result, the host vehicle position, and the traveling route, display information for generating a traveling direction display (TD) indicating the traveling direction of the host vehicle at the intersection to be passed (IS), which is an intersection scheduled to be passed and exists in the traveling direction of the host vehicle on the road being traveled (RR), the road on which the host vehicle is traveling, and superimposing the traveling direction display on the intersection to be passed, including The display information generation process includes a center position estimation process for estimating the position of the intersection center position (PC), which is the center position of the intersection to be passed, at least in the road extension direction orthogonal to the road width direction of the road being traveled, based on the recognition result, Based on the estimated intersection center position, the traveling direction display is superimposed on the intersection to be passed. Claim 21 to The display control program described is a program executed by a display control device (12) configured to control the display operation of a display device (11) that superimposes a virtual image (IM) on the foreground of the host vehicle when mounted on the host vehicle (1). This display control program, as a process executed by the display control device, An object recognition result acquisition process for acquiring an object recognition result based on the foreground image captured by the image capturing device (13), A host vehicle position acquisition process for acquiring the host vehicle position which is the position of the host vehicle, A route information acquisition process for acquiring the traveling route of the host vehicle, Based on the recognition result, the position of the host vehicle, and the travel route, generation of display information for superimposing a travel direction display (TD) indicating the planned travel direction of the host vehicle at an intersection to be passed (IS), which is an intersection with a planned passage existing at the destination of the host vehicle on a road under travel (RR), the road on which the host vehicle is traveling, on the intersection to be passed; a display information generation process including the display information generation process includes a center position estimation process of estimating a position of a center position of the intersection (PC), which is the center position of the intersection to be passed, in at least a road extension direction orthogonal to the road width direction of the road under travel, based on the recognition result Based on the estimated center position of the intersection, the travel direction display is superimposed and displayed on the intersection to be passed When the recognition result includes a traffic signal (TL) provided at the intersection to be passed, a first crosswalk (ZC1) provided at the intersection to be passed along the road width direction, and a second crosswalk (ZC2) provided at the intersection to be passed along the road width direction at a different position from the first crosswalk in the road extension direction, the center position estimation process estimates an intermediate position between the first crosswalk and the second crosswalk in the road extension direction as the position of the center position of the intersection in the road extension direction Claim 23 to The display control program described is a program executed by a display control device (12) configured to control a display operation of a display device (11) that superimposes a virtual image (IM) on the foreground of the host vehicle when mounted on the host vehicle (1) As a process executed by the display control device, this display control program a recognition result acquisition process of acquiring a recognition result of an object based on an image of the foreground captured by an image capturing device (13); a host vehicle position acquisition process of acquiring a host vehicle position, which is the position of the host vehicle; a route information acquisition process of acquiring the travel route of the host vehicle Based on the recognition result, the position of the host vehicle, and the travel route, generation of display information for superimposing a travel direction display (TD) indicating the planned travel direction of the host vehicle at an intersection to be passed (IS), which is an intersection with a planned passage existing at the travel destination of the host vehicle on the road being traveled (RR), on the intersection to be passed. including the display information generation process includes a center position estimation process of estimating the position of the intersection center position (PC), which is the center position of the intersection to be passed, in at least the road extension direction orthogonal to the road width direction of the road being traveled, based on the recognition result. Based on the estimated intersection center position, the travel direction display is superimposed on the intersection to be passed. the center position estimation process Based on the position of the recognized road stud (RM). estimates the center position of a crosswalk (ZC) provided along the road extension direction in the road extension direction as the position of the intersection center position in the road extension direction. Claim 25 to The display control program described is a program executed by a display control device (12) configured to control the display operation of a display device (11) mounted on the host vehicle (1) to superimpose a virtual image (IM) on the foreground of the host vehicle. As a process executed by the display control device, this display control program includes a recognition result acquisition process of acquiring a recognition result of an object based on an image of the foreground captured by an image capturing device (13), a host vehicle position acquisition process of acquiring the position of the host vehicle, which is the position of the host vehicle, a route information acquisition process of acquiring the travel route of the host vehicle, Based on the recognition result, the position of the host vehicle, and the travel route, generation of display information for superimposing a travel direction display (TD) indicating the planned travel direction of the host vehicle at an intersection to be passed (IS), which is an intersection with a planned passage existing at the travel destination of the host vehicle on the road being traveled (RR), on the intersection to be passed. including the display information generation process includes a center position estimation process of estimating a position of an intersection center position (PC), which is a center position of the intersection to be passed, in at least a road extension direction orthogonal to a road width direction of the road being traveled, based on the recognition result. Based on the estimated intersection center position, the forward direction display is superimposed and displayed on the intersection to be passed. The center position estimation process estimates the intersection center position based on a position of a recognized road stud (RM). The display control program according to claim 27 is a program executed by a display control device (12) configured to control a display operation in a display device (11) that superimposes a virtual image (IM) on the foreground of the host vehicle when mounted on the host vehicle (1). As a process executed by the display control device, this display control program a recognition result acquisition process of acquiring a recognition result of a target based on an image of the foreground captured by an image capturing device (13); a host vehicle position acquisition process of acquiring the position of the host vehicle, which is the position of the host vehicle; a route information acquisition process of acquiring the travel route of the host vehicle. Based on the recognition result, the position of the host vehicle, and the travel route, display information generation processing is performed to generate display information for superimposing a travel direction display (TD) indicating the planned travel direction of the host vehicle at an intersection (IS) where a traffic intersection is planned to exist in the running road (RR), which is the road on which the host vehicle is traveling, on the traffic intersection; including; the display information generation processing includes center position estimation processing for estimating, based on the recognition result, the position of the intersection center position (PC), which is the center position of the traffic intersection, in at least the road extension direction orthogonal to the road width direction of the running road; Based on the estimated intersection center position, the travel direction display is superimposed on the traffic intersection; the center position estimation processing estimates the center position of the crosswalk (ZC) provided along the road extension direction in the road extension direction as the position of the intersection center position in the road extension direction; Based on the driving state of the host vehicle, the estimation result of the intersection center position is corrected. The display control program according to claim 29 is a program executed by a display control device (12) configured to control a display operation of a display device (11) mounted on a host vehicle (1) to superimpose a virtual image (IM) on the foreground of the host vehicle. This display control program, as a process executed by the display control device, includes recognition result acquisition processing for acquiring a recognition result of an object based on an image of the foreground captured by an image capturing device (13); host vehicle position acquisition processing for acquiring the position of the host vehicle, which is the position of the host vehicle; route information acquisition processing for acquiring the travel route of the host vehicle; Based on the recognition result, the position of the host vehicle, and the travel route, display information generation processing is performed to generate display information for superimposing a travel direction display (TD) indicating the planned travel direction of the host vehicle at an intersection (IS) where a traffic intersection is planned to exist in the running road (RR), which is the road on which the host vehicle is traveling, on the traffic intersection; including; the display information generation processing includes center position estimation processing for estimating, based on the recognition result, the position of the intersection center position (PC), which is the center position of the traffic intersection, in at least the road extension direction orthogonal to the road width direction of the running road; Based on the estimated intersection center position, the travel direction display is superimposed on the intersection where passage is planned. The center position estimation process estimates the center position in the road extension direction of a crosswalk (ZC) provided along the road extension direction as the position in the road extension direction of the intersection center position. Based on the estimation result of the past intersection center position at an intersection with a traffic signal, the estimation result of the intersection center position at an intersection without a traffic signal is corrected. Claim to 30 The described display control program is a program executed by a display control device (12) configured to control a display operation in a display device (11) that superimposes a virtual image (IM) on the foreground of the host vehicle (1) when mounted on the host vehicle. As processing executed by the display control device, this display control program acquires a recognition result of an object based on an image of the foreground captured by an image capturing device (13), a recognition result acquisition process, and acquires the host vehicle position, which is the position of the host vehicle, a host vehicle position acquisition process, and acquires the travel route of the host vehicle, a route information acquisition process, and based on the recognition result, the host vehicle position, and the travel route, generates display information for superimposing a travel direction display (TD) indicating the planned travel direction of the host vehicle at a planned intersection (IS), which is an intersection where passage is planned and exists in the direction of travel of the host vehicle on a road (RR) on which the host vehicle is traveling, on the planned intersection, a display information generation process, and includes The display information generation process includes a center position estimation process of estimating, based on the recognition result, the position of the intersection center position (PC), which is the center position of the planned intersection, in at least the road extension direction orthogonal to the road width direction of the road on which the vehicle is traveling, and based on the estimated intersection center position, superimposes the travel direction display on the planned intersection, and the center position estimation process is before corrects the estimation result of the intersection center position based on the driving state of the host vehicle while Based on the estimation result of the past intersection center position at an intersection with a traffic signal, the estimation result of the intersection center position at an intersection without a traffic signal is corrected . Claim to 32 The described display control program is a program executed by a display control device (12) configured to control a display operation in a display device (11) that superimposes a virtual image (IM) on the foreground of the host vehicle (1) when mounted on the host vehicle. As processing executed by the display control device, this display control program acquires a recognition result of a target based on an image of the foreground imaged by an image capturing device (13), which is a recognition result acquisition process, acquires a host vehicle position, which is the position of the host vehicle, which is a host vehicle position acquisition process, acquires a travel route of the host vehicle, which is a route information acquisition process, Based on the recognition result, the host vehicle position, and the travel route, display information for generating a travel direction display (TD) indicating a planned travel direction of the host vehicle at a planned intersection (IS), which is an intersection where passage is planned and exists in the travel destination of the host vehicle on a road (RR) on which the host vehicle is traveling, and superimposing the travel direction display on the planned intersection is generated, which is a display information generation process, and includes The display information generation process includes a center position estimation process of estimating a position of a center position (PC) of the planned intersection in at least a road extension direction orthogonal to a road width direction of the road on which the vehicle is traveling, based on the recognition result, Based on the estimated center position of the intersection, the travel direction display is superimposed on the planned intersection, The center position estimation process corrects an estimation result of the intersection center position at an intersection without a traffic signal, based on an estimation result of the intersection center position in the past at an intersection with a traffic signal.

[0008] Note that in each column of the application documents, each element may be assigned a reference sign with parentheses. In this case, the reference sign merely shows an example of the correspondence relationship between the same element and the specific configuration described in the following embodiments. Therefore, the present invention is not limited by the description of the reference sign at all.

Brief Description of the Drawings

[0009]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Mode for Carrying Out the Invention

[0010] (Embodiment) Hereinafter, embodiments of the present invention will be described with reference to the drawings. Note that, if the description of various modified examples applicable to one embodiment is inserted in the middle of the series of descriptions regarding the embodiment, there is a possibility that the understanding of the embodiment may be hindered. For this reason, the modified examples will be described collectively after the series of descriptions regarding the embodiment, rather than in the middle of the series of descriptions regarding the embodiment.

[0011] (Overall Configuration) Referring to FIG. 1, the vehicle 1 is a so-called four-wheel automobile and includes a vehicle body 2 that is substantially rectangular in a plan view when viewed from vertically above the vehicle 1. For the convenience of the following description, the concepts of "front", "rear", "left", "right", "up" and "down" in the vehicle 1 are defined as shown by the arrows in FIG. 1. These direction concepts are based on the driver D, who is a passenger boarding the driver's seat in the vehicle 1 during forward travel (i.e., during travel in a shift range other than reverse). The front-rear direction is synonymous with the vehicle's overall length direction. Also, the left-right direction is synonymous with the vehicle width direction. Further, the up-down direction is synonymous with the vehicle height direction. The vehicle's overall length direction is a direction orthogonal to both the vehicle width direction and the vehicle height direction. The vehicle height direction is a direction that defines the vehicle height of the vehicle 1 and is parallel to the direction of the gravitational force when the vehicle 1 is stably placed on a horizontal plane in a drivable state.

[0012] In the vehicle body 2, a passenger compartment 3, which is an internal space where passengers board, is covered by a front windshield 4 at the front. The front windshield 4 is formed in a plate shape by translucent glass or synthetic resin. The front windshield 4 is inclined and arranged such that, in a side view seen with a line of sight parallel to the vehicle width direction, the upper end portion is located further rearward than the lower end portion.

[0013] Below the front windshield 4 in the passenger compartment 3, a dashboard 5 is arranged. A meter panel 6 is provided on the dashboard 5. Also, a steering column 7 extends from the dashboard 5 toward the driver D. A steering wheel 8 is attached to the steering column 7.

[0014] The vehicle 1 is equipped with an in-vehicle system 10. The vehicle 1 equipped with the in-vehicle system 10 is hereinafter referred to as the "own vehicle 1". The in-vehicle system 10 is configured to be able to execute various controls during the operation of the own vehicle 1 and various display operations associated therewith. The in-vehicle system 10 is provided to form an in-vehicle network compliant with a predetermined communication standard such as CAN (International Registration Mark: International Registration Number 1048262A). CAN (International Registration Mark) is an abbreviation for Controller Area Network.

[0015] The in-vehicle system 10 is configured such that the display operation in the head-up display device 11 that forms the virtual image IM using AR technology is controlled by the display control device 12. AR is an abbreviation for Augmented Reality. The head-up display device 11 is a display device that, when mounted on the own vehicle 1, superimposes and displays the virtual image IM on the foreground (e.g., road surface, etc.) of the own vehicle 1 visible through the front windshield 4, and is housed in the dashboard 5 below the front windshield 4.

[0016] Figure 2 schematically shows the functional block configuration of the in-vehicle system 10. Hereinafter, with reference to FIGS. 1 and 2, the schematic configuration of the in-vehicle system 10 will be described. In addition to the head-up display device 11 and the display control device 12, the in-vehicle system 10 includes an image capturing device 13, a recognition unit 14, a vehicle state sensor 15, and a navigation device 16.

[0017] The image capturing device 13 is a so-called front camera, which is provided to be able to capture images of the front and the front sides of the host vehicle 1. Specifically, the image capturing device 13 is a digital camera device and includes an image sensor such as a CCD or a CMOS. CCD is an abbreviation for Charge Coupled Device. CMOS is an abbreviation for Complementary Metal Oxide Semiconductor. The recognition unit 14 is configured to perform image recognition of a target included in the foreground based on the image of the foreground captured by the image capturing device 13. The "target" includes not only road markings such as the road division line LD, the stop line LL, and the crosswalk ZC shown in FIG. 3 and the like, but also three-dimensional objects such as the traffic signal TL. In the present embodiment, the image capturing device 13 and the recognition unit 14 are configured to be able to detect the three-dimensional position of the target relative to the host vehicle 1 by using monocular stereo or multi-eye stereo technology. The recognition unit 14 is provided so as to be able to provide the image recognition result of the target to each unit including the display control device 12.

[0018] The vehicle state sensor 15 is provided to detect the driving state and driving environment of the host vehicle 1. The "driving state" includes various quantities related to the driving operation state by the driver D or the driving automation system, such as, for example, the accelerator opening, the braking amount, the shift position, the steering angle, etc. Further, the "driving state" includes physical quantities related to the behavior of the host vehicle 1, such as, for example, the vehicle speed (i.e., the traveling speed of the host vehicle 1), the angular velocity, the longitudinal acceleration, the lateral acceleration, etc. The "driving environment" includes the natural environment such as the outside air temperature, the rainfall amount, the illuminance, etc. Further, the "driving environment" includes the presence states of moving objects and stationary objects within a predetermined detection range around the host vehicle 1. The "moving objects" include pedestrians, cyclists, animals, and other vehicles in operation. The "stationary objects" include road debris, guardrails, curbs, other parked vehicles, road signs, and in addition, structures such as walls and buildings. The moving objects and the stationary objects are included in the "targets". That is, the vehicle state sensor 15 is a general term for various sensors such as an accelerator opening sensor, a steering angle sensor, a wheel speed sensor, an angular velocity sensor, an acceleration sensor, an outside air temperature sensor, a rain drop sensor, an illuminance sensor, a radar sensor, an ultrasonic sensor, etc. (excluding the image capturing device 13) for the sake of simplification of illustration and description.

[0019] The navigation device 16 is configured to calculate the host vehicle position and the traveling route of the host vehicle 1 based on the map information and the positioning information by the positioning satellites. The "host vehicle position" is the current position of the host vehicle 1. The "traveling route" is the planned traveling route of the host vehicle 1 from the host vehicle position to the destination. Further, the navigation device 16 has links connecting between nodes indicating intersections as branch information, and is configured to be able to calculate the angle formed by the traveling direction of the host vehicle 1 and the link.

[0020] (Display control device) The display control device 12 has a configuration as an HCU. HCU is an abbreviation for HMI Control Unit. HMI is an abbreviation for Human Machine Interface. The display control device 12 is configured as a so-called in-vehicle computer including a processor and a memory programmed to execute one or more functions embodied by a computer program. The display control device 12 includes a recognition result acquisition unit 121, a host vehicle position acquisition unit 122, a route information acquisition unit 123, a display information generation unit 124, and a center position estimation unit 125 as a functional configuration or functional unit realized on a microcomputer.

[0021] FIG. 3 shows an example of a display in the head-up display device 11 by the display control device 12 according to the present embodiment shown in FIG. 2. Hereinafter, an outline of the functional configuration or functional unit in the display control device 12 according to the present embodiment will be described with reference to FIGS. 1 to 3.

[0022] The recognition result acquisition unit 121 is configured to acquire, from the recognition unit 14, the recognition result of a target based on the image of the foreground captured by the image capturing device 13. The host vehicle position acquisition unit 122 is configured to acquire the host vehicle position from the navigation device 16. The route information acquisition unit 123 is configured to acquire the travel route from the navigation device 16.

[0023] The display information generation unit 124 is configured to generate display information for superimposing the travel direction display TD on the intersection to be passed IS based on the recognition result of the target obtained by the recognition result acquisition unit 121, the position of the host vehicle obtained by the host vehicle position acquisition unit 122, and the travel route obtained by the travel route information acquisition unit 123. The intersection to be passed IS is an intersection that the host vehicle 1 is scheduled to pass through and exists at the destination of the host vehicle 1 on the road RR being traveled. The road RR being traveled is the road on which the host vehicle 1 is traveling and can also be referred to as a "road-like road". A "road-like road" is a road that can be traveled "as is" without the need for right or left turns and may include curves and cranks in addition to straight driving sections. The travel direction display TD is a virtual image display element indicating the planned travel direction of the host vehicle 1 at the intersection to be passed IS (i.e., right turn, straight ahead, left turn, etc.). FIG. 3 corresponds to a display example when the host vehicle 1 is approaching the intersection to be passed IS and is scheduled to make a right turn at such an intersection to be passed IS. In the example of FIG. 3, it is assumed that there are no other intersections between the host vehicle 1 and the intersection to be passed IS. That is, the intersection to be passed IS is the intersection that the host vehicle 1 is about to pass through this time (i.e., from the current host vehicle position for the first time). The same shall apply to other examples from FIG. 4 onwards.

[0024] The display information generation unit 124 includes a center position estimation unit 125. Based on the recognition result of the target obtained by the recognition result acquisition unit 121, the center position estimation unit 125 estimates at least the front and rear positions (i.e., the positions in the road extension direction of the traveling road RR) of the intersection center position PC, which is the center position of the planned intersection IS. The road extension direction is a direction orthogonal to the road width direction of the traveling road RR. That is, the road extension direction is the tangent direction of the locus of the center position in the road width direction of the traveling road RR, and can also be referred to as the "along-the-road direction". The "along-the-road direction" is the direction in which a link series in the "along-the-road" relationship with the link on which the host vehicle 1 is currently traveling extends, and is a direction parallel to the traveling direction of the host vehicle 1 at the host vehicle position. The road width direction can also be referred to as the road breadth direction or the vehicle width direction. The position of the intersection center position PC in the road width direction of the traveling road RR is referred to as the "lateral position". Then, based on the intersection center position PC estimated by the center position estimation unit 125, the display information generation unit 124 superimposes and displays the traveling direction display TD on the planned intersection IS.

[0025] (Operation Outline) Hereinafter, the operations of the in-vehicle system 10 and the display control device 12 according to the present embodiment, as well as the outlines of the control method and control program executed thereby, will be described together with the effects achieved by the present embodiment.

[0026] Based on the foreground image captured by the image capturing device 13, the recognition unit 14 performs image recognition on the targets included in the foreground that can be visually recognized by the driver D through the front windshield 4 in front of and on the front side of the host vehicle 1. The image recognition result of the target by the recognition unit 14 is used not only for the automatic driving operation or ADAS operation in the host vehicle 1, but also for the display operation by the head-up display device 11.

[0027] The display control device 12 superimposes various types of information on the foreground based on signals or information acquired from the recognition unit 14, the vehicle state sensor 15, and the navigation device 16. Specifically, the recognition result acquisition unit 121 acquires, from the recognition unit 14, the recognition result of a target based on the foreground image captured by the image capturing device 13. The host vehicle position acquisition unit 122 acquires the host vehicle position from the navigation device 16. The route information acquisition unit 123 acquires the travel route from the navigation device 16. The display information generation unit 124 generates display information for superimposing various display elements on the foreground based on the recognition result of the target acquired by the recognition result acquisition unit 121, the host vehicle position acquired by the host vehicle position acquisition unit 122, and the travel route acquired by the route information acquisition unit 123. Then, for example, as shown in FIG. 3, when the host vehicle 1 is approaching a planned intersection IS where a right turn is planned, a travel direction display TD indicating a right turn at such a planned intersection IS is superimposed on the planned intersection IS.

[0028] By the way, with the position accuracy of a conventional in-vehicle navigation device 16, there were cases where the front and rear positions of an intersection that is the target for guiding a route or a travel direction deviated. The cause of this is, for example, a decrease in the accuracy of host vehicle position estimation when a positioning signal such as GPS could not be received or was difficult to receive in a high-rise building, a tunnel, a multi-story parking lot, etc. Or, the cause of this is, for example, an error in host vehicle position estimation due to the actual road shape changing from the map when performing map matching. FIG. 4 shows an example of the superimposed display of the travel direction display TD when the intersection center position PC is erroneously calculated as the false center position PE on the front side thereof due to an error in host vehicle position estimation. In this case, the travel direction display TD is superimposed based on the false center position PE on the front side thereof instead of the true intersection center position PC. Then, the travel direction display TD is superimposed in a manner that gives a sense of discomfort, such that the host vehicle 1 makes a right turn in front of the planned intersection IS.

[0029] Therefore, in the present embodiment, the display information generation unit 124, that is, the center position estimation unit 125, estimates the intersection center position PC based on the image recognition result of a target (for example, a stop line LL, a traffic signal TL, a crosswalk ZC, etc.) provided corresponding to the intersection IS where the vehicle intends to pass. Specifically, for example, as shown in FIG. 3, at an intersection with a traffic signal (hereinafter also referred to as an "intersection with a traffic signal" or a "traffic signal-controlled intersection") where a traffic signal TL is provided, a stop line LL is permanently installed on the near side, and a traffic signal TL is permanently installed on the far side. Therefore, the center position estimation unit 125 estimates the intermediate position (specifically, the center position) between the front and rear positions of the stop line LL provided on the near side (i.e., the side of the host vehicle 1) at the intersection IS where the vehicle intends to pass and the front and rear positions of the traffic signal TL provided on the far side (i.e., the side opposite to the side of the host vehicle 1) at the intersection IS where the vehicle intends to pass as the front and rear positions of the intersection center position PC. Then, the display information generation unit 124 superimposes and displays the travel direction display TD on the intersection IS where the vehicle intends to pass based on the intersection center position PC estimated by the center position estimation unit 125.

[0030] According to the present embodiment, even when an error occurs in the detection or estimation of the position of the host vehicle by the navigation device 16, it is possible to estimate or calculate the intersection center position PC, which is the reference position for the superimposed display of the travel direction display TD with respect to the intersection IS where the vehicle intends to pass, based on the image recognition result of the target provided at the intersection IS where the vehicle intends to pass, that is, the three-dimensional position. Therefore, according to the present embodiment, even when an error occurs in the estimation or calculation of the position of the host vehicle, it is possible to suppress the occurrence of a display position deviation.

[0031] Also, regarding the lateral position of the intersection center position PC, by using the recognition result of the road marking LD on the traveling road RR during travel, it becomes possible to suppress the occurrence of a display position deviation of the travel direction display TD. In particular, as shown in FIG. 3, when the road markings LD extending in the direction along the current traveling direction of the host vehicle 1 are provided in parallel to each other on both the left and right sides of the host vehicle 1, the width and inclination of the road or lane can be well recognized based on these recognition results. Therefore, by comparing the inclinations of the road markings LD on both the left and right sides of the host vehicle 1 and calculating the lateral position of the intersection center position PC using these recognition results if the directions are the same, the travel direction display TD can be superimposed and displayed in an appropriate manner with respect to the intersection IS to be passed through.

[0032] FIG. 5 shows an example of a calculation flow of the front-rear position at the intersection center position PC corresponding to the above specific example. When obtaining the image recognition result of the traffic signal TL provided on the traveling road RR for the host vehicle 1 (that is, not the one provided on the intersecting road or the oncoming lane side that intersects the traveling road RR), search for the one with the lateral position within a predetermined value (for example, 8 m) and the minimum depth (that is, the front-rear direction distance from the host vehicle 1). Also, when obtaining the image recognition result of the stop line LL, search for the one with the minimum lateral position. If there are multiple candidates, search for all candidates. Then, when the depth of the traffic signal TL on the rear side is within a predetermined value (for example, 80 m) and both such a traffic signal TL and the stop line LL on the front side are detected, estimate the center position between the front-rear position of the traffic signal TL on the rear side and the front-rear position of the stop line LL on the front side as the front-rear position of the intersection center position PC.

[0033] FIG. 6 shows an example of a calculation flow of the lateral position at the intersection center position PC. The center position estimation unit 125 estimates the lateral position of the intersection center position PC based on the running position of the host vehicle 1 in the road width direction on the road RR during running, which is estimated based on the recognition result. Specifically, as shown in FIG. 6, the lane in which the host vehicle 1 is running is specified based on the image recognition result and the map information. That is, a lane specific value in the lane in which the host vehicle 1 is running is acquired. The lane specific value, also referred to as "lane ID", has an initial value of 1 and is sequentially numbered incrementally by 1 from right to left. Also, based on the image recognition result, the lane width is acquired. Further, based on the image recognition result of the right road demarcation line LD in the host lane, which is the lane in which the host vehicle 1 is running, the lateral position of such road demarcation line LD is acquired. And based on these, the lateral position of the intersection center position PC is estimated. That is, it is possible to calculate an estimated value of the lateral position of the intersection center position PC by offsetting the value of the lateral position of the right road demarcation line LD in the host lane in the road width direction using the lane width according to the lane specific value.

[0034] (Modification example) The present invention is not limited to the above-described embodiment. Therefore, the above-described embodiment can be appropriately changed. Hereinafter, typical modification examples will be described. In the description of the following modification examples, the differences from the above-described embodiment will be mainly described. Also, in the above-described embodiment and the modification examples, the same reference numerals are assigned to the parts that are identical or equivalent to each other. Therefore, in the description of the following modification examples, regarding the components having the same reference numerals as those in the above-described embodiment, the description in the above-described embodiment can be appropriately incorporated as long as there is no technical contradiction or special additional explanation.

[0035] The present invention is not limited to the specific device configuration shown in the above embodiments. That is, for example, the vehicle 1 equipped with the in-vehicle system 10 is not limited to a passenger car. Specifically, such a vehicle 1 may be a large vehicle such as a freight truck. There is no particular limitation on the number of wheels either, and it may be a three-wheeled vehicle, or a six-wheeled or eight-wheeled vehicle such as a freight truck. The type of the vehicle 1 may be a conventional vehicle equipped only with an internal combustion engine, an electric vehicle or a fuel cell vehicle not equipped with an internal combustion engine, or a so-called hybrid vehicle. The shape and structure of the vehicle body 2 in the vehicle 1 are also not limited to a box shape, that is, a substantially rectangular shape in plan view. There is no particular limitation on the use of the vehicle 1, the position of the steering wheel 8, the number of passengers, etc. That is, the driver D may be any in-vehicle passenger who is in charge of or executes a dynamic driving task. In other words, as long as the driver D can execute a driving operation, there is no particular limitation on the seating position of the driver D. Also, instead of or together with the steering wheel 8, any operation device such as a joystick may be used.

[0036] As the communication standard constituting the in-vehicle system 10, those other than CAN (registered trademark), for example, FlexRay (registered trademark), etc. may also be adopted. Also, the communication standard constituting the in-vehicle system 10 is not limited to one type. For example, the in-vehicle system 10 may have a sub-network line compliant with a communication standard such as LIN. LIN is an abbreviation for Local Interconnect Network.

[0037] The display control device 12 is not limited to a configuration as a so-called in-vehicle computer equipped with a processor and a memory. Specifically, for example, all or part of the display control device 12 may be a configuration equipped with a digital circuit configured to enable the above-described operations, such as an ASIC or an FPGA. ASIC is an abbreviation for Application Specific Integrated Circuit. FPGA is an abbreviation for Field Programmable Gate Array.

[0038] The program according to the present invention, which enables various operations, procedures, or processes described in the above embodiments to be executed, can be stored in a non-transitory tangible storage medium readable by a computer as instructions executed by the computer. That is, each of the above functional configurations and methods can also be represented as a computer program including procedures for realizing the same, or a non-transitory tangible storage medium storing the program. Such a program can be downloaded or upgraded via V2X communication. V2X is an abbreviation for Vehicle to X. Alternatively, such a program can be downloaded or upgraded via terminal devices provided in a manufacturing factory, a maintenance factory, a sales store, etc. of Vehicle 1. The storage destination of such a program may be a memory card, an optical disk, a magnetic disk, etc.

[0039] The recognition unit 14 and the recognition result acquisition unit 121 can be integrated. That is, the recognition result acquisition unit 121 may be configured to perform image recognition of an object included in the foreground based on an image of the foreground captured by the image capturing device 13. In other words, the present invention can be favorably applied also to a vehicle 1 not provided with an automatic driving system or an ADAS system that uses the image recognition result of the foreground captured by the image capturing device 13 for driving control or the like.

[0040] The present invention is not limited to the specific operation examples shown in the above embodiments. For example, the stop line LL on the near side at the intersection IS where passage is planned may not be recognized by image recognition because it has become thin due to wear or is blocked by a preceding other vehicle. In such a case, as shown in FIG. 7, if a rear signal TL2 is provided at the intersection IS where passage is planned in addition to the near signal TL1, it is possible to estimate the intermediate position between the front and rear positions of these (i.e., for example, the central position) as the front and rear positions of the intersection center position PC. That is, the center position estimation unit 125 estimates the intermediate position between the front and rear positions of the near signal TL1 provided on the own vehicle 1 side at the intersection IS where passage is planned and the front and rear positions of the rear signal TL2 provided on the side opposite to the own vehicle 1 side at the intersection IS where passage is planned as the front and rear positions of the intersection center position PC. Thereby, even when the stop line LL on the near side at the intersection IS where passage is planned cannot be recognized by image recognition, it is possible to satisfactorily suppress the occurrence of display position deviation.

[0041] As shown in FIG. 8, when the traffic signal TL, the near crosswalk ZC1 and the rear crosswalk ZC2 arranged side by side in the road direction are recognized, there is a high possibility that an intersection exists between the near crosswalk ZC1 and the rear crosswalk ZC2. Therefore, in this case, it is possible to estimate the intermediate position between the front and rear positions of the near crosswalk ZC1 and the rear crosswalk ZC2 (i.e., for example, the central position) as the front and rear positions of the intersection center position PC. That is, when the recognition result includes the traffic signal TL provided at the intersection IS where passage is planned, the near crosswalk ZC1 provided at the intersection IS where passage is planned along the road width direction, and the rear crosswalk ZC2 provided at the intersection IS where passage is planned along the road width direction at a different position from the near crosswalk ZC1 in the road extension direction, the center position estimation unit 125 estimates the intermediate position between the front and rear positions of the near crosswalk ZC1 and the front and rear positions of the rear crosswalk ZC2 as the front and rear positions of the intersection center position PC. Thereby, it is possible to satisfactorily suppress the occurrence of display position deviation.

[0042] FIG. 9 shows an example where the planned intersection IS at which the host vehicle 1 is to turn right is an intersection without traffic signals and the road RR being traveled on is a priority road. Specifically, in the example of FIG. 9, there are no stop lines LL or crosswalks ZC that existed in the example of FIG. 3 in front of or behind the planned intersection IS that the host vehicle 1 is approaching. On the other hand, a crosswalk ZC is provided along the road extension direction of the road RR being traveled on in the intersecting road that intersects the road RR being traveled on. In this case, it is possible to estimate the intersection center position PC at the planned intersection IS based on the center position of the crosswalk ZC provided on the intersecting road. That is, the center position estimation unit 125 estimates the center position of the crosswalk ZC provided along the road extension direction, i.e., the front-rear direction, of the road RR being traveled on as the front-rear position of the intersection center position PC. Thereby, it becomes possible to satisfactorily suppress the occurrence of display position deviation. When such a crosswalk ZC is provided on both the left and right sides as viewed from the host vehicle 1 as shown in FIG. 9, it is possible to estimate the intermediate position in the lateral direction between the two as the lateral position of the intersection center position PC.

[0043] In an intersection with traffic signals as shown in FIG. 3 and the like, it is possible to estimate the intersection center position PC with high accuracy based on the estimation result of the relative position with respect to the host vehicle 1 using the image recognition result of the traffic signal TL and / or the stop line LL. Therefore, every time the host vehicle 1 passes through an intersection one after another, it is possible to statistically process, i.e., learn, the difference between the pseudo center position PE (see FIG. 4) based on the host vehicle position acquired from the navigation device 16 and the estimation result of the intersection center position PC using the image recognition result. Such a learning result can also be applied to the case of an intersection without traffic signals as shown in FIG. 9. Therefore, the center position estimation unit 125 may correct the estimation result of the intersection center position PC at the intersection without traffic signals based on the past estimation result of the intersection center position PC at the intersection with traffic signals. Thereby, it also becomes possible to satisfactorily suppress the occurrence of display position deviation at the intersection without traffic signals.

[0044] As shown in FIG. 10, there may be a road stud RM called a "cat's eye" laid or buried at the intersection center position PC. The relative position of this road stud RM with respect to the host vehicle 1 based on the image recognition result can be regarded as almost the same as the true intersection center position PC. Therefore, the center position estimation unit 125 can estimate the intersection center position PC based on the recognized position of the road stud RM. As a result, it becomes possible to more favorably suppress the occurrence of display position deviation.

[0045] When the host vehicle 1 approaches the intersection IS to be traveled through, as shown in FIG. 11, landmarks such as the stop line LL become out of the imaging range and are no longer recognized. In this case, the center position estimation unit 125 can correct the estimation result of the intersection center position PC based on the driving state of the host vehicle 1 (for example, vehicle speed, yaw rate, steering angle, etc.). As a result, it becomes possible to maintain a state in which the occurrence of display position deviation is favorably suppressed until the host vehicle 1 passes through the intersection IS to be traveled through.

[0046] Referring to FIG. 12, the center position estimation unit 125 may correct the estimation result of the intersection center position PC based on the behavior of the preceding vehicle VA, which is another vehicle in front of the host vehicle 1. That is, the center position estimation unit 125 can correct the estimation result of the intersection center position PC using at least any one of the position, orientation, travel trajectory, and turn signal orientation of the preceding vehicle VA. Specifically, for example, it is possible to set the position shifted backward by the vehicle width at the front position along the lane of a left / right turning vehicle as the front-rear position, and the position shifted by half the lane width at the position waiting to turn right as the left-right position. As a result, it becomes possible to favorably suppress the occurrence of display position deviation.

[0047] As described above, the plurality of estimation methods for the intersection center position PC by the center position estimation unit 125 can be used in an overlapping manner. That is, the estimation result of the intersection center position PC may be obtained by statistically processing the positions estimated by the plurality of estimation methods. Specifically, for example, the estimation result of the intersection center position PC may be calculated using the average value or the weighted average value of the estimated value based on the stop line LL on the near side and the traffic signal TL on the far side, and the estimated value using the crosswalk ZC.

[0048] Each of the above-described examples shows a display example for the planned intersection IS of the four-way intersection, which is the nearest intersection to the host vehicle 1. However, the present invention is not limited to such an aspect. That is, for example, when there is another planned intersection IS going straight between the host vehicle 1 and the planned intersection IS with a planned right or left turn, the travel direction display TD may be displayed for all the planned intersections IS. Alternatively, the planned intersection IS may be provided at a three-way intersection, a five-way intersection, or the like. The present invention can be suitably applied to these scenarios as well.

[0049] In the specification and the claims, expressions such as "acquire", "calculate", "estimate", "detect", "sense", "extract", etc., which are similar or related to each other, can be appropriately replaced with each other as long as there is no technical contradiction or inconvenience. The same applies to "generate", "output", etc.

[0050] It goes without saying that the elements constituting the above-described embodiments are not necessarily essential except in cases where it is explicitly stated that they are particularly essential and cases where they are considered to be clearly essential in principle. Also, when numerical values such as the number, quantity, range, etc. of the components are mentioned, the present invention is not limited to the specific numerical values except in cases where it is explicitly stated that they are particularly essential and cases where they are clearly limited to specific numerical values in principle. Similarly, when the shape, direction, positional relationship, etc. of the components are mentioned, the present invention is not limited to the shape, direction, positional relationship, etc. except in cases where it is explicitly stated that they are particularly essential and cases where they are clearly limited to specific shapes, directions, positional relationships, etc. in principle.

[0051] The modification examples are not limited to the above examples. For example, all or part of one of the plurality of embodiments and all or part of another one can be combined with each other as long as they do not technically conflict. There is no particular limitation on the number of combinations. Similarly, all or part of one of the plurality of modification examples and all or part of another one can be combined with each other as long as they do not technically conflict. Furthermore, all or part of the above embodiments and all or part of the above modification examples can be combined with each other as long as they do not technically conflict.

Explanation of Reference Numerals

[0052] 1 Own vehicle 11 Head-up display device (display device) 12 Display control device 13 Image pickup device 121 Recognition result acquisition unit 122 Own vehicle position acquisition unit 123 Route information acquisition unit 124 Display information generation unit 125 Center position estimation unit TD Travel direction display

Claims

1. A display control device (12) configured to control a display operation of a display device (11) that is mounted on a host vehicle (1) to superimpose and display a virtual image (IM) on the front view of the host vehicle, comprising a recognition result acquisition unit (121) that acquires a recognition result of a target based on an image of the front view captured by an image capturing device (13), a host vehicle position acquisition unit (122) that acquires a host vehicle position that is the position of the host vehicle, a route information acquisition unit (123) that acquires a traveling route of the host vehicle, and a display information generation unit (124) that generates display information for superimposing and displaying a traveling direction display (TD) indicating a planned traveling direction of the host vehicle at a planned intersection (IS), which is an intersection where passage is planned and exists in the traveling direction of the host vehicle on a traveling road (RR) that is the road on which the host vehicle is traveling, on the planned intersection based on the recognition result, the host vehicle position, and the traveling route, and comprising: wherein the display information generation unit comprises a center position estimation unit (125) that estimates a position of a center position (PC) of the planned intersection, which is the center position of the planned intersection, at least in a road extension direction orthogonal to a road width direction of the traveling road based on the recognition result, superimposes and displays the traveling direction display on the planned intersection based on the estimated center position of the intersection, and the center position estimation unit estimates an intermediate position between a position of a stop line (LL) provided on the host vehicle side at the planned intersection in the road extension direction and a position of a traffic signal (TL) provided on the side opposite to the host vehicle side at the planned intersection in the road extension direction as the position of the center position of the intersection in the road extension direction. Display control device.

2. The center position estimation unit estimates, as the position of the intersection center position in the road extension direction, an intermediate position between the position of the near-side traffic signal (TL1) provided on the host vehicle side at the intersection to be traveled and the position of the far-side traffic signal (TL2) provided on the side opposite to the host vehicle side at the intersection to be traveled, in the road extension direction. The display control device according to claim 1.

3. When the recognition result includes a traffic signal (TL) provided at the intersection to be traveled, a first crosswalk (ZC1) provided at the intersection to be traveled along the road width direction, and a second crosswalk (ZC2) provided at the intersection to be traveled along the road width direction at a position different from that of the first crosswalk in the road extension direction, the center position estimation unit estimates, as the position of the intersection center position in the road extension direction, an intermediate position between the first crosswalk and the second crosswalk in the road extension direction. The display control device according to claim 1 or 2.

4. A display control device (12) configured to control a display operation of a display device (11) that is mounted on a host vehicle (1) and superimposes a virtual image (IM) on the foreground of the host vehicle. A recognition result acquisition unit (121) that acquires a recognition result of a target based on an image of the foreground captured by an image capturing device (13). A host vehicle position acquisition unit (122) that acquires the position of the host vehicle, which is the position of the host vehicle. A route information acquisition unit (123) that acquires the traveling route of the host vehicle. Based on the recognition result, the host vehicle position, and the traveling route, display information for superimposing a traveling direction display (TD) indicating the traveling planned direction of the host vehicle at an intersection to be traveled (IS), which is an intersection where the host vehicle plans to travel and exists in the traveling direction of the host vehicle on a traveling road (RR), which is the road on which the host vehicle is traveling, on the intersection to be traveled, is generated by a display information generation unit (124). Comprising The display information generation unit Based on the recognition result, a center position estimation unit (125) is provided to estimate the position of the intersection center position (PC), which is the center position of the planned intersection, in at least the road extension direction orthogonal to the road width direction of the road being traveled on. Based on the estimated intersection center position, the direction-of-travel display is superimposed and displayed on the planned intersection. When the recognition result includes a traffic signal (TL) provided at the planned intersection, a first crosswalk (ZC1) provided at the planned intersection along the road width direction, and a second crosswalk (ZC2) provided at the planned intersection along the road width direction at a different position from the first crosswalk in the road extension direction, the center position estimation unit estimates the intermediate position between the first crosswalk and the second crosswalk in the road extension direction as the position of the intersection center position in the road extension direction. Display control device. Claim 5 The center position estimation unit estimates the center position of a crosswalk (ZC) provided along the road extension direction as the position of the intersection center position in the road extension direction. The display control device according to any one of claims 1 to 4. Claim 6 A display control device (12) configured to control the display operation of a display device (11) mounted on the host vehicle (1) to superimpose and display a virtual image (IM) on the foreground of the host vehicle. A recognition result acquisition unit (121) that acquires a recognition result of an object based on an image of the foreground captured by an image capturing device (13). A host vehicle position acquisition unit (122) that acquires the host vehicle position, which is the position of the host vehicle. A route information acquisition unit (123) that acquires the travel route of the host vehicle. Based on the recognition result, the position of the host vehicle, and the travel route, display information generation unit (124) generates display information for superimposing a travel direction display (TD) indicating the planned travel direction of the host vehicle at an intersection with a planned passage (IS), which is an intersection with a planned passage existing at the destination of the host vehicle on the travel road (RR), which is the road on which the host vehicle is traveling, on the intersection with the planned passage. comprising The display information generation unit comprises a center position estimation unit (125) that estimates the position of the intersection center position (PC), which is the center position of the intersection with the planned passage, at least in the road extension direction orthogonal to the road width direction of the travel road, based on the recognition result. Based on the estimated intersection center position, the travel direction display is superimposed on the intersection with the planned passage. The center position estimation unit estimates the center position of a crosswalk (ZC) provided along the road extension direction in the road extension direction as the position of the intersection center position in the road extension direction based on the position of the recognized road stud (RM). Display control device.

7. The center position estimation unit estimates the intersection center position based on the position of the recognized road stud (RM). The display control device according to any one of claims 1 to 5.

8. A display control device (12) configured to control a display operation in a display device (11) mounted on a host vehicle (1) to superimpose a virtual image (IM) on the foreground of the host vehicle, a recognition result acquisition unit (121) that acquires a recognition result of an object based on an image of the foreground captured by an image capturing device (13); a host vehicle position acquisition unit (122) that acquires the position of the host vehicle, which is the position of the host vehicle; a route information acquisition unit (123) that acquires the travel route of the host vehicle; Based on the recognition result, the position of the host vehicle, and the travel route, a display information generation unit (124) generates display information for superimposing a travel direction display (TD) indicating the planned travel direction of the host vehicle at a planned intersection (IS), which is an intersection with a planned passage existing at the destination of the host vehicle on a running road (RR), which is the road on which the host vehicle is running, on the planned intersection. comprising The display information generation unit is provided with a center position estimation unit (125) that estimates the position of the intersection center position (PC), which is the center position of the planned intersection, at least in the road extension direction orthogonal to the road width direction of the running road, based on the recognition result. Based on the estimated intersection center position, the travel direction display is superimposed on the planned intersection. The center position estimation unit estimates the intersection center position based on the position of the recognized road stud (RM). A display control device.

9. The center position estimation unit corrects the estimation result of the intersection center position based on the driving state of the host vehicle. The display control device according to any one of claims 1 to 8.

10. A display control device (12) configured to control a display operation in a display device (11) that superimposes a virtual image (IM) on the foreground of a host vehicle (1) when mounted on the host vehicle, a recognition result acquisition unit (121) that acquires a recognition result of a target based on an image of the foreground captured by an image capturing device (13); a host vehicle position acquisition unit (122) that acquires the position of the host vehicle, which is the position of the host vehicle; a route information acquisition unit (123) that acquires the travel route of the host vehicle; Based on the recognition result, the position of the host vehicle, and the traveling route, display information generation unit (124) generates display information for superimposing a traveling direction display (TD) indicating the planned traveling direction of the host vehicle at an intersection with a planned passage (IS), which is an intersection with a planned passage existing ahead of the host vehicle on the traveling road (RR), which is the road on which the host vehicle is traveling, on the intersection with the planned passage. comprising The display information generation unit includes a center position estimation unit (125) that estimates the position of the intersection center position (PC), which is the center position of the intersection with the planned passage, at least in the road extension direction orthogonal to the road width direction of the traveling road, based on the recognition result. Based on the estimated intersection center position, the traveling direction display is superimposed on the intersection with the planned passage. The center position estimation unit estimates the center position of the crosswalk (ZC) provided along the road extension direction in the road extension direction as the position of the intersection center position in the road extension direction, and corrects the estimation result of the intersection center position based on the driving state of the host vehicle. A display control device.

11. The center position estimation unit corrects the estimation result of the intersection center position at an intersection without a traffic signal based on the estimation result of the intersection center position in the past at an intersection with a traffic signal. The display control device according to any one of claims 1 to 10.

12. A display control device (12) configured to control a display operation in a display device (11) that superimposes a virtual image (IM) on the foreground of a host vehicle (1) when mounted on the host vehicle, a recognition result acquisition unit (121) that acquires a recognition result of an object based on an image of the foreground captured by an image capturing device (13), a host vehicle position acquisition unit (122) that acquires the position of the host vehicle, which is the position of the host vehicle, a route information acquisition unit (123) that acquires the traveling route of the host vehicle. Based on the recognition result, the position of the host vehicle, and the travel route, a display information generation unit (124) generates display information for superimposing a travel direction display (TD) indicating the planned travel direction of the host vehicle at an intersection (IS) with a planned passage, which is the destination of the host vehicle on the running road (RR), which is the road on which the host vehicle is running, on the intersection with the planned passage. comprising the display information generation unit comprises a center position estimation unit (125) that estimates the position of the intersection center position (PC), which is the center position of the intersection with the planned passage, at least in the road extension direction orthogonal to the road width direction of the running road based on the recognition result. Based on the estimated intersection center position, the travel direction display is superimposed on the intersection with the planned passage. the center position estimation unit estimates the center position of the pedestrian crossing (ZC) provided along the road extension direction in the road extension direction as the position of the intersection center position in the road extension direction. Based on the estimation result of the past intersection center position at an intersection with a traffic signal, corrects the estimation result of the intersection center position at an intersection without a traffic signal. A display control device.

13. A display control device (12) configured to control the display operation of a display device (11) mounted on a host vehicle (1) to superimpose a virtual image (IM) on the foreground of the host vehicle, a recognition result acquisition unit (121) that acquires a recognition result of an object based on an image of the foreground captured by an image capturing device (13); a host vehicle position acquisition unit (122) that acquires the position of the host vehicle, which is the position of the host vehicle; a route information acquisition unit (123) that acquires the travel route of the host vehicle; Based on the recognition result, the position of the host vehicle, and the travel route, display information generation unit (124) generates display information for superimposing a travel direction display (TD) indicating the planned travel direction of the host vehicle at an intersection with a planned passage (IS), which is an intersection with a planned passage existing in the travel destination of the host vehicle on a travel road (RR), which is the road on which the host vehicle is traveling, on the intersection with the planned passage. comprising The display information generation unit includes a center position estimation unit (125) that estimates the position of the intersection center position (PC), which is the center position of the intersection with the planned passage, at least in the road extension direction orthogonal to the road width direction of the travel road, based on the recognition result. Based on the estimated intersection center position, the travel direction display is superimposed on the intersection with the planned passage. The center position estimation unit corrects the estimation result of the intersection center position based on the driving state of the host vehicle, and corrects the estimation result of the intersection center position at an intersection without a traffic signal based on the past estimation result of the intersection center position at an intersection with a traffic signal. A display control device.

14. The center position estimation unit estimates the position of the intersection center position in the road width direction based on the travel position of the host vehicle in the road width direction on the travel road, which is estimated based on the recognition result. The display control device according to any one of claims 1 to 13.

15. A display control device (12) configured to control a display operation in a display device (11) that superimposes a virtual image (IM) on the foreground of the host vehicle when mounted on the host vehicle (1). A recognition result acquisition unit (121) that acquires a recognition result of a target based on an image of the foreground captured by an image capturing device (13). A host vehicle position acquisition unit (122) that acquires the position of the host vehicle, which is the position of the host vehicle. A route information acquisition unit (123) that acquires the travel route of the host vehicle; Based on the recognition result, the host vehicle position, and the travel route, display information for superimposing a travel direction display (TD) indicating the planned travel direction of the host vehicle at a planned intersection (IS), which is an intersection where passage is planned and exists in the travel destination of the host vehicle on the travel road (RR) that is the road on which the host vehicle is traveling, on the planned intersection is generated by a display information generation unit (124); Comprising; The display information generation unit; Based on the recognition result, a center position estimation unit (125) that estimates the position of the intersection center position (PC), which is the center position of the planned intersection, in at least the road extension direction orthogonal to the road width direction of the travel road is provided; Based on the estimated intersection center position, the travel direction display is superimposed and displayed on the planned intersection; The center position estimation unit corrects the estimation result of the intersection center position at an unsignalized intersection based on the estimation result of the past intersection center position at a signalized intersection; A display control device.

16. The center position estimation unit corrects the estimation result of the intersection center position based on the behavior of another vehicle (VA) in front of the host vehicle; The display control device according to any one of claims 1 to 15.

17. The route information acquisition unit acquires the travel route from a navigation device (16); The navigation device has a link connecting nodes indicating intersections as branch information, and calculates an angle formed by the travel direction of the host vehicle and the link; The display control device according to any one of claims 1 to 16.

18. A display control program executed by a display control device (12) configured to control a display operation in a display device (11) that superimposes a virtual image (IM) on the foreground of the host vehicle when mounted on the host vehicle (1); The processes executed by the display control device include: a recognition result acquisition process of acquiring a recognition result of a target based on an image of the foreground imaged by the image capturing device (13); a host vehicle position acquisition process of acquiring the position of the host vehicle, which is the position of the host vehicle; a route information acquisition process of acquiring the traveling route of the host vehicle; based on the recognition result, the host vehicle position, and the traveling route, display information for superimposing and displaying a traveling direction display (TD) indicating the planned traveling direction of the host vehicle at a planned intersection (IS), which is an intersection where passage is planned and exists in the direction of travel of the host vehicle on a traveling road (RR), which is the road on which the host vehicle is traveling, on the planned intersection; and includes: The display information generation process includes: a center position estimation process of estimating, based on the recognition result, a position of a center position (PC) of the planned intersection in at least a road extension direction orthogonal to a road width direction of the traveling road; superimposing and displaying the traveling direction display on the planned intersection based on the estimated center position of the intersection; The center position estimation process estimates an intermediate position between a position of a stop line (LL) provided on the host vehicle side at the planned intersection in the road extension direction and a position of a traffic signal (TL) provided on the side opposite to the host vehicle side at the planned intersection in the road extension direction as the position of the center position of the intersection in the road extension direction. A display control program.

19. The center position estimation process estimates an intermediate position between a position of a front traffic signal (TL1) provided on the host vehicle side at the planned intersection in the road extension direction and a position of a rear traffic signal (TL2) provided on the side opposite to the host vehicle side at the planned intersection in the road extension direction as the position of the center position of the intersection in the road extension direction. The display control program according to claim 18.

20. When the center position estimation process includes, in the recognition result, a traffic signal (TL) provided at the intersection where passage is scheduled, a first crosswalk (ZC1) provided at the intersection where passage is scheduled along the road width direction, and a second crosswalk (ZC2) provided at the intersection where passage is scheduled along the road width direction at a position different from that of the first crosswalk in the road extension direction, the intermediate position in the road extension direction between the first crosswalk and the second crosswalk is estimated as the position in the road extension direction of the intersection center position. The display control program according to claim 18 or 19.

21. A display control program executed by a display control device (12) configured to control a display operation of a display device (11) that superimposes a virtual image (IM) on the foreground of a host vehicle (1) when mounted on the host vehicle. The process executed by the display control device includes: A recognition result acquisition process of acquiring a recognition result of a target based on an image of the foreground captured by an image capturing device (13); A host vehicle position acquisition process of acquiring a host vehicle position that is the position of the host vehicle; A route information acquisition process of acquiring a traveling route of the host vehicle; Based on the recognition result, the host vehicle position, and the traveling route, display information for superimposing a traveling direction display (TD) indicating a traveling direction of the host vehicle at an intersection where passage is scheduled (IS), which is an intersection where passage is scheduled and exists ahead of the host vehicle on a traveling road (RR) that is the road on which the host vehicle is traveling, on the intersection where passage is scheduled is generated. including The display information generation process includes: a center position estimation process of estimating, based on the recognition result, at least a position in a road extension direction orthogonal to the road width direction of a traveling road of the intersection center position (PC) of the intersection where passage is scheduled. Based on the estimated intersection center position, the travel direction display is superimposed on the planned intersection to be passed through. When the center position estimation process includes, in the recognition result, a traffic signal (TL) provided at the planned intersection to be passed through, a first crosswalk (ZC1) provided at the planned intersection along the road width direction, and a second crosswalk (ZC2) provided at the planned intersection along the road width direction at a position different from that of the first crosswalk in the road extension direction, the intermediate position in the road extension direction between the first crosswalk and the second crosswalk is estimated as the position in the road extension direction of the intersection center position. Display control program.

22. The center position estimation process estimates the center position in the road extension direction of a crosswalk (ZC) provided along the road extension direction as the position in the road extension direction of the intersection center position. The display control program according to any one of claims 18 to 21.

23. A display control program executed by a display control device (12) configured to control a display operation in a display device (11) that is mounted on a host vehicle (1) and superimposes a virtual image (IM) on a foreground of the host vehicle, The process executed by the display control device is A recognition result acquisition process for acquiring a recognition result of a target based on an image of the foreground captured by an image capturing device (13), A host vehicle position acquisition process for acquiring a host vehicle position that is the position of the host vehicle, A route information acquisition process for acquiring the travel route of the host vehicle, Based on the recognition result, the host vehicle position, and the travel route, display information for superimposing a travel direction display (TD) indicating the planned travel direction of the host vehicle at a planned intersection (IS), which is an intersection where passage is planned and exists in the travel destination of the host vehicle on a travel road (RR) on which the host vehicle is traveling, on the planned intersection is generated. including the display information generation process includes a center position estimation process for estimating a position of an intersection center position (PC), which is a center position of the intersection to be passed, in at least a road extension direction orthogonal to a road width direction of the road on which the vehicle is traveling, based on the recognition result. Based on the estimated intersection center position, the forward direction display is superimposed and displayed on the intersection to be passed. The center position estimation process estimates, based on a position of a recognized road stud (RM), a center position of a crosswalk (ZC) provided along the road extension direction in the road extension direction as the position of the intersection center position in the road extension direction. Display control program.

24. The center position estimation process estimates the intersection center position based on a position of a recognized road stud (RM). The display control program according to any one of claims 18 to 22.

25. A display control program executed by a display control device (12) configured to control a display operation of a display device (11) that superimposes and displays a virtual image (IM) in front of a host vehicle when mounted on the host vehicle (1). The process executed by the display control device a recognition result acquisition process for acquiring a recognition result of a target based on an image of the foreground captured by an image capturing device (13); a host vehicle position acquisition process for acquiring a host vehicle position, which is a position of the host vehicle; a route information acquisition process for acquiring a traveling route of the host vehicle; Based on the recognition result, the host vehicle position, and the traveling route, display information for generating a forward direction display (TD) indicating a forward direction of the host vehicle at an intersection to be passed (IS), which is an intersection with a planned passage existing ahead of the host vehicle on a road on which the host vehicle is traveling (RR), is generated and superimposed and displayed on the intersection to be passed. including The display information generation process includes: Based on the recognition result, a center position estimation process for estimating the position of the intersection center position (PC), which is the center position of the intersection to be passed, in at least the road extension direction orthogonal to the road width direction of the currently traveled road. Based on the estimated intersection center position, the forward direction display is superimposed and displayed on the intersection to be passed. The center position estimation process estimates the intersection center position based on the position of the recognized road stud (RM). Display control program.

26. The center position estimation process corrects the estimation result of the intersection center position based on the driving state of the host vehicle. The display control program according to any one of claims 18 to 25.

27. A display control program executed by a display control device (12) configured to control the display operation of a display device (11) mounted on the host vehicle (1) to superimpose and display a virtual image (IM) on the foreground of the host vehicle. The process executed by the display control device includes: A recognition result acquisition process for acquiring a recognition result of a target based on an image of the foreground captured by an image capturing device (13). A host vehicle position acquisition process for acquiring the host vehicle position, which is the position of the host vehicle. A route information acquisition process for acquiring the traveling route of the host vehicle. Based on the recognition result, the host vehicle position, and the traveling route, display information for superimposing and displaying a forward direction display (TD) indicating the planned traveling direction of the host vehicle at an intersection to be passed (IS), which is an intersection with a planned passage existing in the forward direction of the host vehicle on the currently traveled road (RR), on the intersection to be passed. including The display information generation process includes: Based on the recognition result, it includes a center position estimation process for estimating the position of the intersection center position (PC), which is the center position of the planned intersection, at least in the road extension direction orthogonal to the road width direction of the currently traveling road. Based on the estimated intersection center position, the travel direction display is superimposed and displayed on the planned intersection. The center position estimation process is as follows. The center position in the road extension direction of a crosswalk (ZC) provided along the road extension direction is estimated as the position of the intersection center position in the road extension direction. Based on the driving state of the host vehicle, the estimation result of the intersection center position is corrected. Display control program.

28. The center position estimation process corrects the estimation result of the intersection center position at an unsignalized intersection based on the estimation result of the intersection center position in the past at a signalized intersection. The display control program according to any one of claims 18 to 27.

29. A display control program executed by a display control device (12) configured to control the display operation of a display device (11) mounted on the host vehicle (1) to superimpose and display a virtual image (IM) on the foreground of the host vehicle. The process executed by the display control device is as follows. A recognition result acquisition process for acquiring a recognition result of a target based on an image of the foreground captured by an image capturing device (13). A host vehicle position acquisition process for acquiring the host vehicle position, which is the position of the host vehicle. A route information acquisition process for acquiring the traveling route of the host vehicle. Based on the recognition result, the host vehicle position, and the traveling route, display information for superimposing and displaying a travel direction display (TD) indicating the planned travel direction of the host vehicle at a planned intersection (IS), which is an intersection where the host vehicle plans to pass and exists in the travel destination of the host vehicle on the currently traveling road (RR), on the planned intersection is generated. including the display information generation process includes a center position estimation process of estimating a position of the intersection center position (PC), which is the center position of the intersection to be passed, in at least the road extension direction orthogonal to the road width direction of the road on which the vehicle is traveling, based on the recognition result based on the estimated intersection center position, superimposing and displaying the traveling direction display on the intersection to be passed the center position estimation process estimates the center position in the road extension direction of a crosswalk (ZC) provided along the road extension direction as the position in the road extension direction of the intersection center position corrects the estimation result of the intersection center position at an intersection without a traffic signal based on the past estimation result of the intersection center position at an intersection with a traffic signal display control program

30. A display control program executed by a display control device (12) configured to control a display operation in a display device (11) that superimposes and displays a virtual image (IM) in the foreground of a host vehicle (1) when mounted on the host vehicle, The process executed by the display control device a recognition result acquisition process of acquiring a recognition result of a target based on an image of the foreground captured by an image capturing device (13), a host vehicle position acquisition process of acquiring a host vehicle position that is the position of the host vehicle, a route information acquisition process of acquiring a traveling route of the host vehicle, based on the recognition result, the host vehicle position, and the traveling route, display information for superimposing and displaying a traveling direction display (TD) indicating a traveling planned direction of the host vehicle at an intersection to be passed (IS), which is an intersection where the host vehicle is scheduled to pass and exists ahead of the host vehicle on a road on which the host vehicle is traveling (RR), on the intersection to be passed, a display information generation process including the display information generation process Based on the recognition result, a center position estimation process is included to estimate the position of the intersection center position (PC), which is the center position of the intersection to be passed, in at least the road extension direction orthogonal to the road width direction of the road being traveled on. Based on the estimated intersection center position, the travel direction display is superimposed and displayed on the intersection to be passed. The center position estimation process includes: Based on the driving state of the host vehicle, the estimation result of the intersection center position is corrected, and Based on the estimation result of the intersection center position in the past at an intersection with a traffic signal, the estimation result of the intersection center position at an intersection without a traffic signal is corrected. A display control program.

31. The center position estimation process estimates the position of the intersection center position in the road width direction based on the driving position of the host vehicle in the road width direction of the road being traveled, which is estimated based on the recognition result. The display control program according to any one of claims 18 to 30.

32. A display control program executed by a display control device (12) configured to control the display operation of a display device (11) mounted on a host vehicle (1) to superimpose and display a virtual image (IM) in the foreground of the host vehicle. The process executed by the display control device includes: A recognition result acquisition process for acquiring a recognition result of a target based on an image of the foreground captured by an image capturing device (13), A host vehicle position acquisition process for acquiring the host vehicle position, which is the position of the host vehicle, A route information acquisition process for acquiring the travel route of the host vehicle, and Based on the recognition result, the position of the host vehicle, and the travel route, display information generation processing for generating display information for superimposing a travel direction display (TD) indicating the planned travel direction of the host vehicle at an intersection with a planned passage (IS), which is an intersection with a planned passage existing at the destination of the host vehicle on a travel road (RR) that is the road on which the host vehicle is traveling, on the intersection with the planned passage is performed. including The display information generation processing includes center position estimation processing for estimating the position of the intersection center position (PC), which is the center position of the intersection with the planned passage, at least in the road extension direction orthogonal to the road width direction of the travel road, based on the recognition result. Based on the estimated intersection center position, the travel direction display is superimposed on the intersection with the planned passage. The center position estimation processing corrects the estimation result of the intersection center position at an intersection without a traffic signal based on the estimation result of the intersection center position in the past at an intersection with a traffic signal. Display control program.

33. The center position estimation processing corrects the estimation result of the intersection center position based on the behavior of another vehicle (VA) in front of the host vehicle. The display control program according to any one of claims 18 to 32.

34. The route information acquisition processing acquires the travel route from a navigation device (16) that has a link connecting nodes indicating intersections as branch information and calculates an angle formed by the travel direction of the host vehicle and the link. The display control program according to any one of claims 18 to 33.

Images