Display control device, head-up display device, program, and display system for vehicles

Abstract

To further improve the effect of attention arousal for speed suppression or the like by displaying complementary content for further attention arousal with respect to a road surface sign such as an image hump that calls attention arousal to a crew member.SOLUTION: A display control device 30 controls a head-up display device (HUD device 10) by which an image is projected onto a display area VA virtually set ahead of a vehicle 1 and is superposed in a front field of view of the vehicle to be made visible for a crew member. The display control device comprises a control section 31 configured to: recognize a specific road surface sign calling attention arousal to the crew member from a picked-up image obtained by imaging the front field of view of the vehicle; generate content calling further attention arousal to the crew member in accordance with the recognized specific road surface sign; and superpose the generated content on the specific road surface sign and display it on the head-up display device (HUD device 10).SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to a display control device and the like that controls a head-up display device that projects an image onto a virtual display area set in front of a vehicle, superimposes the image on the front view of the vehicle, and allows an occupant to visually recognize it.

Background Art

[0002] For example, in a narrow lane such as a residential area, an image hump that utilizes a stereoscopic illusion effect may be used on the road surface to call the attention of an occupant (generally, the driver of the vehicle) such as speed suppression. An image hump is a type of road marking that gives a three-dimensional appearance by changing the color and material of the pavement without creating physical unevenness on the road surface. In particular, "visual narrowing" that gives the occupant an impression of a narrow section and promotes a speed reduction, such as a dotted line with broken white lines drawn on both sides of the lane on a downhill slope or a curve, is also known, and this is also a type of road marking.

[0003] On the other hand, for example, as described in Patent Document 1, a head-up display device (hereinafter, simply referred to as a HUD device) that can visually recognize an image superimposed on the foreground of a vehicle is also known. According to this HUD device, since the occupant who is the viewer can visually recognize the information shown in the image without significantly shifting the line of sight from the foreground, it is possible to reduce the load of line-of-sight movement and contribute to safe driving. In such a HUD device, in addition to a 2D image expressed in a planar manner, it is also possible to make the occupant feel a three-dimensional sense with a 3D image expressed in a three-dimensional manner (see paragraph

[0027] ).

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] By the way, although the road markings typified by the above-described "image hump" are recognized as effective, they are merely a response by road painting, so they lack a three-dimensional effect and have a problem that the effect of attracting attention is lower than that of an actual hump.

[0006] The present invention has been made to solve the above-described problems, and by displaying supplementary content that further prompts attention to road markings such as an image hump that prompts attention to the driver, the effect of prompting attention such as speed suppression is further enhanced, and an object is to provide a display control device and the like.

[0007] Other objects of the present invention will become apparent to those skilled in the art by referring to the embodiments and best mode exemplified below and the accompanying drawings.

Means for Solving the Problems

[0008] Hereinafter, in order to easily understand the outline of the present invention, embodiments according to the present invention will be exemplified.

[0009] A first aspect is a display control device that controls a head-up display device that projects an image onto a display area virtually set in front of a vehicle, superimposes the image on a forward field of view of the vehicle, and allows an occupant to visually recognize the image, the display control device recognizing a specific road marking that prompts attention to the occupant from a captured image obtained by capturing the forward field of view of the vehicle, generating content that further prompts attention to the occupant according to the recognized specific road marking, and performing control to superimpose the content on the specific road marking and display it on the head-up display device, and having a control unit.

[0010] In the first aspect, the control unit generates content that prompts further attention to the occupant (mainly the driver of the vehicle) according to a specific road marking recognized from the captured image, and performs control to superimpose the generated content on the specific road marking and display it on the head-up display device. Here, the "specific road marking" refers to, for example, an image hump using a stereoscopic illusion effect or a dotted line using visual narrowing. Also, the "content that prompts further attention" is, for example, the content that prompts attention such as speed suppression, shown in FIGS. 4(a), (b), and (c) as an example, and is expressed by a 3D image with a three-dimensional effect that imitates a specific road marking, a 2D image with a sense of perspective, or character information. Therefore, according to the first aspect, information (three-dimensional effect and sense of perspective) that is difficult to obtain from a specific road marking can be visually recognized by the occupant through the content that prompts further attention, so that the effect of prompting attention such as speed suppression can be further enhanced and contribute to safe driving.

[0011] In a second aspect subordinate to the first aspect, when the control unit does not recognize the specific road marking from the captured image acquired again after displaying the content, the control unit may perform control to stop the display of the content on the head-up display device.

[0012] In the second aspect, after the control unit displays the content that prompts further attention, when the specific road marking is not recognized from the captured image acquired again, the control unit performs control to stop the display of the content. Therefore, since the display of the content that prompts further attention is stopped in a state where the specific road marking is not shown, a forward field of view can be secured accordingly, and it can contribute even more to safe driving.

[0013] In a third aspect dependent on the first aspect, after the control unit displays the content, when the specific road marking is recognized from the captured image acquired again, the control unit obtains a difference between the position information of the specific road marking acquired immediately before and the position information of the specific road marking newly acquired, and based on the difference, may perform control to relocate the content in the display area and display it on the head-up display device.

[0014] In the third aspect, after the control unit displays content that prompts further attention, when a specific road marking is recognized from the captured image acquired again, the control unit performs control to change and display the display position of supplementary content that prompts further attention based on the difference between the position information of the specific road marking acquired immediately before and the position information of the specific road marking newly acquired. Therefore, according to the third aspect, the supplementary content that prompts further attention can follow the movement of the specific road marking and be displayed in a superimposed manner. In this case, it is possible to prevent the divergence between the specific road marking that prompts attention and the supplementary content that prompts further attention, and it is possible to more effectively prompt the driver to pay attention, including speed suppression.

[0015] In a fourth aspect dependent on the first aspect, as the vehicle moves, the control unit changes the display position of the specific road marking and displays it on the head-up display device, causes the content to follow the movement of the specific road marking, and displays it in a superimposed manner on the head-up display device. When the specific road marking disappears from the forward field of view, the control unit may perform control to fade out the specific road marking and the content from the display area.

[0016] In the fourth aspect, the control unit changes and displays the display position of a specific road marking as the vehicle moves, and causes content that prompts further attention to follow the movement of the specific road marking and superimposes and displays it. When the specific road marking disappears from the forward field of view, the control unit performs control to fade out the specific road marking and the supplementary content from the display area, thereby preventing the divergence between the road marking and the content and not making the occupant feel unnatural, and effectively prompting attention including speed suppression.

[0017] In a fifth aspect that depends on any one of the first to fourth aspects, the control unit may perform control to generate the content composed of a 3D image imitating the specific road marking, a 2D image imitating the specific road marking, or character information and display it on the head-up display device.

[0018] In the fifth aspect, the control unit generates content that prompts further attention, which is composed of a 3D image imitating the specific road marking, a 2D image imitating the specific road marking, or character information, and performs control to superimpose and display it on the specific road marking. Therefore, according to the fifth aspect, the content that prompts further attention is not limited to a 3D image that is displayed three-dimensionally, and by expressing it in a planar display or character information that gives a sense of perspective according to the driving environment of the vehicle, the readability when the occupant visually recognizes it can be ensured.

[0019] Here, the "3D image" refers to, for example, as shown in FIG. 1, an object that has one end on the XY plane of the display distance 103 and the other end on the XY plane of the display distance 104 that is farther from the occupant than the display distance 103 and is imaged as a three-dimensional object having a volume, and makes the occupant feel a three-dimensional sense. The "2D image" refers to, for example, as shown in FIG. 1, an object that has one end on the XY plane of the display distance 103 and the other end on the XY plane of the display distance 104 that is farther from the occupant than the display distance 103 and is imaged two-dimensionally, and makes the occupant feel a sense of perspective. Since the entire display is clearly recognized by adjusting the focus of the occupant's eyes, it refers to a planar image that can give a three-dimensional impression.

[0020] A sixth aspect is a head-up display device that projects an image onto a display area virtually set in front of a vehicle, superimposes the image on a forward field of view of the vehicle, and allows an occupant to visually recognize the image. The head-up display device includes an image display unit that displays the image, and a control unit that recognizes a specific road marking that prompts the occupant to pay attention from a captured image obtained by capturing the forward field of view of the vehicle, generates content that further prompts the occupant to pay attention according to the recognized specific road marking, and performs control to superimpose the content on the specific road marking and display the content on the image display unit.

[0021] In the sixth aspect, the control unit recognizes a specific road marking from the acquired captured image, generates content that further prompts the occupant to pay attention from the recognized specific road marking, and performs control to superimpose the generated content on the specific road marking and display the content on the image display unit. Therefore, information (such as three-dimensional effect and perspective) that is difficult to obtain from the specific road marking can be visually recognized by the occupant, and a head-up display device that can further enhance the effect of prompting attention such as speed suppression and contribute to safe driving can be provided.

[0022] A seventh aspect is a program for a display control device that controls a head-up display device that projects an image onto a display area virtually set in front of a vehicle, superimposes the image on a forward field of view of the vehicle, and allows an occupant to visually recognize the image. The program causes a processor included in the display control device to execute a process of recognizing a specific road marking from a captured image obtained by capturing the forward field of view of the vehicle, a process of generating content that further prompts the occupant to pay attention according to the recognized specific road marking, and a process of superimposing the content on the specific road marking and displaying the content on the head-up display device.

[0023] In the seventh aspect, in order to perform control to recognize a specific road marking from the acquired captured image, generate content that further alerts the driver from the recognized specific road marking, and display the generated content superimposed on the specific road marking, it is possible to provide a program that allows the driver to visually recognize information (such as three-dimensional and perspective) that is difficult to obtain from the road marking, further enhance the effect of alerting such as speed suppression, and contribute to safe driving.

[0024] The eighth aspect is a vehicle display system including an imaging device that captures the front view of the vehicle, a head-up display device that projects an image onto a display area virtually set in front of the vehicle and superimposes the image on the front view of the vehicle for the driver to visually recognize, and a display control device that controls the head-up display device. The display control device recognizes a specific road marking that alerts the driver from the captured image acquired from the imaging device, generates content that further alerts the driver according to the recognized specific road marking, and performs control to superimpose the content on the specific road marking and display it on the head-up display device.

[0025] In the eighth aspect, since the display control device generates content that further alerts the driver from a specific road marking obtained by recognizing the captured image acquired from the imaging device, and performs control to superimpose the generated content on the specific road marking and display it on the head-up display device, it is possible to allow the driver to visually recognize information (such as three-dimensional and perspective) that is difficult to obtain from the road marking, further enhance the effect of alerting such as speed suppression, and provide a vehicle display system that contributes to safe driving.

Brief Description of the Drawings

[0026]

Figure 1

Figure 2

Figure 3A

Figure 3B

Figure 4

Figure 5

Mode for Carrying Out the Invention

[0027] The best mode described below is used to easily understand the present invention. Therefore, those skilled in the art should note that the present invention is not unduly limited by the embodiments described below (hereinafter referred to as the present embodiments).

[0028] [Configuration of the Embodiment] (Configuration of the Vehicle Display System 300) FIG. 1 is a diagram cited to explain the configuration of the vehicle display system 300 of the present embodiment and the image displayed by the head-up display device (hereinafter simply referred to as the HUD device 10). In FIG. 1, the front-rear direction of the vehicle 1 is the Z direction (the front direction is the positive Z direction), the direction along the left-right direction (the width direction of the vehicle 1) of the vehicle 1 is the X direction (the left direction is the positive X direction), and the up-down direction is the Y direction (the up direction is the positive Y direction).

[0029] In FIG. 1, the vehicle display system 300 of the present embodiment includes a storage device 20, a display control device 30, a peripheral monitoring device 50, a communication device 60, and a vehicle ECU (Electronic Control Unit) 80, which are connected via an I / O interface 90 so as to be able to communicate bidirectionally with each other.

[0030] The storage device 20 is composed of, for example, a static RAM, a DRAM, and a flash RAM, and images related to specific road markings such as image humps and image dots are stored as a database (road marking DB 200). Note that the specific road markings may include, for example, polygon graphic data in order to realize a three-dimensionally decorated display. Note that the storage device 20 may be incorporated in the display control device 30 without being connected via the I / O interface 90. The road marking DB 200 is used as a means for pattern matching when the display control device 30 (control unit 31) recognizes whether a specific road marking is included in the captured image of the front view of the vehicle 1.

[0031] The display control device 30 controls the HUD device 10 that projects an image onto a display area VA virtually set in front of the vehicle 1 and overlays the image on the front view of the vehicle 1 for the occupant to visually recognize. Specifically, the display control device 30 recognizes a specific road marking that prompts the occupant's attention, such as an image hump or a dot line, from the captured image obtained by capturing the front view of the vehicle 1, generates content (hereinafter referred to as supplementary content) that further prompts the occupant's attention according to the recognized specific road marking, and performs control to display the supplementary content superimposed on the specific road marking on the HUD device 10. More details will be described later.

[0032] The peripheral monitoring device 50 includes, in addition to a camera (also referred to as an imaging unit, including a camera that captures the face (eyes) of the occupant) that captures the surrounding driving environment including the front view of the vehicle 1, LiDAR (Light Detection And Ranging) and millimeter-wave radar. Here, the camera captures at least the front view of the vehicle 1. The LiDAR uses near-infrared light, visible light, or ultraviolet light to irradiate light, for example, on an obstacle existing in front of the vehicle 1 captured by the camera, captures the reflected light with an optical sensor, and determines the distance to the obstacle based on the time difference. Also, the millimeter-wave radar can measure the distance, speed, or angle to an obstacle or the like using a radar in the millimeter-wave band. The detection information detected by the peripheral monitoring device 50 is transferred to the display control device 30 of the present embodiment via the I / O interface 90.

[0033] The communication device 60 can acquire map data and others through communication with an external center (not shown) via a V2X (Vehicle to X)-type communication system (not shown). Here, the map data is mapping data that is digitized to represent the driving environment of the vehicle 1. As the mapping data, it is preferably digital data of a particularly high-precision dynamic map. Note that the dynamic map is a digital map that combines a vast amount of dynamic information that changes moment by moment, such as traffic regulations, construction information, accidents, traffic jams, pedestrian and signal information, and static information such as high-precision three-dimensional position information (road surface information, lane information, three-dimensional structures).

[0034] The vehicle ECU 80 is, for example, a driving support ECU (electronic control unit) that supports the driver's operation, and includes at least one of a driving support function for supporting the driver's driving operation and an automatic driving function capable of substituting for the driver's driving operation. Based on the detection information acquired from the surrounding monitoring device 50, it recognizes the driving environment in front of the vehicle 1, and in addition to controlling the vehicle 1 according to the analysis result of the recognized detection information, it is also possible to provide various display information indicating the state of the vehicle 1 to the display control device 30 of the present embodiment via the I / O interface 90. As the vehicle ECU 80, an electronic control unit that controls a drive system such as an engine or a motor, a braking system such as a brake, and a steering system such as a steering is also connected.

[0035] Although not shown in FIG. 1, the I / O interface 90 is also connected to a behavior sensor including an IMU (Inertial Measurement Unit) that detects the behavior of the vehicle 1, a vehicle speed sensor, or a steering angle sensor that detects the steering angle of the steering wheel. The IMU can measure (detect translational motion in three axial directions from acceleration [m / s 2 , and rotational motion from angular velocity [deg / s]) the driving situation and attitude of the vehicle 1 using a three-axis acceleration sensor and a three-axis angular velocity sensor (gyro sensor). The information detected by these behavior sensors is transferred to the vehicle ECU 80 via the I / O interface 90 and then to the display control device 30 of the present embodiment.

[0036] The I / O interface 90 performs communication (also referred to as CAN communication) between the communication device 60 and the vehicle ECU 80, in addition to the display control device 30 of the present embodiment, according to, for example, the CAN (Controller Area Network) standard. Note that the communication standard adopted by the I / O interface 90 is not limited to CAN. For example, it includes wired communication interfaces such as CAN FD (CAN with Flexible Data Rate), LIN (Local Interconnect Network), Ethernet (registered trademark), MOST (Media Oriented Systems Transport: MOST is a registered trademark), UART (Universal Asynchronous Receiver Transmitter), or USB (Universal Serial Bus), or in-vehicle communication (internal communication) interfaces that are short-range wireless communication interfaces within several tens of meters, such as personal area networks (PAN) like Bluetooth (registered trademark) networks and local area networks (LAN) like 802.11x Wi-Fi (registered trademark) networks.

[0037] Further, the I / O interface 90 may include an out-of-vehicle communication (external communication) interface such as a wide-area communication network (e.g., an Internet communication network) according to cellular communication standards such as wireless wide-area network (WWAN0, IEEE802.16-2004 (WiMAX: Worldwide Interoperability for Microwave Access)), IEEE802.16e-based (Mobile WiMAX), 4G, 4G-LTE, LTE Advanced, 5G.

[0038] [Configuration of the display control device 30] As shown in FIG. 1, the display control device 30 of this embodiment is composed of a control unit 31 and an image storage unit 32. In the image storage unit 32, in addition to the image representing the supplementary content generated by the control unit 31, there are an image showing a specific road surface marking recognized by the control unit 31, an image representing a specific road surface marking which is a display element generated by the control unit 31, display information such as an image (virtual image) representing the supplementary content, etc. are stored. Also, in the image storage unit 32, a VRA (Video RAM) area is allocated to a partial area, and the display elements of specific road surface markings and supplementary content (virtual images) laid out in this VRAM area (a copy of the display elements set in the display area VA) are written and displayed on the HUD device 10. Note that the display positions of these display elements laid out in the display area VA are sequentially updated as the vehicle 1 moves.

[0039] The control unit 31 can recognize a captured image obtained by photographing the front view of the vehicle 1 by the display control device 30 or a specific road surface marking acquired from the outside via the communication device 60, and generate supplementary content (virtual image) represented by a 3D image or the like that prompts the occupant to take further caution according to the recognized specific road surface marking, and perform control to superimpose the supplementary content generated for the specific road surface marking and display it on the HUD device 10.

[0040] Here, the "specific road surface marking" is, for example, a road surface marking RS such as an image hump using a stereoscopic illusion effect as shown in FIGS. 4(a), (b), and (c) for an example, and the "content that prompts further caution" is supplementary content AC (AC', AC'') that prompts caution such as speed suppression, and this supplementary content AC (AC', AC'') is represented by a 3D image, 2D image, character information, or the like that can obtain a three-dimensional sense and a sense of perspective.

[0041] Also, when the control unit 31 does not recognize the specific road surface marking RS from the captured image acquired again after displaying the supplementary content AC (AC', AC''), the control unit 31 can perform control to stop the display of the supplementary content AC (AC', AC'') on the HUD device 10.

[0042] Further, after the supplementary content AC (AC´, AC´´) is displayed, when a specific road marking RS is recognized from the captured image acquired again, the control unit 31 obtains the difference between the position information of the specific road marking RS acquired immediately before and the position information of the newly acquired specific road marking RS, and based on the difference, relocates the supplementary content AC (AC´, AC´´) to the HUD device 10 and performs control to display it on the HUD device 10.

[0043] In addition, as the vehicle 1 moves, the control unit 31 changes the display position of the specific road marking RS and displays it on the HUD device 10, causes the supplementary content AC (AC´, AC´´) to follow the movement of the specific road marking RS, superimposes and displays it on the HUD device 10, and when the specific road marking RS disappears from the forward field of view, performs control to fade out the specific road marking RS and the supplementary content AC (AC´, AC´´) from the display area.

[0044] Further, the control unit 31 can perform control to generate supplementary content AC (AC´, AC´´) composed of a three-dimensional image (3D) imitating the specific road marking RS, a two-dimensional image imitating the specific road marking RS, or character information and display it on the HUD device 10.

[0045] Therefore, the control unit 31 includes a main control unit 310, a captured image acquisition unit 311, an image recognition unit 312, a display distance determination unit 313, a display position calculation unit 314, a display information generation unit 315, and a display control unit 316.

[0046] The captured image acquisition unit 311 outputs the captured image of the forward field of view of the vehicle 1 acquired via the peripheral monitoring device 50 (imaging unit) to the image recognition unit 312. The image recognition unit 312 recognizes a specific road marking RS such as an image hump from the captured image, and outputs the image of the recognized specific road marking RS and its position information, etc. to the main control unit 310.

[0047] The display distance determination unit 313 determines the display distance 100 of an image 70 including a road marking RS, which is a display element arranged in the display area VA, and supplementary content AC (AC', AC''), and specifically, it is possible to determine the display distance 100 of the image 70 (specifically, each image 71, 72, 73 within the image display area VA). The display distance determination unit 313 considers the position information of a specific road marking RS, such as an image hump recognized by the image recognition unit 312, or the driving situation of the vehicle 1 acquired via the peripheral monitoring device 50 or the vehicle ECU 80 (for example, various situations such as driving at night), and determines at what display distance and in what spatial arrangement to display the specific road marking RS and the supplementary content AC (AC', AC''), and outputs the result to the main control unit 310.

[0048] The control of the display distance refers to the control of how far in front of the vehicle 1 a specific road marking RS and supplementary content AC (AC´, AC´´) are displayed as seen by the occupant. That is, the control of the display distance means that the control unit 31 controls the HUD device 10 so that the specific road marking RS and the supplementary content AC (AC´, AC´´) are visually recognized at a position a predetermined distance ahead of the vehicle 1. Regarding the control of the display distance, it is realized by the control unit 31 performing display control on an image (here, the supplementary content AC (AC´, AC´´)) within the display area VA in a manner with depth. To display the supplementary content AC (AC´, AC´´) in a manner with depth, for example, based on the distance (position information) to the specific road marking RS, the supplementary content AC (AC´, AC´´) is drawn and controlled using a perspective method (for example, the one-point perspective method). Also, by tilting the projection surface of the HUD device 10, the virtual surface (display area VA corresponding to the projection surface) on which the supplementary content AC is displayed can be set to be tilted forward with respect to the vertical direction of the vehicle 1, thereby giving depth to the supplementary content AC (AC´, AC´´). In this case, considering the projection of the image to be visually recognized from the occupant's viewpoint onto the virtual surface, the control unit 31 may perform drawing control on the supplementary content AC (AC´, AC´´) in a manner with depth. Note that as the occupant's viewpoint position, the control unit 31 may use the viewpoint position set by default, or may use the viewpoint information obtained by photographing the occupant's face image with an in-vehicle camera (not shown) and recognizing the occupant's viewpoint position.

[0049] The display position calculation unit 314 calculates the display position so as to display an image of a specific road marking RS recognized at an arbitrary position within the display area VA and supplementary content AC, and controls the HUD device 10 so that the supplementary content AC (AC´, AC´´) is superimposed and visually recognized at the position of the recognized specific road marking RS, as shown in FIGS. 4(a), (b), and (c), for example. The display position calculation unit 314 displays the supplementary content AC (AC´, AC´´) in a virtual image in accordance with the position of the specific road marking RS recognized from the captured image (front view of the vehicle 1) acquired from the surrounding monitoring device 50 (imaging unit). To do so, the display position calculation unit 314 calculates the display position of the supplementary content AC (AC´, AC´´) based on the position of the recognized specific road marking RS and a preset correction value. Here, the correction value is a value set based on the deviation amount between the position of the instrument panel 4 where the viewpoint of the occupant sitting in the driver's seat is assumed to be located and the mounting position of the imaging unit (the camera of the surrounding monitoring device 50). That is, the display position calculation unit 314 calculates the display position when displaying the supplementary content AC (AC´, AC´´) on the projection plane (display area VA) based on the coordinate system with the mounting position of the imaging unit as a reference and the coordinate system with the position of the instrument panel 4 as a reference, that is, the display position based on the viewpoint position of the occupant. A technique for specifying the display position when displaying an image on the projection plane based on the above-described viewpoint position of the imaging unit and the viewpoint position of the occupant, that is, the display position when displaying an image based on the viewpoint position of the occupant, is described in detail in, for example, Japanese Patent Application Laid-Open No. 2021-142770.

[0050] Incidentally, since the viewpoint position of the imaging unit is different from the viewpoint position of the occupant, there is also a known technique of converting a display image generated at the viewpoint position of the imaging unit into a display image adjusted to the viewpoint position of the occupant using three-dimensional view conversion (see, for example, Japanese Patent Application Laid-Open No. 2017-185988). View conversion is a drafting method for expressing an image generated in a state seen from a certain viewpoint position in a state seen from a viewpoint position in a different three-dimensional space. Specifically, using a parallel transformation matrix, the position of the image in the reference coordinate system in which the figure is generated is translated in parallel so as to match the coordinate system based on the viewpoint position of the person viewing the figure, and using a rotation transformation matrix, the image after parallel translation is rotated toward the viewpoint position. By using this view conversion generally used in the field of three-dimensional graphics, it becomes possible to generate, convert, and display an image in real time so as to overlap the occupant's field of view following changes in the forward field of view accompanying the movement of the vehicle 1 and changes in the occupant's viewpoint position.

[0051] The display information generation unit 315 generates supplementary content AC (AC', AC'') that prompts the occupant to pay more attention according to the specific road marking RS recognized by the image recognition unit 312 and outputs it to the main control unit 310. Based on this supplementary content AC (AC', AC''), the HUD device 10 (projection unit 40: see FIG. 2) can be driven. The supplementary content AC includes display distance data for driving the projection unit 40 to display the image 70 at the display distance 100 determined by the display distance determination unit 313. Incidentally, the supplementary content AC (AC', AC'') generated here is, for example, a 3D image AC imitating a specific road marking RS as shown in an example of its expression form in FIG. 4(a) (the image and shape match those of the road marking RS stored in the road marking DB200 of the storage device 20), or a similar 3D image AC' that does not reach the shape match as shown in an example of its expression form in FIG. 4(b), or a 2D image AC'' imitating a specific road marking RS or character information "!" that prompts attention as shown in an example of its expression form in FIG. 4(c).

[0052] The display control unit 316 reads out, in synchronization with the display timing, display information (screen information stored in a display area VA having specific road markings RS and supplementary contents AC (AC', AC'' ) as display elements), which is written in synchronization with the update timing by the display information generation unit 315, from a VRAM area assigned to a partial area of the image memory unit 32, and displays it on the HUD device 10. Further, when a specific road marking RS disappears from the forward field of view, the display control unit 316 also performs control to fade out the specific road marking RS and the supplementary contents AC (AC', AC'' ) from the display area VA.

[0053] Note that, in addition to serving as an interface for information exchange with the storage device 20, the peripheral monitoring device 50, and the communication device 60, which are connected via the I / O interface 90, the main control unit 310 realizes a function for performing control to generate supplementary contents AC composed of, for example, a 3D image that recognizes a specific road marking RS based on a captured image obtained by capturing the forward field of view of the vehicle 1 or information acquired from the outside via the communication device 60 and prompts the occupant to pay further attention according to the recognized specific road marking RS, and to display, on the HUD device 10, the generated supplementary contents AC superimposed on the recognized specific road marking RS. To this end, the main control unit 310 performs sequence control of the above-described captured image acquisition unit 311, the image recognition unit 312, the display distance determination unit 313, the display position calculation unit 314, the content generation unit 315, and the display control unit 316.

[0054] Note that, in order to perform the above-described control, the control unit 31 has, for example, a processor with a built-in memory (ROM / RAM) or an externally attached processor. Based on a captured image obtained by capturing the forward view of the vehicle 1 or information acquired from the outside via the communication device 60 by the processor, a specific road marking RS is recognized, and when supplementary content AC (AC′, AC″) consisting of a 3D image or the like that prompts the occupant to pay further attention according to the recognized specific road marking RS is generated, a graphics controller that superimposes the generated supplementary content AC (AC′, AC″) on the recognized specific road marking RS and displays it on the HUD device 10 is included. Then, the processor executes a program recorded in the ROM and operates in cooperation with the graphics controller (GDC) to execute each of the above-described functions. Further, at least a part of the above-described functions can also be realized by hardware such as an FPGA (Field Programmable Gate Array) or a logic circuit without depending on the processor or the GDC.

[0055] (Configuration of HUD device 10) The HUD device 10 has a projection unit 40 that projects display light L. The HUD device 10 projects the display light L toward the projection target unit 2 that transmits part of the light and reflects part of the light. This projection target unit 2 is constituted by a part of the front windshield of the vehicle 1. An eyebox 4 can be formed by the display light L reflected by the projection target unit 2 in a region where the eyes 3 of the occupant are assumed to be placed. By placing the eyes 3 inside this eyebox 4, the occupant can visually recognize the entire image 70 displayed by the HUD device 10, and when the eyes 3 move out of the eyebox 4, a part of the image 70 becomes unrecognizable (difficult to recognize). The HUD device 10 can adjust the positions where the images 71, 72, 73 are displayed in the X direction, Y direction, and Z direction. That is, the HUD device 10 can particularly adjust the display distance 100, which is the distance in the depth direction Z until the images 71, 72, 73 are formed from the eyebox 4, and can arrange (display) the images 71, 72, 73 in the display area VA, which is a three-dimensional space.

[0056] (Description of the Image) Next, the image 70 displayed by the HUD device 10 will be described. The image 70 has, for example, a first 2D image 71 and a second 2D image 72 that are represented two-dimensionally, and a 3D image 73 that is represented three-dimensionally. The first 2D image 71 is imaged two-dimensionally on the XY plane at the display distance 101 and is an image that does not give the occupant a sense of perspective. According to such a first 2D image 71, the entire display can be clearly recognized generally without adjusting the focus of the eyes. Further, the second 2D image 72 has one end on the XY plane at the display distance 103 and the other end on the XY plane at the display distance 104 that is farther from the occupant than the display distance 103, is imaged two-dimensionally, and is an image that gives the driver a sense of perspective. Such a second 2D image 72 can give a three-dimensional impression because the entire display is clearly recognized by adjusting the focus of the driver's eyes. An example of the 3D image 73 has one end on the XY plane at the display distance 102 and the other end on the XY plane at the display distance 103 that is farther from the driver than the display distance 102, and is imaged as a three-dimensional object having a volume, and is an image that gives the driver a sense of three-dimensionality.

[0057] In other words, increasing the display distance 100 means imaging the image 70 at a position farther from the instrument box 4. For example, it means bringing the second 2D image 72 displayed at the display distance 101 and the 3D image 73 displayed between the display distances 102 and 103 closer to the display distance 104 on the far side. Also, decreasing the display distance 100 means, in other words, imaging the image 70 at a position closer to the instrument box 4. For example, it means bringing the second 2D image 72 displayed at the display distance 104 and the 3D image 73 displayed between the display distances 102 and 103 closer to the display distance 101 on the near side.

[0058] (Configuration of the Projection Unit 40) Next, refer to FIG. 2. FIG. 2 is a diagram showing a configuration example of the projection unit 40 in FIG. 1. The projection unit 40 included in the HUD device 10 is composed of a stereoscopic image display unit 41 and a relay optical unit 42. The stereoscopic image display unit 41 forms a first 3D real image 43, and the relay optical unit 42 enlarges the first 3D real image 43 to form a second 3D real image 44, and projects the display light L of this second 3D real image 44 toward the external projection unit 2.

[0059] The stereoscopic image display unit 41 in FIG. 2 generates a first 3D real image 43 formed in three dimensions, and includes an image projection unit 45 and a vibrating screen 46.

[0060] The image projection unit 45 is a projector that emits video light (not shown) representing the image included in the display data based on the display data input from the display control device 30. According to the display distance data included in the display data, the timing of displaying the image is adjusted, and the image projected in synchronization with the vibration position of the vibrating screen 46 is switched at high speed. In other words, the image projection unit 45 projects an image suitable for the vibration position of the vibrating screen 46 onto the vibrating screen 46 based on the display distance data.

[0061] The vibrating screen 46 is, for example, a polycarbonate diffusing film that diffuses the video of the image projection unit 45 within a certain angular range. It receives the display light from the image projection unit 45 to form a real image, and reciprocates along the optical axis of the display light emitted by the image projection unit 45. The vibrating screen 46 can transmit a signal indicating the vibration position to the image projection unit 45 continuously or intermittently. The image projection unit 45 may adjust the timing of displaying the image 70 so that the image 70 is visually recognized at a position corresponding to the display distance data included in the display data based on the signal indicating the vibration position.

[0062] The HUD device 10 adjusts the length of the image 70 in the depth direction Z while keeping the amplitude of the vibrating screen 46 in the screen vibration direction constant under the control of the display control device 30 of the present embodiment. That is, the display light L is not emitted during the period when the vibrating screen 46 is located in the first range of its amplitude, and the display light L is emitted during the period when the vibrating screen 46 is located in the second range of its amplitude. By adjusting the positions and ratios of the first range and the second range, the position and length of the image 70 in the depth direction Z are adjusted. Specifically, the vibrating screen 46 vibrates at a frequency of 60 [Hz] or more, and within this period of 1 / 60 [sec], the image projection unit 45 projects different videos of a plurality of frames to form different videos (real images) of a plurality of frames at each vibration position. That is, the HUD device 10 can generate the first 3D real image 43 by overlapping the real images of a plurality of frames in the vibration direction of the vibrating screen 46. Note that the HUD device 10 may adjust the length of the image 70 in the depth direction Z by changing the amplitude of the vibrating screen 46.

[0063] The relay optical unit 42 in FIG. 2 receives the light of the first 3D real image 43 generated by the stereoscopic image display unit 41, forms an intermediate second 3D real image 44 by magnifying the first 3D real image 43, and then projects the display light L, which is the light of the second 3D real image 44, toward the projection unit 2. The relay optical unit 42 includes, for example, a first relay optical unit 47 composed of a lens group that receives the light of the first 3D real image 43 generated by the stereoscopic image display unit 41, a second relay optical unit 48 that reflects the light passing through the first relay optical unit 47 and forms a second 3D real image 44 by magnifying the first 3D real image 43 in cooperation with the optical power of the first relay optical unit 47, and a third relay optical unit 49 that reflects the display light L, which is the light of the second 3D real image 44, toward the projection unit 2.

[0064] The first relay optical unit 47 has a function of magnifying each video imaged at each vibration position of the vibrating screen 46 in the first 3D real image 43 at different magnifications. In FIG. 2, it is schematically illustrated as a single lens, but in reality, it is composed of a composite lens formed by synthesizing a plurality of thin film lenses not shown.

[0065] The second relay optical unit 48 is composed of, for example, a mirror having a concave reflecting surface with positive optical power, receives the light of the first 3D image 43 from the first relay optical unit 47, reflects the incident light toward the third relay optical unit 49, and forms a second 3D image 44 that enlarges the first 3D image 43 in cooperation with the optical power of the first relay optical unit 47 between the second relay optical unit 48 and the third relay optical unit 49. Note that the second relay optical unit 48 may be omitted by giving the first relay optical unit 47 the optical action that the second relay optical unit 48 is responsible for.

[0066] The third relay optical unit 49 is a concave mirror that reflects the display light L of the second 3D image 44 toward the projection unit 2, and has a function of correcting image distortion due to the curved surface shape of the projection unit 2 and a function of enlarging the second 3D image 44.

[0067] Note that the projection unit 40 may adopt a known 3D display method such as a parallax division method including a parallax barrier method or a lenticular lens method, a spatial reproduction method including a light field method or a hologram method, for example, a transmissivity adjustment screen method in which a plurality of screens having a light control layer with adjustable transmissivity are stacked in the thickness direction as disclosed in JP-A-2016-212318, and a projector projects an image while rapidly switching the projected image toward the plurality of screens, and in response to the rapid switching of the projected image, the plurality of screens respectively adjust the light control rate as appropriate to display a 3D real image inside, for example, a method in which a plurality of liquid crystal display elements are stacked in the thickness direction to display a 3D real image inside as disclosed in JP-A-2004-168230, or a light field display method.

[0068] [Operation of the Embodiment] FIG. 3A and FIG. 3B are flowcharts showing the main operation procedures of the display control device 30 according to the present embodiment. Further, FIG. 4 is a diagram showing an example of the screen layout of display elements generated by the display control device 30 according to the present embodiment. FIG. 4(a) shows a display example when supplementary content is represented in 3D, FIG. 4(b) shows a display example when similar supplementary content is represented in 3D, and FIG. 4(c) shows a display example when the supplementary content is represented in 2D and further represented by characters. In FIGS. 4(a), 4(b), and 4(c), RS is an image showing a specific road marking, AC is supplementary content represented by a 3D image having a shape that matches a shape imitating the specific road marking RS, AC' is supplementary content represented by a similar 3D image that does not reach shape matching, and AC'' is supplementary content represented by a 2D image imitating the specific road marking RS and to which character information is also added. Also, LA indicates a lane.

[0069] Hereinafter, with reference to FIGS. 3A, 3B, 4(a), 4(b), and 4(c), the operation of the display control device 30 of the present embodiment shown in FIG. 1 will be described in detail.

[0070] In FIG. 3A, the display control device 30 of the present embodiment, under the sequence control by the main control unit 310, first, the imaging image acquisition unit 311 acquires an image of the front view of the vehicle 1 photographed by the peripheral monitoring device 50 (imaging unit) (step ST101). Next, when the image recognition unit 312 recognizes a specific road marking RS that prompts the occupant to pay attention, such as an image hump, from the acquired image of the front view of the vehicle 1 by image processing such as pattern matching with the images stored in the road marking DB200 of the storage device 20 (step ST102 “YES”), the display information generation unit 315 generates supplementary content AC (AC', AC'') represented by a 3D image or the like that further prompts the occupant to pay attention (step ST103).

[0071] Here, the "supplementary content that prompts further attention" is, based on the fact that the recognized captured image (specific road marking RS), even if its validity is recognized, is ultimately a response by road painting and thus lacks a three-dimensional effect, a 3D image or a 2D image that emphasizes the three-dimensional effect and imitates the specific road marking RS, such as those shown in FIGS. 4(a), 4(b), and 4(c) described later. Note that it may include character information.

[0072] Note that when a specific road marking RS is not recognized (step ST102 "NO"), the process of step ST101 is repeatedly executed until a specific road marking is recognized.

[0073] Next, the display distance determination unit 313 determines the display distance 100 of the image 70 including the road marking RS and the supplementary content AC (AC', AC'') which are display elements arranged in the display area VA. Specifically, it determines the display distance 100 of the image 70 (specifically, each image 71, 72, 73 within the image display area VA). The display distance determination unit 313 considers the position information of a specific road marking RS, such as an image hump recognized by the image recognition unit 312, in the forward field of view, or the driving situation of the vehicle 1 (for example, various situations such as driving at night) acquired via the surrounding monitoring device 50 or the vehicle ECU 80, and determines at what display distance and in what spatial arrangement to display the specific road marking RS and the supplementary content AC (AC', AC''), and outputs the result to the main control unit 310 (step ST104).

[0074] Also, in step ST104, the display position calculation unit 314 calculates the display position so as to superimpose and display the supplementary content AC (AC', AC'') generated by the display information generation unit 315 on the image of the recognized specific road surface marking RS at an arbitrary position within the display area VA. The display distance determined by the display distance determination unit 313 and the display position calculated by the display position calculation unit 314 are information necessary to control the HUD device 10 so that the supplementary content AC, AC', AC'' is visually recognized as superimposed on the specific road surface marking RS, as shown in FIGS. 4(a), (b), and (c), for example, at the position of the specific road surface marking RS recognized by the image recognition unit 312. Note that the display position calculation unit 314 displays the supplementary content AC (AC', AC'') in a virtual image in accordance with the position of the specific road surface marking RS recognized from the captured image (front view of the vehicle 1) acquired from the surrounding monitoring device 50 (imaging unit). Therefore, the display position of the supplementary content AC (AC', AC'') is calculated based on the position information of the recognized specific road surface marking RS and a preset correction value. Here, the correction value is a value set based on the deviation amount between the position of the glove box 4 where the viewpoint of the occupant seated in the driver's seat is assumed to be located and the mounting position of the imaging unit (surrounding monitoring device 50). That is, the display position calculation unit 314 calculates the display position when displaying the supplementary content AC (AC', AC'') on the projection plane (display area VA) based on the coordinate system with the mounting position of the imaging unit as a reference and the coordinate system with the position of the glove box 4 as a reference, that is, the display position based on the viewpoint position of the occupant.

[0075] Subsequently, under the sequence control of the main control unit 310, the display control unit 316 waits for the arrival of the display timing (step ST105 “YES”), drives the projection unit 40 (see FIG. 2) of the HUD device 10, and superimposes the supplementary content AC (AC', AC'') on the specific road marking RS and displays it on the HUD device 10 (step ST106). The supplementary content AC (AC', AC'') includes information on the display distance for driving the projection unit 40 to display the image 70 at the display distance 100 determined by the display distance determination unit 313. The image projection unit 45 adjusts the timing of displaying the image 70 according to the display distance information included in the display information input from the display control unit 316, and rapidly switches the image 70 projected in synchronization with the vibration position of the vibration screen 46. In other words, based on the display distance 100, the image projection unit 45 adjusts the length of the image 70 in the depth direction Z while keeping the amplitude of the vibration screen 46 in the vibration screen direction constant. That is, the display light L is not emitted during the period when the vibration screen 46 is located within the first range of its amplitude, and the display light L is emitted during the period when the vibration screen 46 is located within the second range of its amplitude. By adjusting the positions and ratios of the first range and the second range, the position and length of the image 70 in the depth direction Z are adjusted.

[0076] As described above, the display control device 30 of the present embodiment generates and displays, in real time, an image showing a specific road marking RS that has been recognized and its supplementary content AC (AC', AC'') so as to overlap the occupant's field of view following the change in the forward field of view accompanying the movement of the vehicle 1 and the change in the viewpoint position of the occupant. At this time, as the vehicle 1 travels in its traveling direction, the specific road marking RS is displayed approaching the front side of the screen. However, the display position of the supplementary content AC (AC', AC'') is changed so as to move to the front side of the screen in accordance with the specific road marking RS on the forward field of view so as to overlap and follow it. When the specific road marking RS disappears from the forward field of view (step ST107 “YES”), the display control unit 316 performs control to fade out the supplementary content AC from within the display area VA and finally make it invisible (for example, gradually reduce the luminance) (step ST108). Note that the display control unit 316 may perform the above-described fade-out in combination with a slide-out that erases the specific road marking RS and the supplementary content AC (AC', AC'') so as to be cut off outside the display area VA.

[0077] Next, in FIG. 3B, after the display control unit 316 of the display control device 30 of the present embodiment superimposes and displays the supplementary content AC (AC', AC'') on the specific road marking RS on the HUD device 10, under the sequence control by the main control unit 310, the imaging image acquisition unit 311 acquires again the image of the front view of the vehicle 1 captured by the peripheral monitoring device 50 (imaging unit) (step ST109). Then, the image recognition unit 312 recognizes from the acquired image of the front view of the vehicle 1 a specific road marking RS that prompts the driver's attention through image processing such as pattern matching with the image stored in the road marking DB200 of the storage device 20. Here, when the specific road marking RS can be recognized (step ST110 “YES”), the difference between the position information of the specific road marking RS acquired immediately before and the position information of the newly acquired specific road marking RS is calculated, and based on this difference, the display information generation unit 315 controls the supplementary content AC (AC', AC'') generated to be rearranged in the display area VA (step ST111). In response to this, the display control unit 316 waits for the arrival of the display timing (step ST112 “YES”), and controls to superimpose the supplementary content AC (AC', AC'') on the image showing the rearranged specific road marking RS and display it on the HUD device 10 (step ST113).

[0078] On the other hand, in step ST110, when the specific road marking RS is not recognized from the captured image acquired again (step ST110 “NO”), the display control unit 316 controls to stop the display of the supplementary content AC (AC', AC'') on the HUD device 10 (step ST114). In this way, since the display of the supplementary content AC (AC', AC'') that prompts further attention is stopped in a state where the specific road marking RS is not shown, the driver can secure the front view accordingly, which can further contribute to safer driving.

[0079] As described above, according to the display control device 30 of the present embodiment, the control unit 31 generates supplementary content AC (AC', AC'') that prompts the occupant to pay further attention according to a specific road marking RS recognized from the captured image, and superimposes the generated supplementary content AC (AC', AC'') on the specific road marking RS and controls the HUD device 10 to display it. Therefore, information (stereoscopic effect and perspective) that is difficult to obtain from an image showing the specific road marking RS can be visually recognized by the occupant through the supplementary content AC (AC', AC''), so that the effect of attention prompting such as speed suppression can be further enhanced, contributing to safe driving.

[0080] (Modification example) According to the display control device 30 of the above-described present embodiment, the display control device 30 (control unit 31) generates supplementary content AC (AC', AC'') that prompts the occupant to pay further attention according to a specific road marking RS recognized from the captured image, and superimposes the generated supplementary content AC (AC', AC'') on this specific road marking RS and controls the HUD device 10 to display it. However, by incorporating these functions into the HUD device 10, that is, by incorporating a function of capturing a captured image of the forward field of view, a function of recognizing the captured image, and a function of generating supplementary content AC (AC', AC''), superimposing it on a specific road marking RS, the functions of the display control device 30 can be realized by the HUD 10 alone. In this case, the processing load of the display control device 30 can be reduced.

[0081] In this case, the HUD device 10A of the present embodiment is configured by, for example, a control unit 10a and an image display unit 10b as shown in FIG. 5. The control unit 10a generates supplementary content AC (AC', AC'') that prompts the occupant to pay further attention according to a specific road marking RS recognized from the captured image, and superimposes the generated supplementary content AC (AC', AC'') on this specific road marking RS and controls the image display unit 10b to display it.

[0082] Note that the image display unit 10b mainly includes a light source composed of light-emitting diodes mounted on a wiring board, a projection unit 110 (having the same configuration as the projection unit 40 in FIG. 2) including a relay optical system, and a TFT (Thin Film Transistor Liquid Crystal) type liquid crystal display element 111 located on the emission side (directly above) of the light source so as to form display light by transmitting illumination light from the light source. The liquid crystal display element 111 can output display light by transmitting the light emitted from the light source. The liquid crystal display element 111 forms a desired image based on display image data (driving signal) generated under the control of the control unit 10a, and overlaps the image formed in a display area VA virtually set in front of the vehicle 1 with the front view, and is a display device that allows a viewer to view it.

[0083] Note that, in order to perform the above-described control, the control unit 10a includes, for example, a processor (CPU) with a built-in or externally attached memory (ROM / RAM). The processor recognizes a specific road marking RS acquired from the outside through a captured image obtained by photographing the front view of the vehicle 1, and according to the recognized specific road marking RS, for example, as shown in FIGS. 4(a), (b), and (c) which show an example of its expression form, when supplementary content AC (AC', AC'') composed of a 3D image or the like that prompts the occupant to pay more attention is generated, a graphics controller (GPU) that superimposes the generated supplementary content AC (AC', AC'') on the recognized specific road marking RS and displays it on the image display unit 10b is included. Then, the processor executes a program recorded in the ROM and operates in cooperation with the GPC to execute each of the above-described functions. Also, at least a part of the above-described functions can be realized by an FPGA, a logic circuit, or the like without depending on the processor.

[0084] [Effects of the Embodiment] As described above, the display control device according to the present embodiment is, for example, as shown in FIG. 1, a display control device 30 that controls a head-up display device (HUD device 10) that projects an image onto a display area VA virtually set in front of the vehicle 1 and superimposes the image on the forward view of the vehicle 1 for the occupant to visually recognize. And the display control device 30 recognizes a specific road marking RS (see FIGS. 4(a), (b), and (c)) that prompts the occupant to pay attention from a captured image obtained by capturing the forward view of the vehicle 1, and generates content (supplementary content AC (AC', AC'')) that further prompts the occupant to pay attention according to the recognized specific road marking RS, and has a control unit 31 that performs control to superimpose the generated content (supplementary content AC (AC', AC'')) on this specific road marking RS and display it on the HUD device 10.

[0085] According to the display control device 30 of the present embodiment, the control unit 31 generates supplementary content AC that further prompts the occupant to pay attention according to the specific road marking RS recognized from the captured image, and performs control to superimpose the generated supplementary content AC (AC', AC'') on this specific road marking RS and display it on the HUD device 10. For this reason, information (three-dimensional feeling and sense of perspective) that is difficult to obtain from the specific road marking RS can be visually recognized by the occupant through the supplementary content AC (AC', AC'') that further prompts the occupant to pay attention, so that the effect of prompting the occupant to pay attention such as speed suppression can be further enhanced, contributing to safe driving.

[0086] Also, in the display control device 30 of the present embodiment, for example, as shown in FIG. 3B, the control unit 31, after displaying the supplementary content AC (AC', AC'') that further prompts the occupant to pay attention, if the specific road marking RS is not recognized from the captured image acquired again (step ST110 “NO”), performs control to stop the display of the supplementary content on the HUD device 10 (step ST114). In this way, since the display of the supplementary content AC (AC', AC'') that further prompts the occupant to pay attention is stopped in a state where the specific road marking RS is not shown, the forward view can be secured by that much, and the occupant can contribute more to safe driving.

[0087] Also, in the display control device 30 of the present embodiment, for example, as shown in FIG. 3B, after the control unit 31 displays supplementary content AC (AC', AC'') that prompts further attention, when a specific road marking RS is recognized from the captured image acquired again (step ST110 “YES”), the control unit 31 calculates the difference between the position information of the specific road marking RS acquired immediately before and the position information of the newly acquired specific road marking RS, and based on the calculated difference, controls to relocate the supplementary content AC (AC', AC'') and display it on the HUD device 10 (steps ST112, ST113). Therefore, the supplementary content AC (AC', AC'') that prompts further attention can follow the movement of the specific road marking RS and be superimposed and displayed. In this case, it is possible to prevent the deviation between the specific road marking RS that prompts attention and the supplementary content AC (AC', AC'') that prompts further attention, and it is possible to more effectively prompt the driver to pay attention including speed suppression and the like.

[0088] Also, in the display control device 30 of the present embodiment, for example, as shown in FIG. 3A, as the vehicle 1 moves, the control unit 31 changes the display position of the specific road marking RS and displays it on the HUD device 10, and causes the supplementary content AC (AC', AC'') to follow the movement of the specific road marking RS and displays it on the HUD device 10 (steps ST101 to ST106). When the specific road marking RS disappears from the forward field of view (step ST107 “YES”), the control unit 31 controls to fade out the specific road marking RS and the supplementary content AC (AC', AC'') from the display area VA (step ST108). Therefore, it is possible to prevent the deviation between the specific road marking and the content, without making the driver feel unnatural, and effectively prompt the driver to pay attention including speed suppression.

[0089] Also, in the display control device 30 of the present embodiment, for example, as shown in FIGS. 4(a) and 4(b), the control unit 31 generates a 3D image (AC, AC') imitating a specific road marking RS, as shown in FIG. 4(c), a 2D image (AC'') imitating a specific road marking RS, or supplementary content AC (AC', AC'') consisting of the character information "!" and controls the HUD device 10 to display it. In this way, the content AC (AC', AC'') that prompts further attention is not limited to a 3D image that is displayed three-dimensionally. By expressing it in a planar display or character information that gives a sense of perspective according to the driving environment of the vehicle 1, the readability when the occupant views it can be ensured.

[0090] Here, the "3D image" refers to, for example, as shown in FIG. 1, a three-dimensional object that has one end on the XY plane of the display distance 102 and the other end on the XY plane of the display distance 103 that is farther from the occupant than the display distance, and is imaged as a solid object having a volume, giving the occupant a sense of three-dimensionality. The "2D image" refers to, for example, as shown in FIG. 1, an image that has one end on the XY plane of the display distance 103 and the other end on the XY plane of the display distance 104 that is farther from the occupant than the display distance 103, is imaged two-dimensionally, gives the occupant a sense of perspective, and since the entire display is clearly recognized by adjusting the focus of the occupant's eyes, it refers to a planar image that can give a three-dimensional impression.

[0091] Further, as shown in FIG. 1, for example, the head-up display device of the present embodiment is a HUD device 10A (see FIG. 5) that projects an image onto a display area VA virtually set in front of the vehicle 1 and superimposes the image on the forward field of view of the vehicle 1 for the occupant to visually recognize. And, as shown in FIG. 5, for example, the HUD device 10A includes an image display unit 10b that displays an image, recognizes a specific road marking RS that prompts the occupant to pay attention from a captured image obtained by photographing the forward field of view of the vehicle 1, and generates content (supplementary content AC (AC', AC'')) that further prompts the occupant to pay attention according to the recognized specific road marking RS, and performs control to superimpose the content (supplementary content AC (AC', AC'')) generated for the specific road marking RS on the image display unit 10b for display.

[0092] According to the HUD device 10A of the present embodiment, the control unit 10a recognizes a specific road marking RS from the acquired captured image, generates supplementary content AC (AC', AC'') that further prompts the occupant to pay attention from the recognized specific road marking, and performs control to superimpose the content (supplementary content AC (AC', AC'')) generated for the specific road marking RS on the image display unit 10b for display. Therefore, information (stereoscopic effect and sense of perspective) that is difficult to obtain from the specific road marking RS can be visually recognized by the occupant, and a HUD device 10A can be provided that further enhances the effect of prompting attention such as speed suppression and contributes to safe driving.

[0093] Also, as shown in FIG. 1 for example, the program of this embodiment is a program of a display control device 30 that controls a HUD device 10 which projects an image onto a display area VA virtually set in front of the vehicle 1 and superimposes the image on the forward field of view of the vehicle 1 for the occupant to visually recognize (recorded in the ROM as described above). Then, the program causes a processor implemented in the display control device 30 (control unit 31) to perform, for example, as shown in FIG. 3A, a process of recognizing a specific road marking RS from a captured image obtained by capturing the forward field of view of the vehicle 1 (steps ST101 and ST102 in FIG. 3A), a process of generating content (supplementary content AC (AC', AC'')) that prompts the occupant to take further caution according to the recognized specific road marking RS (step ST103), and a process of superimposing content (AC (see FIG. 4(a)), AC' (see FIG. 4(b)), AC'' (see FIG. 4(c)) that prompts the occupant to take further caution for the specific road marking RS that prompts the occupant to take caution and displaying it on the HUD device 10 (steps ST104 to ST106).

[0094] According to the program of this embodiment, since control is performed to recognize a specific road marking RS from a captured image acquired from a peripheral monitoring device 50 (imaging unit) and generate and display supplementary content AC (AC', AC'') that prompts the occupant to take further caution from the recognized specific road marking RS, it is possible to provide a program that allows the occupant to visually recognize information (stereoscopic effect and sense of perspective) that is difficult to obtain from the road marking RS, further enhance the effect of caution such as speed suppression, and contribute to safe driving.

[0095] In addition, as shown in FIG. 1 for example, the vehicle display system according to the present embodiment includes an imaging device (e.g., the surrounding monitoring device 50) that captures the forward view of the vehicle 1, a head-up display device (HUD device 10) that projects an image onto a display area VA virtually set in front of the vehicle 1 and superimposes the image on the forward view of the vehicle 1 for the occupant to visually recognize, and a display control device 30 that controls the head-up display device (HUD device 10). The vehicle display system 300 is such that the display control device 30 recognizes a specific road marking RS (see FIGS. 4(a), (b), and (c)) that prompts the occupant to pay attention from the captured image obtained from the imaging device (surrounding monitoring device 50), generates content (supplementary content AC, AC', AC'') that further prompts the occupant to pay attention according to the recognized specific road marking RS, and controls to superimpose the content (supplementary content AC, AC', AC'') generated for the specific road marking RS on the head-up display device (HUD device 10) for display.

[0096] According to the vehicle display system 300 of the present embodiment, the display control device 30 generates supplementary content (AC, AC', AC'') that further prompts the occupant to pay attention from a specific road marking RS obtained by recognizing the captured image acquired from the imaging unit (surrounding monitoring device 50), and by controlling to superimpose the generated supplementary content (AC, AC', AC'') on the specific road marking RS and display it on the HUD device 10, information (stereoscopic effect and sense of perspective) that is difficult to obtain from the road marking RS can be visually recognized by the occupant. Therefore, a vehicle display system 300 can be provided that further enhances the effect of prompting attention such as speed suppression and contributes to safe driving.

[0097] The present invention is not limited to the above-described exemplary embodiments, and those skilled in the art will be able to easily modify the above-described exemplary embodiments to the extent included in the scope of the claims.

Description of Reference Numerals

[0098] 1... Vehicle, 2... Projection target, 3... Eye, 4... Eyebox, 10, 10A... HUD device, 10a... Control unit (HUD), 10b... Image display unit, 20... Memory device, 30... Display control device, 31... Control unit, 32... Image memory unit, 40, 110... Projection unit, 50... Peripheral monitoring device, 60... Communication device, 70... Image, 80... Vehicle ECU, 90... I / O interface, 100(101, 102, 103, 104)... Display distance, 200... Road marking DB, 300... Vehicle display system, 310... Main control unit, 311... Captured image acquisition unit, 312... Image recognition unit, 313... Display distance determination unit, 314... Display position calculation unit, 315... Display information generation unit, 316... Display control unit, VA... Display area, RS... Specific road marking, AC, AC´, AC´´... Supplementary content

Claims

1. A display control device for controlling a head-up display device that projects an image onto a virtual display area set in front of a vehicle and overlays the image on the forward view of the vehicle for the driver to visually recognize, the display control device comprising: a control unit configured to recognize a specific road marking that prompts the driver to pay attention from a captured image obtained by capturing the forward view of the vehicle, generate content that further prompts the driver to pay attention according to the recognized specific road marking, and perform control to superimpose the content on the specific road marking and display it on the head-up display device.

2. The control unit, when the specific road marking is not recognized from the captured image acquired again after displaying the content, performs control to stop displaying the content on the head-up display device. The display control device according to Claim 1.

3. The control unit, when the specific road marking is recognized from the captured image acquired again after displaying the content, obtains a difference between the position information of the specific road marking acquired immediately before and the position information of the specific road marking newly acquired, and based on the difference, performs control to relocate the content in the display area and display it on the head-up display device. The display control device according to Claim 1.

4. The control unit, as the vehicle moves, changes the display position of the specific road marking and displays it on the head-up display device, causes the content to follow the movement of the specific road marking and superimposes and displays it on the head-up display device, and when the specific road marking disappears from the forward view, performs control to fade out the specific road marking and the content from the display area. The display control device according to Claim 1.

5. The control unit, generates the content consisting of a 3D image imitating the specific road marking, a 2D image imitating the specific road marking, or character information, and performs control to superimpose the content on the road marking and display it on the head-up display device. The display control device according to any one of Claims 1 to 4.

6. A head-up display device that projects an image onto a virtual display area set in front of a vehicle and overlays the image on the forward view of the vehicle for the driver to visually recognize, comprising an image display unit that displays the image, A head-up display device having a control unit that recognizes a specific road marking that prompts the driver to pay attention from a captured image obtained by capturing the forward view of the vehicle, generates content that further prompts the driver to pay attention according to the recognized specific road marking, and performs control to display the content superimposed on the specific road marking on the image display unit.

7. A program for a display control device that controls a head-up display device that projects an image onto a display area virtually set in front of a vehicle and superimposes the image on the forward view of the vehicle for the driver to visually recognize, wherein the processor of the display control device performs a process of recognizing a specific road marking from a captured image obtained by capturing the forward view of the vehicle, a process of generating content that further prompts the driver to pay attention according to the recognized specific road marking, and a process of superimposing the content on the specific road marking and displaying it on the head-up display device.

8. A vehicle display system including an imaging device that captures the forward view of the vehicle, a head-up display device that projects an image onto a display area virtually set in front of the vehicle and superimposes the image on the forward view of the vehicle for the driver to visually recognize, and a display control device that controls the head-up display device, wherein the display control device recognizes a specific road marking that prompts the driver to pay attention from the captured image acquired from the imaging device, generates content that further prompts the driver to pay attention according to the recognized specific road marking, and performs control to superimpose the content on the specific road marking and display it on the head-up display device.

Images