Display device

Abstract

The present invention prevents giving a strange feeling to a user who visually recognizes a notification image. This display device displays a superimposed image which is visually recognized as a virtual image A superimposed on a scene ahead of a vehicle. The display device is provided with an acquisition means, a drawing control means, and an identifying means. The acquisition means acquires object information including location information about a forward object F. The drawing control means draws a notification image C at a corresponding location corresponding to the location information on the basis of the object information. The identifying means identifies a display delay factor that causes a delay in displaying the notification image C, on the basis of at least one of a drawing control load of the drawing control means and a transmission delay of the object information. When a value represented by the identified display delay factor is greater than or equal to a predetermined value, the drawing control means executes an image enlargement process for drawing the notification image C in a size larger than a predetermined standard size.

Description

[Technical field]

[0001] The present invention relates to a display device. [Background technology]

[0002] Patent documents 1 to 3 disclose display devices that display an image that is visually recognized by a user (mainly the driver of the vehicle) as a virtual image that is superimposed on the scenery ahead of the vehicle. The display devices disclosed in Patent documents 1 to 3 display an alarm image for notifying the user of the presence of a forward object as a virtual image at a position corresponding to the position of the forward object ahead of the vehicle (for example, a position superimposed on the forward object). Each of these display devices performs unique control to prevent the alarm image from shifting from a desired display position due to external factors such as vibrations applied to the vehicle or the behavior of the forward object, which would cause the user to feel uncomfortable. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2015-80988 A [Patent Document 2] JP 2017-13590 A [Patent Document 3] JP 2017-171226 A Summary of the Invention [Problem to be solved by the invention]

[0004] The causes of the notification image being shifted from the desired display position include not only the above-mentioned external factors but also internal factors such as the load of drawing control and sensing processing of a forward object, etc. Therefore, there is room for improvement in preventing the user from feeling uncomfortable due to the shift in the display position of the notification image.

[0005] The present invention has been made in consideration of the above-described circumstances, and has an object to provide a display device that can suppress the sense of discomfort felt by a user viewing an announcement image. [Means for solving the problem]

[0006] In order to achieve the above object, the display device according to the present invention comprises: A display device that displays a superimposed image that is visually recognized by a user as a virtual image that is superimposed on a view ahead of a vehicle, an acquisition means for acquiring object information relating to a forward object present in front of the vehicle, the object information including at least position information of the forward object; a drawing control means for drawing a notification image for notifying information about the forward object at a corresponding position corresponding to the position information within a display area of ​​the superimposed image based on the object information acquired by the acquisition means; and a determination unit that determines a display delay factor that causes the notification image drawn based on the object information acquired at a predetermined timing to be displayed with a delay from the predetermined timing, the cause of the display delay is identified based on at least one of a drawing control load imposed by the drawing control means and a transmission delay of the object information; The drawing control means executes an image enlargement process for drawing the notification image at a size larger than a predetermined reference size when a value indicated by the identified display delay factor is equal to or larger than a predetermined value. When the preceding object is a moving object moving away from the vehicle at a relative speed equal to or greater than a predetermined speed, the image enlargement process is not executed on the notification image for notifying information about the moving object. In order to achieve the above object, a display device according to the present invention comprises: A display device that displays a superimposed image that is visually recognized by a user as a virtual image that is superimposed on a view ahead of a vehicle, an acquisition means for acquiring object information relating to a forward object present in front of the vehicle, the object information including at least position information of the forward object; a drawing control means for drawing a notification image for notifying information about the forward object at a corresponding position corresponding to the position information within a display area of ​​the superimposed image based on the object information acquired by the acquisition means; and a determination unit that determines a display delay factor that causes the notification image drawn based on the object information acquired at a predetermined timing to be displayed with a delay from the predetermined timing, the cause of the display delay is identified based on at least one of a drawing control load imposed by the drawing control means and a transmission delay of the object information; the drawing control means, when a value indicated by the identified display delay factor is equal to or greater than a predetermined value, executes an image enlargement process for drawing the notification image at a size larger than a predetermined reference size; The identification means identifies the drawing control load as a cause of the display delay based on at least one of the area, number, and dimension of a content image, which is an image drawn in the display area by the drawing control means and includes the notification image. Effect of the Invention

[0007] According to the present invention, it is possible to prevent a user viewing a notification image from feeling uncomfortable. [Brief description of the drawings]

[0008] [Figure 1A] 1 is a diagram showing an example of a manner in which a head-up display (HUD) device according to a first embodiment of the present invention is mounted on a vehicle. [Figure 1B] 1 is a diagram showing the relationship between a user's viewpoint position and a virtual image. [Diagram 2] FIG. 1 is a block diagram of a vehicle display system. [Figure 3A] 5A to 5C are diagrams illustrating a display example of a notification image according to the first embodiment. [Figure 3B] 11A and 11B are diagrams illustrating an example of occurrence of display deviation. [Figure 4] 10A to 10C are diagrams illustrating an example of a display of a notification image when the image enlargement process according to the first embodiment is executed. [Figure 5A] 11A and 11B are diagrams illustrating an example of occurrence of display deviation. [Figure 5B] 10A to 10C are diagrams illustrating an example of a display of a notification image when the image enlargement process according to the first embodiment is executed. [Figure 6A] FIG. 13 is a diagram showing an example of a specific type of notification image. [Figure 6B] FIG. 13 is a diagram showing an example of a notification image of a non-specific type. [Figure 7A] 3A to 3C are diagrams illustrating examples of configurations of various tables used in the display control process according to the first embodiment. [Figure 7B] 3A to 3C are diagrams illustrating examples of configurations of various tables used in the display control process according to the first embodiment. [Figure 8A] 3A to 3C are diagrams illustrating examples of configurations of various tables used in the display control process according to the first embodiment. [Figure 8B] 3A to 3C are diagrams illustrating examples of configurations of various tables used in the display control process according to the first embodiment. [Figure 9] 4 is a flowchart of a display control process according to the first embodiment. [Figure 10A]13A and 13B are diagrams illustrating a state in which display deviation occurs in a notification image according to the second embodiment. [Figure 10B] 13A to 13C are diagrams illustrating an example of a display of a notification image when an image position adjustment process according to the second embodiment is executed. [Figure 11A] 11A and 11B are diagrams for explaining a specific moving state of a forward object; [Figure 11B] 11A and 11B are diagrams for explaining a specific moving state of a forward object; [Figure 12] 13 is a diagram showing an example of the configuration of a movement amount determination table used in the display control process according to the second embodiment. FIG. [Figure 13] 10 is a flowchart of a display control process according to the second embodiment. [Figure 14A] 13A and 13B are diagrams illustrating a state in which a display delay occurs in a notification image according to the third embodiment. [Figure 14B] 13A to 13C are diagrams illustrating an example of a display of a notification image when a line width enlargement process according to the third embodiment is executed. [Figure 15A] 11A and 11B are diagrams illustrating an example of occurrence of display deviation. [Figure 15B] 13A to 13C are diagrams illustrating an example of a display of a notification image when a line width enlargement process according to the third embodiment is executed. [Figure 16] 13 is a diagram showing an example of the configuration of an enlargement amount determination table used in the display control process according to the third embodiment. FIG. [Figure 17] 13 is a flowchart of a display control process according to the third embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] A display device according to an embodiment of the present invention will be described with reference to the drawings.

[0010] (First embodiment) The display device according to this embodiment is a HUD (Head-Up Display) device 10 included in a vehicle display system 100 shown in Fig. 2. As shown in Fig. 1A, the HUD device 10 is provided inside a dashboard 2 of a vehicle 1 (hereinafter also referred to as the vehicle 1), and notifies a user 4 (mainly the driver of the vehicle 1) of not only information related to the vehicle 1 (hereinafter referred to as vehicle information) but also information other than the vehicle information in an integrated manner. Note that the vehicle information includes not only information about the vehicle 1 itself but also information outside the vehicle 1 related to the operation of the vehicle 1.

[0011] The vehicular display system 100 is a system configured within a vehicle 1, and includes a HUD device 10, a forward object detection unit 40, a viewpoint detection unit 50, an ECU (Electronic Control Unit) 60, and a car navigation device 70, as shown in FIG. 2.

[0012] 1A, the HUD device 10 emits display light Q toward a windshield 3 of a vehicle 1. The display light Q reflected by the windshield 3 proceeds toward a user 4. By placing the user 4's viewpoint within an eyebox 5, the user 4 can visually recognize an image represented by the display light Q displayed in front of the windshield 3 as a virtual image A. This allows the user 4 to visually recognize the virtual image A superimposed on the scenery ahead.

[0013] 1B, the HUD device 10 displays a virtual image A on a virtual plane set in front of the vehicle 1. The virtual plane is set at a position a predetermined distance P (for example, about 5 to 10 m) ahead of the vehicle 1 from a viewpoint position 4a when the user 4 places his / her viewpoint inside an eye box 5. The virtual plane corresponds to a display surface of an image on the display unit 20. The virtual plane and the eye box 5 are set based on the size of the display surface and an optical system formed by various mirrors and the windshield 3 inside the HUD device 10.

[0014] The HUD device 10 includes a display unit 20 and a control unit 30 shown in FIG. 2, and a reflector unit (not shown).

[0015] The display unit 20 displays a superimposed image that is visually recognized by the user 4 as a virtual image A under the control of the control device 30. The display unit 20 has, for example, a TFT (Thin Film Transistor) type LCD (Liquid Crystal Display) and a backlight that illuminates the LCD from behind. The backlight is configured to include, for example, an LED (Light Emitting Diode). The display unit 20 generates display light Q by displaying an image on the LCD illuminated by the backlight under the control of the control device 30. The generated display light Q is reflected by a reflecting unit and then emitted toward the windshield 3. The reflecting unit is configured, for example, of two mirrors, a folding mirror and a concave mirror. The folding mirror folds the display light Q emitted from the display unit 20 and directs it toward the concave mirror. The concave mirror reflects the display light Q from the folding mirror toward the windshield 3 while enlarging it. As a result, the virtual image A visually recognized by the user 4 is an enlarged image displayed on the display unit 20. The type and number of mirrors constituting the reflecting portion can be changed as desired depending on the design.

[0016] In the following, the display unit 20 displaying an image to be viewed by the user 4 as the virtual image A is also referred to as "displaying a superimposed image." Furthermore, the control device 30 controlling the display of the display unit 20 is also referred to as "performing display control of a superimposed image." Furthermore, the display unit 20 is not limited to one using an LCD, and may be one using a display device such as an OLED (Organic Light Emitting Diodes), a DMD (Digital Micro mirror Device), or an LCOS (Liquid Crystal On Silicon) as long as it can display a superimposed image.

[0017] The control device 30 is made up of a microcomputer that controls the overall operation of the HUD device 10, and includes a control unit 31 and a storage unit 32. The control device 30 also includes, as components not shown, a driver for driving the display unit 20 and an input / output circuit for communicating with various systems in the vehicle 1.

[0018] The storage unit 32 is composed of a ROM (Read Only Memory) that stores operation programs and various image data in advance, and a RAM (Random Access Memory) that temporarily stores various calculation results. The ROM of the storage unit 32 stores data of operation programs for executing a display control process described below, and data of various tables and formulas used when executing the display control process. The control unit 31 includes a CPU (Central Processing Unit) 31a that executes the operation programs stored in the ROM of the storage unit 32, and a GDC (Graphics Display Controller) 31b that cooperates with the CPU 31a to execute image processing. The GDC 31b is composed of, for example, a GPU (Graphics Processing Unit), an FPGA (Field-Programmable Gate Array), an ASIC (Application Specific Integrated Circuit), etc. The GDC 31b is capable of both drawing processing using vector images and drawing processing using raster images (bitmap images). The configurations of the control device 30 and the control unit 31 are arbitrary as long as they fulfill the functions described below.

[0019] The control unit 31 drives and controls the display unit 20. For example, the control unit 31 drives and controls the backlight of the display unit 20 with the CPU 31a, and drives and controls the LCD of the display unit 20 with the GDC 31b that operates in cooperation with the CPU 31a.

[0020] The CPU 31a of the control unit 31 cooperates with the GDC 31b to control the superimposed image based on various image data stored in the ROM of the storage unit 32. The GDC 31b determines the control content of the display operation of the display unit 20 based on a display control command from the CPU 31a. The GDC 31b reads image part data required to configure one screen to be displayed on the display unit 20 from the ROM and transfers it to the RAM of the storage unit 32. The GDC 31b also uses the RAM to create picture data for one screen based on the image part data and various image data received by communication from the outside of the HUD device 10. Then, when the GDC 31b completes one screen of picture data in the RAM, it transfers it to the display unit 20 in accordance with the image update timing (frame rate). As a result, the superimposed image visually recognized by the user 4 as the virtual image A is displayed on the display unit 20. Also, a layer is assigned in advance to each image constituting the image visually recognized as the virtual image A, and the control unit 31 is capable of individual display control of each image. By such control, the control unit 31 draws the content image within the display area of ​​the virtual image A. Note that the rectangular frame indicated by the virtual image A in Fig. 1B, Fig. 3, etc. indicates the display area of ​​the virtual image A, and the content image is an image that is visually recognized as part of the virtual image A within the display area. This is synonymous with the content image being displayed within the display area of ​​the superimposed image on the display unit 20 that displays the superimposed image visually recognized by the user 4 as the virtual image A.

[0021] The content image is any image drawn in the display area of ​​the virtual image A, and includes an alarm image C for notifying information about a forward object F present in front of the vehicle 1. The alarm image C is displayed at a corresponding position corresponding to position information (information indicating the position of the forward object F) described later in the display area of ​​the virtual image A under the control of the control unit 31. Here, the corresponding position as the display position of the alarm image C in the display area of ​​the virtual image A is a position where the alarm image C is visually recognized by the user 4 in a state where it is superimposed on at least a part of the forward object F or adjacent to the forward object F in an ideal state where there is no display deviation described later. The corresponding position can be appropriately set according to the type of the forward object F. FIG. 3A is an example in which an alarm image C is displayed to notify the presence of the forward object F as a preceding vehicle. Note that the content image includes not only the alarm image C but also an image showing vehicle information. For example, the image showing vehicle information is an image A1 showing the vehicle speed and an image A2 showing the speed limit of the driving lane, as shown in FIG. 3B. Furthermore, the forward object F that is the subject of notification by the notification image C is not limited to the preceding vehicle, but may be a pedestrian, a building, or part of a road (for example, a dividing line such as a solid or dashed white line, an intersection, or a fork in the road), and can be set arbitrarily.

[0022] The control unit 31 also communicates with each of the forward object detection unit 40, the viewpoint detection unit 50, the ECU 60, and the car navigation device 70. As the communication, for example, a communication method such as CAN (Controller Area Network), Ethernet (registered trademark), MOST (Media Oriented Systems Transport) (registered trademark), or LVDS (Low Voltage Differential Signaling) can be applied.

[0023] The forward object detection unit 40 detects a forward object F located in front of the vehicle 1, and supplies "object information" including at least position information of the forward object F as information related to the detected forward object F to the control unit 31. The position information included in the object information is, for example, data of coordinates (x, y, z) in which the traveling direction of the vehicle 1 is the x-axis, the left-right direction of the vehicle 1 is the y-axis, and the height direction of the vehicle 1 is the z-axis, as shown in FIG. 1B. The origin of the coordinates may be any position set, such as a representative point at the tip of the vehicle 1 or the center point of the vehicle 1.

[0024] The forward object detection unit 40 is configured to include, for example, one or a combination of various sensors described below, and a processing unit that processes information detected by the sensors. For example, a LiDAR (Light Detection And Ranging) sensor may be used as the sensor constituting the forward object detection unit 40. The forward object detection unit 40 removes unnecessary points from point cloud data measured by the LiDAR, and then performs a clustering process to detect one or more clusters (point cloud sets). The forward object detection unit 40 also uses a classifier to determine whether or not a cluster that is estimated to have measured a preset forward object F among the detected clusters is a forward object F. For example, Euclidean Clustering in a PCL (Point Cloud Library) may be used as the clustering process. Also, a support vector machine (SVM) may be used as the classifier. In this way, the forward object detection unit 40 supplies object information including position information (including information indicating the size of the forward object F) and type information indicating the type of the forward object F (leading vehicle, pedestrian, building, etc.) to the control unit 31 for the forward object F detected in this way. Furthermore, the forward object detection unit 40 may be configured to include an imaging device that captures an image of a forward scene including a road surface on which the vehicle 1 is traveling. The imaging device may be configured, for example, as a stereo camera. The forward object detection unit 40 calculates position information and type information of the forward object F by performing image analysis on the data of the image captured by the imaging device using a known method such as a pattern matching method in a processing unit. Then, the forward object detection unit 40 supplies this information to the control unit 31 as object information. Note that the forward object detection unit 40 may be configured to include a sonar, an ultrasonic sensor, a millimeter wave radar, or the like that detects the forward object F.

[0025] The control unit 31 acquires object information from the forward object detection unit 40, and specifies the position of the forward object F in the display area of ​​the virtual image A based on the acquired object information. Furthermore, when there are multiple forward objects F detected by the forward object detection unit 40, the control unit 31 assigns identification information (ID) to each of the multiple forward objects F, and specifies the position of a specific forward object F based on the position information corresponding to the ID. Note that if the position information (coordinates) of the forward object F is known, the direction of the forward object F relative to the vehicle 1 can be specified. Furthermore, if the position information is known, the distance from the vehicle 1 to the forward object F can be calculated. The distance can be differentiated with respect to time to calculate the relative speed of the forward object F relative to the vehicle 1. The specification of the direction of the forward object F and the calculation of the distance and relative speed may be performed by the forward object detection unit 40 or the control unit 31.

[0026] The viewpoint detection unit 50 has a known configuration for detecting the viewpoint position (including the line of sight) of the user 4, and includes, for example, an infrared camera that captures an image of the face of the user 4, and a viewpoint position analysis unit that analyzes the data of the image captured by the infrared camera (imaging data). The viewpoint position analysis unit identifies the viewpoint position of the user 4 by performing image analysis on the imaging data using a known method such as a pattern matching method, and supplies information indicating the viewpoint position to the control unit 31.

[0027] The ECU 60 controls each part of the vehicle 1 and supplies vehicle information related to the vehicle 1 to the control unit 31. The control unit 31 may directly acquire vehicle speed information from various sensors such as a vehicle speed sensor. The vehicle information related to the vehicle 1 includes, for example, vehicle speed, engine RPM, and warning information (low fuel, abnormal engine oil pressure, etc.). The control unit 31 controls the display of the superimposed image based on the vehicle information acquired from the ECU 60, and can also display an image showing predetermined vehicle information within the display area of ​​the virtual image A. In other words, an image showing vehicle information other than the notification image C can also be displayed within the display area of ​​the virtual image A.

[0028] The car navigation device 70 includes a GPS controller that calculates the position of the vehicle 1 based on a GPS (Global Positioning System) signal received from an artificial satellite or the like. The car navigation device 70 has a storage unit that stores map data, and based on the position information of the vehicle 1 from the GPS controller, reads out map data in the vicinity of the current position from the storage unit and determines a guide route to a destination set by the user 4. The car navigation device 70 then outputs information on the position of the vehicle 1 and the determined guide route to the control unit 31. The car navigation device 70 also references the map data to output information indicating the name and type of facilities ahead of the vehicle 1, the distance between the facilities and the vehicle 1, and the like to the control unit 31. In the map data, various information such as road shape information (lanes, road width, number of lanes, intersections, curves, branching roads, etc.), regulation information on road signs such as speed limits, and information on each lane when there are multiple lanes are associated with position data. The car navigation device 70 outputs these various pieces of information to the control unit 31 as navigation data. Here, when the forward object F to be notified by the notification image C is a building or an intersection, the navigation data includes object information on the forward object F, information for displaying a guide route as the notification image C, and information for the control unit 31 to specify the distance from the vehicle 1 to the forward object F. That is, the control unit 31 may obtain the object information from the car navigation device 70. The notification image C may be an image showing a POI (Point of Interest) such as a destination or a facility that is set in advance by the user 4 in the car navigation device 70. Note that the car navigation device 70 is not limited to being mounted on the vehicle 1, and may be realized by a mobile terminal (such as a smartphone or a tablet PC (Personal Computer)) having a car navigation function and communicating with the control unit 31 by wire or wirelessly.

[0029] The control unit 31 calculates the display position of the notification image C within the display area of ​​the virtual image A based on the position information of the forward object F identified based on the object information from the forward object detection unit 40 and information on the viewpoint position of the user 4 input from the viewpoint detection unit 50, and controls the display operation of the display unit 20. As a result, the control unit 31 displays the notification image C at a desired position (the aforementioned corresponding position) relative to the forward object F in the real scene.

[0030] The control unit 31 may not only display the content image including the notification image C in two dimensions (2D), but may also display it in 3D using three-dimensional (3D) image data pre-stored in the ROM of the storage unit 32. The control unit 31 may also execute warping to pre-distort the image displayed on the display surface of the display unit 20 (the screen when the configuration in which the display light Q is projected onto a screen is adopted) in the opposite direction by the amount of distortion caused by the optical meter after the display surface, as a process for correcting distortion of the virtual image A mainly caused by the curved shape of the windshield 3. Furthermore, the control unit 31 may execute dynamic warping, which is warping according to the viewpoint position of the user 4. The dynamic warping realizes distortion correction optimized according to the viewpoint position by using parameters set for each of a plurality of regions obtained by dividing the rectangular region of the eye box 5 into a grid shape.

[0031] Fig. 3A shows a display example of an alert image C for alerting a forward object F. Fig. 3A shows an example in which an alert image C is displayed at a desired display position, in which an elliptical alert image C is displayed on the road surface side of the forward object F, which is a preceding vehicle, so as to be visually recognized superimposed on a part of the forward object F.

[0032] The notification image C displayed according to the position of the forward object F may be displayed with a delay from the desired display timing due to the drawing control load (hereinafter also referred to as the drawing load) and the like. When the display delay of the notification image C occurs, a "display shift" occurs in which the notification image C is shifted from the desired display position depending on the relative speed between the vehicle 1 and the forward object F, which gives a sense of discomfort to the user. FIG. 3B shows an example in which a display shift occurs in the notification image C due to an increase in the number of content images displayed in the display area of ​​the virtual image A and an increase in the drawing load. FIG. 3B shows an example in which a preceding vehicle F1 in the driving lane of the vehicle 1, a preceding vehicle F2 in the lane next to the driving lane, and buildings F3 and F4 located in front of the vehicle 1 are detected as the forward objects F, and a display shift occurs in the notification images C (notification images C1 to C4) that notify each of these four forward objects F. In order to reduce the discomfort felt by the user due to such display misalignment, the control unit 31 according to the first embodiment executes image enlargement processing.

[0033] Hereinafter, a description will be given of main functions of the control unit 31 that executes the image enlargement process. The control unit 31 mainly functions as an acquisition unit, a drawing control unit, and a specification unit.

[0034] The acquisition means acquires the above-mentioned object information from the forward object detection unit 40. The drawing control means draws, in the display area of ​​the superimposed image, an alert image C that is visually recognized by the user 4 at a corresponding position that corresponds to the position (position information) of the forward object F, based on the object information acquired by the acquisition means. The correspondence relationship between the position information and the corresponding position (the display position of the alert image C in the display area of ​​the virtual image A) is stored in advance in the ROM of the storage unit 32.

[0035] In addition, the identification means identifies a display delay factor that causes the notification image C, which is drawn based on the object information acquired by the acquisition means at a predetermined timing, to be displayed with a delay from the predetermined timing. Specifically, the control unit 31 as the identification means identifies the display delay factor as follows.

[0036] (Identifying the cause of display delays) When identifying the delay factor, first, the control unit 31 refers to the predicted drawing load determination table TA1 shown in FIG. 7A, and determines the drawing load (predicted drawing load L) predicted when the drawing prediction means (mainly GDC31b) draws the content image (including the notification image C) to be displayed in the display area of the virtual image A in the future. The predicted drawing load determination table TA1 is referred to for example for one content image, and is configured to correspond to the combination of the content type (2D or 3D) and the drawing screen area S and the predicted drawing load L. The predicted drawing load determination table TA1 is stored in advance in the ROM of the storage unit 32.

[0037] The control unit 31 refers to the predicted drawing load determination table TA1 to determine the predicted drawing load L for one content image (hereinafter referred to as the prediction target image) among the images for one screen created by GDC31b and transferred to the display unit 20 at a predetermined update timing (hereinafter referred to as the prediction target timing) after the next in the frame rate. For example, when the type of the prediction target image is "2D" and its drawing screen area S is "S1 ≦ S < S2", the control unit 31 refers to the predicted drawing load determination table TA1 and determines the predicted drawing load L to be "10%".

[0038] When a plurality of prediction target images are included in the picture data for one screen drawn at the prediction target timing, the control unit 31 determines the predicted drawing load L for the number of prediction target images. Then, the control unit 31 sums up the determined predicted drawing loads L to obtain the total predicted drawing load ΣL. For example, as shown in FIG. 3B, when there are six content images including four notification images C (C1 to C4), an image A1 representing the vehicle speed, and an image representing the speed limit of the driving lane in the display area of the virtual image A, the predicted drawing load L for each of the six content images is determined, and these are added together to obtain the total predicted drawing load ΣL.

[0039] That is, the control unit 31 as the identification means identifies the drawing control load as a cause of the display delay based on the area, number, and dimension of the content image (including the notification image C). Note that the control unit 31 may identify the drawing control load as a cause of the display delay based on at least one of the area, number, and dimension of the content image.

[0040] After determining the total predicted drawing load ΣL, the control unit 31 refers to a predicted delay time determination table TA2 shown in FIG. 7B and predicts a delay time in displaying the content image caused by the drawing load (predicted delay time T L The predicted delay time determination table TA2 determines the total predicted drawing load ΣL and the predicted delay time T L The control unit 31 refers to the predicted delay time determination table TA2 and determines the predicted delay time T L Determine.

[0041] The display delay of the content image is also caused by system configuration other than the drawing load. For example, if the timing when the forward object detection unit 40 senses a predetermined forward object F is t0, the notification image C is drawn based on the object information sensed at t0 at a timing t1 after t0. In this embodiment, the sensing delay time T S Consider the sensing delay time T S is mainly a transmission delay of object information from the forward object detection unit 40 to the control unit 31, and is stored in advance as data in the ROM of the storage unit 32. The control unit 31 determines the predicted delay time T due to the drawing load determined as described above. L and sensing delay time T S The sum of the delay factors (T = T L +T S )

[0042] In addition, the predicted delay time T Lmay be set in advance taking into consideration not only the drawing load but also the display control and display operation of the display unit 20. S may be determined taking into consideration the processing delay when the forward object detection unit 40 calculates the object information. Furthermore, the processing load when the control unit 31 executes the above-mentioned warping or dynamic warping may be considered as a display delay factor caused by the system configuration. The processing load when performing dynamic warping can be predicted in consideration of the number of previously set viewpoint positions, the setting of the number of grids when dividing the eye box 5 into a plurality of regions, and the like, or can be obtained by experiment.

[0043] (Image enlargement processing) As described above, when the cause of display delay is identified by the identification means, the control unit 31, as a drawing control means, executes an image enlargement process to draw the notification image C at a size larger than the reference size if the value indicated by the identified cause of display delay is equal to or greater than a predetermined value.

[0044] The reference size is the size of the notification image C at a predetermined distance P (see FIG. 1B) determined according to the type of notification image C. The reference size is not a fixed size within the display area of ​​the virtual image A, but is variable according to reference information (mathematical formula data or table data) that defines the correspondence between the distance from the host vehicle 1 to the forward object F and the size of the reference size. The reference information is stored in advance in the storage unit 32, and is set, for example, using the law of perspective, so that the reference size becomes smaller as the distance to the forward object F of the notification target becomes longer. The control unit 31 draws the notification image C at the reference size based on the distance from the host vehicle 1 to the forward object F and the reference information.

[0045] When executing image enlargement processing, the control unit 31 as a drawing control means determines the magnification with respect to the reference size in the following manner.

[0046] The control unit 31 determines a magnification (hereinafter, referred to as a first magnification) corresponding to the display delay factor obtained as described above, by referring to the first magnification determination table TA3 shown in Fig. 8A. The first magnification determination table TA3 is configured so that the display delay factor corresponds to the first magnification, and is stored in the ROM of the storage unit 32 in advance.

[0047] Furthermore, when the control unit 31 has determined the first magnification and the relative speed between the vehicle 1 and the approaching forward object F is equal to or greater than a predetermined speed, the control unit 31 refers to a second magnification determination table TA4 shown in Fig. 8B to determine a magnification (hereinafter referred to as a second magnification) corresponding to the relative speed. The second magnification determination table TA4 is configured so that the relative speed corresponds to the second magnification, and is stored in advance in the ROM of the storage unit 32.

[0048] The control unit 31, which has determined the second magnification, determines the product of multiplying the first magnification and the second magnification as the magnification in the image enlargement process. For example, if the determined first magnification is "1.1" and the determined second magnification is "1.2", the control unit 31 determines 1.32 as the magnification in the image enlargement process, since 1.1 x 1.2 = 1.32. In this case, the control unit 31 draws the notification image C that is 1.32 times the standard size according to the above-mentioned standard information. For example, the image enlargement process is performed while maintaining the aspect ratio of the notification image C of the standard size. Note that the shape of the notification image C is not limited to the ellipse in the illustrated example, but may be any other shape such as a rectangle, a circle, a triangle, or a polygon. In addition, the configuration of the notification image C may be any as long as it can notify information about the forward object F, and may be, for example, a character, a symbol, a figure, an icon, or a combination of these.

[0049] 4 shows an example of the display of the notification image C when the image enlargement process is executed. When the value indicated by the display delay factor is equal to or greater than a predetermined value, the image enlargement process is executed to enlarge the notification image C of the reference size at the magnification determined as described above, so that the deviation from the desired display position can be made inconspicuous, and the discomfort felt by the user can be suppressed.

[0050] In particular, when the control unit 31, which functions as a drawing control means, executes an image enlargement process when the vehicle 1 and the forward object F become relatively closer in a specified direction, it draws the notification image C so that it is enlarged and viewed toward the vehicle 1, based on an end B of the notification image C that is viewed at a position far from the vehicle 1 in the specified direction.

[0051] For example, in the state shown in Fig. 4, when the vehicle 1 and the forward object F relatively approach each other in the longitudinal direction, the notification image C is drawn so as to be visually enlarged toward the vehicle 1, based on the end B (the upper end in Fig. 4) visually recognized at a position far from the vehicle 1 in the longitudinal direction, when the image enlargement process is executed. In this way, even if a display deviation occurs in which the notification image C is shifted above the desired display position due to a display delay, the display deviation can be effectively made inconspicuous.

[0052] FIG. 5A shows an example in which the forward object F is a road marking (a white line in the example of the figure), and the notification image C notifying the forward object F is displayed superimposed on the marking to emphasize the presence of the marking. In this case, the forward object F is an immovable object on the ground, but as the vehicle 1 approaches the marking, a display shift of the notification image C may occur as shown in FIG. 5A. In this case in which the vehicle 1 and the forward object F relatively approach each other in the lateral direction, when the image enlargement process is executed, as shown in FIG. 5B, the notification image C is drawn so that it is enlarged and viewed toward the vehicle 1 side, based on the end B (the right end in FIG. 5B) that is viewed at a position farther from the vehicle 1 in the lateral direction. In this way, even if a display shift occurs in which the notification image C is shifted to the right of the desired display position due to a display delay, the display shift can be effectively made inconspicuous.

[0053] (Display control processing) From here on, the display control process executed by the control unit 31 will be described mainly with reference to Fig. 9. The display control process is executed continuously while the ignition of the vehicle 1 is turned on, for example.

[0054] When the display control process is started, first, the control unit 31 determines whether or not a forward object F has been detected (step S101). When object information is supplied from the forward object detection unit 40, the control unit 31 determines that a forward object F has been detected (step S101; Yes), and proceeds to step S102. On the other hand, when a forward object F has not been detected (step S101; No), the control unit 31 waits in step S101.

[0055] Next, the control unit 31 calculates the relative speed between the host vehicle 1 and the forward object F, as described above, based on the object information acquired from the forward object detection unit 40 (step S102).

[0056] Next, the control unit 31 determines whether or not the notification image C for notifying the user of the detected forward object F is a specific type of image (step S103).

[0057] Here, the specific type of image is an image that is acceptable for notifying the vehicle 1 even if the notification image C for notifying information about the forward object F is a stationary object such as a road and is visually recognized as being shifted from the desired display position (the corresponding position described above) in the traveling direction of the vehicle 1, and the type is stored in the ROM of the storage unit 32 in advance. For example, as shown in FIG. 6A, when the forward object F is the road surface in front of the vehicle 1 and the notification image C is simply a guide display indicating going straight, even if a display delay occurs in the notification image C, the user 4 is unlikely to recognize that the notification image C is displayed shifted from the road surface in front. Therefore, in this way, the notification image C such as a guide display indicating going straight is stored in advance as a specific type, and when the notification image C is an image of the specific type, the image enlargement process described later is not executed. On the other hand, as shown in FIG. 6B, when the forward object F is an intersection and the notification image C is a guide display indicating a left turn, the notification image C is not set to a specific type. 6B is viewed out of the direction of travel of the vehicle 1, it will affect the guidance. In addition, when the notification image C provides guidance in a direction other than going straight ahead, the notification image C is not set to a specific type.

[0058] In step S103, if the notification image C is a specific type of image as shown in Fig. 6B (step S103; Yes), the control unit 31 executes normal image control to display the notification image C of a standard size according to the above-mentioned standard information at a corresponding position corresponding to the position information included in the object information (step S104). Note that, if multiple forward objects F are detected in step S101, the control unit 31 displays the notification image C corresponding to each of the multiple forward objects F.

[0059] In step S103, if the notification image C is not a specific type of image (step S103; No), the control unit 31 determines whether the forward object F is moving away at a predetermined relative speed or more based on the relative speed calculated in step S102 (step S103A). Specifically, if the forward object F specified by the object information acquired in step S101 is a moving object such as a preceding vehicle, and the moving object is moving away from the vehicle 1 at a predetermined relative speed or more previously set in the ROM of the storage unit 32 (step S103A; Yes), the control unit 31 executes normal image control (step S104). This is because the moving object is considered to be of low importance to the user 4 at the present time. Note that a moving object is an object that moves relative to the ground, such as a preceding vehicle or a pedestrian.

[0060] In step S103A, if the forward object F is not moving away at a relative speed equal to or greater than the predetermined speed (step S103A; No), the control unit 31 identifies the cause of the display delay (step S105). As described above, the control unit 31 determines the predicted delay time T based on the type (2D or 3D) of the notification image C to be displayed, the predicted drawing load determination table TA1, and the predicted delay time determination table TA2. L Then, the control unit 31 determines the predicted delay time T L and sensing delay time T S The sum of the delay factors (T = T L +T S )

[0061] Next, the control unit 31 determines whether the value T indicating the display delay factor identified in step S105 is equal to or greater than a predetermined value T1 previously defined in the ROM of the storage unit 32 (step S106). If the value T indicating the display delay factor is less than the predetermined value T1 (step S106; No), the control unit 31 executes normal image control (step S104).

[0062] On the other hand, if the value T indicated by the display delay factor is equal to or greater than the predetermined value T1 (step S106; Yes), the control unit 31 determines a magnification for the reference size (step S107). Here, as described above, the control unit 31 refers to the first magnification determination table TA3 to determine a magnification (first magnification) corresponding to the display delay factor identified in step S105. Also, if the relative speed V between the vehicle 1 and the forward object F approaching relatively is equal to or greater than a predetermined speed (V1 in the example of FIG. 8B), the control unit 31 refers to the second magnification determination table TA4 to determine a magnification (second magnification) corresponding to the relative speed V. Then, the control unit 31 determines the product of multiplying the first magnification and the second magnification as the magnification in the image enlargement process. Note that, if the relative speed V is less than the predetermined speed V1, the first magnification is determined as it is as the magnification in the image enlargement process.

[0063] Next, the control unit 31 executes an image enlargement process to draw the notification image C with the reference size enlarged at the magnification determined in step S107 (step S108). As described above, when the vehicle 1 and the forward object F are relatively approaching each other in a predetermined direction, the notification image C is drawn so as to be enlarged and viewed toward the vehicle 1 side, based on an end B of the notification image C that is visually recognized at a position far from the vehicle 1 in the predetermined direction, when the image enlargement process is executed. Note that, when the vehicle 1 and the forward object F are not relatively approaching each other in the predetermined direction, the control unit 31 may enlarge the notification image C based on an arbitrary position. Furthermore, when a plurality of forward objects F are detected in step S101, the control unit 31 executes the image enlargement process for the image of the notification image C, the magnification of which is determined in step S107, among the notification images C displayed corresponding to each of the plurality of forward objects F.

[0064] After executing step S108 or step S104, the control unit 31 executes the process again from step S101. The above display control process is continuously executed until, for example, the HUD device 10 is turned off. This concludes the description of the first embodiment.

[0065] From here, other embodiments in which some processes in the display control process are different from the first embodiment will be described. In the following embodiments, the configuration of the vehicular display system 100 is similar to that of the first embodiment. Therefore, in the following, the same components as in the first embodiment will be described using the same reference numerals as in the first embodiment, and the differences from the first embodiment will be mainly described.

[0066] Second embodiment In the second embodiment, the control unit 31 also displays an alarm image C for notifying a user of a forward object F, for example, as shown in FIG. 3A. The control unit 31 according to the second embodiment executes an image position adjustment process, which will be described later, in order to reduce the discomfort felt by the user due to the occurrence of display deviation as shown in FIG. 3B and FIG. 10A. The control unit 31, which executes a display control process including the image position adjustment process, mainly functions as an acquisition means, a drawing control means, and an identification means. The identification means according to the second embodiment identifies the cause of the display delay in the same manner as in the first embodiment.

[0067] (Image position adjustment process) When the identification means identifies the cause of the display delay, the control unit 31 as drawing control means executes an image position adjustment process for drawing an alarm image C that is visually recognized at an adjusted position obtained by moving a desired display position (the corresponding position described above) by a predetermined movement amount in the direction in which the forward object F moves relative to the vehicle 1, if the value indicated by the identified cause of the display delay is equal to or greater than a predetermined value. When executing the image position adjustment process, the control unit 31 as drawing control means determines the movement amount of the alarm image C as follows.

[0068] When the value indicated by the display delay factor is equal to or greater than a predetermined value, the control unit 31 sets the movement amount to a predetermined value M (hereinafter, M is also referred to as the reference movement amount). Further, the control unit 31 obtains the relative speed V between the host vehicle 1 and the forward object F, and refers to the movement amount determination table TA5 shown in FIG. 12 to determine the movement amount corresponding to the obtained relative speed V. The movement amount determination table TA5 is configured such that the relative speed and the movement amount correspond to each other, and is stored in advance in the ROM of the storage unit 32. For example, when the magnitude (absolute value) of the calculated relative speed V satisfies 0 ≦ V < V1, the control unit 31 sets the movement amount of the notification image C to M as it is. When the magnitude of the calculated relative speed V satisfies V1 ≦ V < V2, the control unit 31 sets the movement amount of the notification image C to α × M (α > 1). That is, the control unit 31 increases the movement amount when the relative speed V is a second value greater than the first value compared to when the relative speed V is the first value.

[0069] Note that the control unit 31 may variably control the reference movement amount M itself so that the movement amount of the notification image C increases as the value indicated by the display delay factor increases. Further, the control unit 31 may determine a coefficient (α or β in the example of FIG. 12) according to the obtained relative speed V, and multiply the determined coefficient by the reference movement amount M to obtain the movement amount of the notification image C corresponding to the relative speed V. Further, the relationship between the relative speed V and the movement amount of the notification image C is stored in advance in the ROM of the storage unit 32 as data of a function (for example, a linear function or a quadratic function), and the control unit 31 may determine the movement amount of the notification image C based on the obtained relative speed V and the function.

[0070] FIG. 10B shows an example of the display of the notification image C when the image position adjustment process is executed. For example, when the display delay factor increases and the forward object F approaches the host vehicle 1, a display shift as shown in FIG. 10A occurs. In order to suppress the discomfort given to the user 4 due to this display shift, when the value indicated by the display delay factor is equal to or greater than a predetermined value, the control unit 31 according to the second embodiment executes an image position adjustment process of drawing the notification image C that is visually recognized at an adjustment position obtained by moving a desired display position (the corresponding position described above) in the direction in which the forward object F moves with respect to the vehicle 1 (downward in the example of FIG. 10) by a predetermined movement amount.

[0071] In particular, the control unit 31, which functions as a drawing control means, corrects the amount of movement when the forward object F is in a specific movement state in which it is approaching the vehicle 1 from a position more than a specific distance away from the vehicle 1 in an intersecting direction (e.g., y direction) that intersects with the vehicle 1's traveling direction (e.g., x direction) so that the amount of movement when performing the image position adjustment process is larger than when the forward object F is not in the specific movement state. Here, the specific movement state of the forward object F will be described with reference to Figs. 11A and 11B. Fig. 11A is an example in which the forward object F approaches the vehicle 1 in a straight line with a relative velocity vector of magnitude v. On the other hand, Fig. 11B is an example in which the forward object F approaches the vehicle 1 in an oblique manner with a relative velocity vector of magnitude v. For the user 4 (mainly the driver), it is more difficult to grasp the sense of distance in the case shown in Fig. 11B than in the case shown in Fig. 11A, even if the magnitude of the relative velocity in both cases is the same at v. Considering this, for example, the control unit 31 determines that the forward object F is in a specific movement state based on the position information (x, y, z) included in the acquired object information, when the forward object F is approaching the vehicle 1 and the y coordinate of the forward object F is at a position that is a specific distance or more away from the vehicle 1. The specific distance is stored in advance in the ROM of the storage unit 32 as a distance in the y direction. If the movement amount determined as described above is Mf, when the control unit 31 determines that the forward object F is in a specific movement state, it adds a correction amount ΔM to Mf and sets the movement amount of the notification image C to (Mf+ΔM). In this way, the notification image C can be displayed closer to the vehicle 1, so that the presence of the forward object F in a specific movement state in which it is assumed that it is difficult for the user 4 to grasp the sense of distance can be properly notified.

[0072] As shown in FIG. 5A, the image position adjustment process according to the second embodiment can be executed even when the forward object F is a road dividing line and the notification image C emphasizes the dividing line to notify the driver. As the vehicle 1 approaches the dividing line, a display shift of the notification image C may occur as shown in FIG. 5A. However, by executing the image position adjustment process according to the second embodiment and drawing the notification image C that is viewed at an adjusted position moved by a determined movement amount in the direction in which the forward object F moves relative to the vehicle 1 (leftward in FIG. 5A), the display shift can be made less noticeable. In this way, even if a display shift occurs in which the notification image C is shifted to the right of the desired display position due to a display delay, the display shift can be made less noticeable.

[0073] (Display control processing) From here on, the display control process according to the second embodiment will be described with reference to Fig. 13. The control unit 31 executes the processes of steps S201 to S206 in the same manner as steps S101 to S106 described in the first embodiment.

[0074] In step S206, if the value T indicated by the display delay factor is equal to or greater than the predetermined value T1 (step S206; Yes), the control unit 31 determines the amount of movement from the corresponding position of the notification image C to the adjustment position (step S207). As described above, the control unit 31 refers to the movement amount determination table TA5 to determine the amount of movement corresponding to the relative speed V calculated in step S202.

[0075] Next, the control unit 31 determines whether or not the vehicle 1 is in a specific moving state in which the forward object F is approaching the vehicle 1 obliquely, as described above (step S207A).

[0076] If the forward object F is not in the specific movement state (step S207A; No), the control unit 31 executes an image position adjustment process to draw the notification image C at an adjustment position moved from the corresponding position by the movement amount determined in step S207 (step S208). Note that if step S207A is No and the process transitions to step S208, the control unit 31 may execute the image position adjustment process even if the forward object F is away from the host vehicle 1.

[0077] If the forward object F is in a specific moving state (step S207A; Yes), the control unit 31, when the movement amount determined in step S207 is Mf, corrects the movement amount of the notification image C to (Mf + ΔM) by adding a correction amount ΔM to Mf (step S207B), and performs image position adjustment processing with the corrected movement amount (step S208).

[0078] After executing step S208 or step S204, the control unit 31 executes the process again from step S201. This completes the description of the second embodiment.

[0079] Third embodiment In the third embodiment, the control unit 31 displays, as an image for notifying a forward object F, a notification image C configured from a linear image (an image visually recognized as a line), as shown in FIG. 14A, for example.

[0080] The notification image C is drawn, for example, by a vector image, and as shown in Fig. 14A, includes a specific linear image Cs visually recognized along a direction intersecting with the direction in which the vehicle 1 and the forward object F face each other (the up-down direction in Fig. 14A), and a predetermined linear image Cn visually recognized as extending from each of both ends of the specific linear image Cs toward the front of the vehicle 1. Note that the predetermined linear image Cn in this embodiment is an image that is not subject to the line width enlargement process described below.

[0081] The notification image C shown in Fig. 14A is an example of a mode in which the tips of the two predetermined linear images Cn are inclined so as to approach each other, creating a sense of perspective and being visually recognized along the road surface ahead. Note that Fig. 14A is an example in which the notification image C is displayed shifted from the desired display position due to the above-mentioned display shift.

[0082] The control unit 31 according to the third embodiment executes a line width enlargement process, which will be described later, in order to reduce the discomfort felt by the user due to the occurrence of the display deviation as shown in Fig. 14A. The control unit 31, which executes the display control process including the line width enlargement process, mainly functions as an acquisition unit, a drawing control unit, and an identification unit. The identification unit according to the third embodiment identifies the cause of the display delay in the same manner as in the first and second embodiments.

[0083] (About line width expansion) When the display delay factor is specified by the specifying means, the control unit 31 as a drawing control means executes a line width enlargement process for drawing the notification image C so that the line width of the specific linear image Cs is enlarged by a predetermined enlargement amount in the direction in which the forward object F moves relative to the vehicle 1 and is visually recognized when the value indicated by the specified display delay factor is equal to or greater than a predetermined value. The line width enlargement process is executed by enlarging a predetermined reference line width by the determined enlargement amount. The reference line width is the width of the linear image (including the specific linear image Cs) at a predetermined distance P (see FIG. 1B) determined according to the type of the notification image C. The reference line width does not have to be fixed within the display area of ​​the virtual image A, and may be variable according to reference line width information (mathematical formula data or table data) that specifies the correspondence relationship between the distance from the vehicle 1 to the forward object F and the reference line width. This reference line width information is stored in advance in the storage unit 32, and is set so that the reference line width becomes narrower as the distance to the forward object F to be notified becomes longer, for example, by using the law of perspective. The control unit 31 draws the notification image C, which is made up of a linear image, with the reference line width based on the distance from the vehicle 1 to the forward object F and the reference line width information.

[0084] When the value indicated by the display delay factor is equal to or greater than a predetermined value, the control unit 31 sets the enlargement amount to a predetermined value E (hereinafter, E is also referred to as the reference movement amount). Further, the control unit 31 obtains the relative speed V between the host vehicle 1 and the preceding object F, and refers to the enlargement amount determination table TA6 shown in FIG. 16 to determine the enlargement amount corresponding to the obtained relative speed V. The enlargement amount determination table TA6 is configured such that the relative speed and the enlargement amount correspond to each other, and is stored in advance in the ROM of the storage unit 32. For example, when the magnitude (absolute value) of the calculated relative speed V satisfies 0 ≦ V < V1, the control unit 31 sets the enlargement amount of the specific linear image Cs to E as it is. When the magnitude of the calculated relative speed V satisfies V1 ≦ V < V2, the control unit 31 sets the enlargement amount of the specific linear image Cs to α × E (α > 1). That is, the control unit 31 increases the enlargement amount when the relative speed V is a second value greater than the first value compared to when the relative speed V is the first value.

[0085] Note that the control unit 31 may variably control the reference enlargement amount E itself so that the enlargement amount of the specific linear image Cs increases as the value indicated by the display delay factor increases. Further, the control unit 31 may determine a coefficient (α or β in the example of FIG. 16) according to the obtained relative speed V, and multiply the determined coefficient by the reference enlargement amount E to obtain the enlargement amount of the specific linear image Cs corresponding to the relative speed V. Further, the relationship between the relative speed V and the enlargement amount of the specific linear image Cs is stored in advance in the ROM of the storage unit 32 as data of a function (for example, a linear function or a quadratic function), and the control unit 31 may determine the enlargement amount of the specific linear image Cs based on the obtained relative speed V and the function.

[0086] FIG. 14B shows a display example of the notification image C when the line width enlargement process is executed. For example, when the display delay factor increases and the preceding object F approaches the host vehicle 1, a display shift as shown in FIG. 14A occurs. In order to suppress the discomfort given to the user 4 due to this display shift, when the value indicated by the display delay factor is equal to or greater than a predetermined value, the control unit 31 according to the third embodiment executes a line width enlargement process of drawing the notification image C so that the line width of the specific linear image Cs is enlarged by the determined enlargement amount in the direction in which the preceding object F moves with respect to the vehicle 1 (the downward direction in the example of FIG. 14) and is visually recognized.

[0087] In particular, the control unit 31, which functions as a drawing control means, corrects the enlargement amount when executing the line width enlargement process so that when the forward object F is in a specific moving state in which it is approaching the vehicle 1 from a position more than a specific distance away from the vehicle 1 in an intersecting direction (e.g., y direction) that intersects with the traveling direction (e.g., x direction) of the vehicle 1, the enlargement amount is larger than when the forward object F is not in the specific moving state, as described in the second embodiment. If the enlargement amount determined as described above is Ef, when the control unit 31 determines that the forward object F is in a specific moving state, it adds a correction amount ΔE to Ef and sets the enlargement amount of the specific linear image Cs to (Ef+ΔE). In this way, the specific linear image Cs in the notification image C can be enlarged further toward the vehicle 1, so that the presence of the forward object F in a specific moving state in which it is assumed that it is difficult for the user 4 to grasp the sense of distance can be properly notified.

[0088] 15A shows an example in which the forward object F is a road division line (a white line in the example of the figure), and an alert image C notifying the forward object F is displayed superimposed on the division line to emphasize the presence of the division line. In the example of FIG. 15A, the alert image C is configured in a substantially rectangular enclosing shape, and of two linear images visually recognized along a direction intersecting the direction in which the vehicle 1 and the forward object F face each other (the direction in which the division line, which is the forward object F in FIG. 15A, extends), the one closer to the vehicle 1 is set as a specific linear image Cs. Among the alert images C configured as linear images, the ones other than the specific linear image Cs are set as predetermined linear images Cn. As the vehicle 1 approaches the lane marking, a display shift of the notification image C may occur as shown in Fig. 15A, but the display shift can be made less noticeable by executing a line width enlargement process for drawing the notification image C so that the line width of the specific linear image Cs is enlarged by a predetermined amount in the direction in which the forward object F moves relative to the vehicle 1 (leftward in Fig. 15B). In this way, even if a display shift occurs in which the notification image C is shifted to the right of the desired display position due to a display delay, the display shift can be made less noticeable.

[0089] (Display control processing) From here on, the display control process according to the third embodiment will be described with reference to Fig. 17. The control unit 31 executes the processes of steps S301 to S306 in the same manner as steps S101 to S106 described in the first embodiment.

[0090] In step S306, if the value T indicated by the display delay factor is equal to or greater than the predetermined value T1 (step S306; Yes), the control unit 31 determines whether the forward object F is approaching the vehicle 1 (step S306A). If the forward object F is not approaching the vehicle 1 (step S306A; No), the control unit 31 does not execute the line width enlargement process, and draws the notification image C with normal image control (step S304). As a result, for the notification image C in which the forward object F is not approaching the vehicle 1 and is considered to be of low importance to the user 4, the control load can be reduced without executing the line width enlargement process. If the forward object F is approaching the host vehicle 1 (step S306A; Yes), the control unit 31 refers to the enlargement amount determination table TA6 and determines the enlargement amount corresponding to the relative speed V obtained in step S302 (step S307).

[0091] Next, the control unit 31 determines whether or not the vehicle 1 is in a specific moving state in which the forward object F is approaching the vehicle 1 obliquely, similarly to step S207A in the second embodiment (step S307A).

[0092] If the forward object F is not in the specific movement state (step S307A; No), the control unit 31 executes a line width enlargement process to enlarge the line width of the specific linear image Cs in the notification image C by the enlargement amount determined in step S307 (step S308).

[0093] If the forward object F is in a specific moving state (step S307A; Yes), the control unit 31 corrects the enlargement amount of the notification image C to (Ef+ΔE) by adding a correction amount ΔE to Ef when the enlargement amount determined in step S307 is Ef (step S307B), and executes line width enlargement processing with the corrected enlargement amount (step S308).

[0094] After executing step S308 or step S304, the control unit 31 executes the process again from step S301. This completes the description of the third embodiment.

[0095] The present invention is not limited to the above-described embodiment and drawings, and can be modified (including the deletion of components) as appropriate without departing from the spirit of the present invention.

[0096] In the above, the value indicated by the display delay factor is the predicted delay time T L and the sensing delay time T S In the above example, the delay time T is the sum of the delay time T and the delay time T. However, the present invention is not limited to this. The display delay factor may be determined based on at least one of the drawing control load by the drawing control means and the transmission delay of the object information. For example, the value indicated by the display delay factor may be the predicted drawing load L shown in FIG. 7A. The control unit 31 may determine the magnification according to the first embodiment, the movement amount according to the second embodiment, and the enlargement amount according to the third embodiment according to the predicted drawing load L. In addition, although an example of determining the value indicated by the display delay factor using table data has been shown above, the control unit 31 may calculate the value indicated by the display delay factor using formula data. The formula data may be stored in advance in the ROM of the storage unit 32, and may be configured to be capable of calculating the value indicated by the display delay factor from various parameters such as the type, area, and number of content images.

[0097] The control unit 31 may execute a combination of the image enlargement process according to the first embodiment, the image position adjustment process according to the second embodiment, and the line width enlargement process according to the third embodiment. For example, the control unit 31 may determine a magnification when executing the image enlargement process and a movement amount when executing the image position adjustment process according to the identified display delay factor, execute the image enlargement process to enlarge the notification image C from a reference size, and execute the image position adjustment process to move the display position of the notification image C by the determined movement amount. In addition, when both the notification image C consisting of a linear image and the notification image C that is not a linear image (for example, a filled-in image) are displayed within the display area of ​​the virtual image A, the control unit 31 may be capable of executing at least one of the image enlargement process and the image position adjustment process, and the line width enlargement process.

[0098] The notification image C described in the third embodiment may have any shape or form as long as it is composed of a linear image including a specific linear image Cs. For example, the notification image C may be a ring-shaped linear image. In this case, any part of the ring-shaped linear image that is visually recognized along a direction intersecting the direction in which the vehicle 1 and the forward object F face each other can be set as the specific linear image Cs, and the other part can be set as the predetermined linear image Cn.

[0099] The specific linear image Cs in the notification image C described in the third embodiment may be a horizontal linear image extending in the left-right direction as viewed from the user 4. In addition, in the line width enlargement process, at least a part of the specific linear image Cs whose width has been enlarged relative to the reference line width may be blurred. For example, the part of the specific linear image Cs whose width has been enlarged from the reference line width may be blurred, or the entire enlarged specific linear image Cs may be blurred. In this way, for example, it is possible to reduce the strange appearance caused by the enlargement of the specific linear image Cs, and also to improve the superimposition on the forward object F. In addition, in the line width enlargement process, the line width of the predetermined linear image Cn (for example, a vertical line-shaped image extending approximately in the up-down direction as viewed from the user 4) may be enlarged together with the enlargement of the line width of the specific linear image Cs. The process of enlarging the line width of the specific linear image Cn together with the enlargement of the line width of the specific linear image Cs in this way is useful, for example, when the forward object F moves obliquely relative to the vehicle 1.

[0100] Although a configuration example in which the forward object detection unit 40 senses the forward object F from the host vehicle 1 has been described above, the present invention is not limited to this. The forward object detection unit 40 may be configured to receive object information related to a forward object F from outside the vehicle 1 and supply the received object information to the control unit 31. The forward object detection unit 40 configured as such may be configured with various modules that enable communication between the vehicle 1 and a wireless network (V2N: Vehicle To Cellular Network), communication between the vehicle 1 and other vehicles (V2V: Vehicle To Vehicle), communication between the vehicle 1 and pedestrians (V2P: Vehicle To Pedestrian), and communication between the vehicle 1 and roadside infrastructure (V2I: Vehicle To Roadside Infrastructure). For example, (i) the forward object detection unit 40 may include a communication module that can directly access a WAN (Wide Area Network), a communication module for communicating with an external device (such as a mobile router) that can access the WAN, an access point of a public wireless LAN (Local Area Network), or the like, and may perform Internet communication, or may include a GPS controller that calculates the position of the vehicle 1 based on a GPS (Global Positioning System) signal received from an artificial satellite or the like. With these configurations, communication by V2N is possible. (ii) The forward object detection unit 40 may also include a wireless communication module that complies with a predetermined wireless communication standard and perform communication by V2V or V2P. (iii) The forward object detection unit 40 may also include a communication device that performs wireless communication with roadside infrastructure, and may obtain forward information of the vehicle 1 from a base station of a Driving Safety Support System (DSSS) via a roadside wireless device installed as infrastructure, for example. This enables communication by V2I.

[0101] That is, the forward object detection unit 40 may be configured to enable communication by V2X (Vehicle To Everything) between the vehicle 1 and the outside of the vehicle 1. The forward object detection unit 40 may be configured to be composed of a combination of a plurality of devices among the above-mentioned imaging devices, sensor groups, and various modules that enable communication by V2X, and to supply object information to the control unit 31 from each of the plurality of devices.

[0102] By tilting the display surface of the display unit 20 (the display surface of an LCD, or a screen if a DMD or LCOS is used), the virtual surface on which the virtual image A is displayed may be tilted forward with respect to the vertical direction of the vehicle 1.

[0103] The projection target of the display light Q is not limited to the windshield 3, and may be a combiner constituted by a plate-shaped half mirror, a hologram element, or the like. The display device that executes the display control process described above is not limited to the HUD device 10. The display device may be configured as a head mounted display (HMD) worn on the head of a user 4 of the vehicle 1. In addition, the display control of the notification image C in the image displayed as a virtual image by the HMD may be executed in a manner similar to that described above. In other words, the display device is not limited to one mounted on the vehicle 1, and may be one that is used in the vehicle 1.

[0104] The HUD device 10 as an example of the display device described above displays a superimposed image that is visually recognized by the user 4 as a virtual image A superimposed on the scenery ahead of the vehicle 1. The HUD device 10 includes a control unit 31 that functions as an acquisition means, a drawing control means, and a specification means. The acquisition means acquires object information that is information about a forward object F present ahead of the vehicle 1 and includes at least position information of the forward object F. The drawing control means draws an announcement image C for notifying information about the forward object F at a corresponding position that corresponds to the position information in the display area of ​​the superimposed image based on the object information acquired by the acquisition means. The specification means specifies a display delay factor that is a factor that causes the announcement image C, which is drawn based on the object information acquired at a predetermined timing, to be displayed with a delay from the predetermined timing. The display delay factor is specified based on at least one of a drawing control load by the drawing control means and a transmission delay of the object information.

[0105] (1-1) In the HUD device 10 according to the first embodiment, when the value indicated by the identified display delay factor is equal to or greater than a predetermined value, the drawing control means executes an image enlargement process to draw the notification image C at a size larger than a predetermined standard size. According to this configuration, as described above, it is possible to prevent the user 4 viewing the notification image C from feeling uncomfortable.

[0106] (1-2) Specifically, when executing the image enlargement process, the drawing control means draws the notification image C at a magnification ratio relative to the reference size determined based on the cause of the display delay.

[0107] (1-3) In addition, when the drawing control means executes the image enlargement process in the case where the vehicle 1 and the forward object F are approaching each other relatively in a specified direction, the drawing control means draws the notification image C so that it is enlarged and viewed toward the vehicle 1, based on an end B of the notification image C that is viewed at a position far from the vehicle 1 in the specified direction. According to this configuration, even if a display delay causes a display deviation in which the notification image C is shifted from a desired display position, the display deviation can be effectively made inconspicuous.

[0108] (1-4) In addition, when the vehicle 1 and the object ahead F are approaching each other relatively and the relative speed V between the vehicle 1 and the object ahead F is equal to or greater than a predetermined speed, the drawing control means draws the notification image C at a magnification relative to a reference size determined based on the display delay factor and the relative speed V when executing the image enlargement process. According to this configuration, the ability of the notification image C to follow the forward object F can be ensured.

[0109] (1-5) Furthermore, when the forward object F is an immovable object relative to the ground, if the notification image C for notifying information about the immovable object is a predetermined specific type of image (an image that is acceptable for making the notification even if it is viewed shifted in the direction of travel of the vehicle 1 from the desired display position), the drawing control means does not perform image enlargement processing on the specific type of image. According to this configuration, it is not necessary to perform image enlargement processing on the specific type of notification image C, so that the control load can be reduced.

[0110] (1-6) Furthermore, when the forward object F is a moving object moving away from the vehicle 1 at a relative speed greater than or equal to a predetermined speed, the drawing control means does not execute image enlargement processing on the notification image C for notifying information about the moving object. According to this configuration, it is not necessary to perform image enlargement processing on the notification image C that is considered to be of low importance to the user 4, so that the control load can be reduced.

[0111] (1-7) Specifically, the identification means identifies the drawing control load as a cause of display delay based on at least one of the area, number, and dimension of a content image, which is an image drawn by the drawing control means within the display area of ​​the superimposed image and includes an alarm image C.

[0112] (2-1) In the HUD device 10 of the second embodiment, when the value indicated by the identified display delay factor is equal to or greater than a predetermined value, the drawing control means executes an image position adjustment process to draw an alarm image C that is visually recognized at an adjusted position obtained by moving the corresponding position (desired display position) by a predetermined movement amount relative to the vehicle 1 in the direction in which the forward object F moves. According to this configuration, as described above, it is possible to prevent the user 4 viewing the notification image C from feeling uncomfortable.

[0113] (2-2) Specifically, when performing the image position adjustment process, the drawing control means determines the amount of movement based on the cause of the display delay.

[0114] (2-3) In addition, the drawing control means controls the amount of movement when performing the image position adjustment process in accordance with the relative speed V between the vehicle 1 and the forward object F, and makes the amount of movement larger when the relative speed V is a second value greater than the first value than when the relative speed V is a first value. According to this configuration, the ability of the notification image C to follow the forward object F can be ensured.

[0115] (2-4) In addition, when the forward object F is in a specific movement state in which it is approaching vehicle 1 from a position that is more than a specific distance away from vehicle 1 in an intersecting direction that intersects with the traveling direction of vehicle 1, the drawing control means corrects the movement amount when performing the image position adjustment process so that it is larger than when the forward object F is not in the specific movement state. According to this configuration, the notification image C can be displayed closer to the vehicle 1, so that the user 4 can be effectively notified of the presence of the forward object F in a specific moving state, which is assumed to be difficult for the user 4 to grasp the sense of distance.

[0116] (2-5) Specifically, the identification means identifies the drawing control load as a cause of display delay based on at least one of the area, number, and dimensions of a content image, which is an image drawn by the drawing control means within the display area of ​​the superimposed image and includes an alarm image C.

[0117] (3-1) In the HUD device 10 according to the third embodiment, the notification image C includes a specific linear image Cs that is configured from linear images and is visually recognized along a direction intersecting the direction in which the vehicle 1 and the forward object F face each other. When the value indicated by the identified display delay factor is equal to or greater than a predetermined value, the drawing control means executes a line width enlargement process for drawing the notification image C such that the line width of the specific linear image Cs is visually enlarged by a predetermined enlargement amount in the direction in which the forward object F moves relative to the vehicle 1. According to this configuration, as described above, it is possible to prevent the user 4 viewing the notification image C from feeling uncomfortable.

[0118] (3-2) Specifically, when executing the line width enlargement process, the drawing control means determines the amount of enlargement based on the cause of the display delay.

[0119] (3-3) In addition, the drawing control means controls the amount of enlargement when executing the line width enlargement process in accordance with the relative speed V between the vehicle 1 and the forward object F, and increases the amount of enlargement when the relative speed V is a second value greater than the first value compared to when the relative speed V is a first value. According to this configuration, the ability of the notification image C to follow the forward object F can be ensured.

[0120] (3-4) In addition, when the forward object F is in a specific movement state, the drawing control means corrects the enlargement amount when executing the line width enlargement process so that the enlargement amount is larger than when the forward object F is not in a specific movement state. According to this configuration, the notification image C can be displayed closer to the vehicle 1, so that the user 4 can be effectively notified of the presence of the forward object F in a specific moving state, which is assumed to be difficult for the user 4 to grasp the sense of distance.

[0121] (3-5) Specifically, the identification means identifies the drawing control load as a cause of display delay based on at least one of the area, number, and dimensions of a content image, which is an image drawn by the drawing control means within the display area of ​​the superimposed image and includes an alarm image C.

[0122] In the above description, in order to facilitate understanding of the present invention, descriptions of well-known technical matters have been omitted as appropriate. [Explanation of symbols]

[0123] 100...Vehicle display system 10…HUD device 20...Display section 30: control device, 31: control unit, 32: storage unit 40...forward object detection unit, 50...viewpoint detection unit, 60...ECU, 70...car navigation device Q: display light, A: virtual image, C: notification image, F: object in front Cs: specific linear image, Cn: predetermined linear image TA1...Predicted drawing load determination table TA2: Predicted delay time determination table TA3…First multiplier determination table TA4: Second multiplier determination table TA5...Movement amount determination table TA6…Expansion amount determination table 1...vehicle, 2...dashboard, 3...windshield, 4...user, 5...eyebox

Claims

1. A display device that displays a superimposed image that is visually recognized by a user as a virtual image that is superimposed on a view ahead of a vehicle, an acquisition means for acquiring object information relating to a forward object present in front of the vehicle, the object information including at least position information of the forward object; a drawing control means for drawing a notification image for notifying information about the forward object at a corresponding position corresponding to the position information within a display area of ​​the superimposed image based on the object information acquired by the acquisition means; and a determination unit that determines a display delay factor that causes the notification image drawn based on the object information acquired at a predetermined timing to be displayed with a delay from the predetermined timing, the cause of the display delay is identified based on at least one of a drawing control load imposed by the drawing control means and a transmission delay of the object information; the drawing control means, when a value indicated by the identified display delay factor is equal to or greater than a predetermined value, executes an image enlargement process for drawing the notification image at a size larger than a predetermined reference size, and, when the forward object is a moving object moving away from the vehicle at a relative speed equal to or greater than a predetermined relative speed, does not execute the image enlargement process for the notification image for notifying information about the moving object. Display device.

2. A display device that displays a superimposed image that is visually recognized by a user as a virtual image that is superimposed on a view ahead of a vehicle, an acquisition means for acquiring object information relating to a forward object present in front of the vehicle, the object information including at least position information of the forward object; a drawing control means for drawing a notification image for notifying information about the forward object at a corresponding position corresponding to the position information within a display area of ​​the superimposed image based on the object information acquired by the acquisition means; and a determination unit that determines a display delay factor that causes the notification image drawn based on the object information acquired at a predetermined timing to be displayed with a delay from the predetermined timing, the cause of the display delay is identified based on at least one of a drawing control load imposed by the drawing control means and a transmission delay of the object information; the drawing control means, when a value indicated by the identified display delay factor is equal to or greater than a predetermined value, executes an image enlargement process for drawing the notification image at a size larger than a predetermined reference size; The determination means determines the drawing control load as the cause of the display delay based on at least one of an area, a number, and a dimension of a content image, which is an image drawn in the display area by the drawing control means and includes the notification image. Display device.

3. When executing the image enlargement process, the drawing control means draws the notification image at a magnification ratio with respect to the reference size determined based on the cause of the display delay. The display device according to claim 1 .

4. and when the vehicle and the forward object are relatively approaching each other in a predetermined direction, the drawing control means, when executing the image enlargement process, draws the notification image so that the notification image is enlarged and viewed toward the vehicle side based on an end of the notification image that is viewed at a position far from the vehicle in the predetermined direction. The display device according to claim 1 .

5. when the vehicle and the forward object are relatively approaching each other and a relative speed between the vehicle and the forward object is equal to or greater than a predetermined speed, the drawing control means draws the notification image at a magnification ratio with respect to the reference size determined based on the cause of the display delay and the relative speed when executing the image enlargement process. The display device according to claim 1 .

6. the drawing control means, when the forward object is an immovable object relative to the ground, and when the notification image for notifying information regarding the immovable object is an image of a predetermined specific type, does not execute the image enlargement process for the image of the predetermined specific type. The display device according to claim 1 .

7. The determination means determines the drawing control load as the cause of the display delay based on at least one of an area, a number, and a dimension of a content image, which is an image drawn in the display area by the drawing control means and includes the notification image. The display device according to claim 1 .

Images