Presentation control method, presentation control device

Abstract

A computer executes a process of generating an image to be presented to the driver in a preset coordinate system, a process of detecting the positions of the driver's eyes and identifying the driver's viewpoint position in the passenger compartment on the basis of the detected eye positions, a process of identifying a display position where the generated image is displayed in the driver's field of view on the basis of the identified viewpoint position, a process of generating a driver's viewpoint image obtained by converting the generated image to a coordinate system based on the identified viewpoint position, a process of displaying the driver's viewpoint image so as to overlap the driver's field of view, a process of detecting the driver's line of sight, a process of determining whether or not the amount of change in the detected line of sight exceeds a preset threshold value, and a process of minimizing the change in the displayed position of the driver's viewpoint image occurring along with the change in the viewpoint position when it is determined that the amount of change in the detected line of sight exceeds the threshold value.

Description

Technical Field

[0001] The present invention relates to a display control method and a display control device.

Background Art

[0002] As a technique for correcting the position of an image displayed on a projection surface such as a windshield according to the position of a driver's viewpoint, for example, there is a technique disclosed in Patent Document 1. In the technique disclosed in Patent Document 1, the position of an image displayed superimposed on a preceding vehicle or the like on the projection surface is constantly corrected according to the movement of the driver's line of sight.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In the technique disclosed in Patent Document 1, since the position of the image on the projection surface is constantly corrected according to the movement of the driver's line of sight, while the driver's line of sight is moving, the position of the image is constantly changing, and a deviation occurs between the position of the object and the image, so there is a possibility that the driver may feel annoyed.

[0005] In view of the above problems, an object of the present invention is to provide a display control method and a display control device capable of reducing the annoyance felt by the driver.

Means for Solving the Problems

[0006] According to one aspect of the present invention, there is provided a display control method and a display control device that cause a computer to execute: a process of generating an image to be presented to a driver in a preset coordinate system; a process of detecting the position of the driver's eyes; a process of specifying a viewpoint position indicating the position of the driver's eyes in the vehicle interior based on the detected eye position; a process of specifying a display position for displaying the generated image in the driver's field of view based on the specified viewpoint position; a process of generating a driver's viewpoint image that is an image obtained by converting the generated image into a coordinate system based on the specified viewpoint position; a process of causing the driver's viewpoint image to be displayed so as to overlap the driver's field of view; a process of detecting the driver's line of sight; a process of determining whether or not a change amount of the detected line of sight exceeds a preset threshold value; and a process of suppressing a change in the display position of the driver's viewpoint image accompanying a change in the viewpoint position when it is determined that the change amount of the detected line of sight exceeds the threshold value.

Advantages of the Invention

[0007] According to the present invention, it is possible to provide a display control method and a display control device that can reduce the annoyance felt by the driver.

Brief Description of the Drawings

[0008]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Modes for Carrying Out the Invention

[0009] Hereinafter, embodiments of the present invention will be described with reference to the drawings. In the description of the drawings, the same or similar parts are denoted by the same or similar reference numerals, and redundant descriptions are omitted. Each drawing is schematic and may differ from the actual one. The embodiments shown below exemplify devices and methods for embodying the technical idea of the present invention, and the technical idea of the present invention is not limited to the devices and methods exemplified in the following embodiments. The technical idea of the present invention can be variously modified within the technical scope described in the claims.

[0010] (First Embodiment) Hereinafter, the first embodiment of the present invention will be described with reference to the drawings. (Configuration) The configuration of the display control device 1 will be described with reference to FIG. 1. The display control device 1 is provided in a vehicle and includes a HUD (Head-Up Display) 10. The vehicle equipped with the HUD 10 is, for example, an electric vehicle equipped with an electric motor as a drive source, an engine vehicle equipped with an internal combustion engine as a drive source, or a hybrid vehicle equipped with both an electric motor and an internal combustion engine as drive sources.

[0011] <hud> The HUD 10 includes a controller 20 and a projector 30. The controller 20 is an electronic control unit (ECU: Electronic Control Unit) that controls the operation of the display control device 1. Further, the controller 20 is an electronic control unit that performs display control of the HUD 10. The controller 20 includes, for example, a processor and peripheral components such as a storage device. The processor may be, for example, a CPU or an MPU. The storage device may include non-temporary tangible storage media such as a register, a cache memory, a ROM used as a main storage device, a RAM, and other memories. Note that the storage device may store, for example, the dominant eye of the vehicle driver.

[0012] The functions of the controller 20 are realized, for example, when the processor executes a computer program stored in the storage device. The specific configuration of the controller 20 will be described later. In addition, the controller 20 receives information input from a radar 2, a camera 3, an IVI (In-Vehicle Infotainment) 4, and a DMS (Driver Monitoring System) 5.

[0013] The radar 2 is formed using, for example, a laser range finder (LRF), a radar, a laser radar such as a LiDAR (Light Detection and Ranging). In addition, the radar 2 detects the relative position between an object existing around the vehicle and the vehicle, and outputs relative position information, which is information including the detected relative position, to the controller 20. Note that the objects existing around the vehicle are, for example, a preceding vehicle traveling ahead, a pedestrian existing around the vehicle, or a light vehicle (such as a bicycle).

[0014] The camera 3 photographs the surrounding environment of the vehicle, and outputs surrounding environment information, which is information including the detected surrounding environment of the vehicle, to the controller 20. Note that the attachment position of the camera 3 is stored in the storage device included in the controller 20. The IVI4 includes, for example, a storage device that stores map data and the like, and a GPS (Global Positioning System) or the like that detects the current position of the vehicle. Further, the IVI4 outputs road environment information, which is navigation information such as the traveling direction at an intersection and information indicating what kind of building (commercial facility, hospital, government office, etc.) the building visible to the driver during driving is, to the controller 20.

[0015] The DMS5 is formed, for example, using an imaging device disposed in the vehicle interior, and detects the position of the eyes, the direction of the line of sight, and the orientation of the face of the driver sitting in the driver's seat. Then, the DMS5 outputs viewpoint position information, which is information indicating the detected position of the eyes, line-of-sight direction information, which is information indicating the detected direction of the line of sight, and head direction information, which is information indicating the detected orientation of the face, to the controller 20. The projector 30 projects an image onto the W / S (windshield) 6. The W / S6 is the inner surface of the front windshield of the vehicle. Note that the three-dimensional shape of the W / S6 is stored in the storage device included in the controller 20.

[0016] (Specific configuration of the controller) As shown in FIG. 2, the controller 20 includes a display target position calculation unit 21, a reference display position calculation unit 22, a reference display image generation unit 23, a viewpoint position calculation unit 24, and a viewpoint correction value calculation unit 25. In addition to this, the controller 20 includes a corrected display position calculation unit 26, an image correction control unit 27, and a corrected display image generation unit 28.

[0017] <Display target position calculation unit> The display target position calculation unit 21 refers to the relative position information and the surrounding environment information, and calculates the three-dimensional coordinates (coordinates on the X-axis, coordinates on the Y-axis, coordinates on the Z-axis) of the object (preceding vehicle, pedestrian, light vehicle) that is the target of the image projected onto the W / S6 as the coordinates indicating the position of the object. The three-dimensional coordinates of the object calculated by the display target position calculation unit 21 are, for example, a coordinate system based on the viewpoint position of the camera 3. Then, the display target position calculation unit 21 outputs display target position information, which is information including the three-dimensional coordinates of the object, to the reference display image generation unit 23.

[0018] <Reference display position calculation unit> The reference display position calculation unit 22 calculates a reference position (reference display position) for displaying an image projected onto the W / S6 with the position of the driver's head and the driver's viewpoint as reference positions. Note that the reference display position calculated by the reference display position calculation unit 22 is calculated with the position of the driver's head and the driver's viewpoint as reference positions, for example, in a state where the driver sitting in the driver's seat drives the vehicle straight ahead (a state where the driver is facing forward). Then, the reference display position calculation unit 22 outputs reference display position information, which is information including the reference display position, to the reference display image generation unit 23.

[0019] <Reference display image generation unit> The reference display image generation unit 23 performs a process of generating an image (virtual image) corresponding to the object with reference to the relative position information, the surrounding environment information, and the display target position information. Then, the reference display image generation unit 23 outputs reference display image information, which is information including the generated image, to the corrected display image generation unit 28. Note that the image (virtual image) corresponding to the object is, for example, an icon IC or the like indicating the presence of the preceding vehicle LV as shown in FIG. 3. Note that the icon IC is formed of a blinking rectangle having an area corresponding to the vehicle width of the preceding vehicle LV, for example. Also, the position where the icon IC is displayed is set, for example, behind the preceding vehicle LV as viewed by the driver. Also, the image generated by the reference display image generation unit 23 is generated, for example, in a coordinate system based on the viewpoint position of the camera 3. Therefore, the reference display image generation unit 23 provided in the controller 20 executes a process of generating an image to be presented to the driver in a preset coordinate system (a coordinate system based on the viewpoint position of the camera 3).

[0020] <Viewpoint position calculation unit> The viewpoint position calculation unit 24 performs a process of calculating the viewpoint position, which is the position of the eyes of the driver sitting in the driver's seat, with reference to the viewpoint position information. Then, the viewpoint position calculation unit 24 outputs the viewpoint position information, which is information including the calculated viewpoint position, to the viewpoint correction value calculation unit 25. Note that the viewpoint position calculated by the viewpoint position calculation unit 24 is the position of both eyes (right eye ER and left eye EL) of the driver, and is calculated in three-dimensional coordinates. In FIG. 3, the head of the driver sitting in the driver's seat is indicated by the reference sign "DH". Therefore, the viewpoint position calculation unit 24 provided in the controller 20 executes a process of detecting the position of the driver's eyes and a process of specifying the viewpoint position indicating the position of the driver's eyes in the passenger compartment based on the detected eye positions.

[0021] <Viewpoint correction value calculation unit> The viewpoint correction value calculation unit 25 calculates a viewpoint correction value, which is the amount of change between the reference line of sight and the current line of sight of the driver sitting in the driver's seat, with reference to the viewpoint position information and the line of sight direction information. Note that the reference line of sight is, for example, the line of sight calculated based on the position of the driver's head and the position of the driver's eyes in a state where the driver sitting in the driver's seat is facing forward. That is, the reference line of sight is, for example, a line of sight facing forward. Then, the viewpoint correction value calculation unit 25 outputs the viewpoint correction value information, which is information including the calculated viewpoint correction value, to the correction display position calculation unit 26 and the image correction control unit 27. Therefore, the viewpoint correction value calculation unit 25 provided in the controller 20 executes a process of detecting the line of sight of the driver.

[0022] <Correction display position calculation unit> The correction display position calculation unit 26 calculates a correction display position, which is the position on the W / S6 where an image is projected corresponding to the current line of sight of the driver, with reference to the reference display image information and the viewpoint correction value information. Then, the correction display position calculation unit 26 outputs the correction display position information, which is information including the calculated correction display position, to the image correction control unit 27 and the correction display image generation unit 28. Therefore, the correction display position calculation unit 26 included in the controller 20 executes a process of specifying a display position for displaying the generated image in the driver's field of view based on the specified viewpoint position.

[0023] <Image correction control unit> The image correction control unit 27 stores an area (displayable area) in which the projector 30 can display an image on the W / S6, coordinates of the center point of the displayable area SE (center point coordinates), and a threshold value (change amount threshold value). As shown in FIG. 3, the displayable area SE is a part of the W / S6 and is set and stored in advance. In the first embodiment, as an example, the case where the displayable area SE is set as a quadrilateral area will be described.

[0024] The center point coordinates are coordinates calculated in three-dimensional coordinates and are stored in advance. The change amount threshold value includes a horizontal angle threshold value φth and a vertical angle threshold value ψth. The horizontal angle threshold value φth is a value obtained by setting an angle along the vehicle width direction (left - right direction) from the center point coordinates, and is a preset value. In the first embodiment, as an example, as shown in FIG. 3, the horizontal threshold band HA (φth×2) is shown as the combined value of the horizontal angle threshold value φth set from the center point coordinates to the left direction and the horizontal angle threshold value φth set from the center point coordinates to the right direction.

[0025] The vertical angle threshold value ψth is a value obtained by setting an angle along the height direction (up - down direction) from the center point coordinates, and is a preset value. In the first embodiment, as an example, as shown in FIG. 3, the vertical threshold band VA (ψth×2) is shown as the combined value of the vertical angle threshold value ψth set from the center point coordinates to the up direction and the vertical angle threshold value ψth set from the center point coordinates to the down direction. Further, the image correction control unit 27 calculates a change amount from the center point coordinates of the line of sight included in the line of sight direction information with reference to the line of sight direction information and the center point coordinates. Specifically, a horizontal change amount φd, which is a change amount in the horizontal direction of the line of sight included in the line of sight direction information with respect to the center point coordinates, is calculated. In addition to this, a vertical change amount ψd, which is a change amount in the vertical direction of the line of sight included in the line of sight direction information with respect to the center point coordinates, is calculated.

[0026] Furthermore, the image correction control unit 27 compares the horizontal change amount φd with the horizontal angle threshold φth and compares the vertical change amount ψd with the vertical angle threshold ψth. When it is determined that the horizontal change amount φd is within the horizontal angle threshold φth and the vertical change amount ψd is within the vertical angle threshold ψth, the image output command information is output to the corrected display image generation unit 28. The image output command information is information including an image output command for outputting the generated image.

[0027] On the other hand, when it is determined that at least one of the condition that the horizontal change amount φd exceeds the horizontal angle threshold φth and the condition that the vertical change amount ψd exceeds the vertical angle threshold ψth is satisfied, the image suppression command information is output to the corrected display image generation unit 28. The image suppression command information is information including an image suppression command for suppressing a change in the position where the driver's viewpoint image described later is displayed. Also, when the image correction control unit 27 determines that at least one of the condition that the horizontal change amount φd exceeds the horizontal angle threshold φth and the condition that the vertical change amount ψd exceeds the vertical angle threshold ψth is satisfied, the image position fixing information is output to the corrected display image generation unit 28. The image position fixing information is information including an image position fixing signal for fixing the position where the driver's viewpoint image is displayed. In addition to this, when the change amount (φd, ψd) that exceeded the threshold (φth, ψth) returns below the threshold, the fixing release information, which is information including a fixing release signal for releasing the fixing of the position where the driver's viewpoint image is displayed, is output to the corrected display image generation unit 28.

[0028] Therefore, the image correction control unit 27 provided in the controller 20 executes a process of determining whether or not the detected change amount of the line of sight exceeds a preset threshold. Further, when the image correction control unit 27 included in the controller 20 determines that the detected amount of change in the line of sight exceeds the threshold value, it executes a process of suppressing a change in the position where the driver's viewpoint image is displayed along with the change in the viewpoint position. Also, when the image correction control unit 27 included in the controller 20 determines that the amount of change (φd, ψd) exceeds the threshold values (φth, ψth), it fixes the position where the driver's viewpoint image is displayed. Further, when the amount of change (φd, ψd) that has exceeded the threshold values (φth, ψth) returns below the threshold values, the fixation of the position where the driver's viewpoint image is displayed is released.

[0029] <Correction display image generation unit> The correction display image generation unit 28 generates a driver's viewpoint image, which is an image obtained by converting the image generated by the reference display image generation unit 23 into a coordinate system based on the driver's current viewpoint position, by referring to the reference display image information and the viewpoint correction value information. Furthermore, when the correction display image generation unit 28 receives an input of image output command information, it performs a process of causing the driver's viewpoint image to be displayed so as to overlap the driver's field of view. The driver's field of view is calculated, for example, by detecting the position of the driver's eyes with the camera 3. Then, the correction display image generation unit 28 generates projection image information, which is information including an image to be projected onto the W / S6, and outputs the generated projection image information to the projector 30. The projector 30 that has received the input of the projection image information projects an image onto the W / S6.

[0030] Therefore, the correction display image generation unit 28 that has received the input of the image output command information generates, as the projection image information, information for causing the driver's viewpoint image (icon IC) to be displayed so as to overlap the driver's field of view according to the driver's current viewpoint position. Also, when the correction display image generation unit 28 receives an input of image position fixation information in addition to the image suppression command information, it generates, as the projection image information, information for fixing the position where the driver's viewpoint image is displayed at the time when it is determined that the amount of change (φd, ψd) exceeds the threshold values (φth, ψth). Furthermore, when the correction display image generation unit 28 receives an input of unlocking information in addition to the image suppression command information, the correction display image generation unit 28 generates, as projection image information, information for releasing the fixation of the position where the driver's viewpoint image is to be displayed.

[0031] Therefore, the correction display image generation unit 28 provided in the controller 20 executes a process of generating a driver's viewpoint image, which is an image obtained by converting the generated image into a coordinate system based on the specified viewpoint position. In addition, the correction display image generation unit 28 provided in the controller 20 executes a process of displaying the driver's viewpoint image so as to overlap with the driver's field of view.

[0032] (Operation) Next, with reference to FIGS. 1 to 3 and using FIG. 4, an example of a display control method performed using the display control device 1 during vehicle travel will be described. During vehicle travel, in step S1, the controller 20 uses the camera 3 to acquire the position (three-dimensional coordinates) of the object as information on the object to be displayed (projected) on the W / S6. In step S2, the controller 20 calculates the reference display position. In step S3, the controller 20 detects, in three-dimensional coordinates, the viewpoint position of the driver sitting in the driver's seat. In step S4, the controller 20 Of the line of sight detects the horizontal change amount φd and the vertical change amount ψd of the driver.

[0033] In step S5, the controller 20 determines whether the horizontal change amount φd is within the horizontal angle threshold φth and whether the vertical change amount ψd is within the vertical angle threshold ψth. If it is determined that the horizontal change amount φd is within the horizontal angle threshold φth and the vertical change amount ψd is within the vertical angle threshold ψth (step S5: Yes), the process proceeds to step S6. On the other hand, if it is determined that at least one of the condition that the horizontal change amount φd exceeds the horizontal angle threshold φth and the condition that the vertical change amount ψd exceeds the vertical angle threshold ψth is satisfied (step S5: No), the process proceeds to step S7. That is, in step S5, it is determined whether the angles (vertical change amount ψd, horizontal change amount φd) of the driver's line of sight are within the specified angles (vertical angle threshold ψth, horizontal angle threshold φth).

[0034] In step S6, the controller 20 calculates and updates the viewpoint correction value, and calculates the corrected display position corresponding to the driver's current line of sight according to the updated viewpoint correction value. In step S7, the controller 20 does not update the viewpoint correction value. Then, the corrected display position corresponding to the line of sight at the time when it is determined that at least one of the condition that the horizontal change amount φd exceeds the horizontal angle threshold φth and the condition that the vertical change amount ψd exceeds the vertical angle threshold ψth is satisfied is calculated. In step S8, the controller 20 causes the projector 30 provided in the HUD 10 to display at the corrected display position calculated in step S6 or step S7 among W / S6, and projects the driver's viewpoint image. Then, the process returns to step S1.

[0035] Note that when proceeding from step S6 to step S8, among W / S6, the icon IC which is the driver's viewpoint image is displayed at the current viewpoint position of the driver inside the displayable area SE. Therefore, when the driver's line of sight moves only inside the displayable area SE, the icon IC is displayed at the position where the driver's line of sight is directed inside the displayable area SE.

[0036] Also, when proceeding from step S7 to step S8, when it is determined that the condition that the change amounts (φd, ψd) within the displayable area SE among W / S6 exceed the threshold values (φth, ψth) is satisfied, an icon IC is displayed at a position corresponding to the line of sight at the determined time. As an example, in the state shown in FIG. 3, when the driver's line of sight exits the displayable area SE and moves significantly to the left and faces another vehicle entering the intersection from the left direction, the icon IC is displayed at the left end of the displayable area SE. Then, from the time when the driver's line of sight that has exited the displayable area SE and moved significantly to the left returns inside the displayable area SE, the icon IC is displayed at the position where the driver's line of sight is directed inside the displayable area SE.

[0037] Note that the above-described first embodiment is an example of the present invention, and the present invention is not limited to the above-described first embodiment. Various modifications can be made according to the design and the like as long as they do not depart from the technical idea of the present invention even in forms other than this embodiment.

[0038] (Effect of the First Embodiment) The display control device according to the first embodiment can achieve the following effects. (1) The computer included in the display control device 1 executes a process of generating an image presented to the driver in a preset coordinate system, a process of detecting the position of the driver's eyes, and a process of specifying a viewpoint position indicating the position of the driver's eyes in the vehicle interior based on the detected eye position. Further, based on the specified viewpoint position, a process of specifying a display position for displaying the generated image in the driver's field of view and a process of generating a driver's viewpoint image that is an image obtained by converting the generated image into a coordinate system based on the specified viewpoint position are executed. In addition, a process of displaying the driver's viewpoint image so as to overlap the driver's field of view, a process of detecting the driver's line of sight, and a process of determining whether or not the change amount of the detected line of sight exceeds a preset threshold value are executed. Then, when it is determined that the change amount of the detected line of sight exceeds the threshold value, a process of suppressing the change in the display position of the driver's viewpoint image accompanying the change in the viewpoint position is executed.

[0039] Therefore, if the amount of change in the driver's line of sight is equal to or less than the threshold value, it becomes possible to display the driver's viewpoint image at a position corresponding to the change in the driver's viewpoint position within the driver's field of view. On the other hand, if the amount of change in the driver's line of sight exceeds the threshold value, it becomes possible not to change the position of the image in order to suppress the change in the position where the driver's viewpoint image is displayed accompanying the change in the viewpoint position. As a result, it becomes possible to provide the display control device 1 that can reduce the annoyance felt by the driver.

[0040] (2) When the computer determines that the amount of change (φd, ψd) exceeds the threshold values (φth, ψth), it fixes the position where the driver's viewpoint image is displayed. Further, when the amount of change (φd, ψd) that has exceeded the threshold values (φth, ψth) returns below the threshold values, the fixation of the position where the driver's viewpoint image is displayed is released. As a result, when the driver moves the line of sight, such as when checking the left and right directions at an intersection or the like, the position where the driver's viewpoint image is displayed is fixed, and when the driver's line of sight returns to the original position, the fixation of the position where the driver's viewpoint image is displayed is released, thereby making it possible to suppress the flickering of the image.

[0041] Also, with the display control method of the first embodiment, it becomes possible to achieve the effects described below. (3) Cause the computer to execute a process of generating an image to be presented to the driver in a preset coordinate system, a process of detecting the position of the driver's eyes, and a process of specifying a viewpoint position indicating the position of the driver's eyes in the vehicle interior based on the detected eye position. Also, cause the computer to execute a process of specifying a display position for displaying the generated image in the driver's field of view based on the specified viewpoint position, and a process of generating a driver's viewpoint image that is an image obtained by converting the generated image into a coordinate system based on the specified viewpoint position. Further, cause the computer to execute a process of displaying the driver's viewpoint image so as to overlap the driver's field of view, a process of detecting the driver's line of sight, and a process of determining whether or not a change amount of the detected line of sight exceeds a preset threshold value. Then, when the computer determines that the detected change amount of the line of sight exceeds the threshold value, cause the computer to execute a process of suppressing a change in the position where the driver's viewpoint image is displayed accompanying a change in the viewpoint position.

[0042] Therefore, if the change amount of the driver's line of sight is equal to or less than the threshold value, it becomes possible to display the driver's viewpoint image at a position corresponding to the change in the driver's viewpoint position in the driver's field of view. On the other hand, if the change amount of the driver's line of sight exceeds the threshold value, it becomes possible to prevent the position of the image from changing in order to suppress the change in the position where the driver's viewpoint image is displayed accompanying the change in the viewpoint position. As a result, it becomes possible to provide a display control method capable of reducing the annoyance felt by the driver.

[0043] (Modification of the First Embodiment) (1) In the first embodiment, when it is determined that the change amounts (φd, ψd) exceed the threshold values (φth, ψth), the position where the driver's viewpoint image is displayed is fixed, and when the change amount exceeding the threshold value returns below the threshold value, the fixing of the position where the driver's viewpoint image is displayed is released. However, the present invention is not limited to this. That is, when it is determined that the change amount exceeds the threshold value, the driver's viewpoint image may be erased, and when the change amount exceeding the threshold value returns below the threshold value, the erased driver's viewpoint image may be redisplayed. In this case, assuming that the driver's line of sight is following another object (for example, a vehicle other than the preceding vehicle), by eliminating the object shown in the driver's viewpoint image (for example, the preceding vehicle), it is possible to reduce the annoyance felt by the driver with respect to the display of W / S6.

[0044] (2) In the first embodiment, when it is determined that the change amounts (φd, ψd) exceed the threshold values (φth, ψth), the position where the driver's viewpoint image is displayed is fixed, and when the change amount that has exceeded the threshold value returns below the threshold value, the fixing of the position where the driver's viewpoint image is displayed is released. However, the present invention is not limited to this. That is, when it is determined that the change amount exceeds the threshold value, the driver's viewpoint image may be faded out and eliminated. In this case, assuming that the driver's line of sight is following another object, by fading out and eliminating the object shown in the driver's viewpoint image, the display of W / S6 changes smoothly, and it is possible to reduce the annoyance felt by the driver with respect to the display of W / S6.

[0045] In addition to this, when the change amount that has exceeded the threshold value returns below the threshold value, the driver's viewpoint image that has been eliminated may be faded in and redisplayed in a time shorter than the time when the driver's viewpoint image that has been eliminated was faded out and eliminated. In this case, when the driver's line of sight returns from another object to the object shown in the driver's viewpoint image, the eliminated driver's viewpoint image is quickly displayed, so that the display of W / S6 changes smoothly according to the change in the line of sight. As a result, it is possible to reduce the annoyance felt by the driver with respect to the display of W / S6.

[0046] Also, when the change amount that has exceeded the threshold value returns below the threshold value, the eliminated driver's viewpoint image may be immediately redisplayed. In this case, when the driver's line of sight returns from another object to the object shown in the driver's viewpoint image, the deleted driver's viewpoint image is instantaneously displayed. Therefore, even when the change speed of the line of sight is high, the display of W / S6 changes instantaneously. This makes it possible to reduce the annoyance felt by the driver with respect to the display of W / S6.

[0047] (3) In the first embodiment, when it is determined that the change amounts (φd, ψd) exceed the threshold values (φth, ψth), the position where the driver's viewpoint image is displayed is fixed. When the change amount that has exceeded the threshold value returns below the threshold value, the fixing of the position where the driver's viewpoint image is displayed is released. However, the present invention is not limited to this. That is, for example, it is also possible to add hysteresis (α) to the threshold value and configure to fix the position where the driver's viewpoint image is displayed when it is determined that the change amounts (φd, ψd) exceed the threshold values (φth + α, ψth + α). Similarly, when the change amounts (φd, ψd) that have exceeded the threshold value return below the threshold values (φth - α, ψth - α), it is also possible to configure to release the fixing of the position where the driver's viewpoint image is displayed. In this case, by setting the value of hysteresis to an arbitrary value, it becomes possible to control the change in the position where the driver's viewpoint image is displayed without being limited to the area of the displayable area SE.

[0048] (Second Embodiment) Hereinafter, the second embodiment of the present invention will be described with reference to the drawings. (Configuration) With reference to FIGS. 1 to 3, the configuration of the second embodiment will be described. In the drawings and the following description, the same components as those in the above-described first embodiment are denoted by the same reference numerals. Also, in the following description, the description of the same components as those in the above-described first embodiment may be omitted. The configuration of the second embodiment is the same as the configuration of the first embodiment, except for the configurations of the image correction control unit 27 and the corrected display image generation unit 28.

[0049] <Image Correction Control Unit> When the image correction control unit 27 determines that the horizontal change amount φd is within the horizontal angle threshold φth and the vertical change amount ψd is within the vertical angle threshold ψth, it outputs the image output command information to the corrected display image generation unit 28. On the other hand, when it is determined that at least one of the condition that the horizontal change amount φd exceeds the horizontal angle threshold φth and the condition that the vertical change amount ψd exceeds the vertical angle threshold ψth is satisfied, the image suppression command information is output to the corrected display image generation unit 28.

[0050] Also, when the image correction control unit 27 determines that at least one of the condition that the horizontal change amount φd exceeds the horizontal angle threshold φth and the condition that the vertical change amount ψd exceeds the vertical angle threshold ψth is satisfied, it outputs the intensity reduction information to the corrected display image generation unit 28 at that time. The intensity reduction information is information including an intensity reduction signal that reduces the degree of emphasis of the driver's viewpoint image. In addition to this, when the change amount (φd, ψd) that exceeded the threshold (φth, ψth) returns below the threshold, intensity restoration information, which is information including an intensity restoration signal that restores the reduced degree of emphasis to its original state, is output to the corrected display image generation unit 28. The degree of emphasis of the driver's viewpoint image is, for example, at least one of the brightness and chroma of the driver's viewpoint image. Two In the first embodiment, as an example, the case where the degree of emphasis of the driver's viewpoint image is the brightness of the driver's viewpoint image will be described.

[0051] Also, when outputting the intensity reduction information, the image correction control unit 27 outputs the time-dependent reduction information including a time-dependent reduction signal that reduces the degree of emphasis of the driver's viewpoint image over time to the corrected display image generation unit 28. Furthermore, when outputting the intensity restoration signal, the image correction control unit 27 outputs the rapid restoration information including a rapid restoration signal that restores the reduced degree of emphasis in a time shorter than the time elapsed when reducing the degree of emphasis of the driver's viewpoint image to the corrected display image generation unit 28.

[0052] Therefore, when the image correction control unit 27 provided in the controller 20 determines that the change amounts (φd, ψd) exceed the threshold values (φth, ψth), the degree of emphasis of the driver's perspective image is decreased. Further, when the change amounts (φd, ψd) that exceeded the threshold values (φth, ψth) return below the threshold values, the decreased degree of emphasis is restored to the original level. Also, when the image correction control unit 27 provided in the controller 20 determines that the change amounts (φd, ψd) exceed the threshold values (φth, ψth), the degree of emphasis is decreased as time passes. Also, when the change amounts (φd, ψd) that exceeded the threshold values (φth, ψth) return below the threshold values, the image correction control unit 27 provided in the controller 20 restores the decreased degree of emphasis to the original level in a time shorter than the time elapsed when the degree of emphasis was decreased.

[0053] <Correction display image generation unit> When the correction display image generation unit 28 receives an input of intensity decrease information in addition to the image suppression command information, it generates, as projection image information, information for decreasing the degree of emphasis of the driver's perspective image at the time when it is determined that the change amounts (φd, ψd) exceed the threshold values (φth, ψth). Also, when the correction display image generation unit 28 receives an input of time-dependent decrease information in addition to the intensity decrease information, it generates, as projection image information, information for decreasing the degree of emphasis of the driver's perspective image as time passes. Furthermore, when the correction display image generation unit 28 receives an input of intensity restoration information in addition to the image suppression command information, it generates, as projection image information, information for displaying the driver's perspective image with the decreased degree of emphasis restored to the original level. Also, when the correction display image generation unit 28 receives an input of rapid restoration information in addition to the intensity restoration information, it generates, as projection image information, information for restoring the decreased degree of emphasis to the original level in a time shorter than the time elapsed when the degree of emphasis of the driver's perspective image was decreased.

[0054] (Operation) With reference to FIGS. 1 to 4 and using FIG. 5, an example of a display control method performed using the display control device 1 during vehicle travel will be described. For the same processing as in the above-described first embodiment, the same step numbers are assigned. During vehicle travel, in step S1, the controller 20 uses the camera 3 to acquire the position (three-dimensional coordinates) of the object as information on the object to be displayed (projected) on the W / S6. In step S2, the controller 20 calculates the reference display position. In step S3, the controller 20 detects the viewpoint position of the driver sitting in the driver's seat in three-dimensional coordinates. In step S4, the controller 20 of the driver Of the line of sight Detects the horizontal change amount φd and the vertical change amount ψd.

[0055] In step S5, the controller 20 determines whether the horizontal change amount φd is within the horizontal angle threshold φth and the vertical change amount ψd is within the vertical angle threshold ψth. When it is determined that the horizontal change amount φd is within the horizontal angle threshold φth and the vertical change amount ψd is within the vertical angle threshold ψth (step S5: Yes), the process proceeds to step S9. On the other hand, when it is determined that at least one of the condition that the horizontal change amount φd exceeds the horizontal angle threshold φth and the condition that the vertical change amount ψd exceeds the vertical angle threshold ψth is satisfied (step S5: No), the process proceeds to step S10. That is, in step S5, it is determined whether the angles (vertical change amount ψd, horizontal change amount φd) of the driver's line of sight are within the specified angles (vertical angle threshold ψth, horizontal angle threshold φth).

[0056] In step S9, the controller 20 calculates and updates the viewpoint correction value, and calculates the corrected display position corresponding to the driver's current line of sight according to the updated viewpoint correction value. In step S10, the controller 20 calculates and updates the viewpoint correction value, and calculates the corrected display position corresponding to the driver's current line of sight according to the updated viewpoint correction value. In addition, processing is performed to reduce the degree of emphasis of the driver's viewpoint image. In step S8, the controller 20 causes the projector 30 included in the HUD 10 to project a driver's viewpoint image on the corrected display position calculated in step S9 or step S10 among W / S6. After that, the process returns to step S1.

[0057] Note that when proceeding from step S9 to step S8, among W / S6, an icon IC, which is a driver's viewpoint image, is displayed at the current viewpoint position of the driver inside the displayable area SE. Therefore, when the driver's line of sight moves only inside the displayable area SE, the icon IC is displayed at the position where the driver's line of sight is directed inside the displayable area SE. Also, when proceeding from step S10 to step S8, among W / S6, an icon IC with reduced luminance is displayed at the current viewpoint position of the driver.

[0058] Note that the above-described second embodiment is an example of the present invention, and the present invention is not limited to the above-described second embodiment. Various changes can be made according to the design and the like as long as they do not deviate from the technical idea of the present invention even in forms other than this embodiment.

[0059] (Effect of the Second Embodiment) With the display control method using the display control device 1 of the second embodiment, the following effects can be achieved. (1) When the computer determines that the change amounts (φd, ψd) exceed the threshold values (φth, ψth), the degree of emphasis of the driver's viewpoint image is decreased. Further, when the change amounts (φd, ψd) that have exceeded the threshold values (φth, ψth) return below the threshold values, the decreased degree of emphasis is restored. As a result, when the driver moves the line of sight, such as when checking the left and right directions at an intersection or the like, the degree of emphasis of the driver's viewpoint image is decreased, and when the driver's line of sight returns to its original position, the decreased degree of emphasis is restored, thereby making it possible to suppress the flickering of the image.

[0060] (2) When the computer determines that the change amounts (φd, ψd) exceed the threshold values (φth, ψth), the computer reduces the degree of emphasis of the driver's viewpoint image over time. As a result, assuming that the driver's line of sight is following another object, by reducing the degree of emphasis of the object shown in the driver's viewpoint image over time, it becomes possible to smoothly change the display of W / S6. This makes it possible to reduce the annoyance that the driver feels with respect to the display of W / S6.

[0061] (3) When the change amounts (φd, ψd) that exceeded the threshold values (φth, ψth) return below the threshold values, the computer returns the reduced degree of emphasis to its original state in a time shorter than the time elapsed when reducing the degree of emphasis of the driver's viewpoint image. As a result, when the driver's line of sight returns from another object to the object shown in the driver's viewpoint image, the reduced degree of emphasis of the driver's viewpoint image quickly returns to its original state, so that the display of W / S6 changes smoothly according to the change in the line of sight. This makes it possible to reduce the annoyance that the driver feels with respect to the display of W / S6.

[0062] (4) The degree of emphasis of the driver's viewpoint image is at least one of the brightness and chroma of the driver's viewpoint image. As a result, by changing at least one of the brightness and chroma of the driver's viewpoint image according to the magnitude relationship between the threshold values (φth, ψth) and the change amounts (φd, ψd), it becomes possible to reduce the annoyance that the driver feels with respect to the display of W / S6.

[0063] (Modification of the Second Embodiment) (1) In the second embodiment, when the change amounts (φd, ψd) that exceeded the threshold values (φth, ψth) return below the threshold values, the configuration is such that the reduced degree of emphasis is returned to its original state in a time shorter than the time elapsed when reducing the degree of emphasis of the driver's viewpoint image. However, the present invention is not limited to this. That is, when the change amount that exceeded the threshold value returns below the threshold value, the configuration may be such that the reduced degree of emphasis is immediately returned to its original state. In this case, when the driver's line of sight returns from another object to the object shown in the driver's viewpoint image, the degree of emphasis of the reduced driver's viewpoint image instantly returns to its original state. Therefore, even when the speed of change of the line of sight is high, the display of W / S6 instantaneously changes according to the change in the line of sight. This makes it possible to reduce the annoyance that the driver feels with respect to the display of W / S6.

[0064] (Other Embodiments) As described above, embodiments of the present invention have been described. However, the discussion and drawings that form part of this disclosure should not be understood to limit the present invention. Various alternative embodiments, examples, and operating techniques will be apparent to those skilled in the art from this disclosure. In addition, the present invention naturally includes various embodiments and the like that are not described herein, such as configurations in which each configuration described in the above embodiments is arbitrarily applied. Therefore, the technical scope of the present invention is defined only by the invention-specific matters according to the legitimate claims based on the above description.

Description of Reference Numerals

[0065] 1... Display control device, 2... Radar, 3... Camera, 4... IVI, 5... DMS, 6... W / S, 10... HUD, 20... Controller, 21... Display target position calculation unit, 22... Reference display position calculation unit, 23... Reference display image generation unit, 24... Viewpoint position calculation unit, 25... Viewpoint correction value calculation unit, 26... Corrected display position calculation unit, 27... Image correction control unit, 28... Corrected display image generation unit, 30... Projector, LV... Leading vehicle, IC... Icon, ER... Right eye, EL... Left eye, DH... Driver's head, SE... Displayable area, HA... Horizontal threshold band, VA... Vertical threshold band< / hud>

Claims

1. A process of generating an image to be presented to a driver in a preset coordinate system, a process of detecting the position of the driver's eyes, a process of specifying a viewpoint position indicating the position of the driver's eyes in the vehicle interior based on the detected eye position, a process of specifying a display position for displaying the generated image in the driver's field of view based on the specified viewpoint position, a process of generating a driver's viewpoint image which is an image obtained by converting the generated image into a coordinate system based on the specified viewpoint position, a process of displaying the driver's viewpoint image so as to overlap with the driver's field of view, a process of detecting the driver's line of sight, a process of determining whether or not a change amount of the detected line of sight exceeds a preset threshold value, when it is determined that the detected change amount of the line of sight exceeds the threshold value, causing a computer to execute a process of suppressing a change in the display position of the driver's viewpoint image accompanying the change in the viewpoint position, wherein the computer when it is determined that the change amount exceeds the threshold value, decreases the degree of emphasis of the driver's viewpoint image over time, when the change amount exceeding the threshold value returns below the threshold value, returns the decreased degree of emphasis to its original state in a time shorter than the time elapsed when decreasing the degree of emphasis, A display control method characterized by the above.

2. The display control method according to claim 1, wherein the computer fixes the display position of the driver's viewpoint image when it is determined that the change amount exceeds the threshold value, and releases the fixation of the display position when the change amount exceeding the threshold value returns below the threshold value.

3. A process of generating an image to be presented to a driver in a preset coordinate system, a process of detecting the position of the driver's eyes, a process of specifying a viewpoint position indicating the position of the driver's eyes in the vehicle interior based on the detected eye position, a process of specifying a display position for displaying the generated image in the driver's field of view based on the specified viewpoint position, a process of generating a driver's viewpoint image which is an image obtained by converting the generated image into a coordinate system based on the specified viewpoint position, a process of displaying the driver's viewpoint image so as to overlap with the driver's field of view, a process of detecting the driver's line of sight, a process of determining whether or not a change amount of the detected line of sight exceeds a preset threshold value, When it is determined that the detected amount of change in the line of sight exceeds the threshold value, a computer that executes a process of suppressing a change in the position where the driver's viewpoint image is displayed accompanying the change in the viewpoint position is provided. The computer When it is determined that the amount of change exceeds the threshold value, reduces the degree of emphasis of the driver's viewpoint image over time. When the amount of change that exceeded the threshold value returns below the threshold value, the reduced degree of emphasis is restored to its original value in a time shorter than the time elapsed when reducing the degree of emphasis. A display control device characterized by the above.

4. A process of generating an image to be presented to the driver in a preset coordinate system, A process of detecting the position of the driver's eyes, A process of specifying a viewpoint position indicating the position of the driver's eyes in the vehicle interior based on the detected eye position, A process of specifying a display position for displaying the generated image in the driver's field of view based on the specified viewpoint position, A process of generating a driver's viewpoint image that is an image obtained by converting the generated image into a coordinate system based on the specified viewpoint position, A process of displaying the driver's viewpoint image so as to overlap the driver's field of view, A process of detecting the driver's line of sight, A process of determining whether or not the detected amount of change in the line of sight exceeds a preset threshold value, When it is determined that the detected amount of change in the line of sight exceeds the threshold value, causes the computer to execute a process of suppressing a change in the position where the driver's viewpoint image is displayed accompanying the change in the viewpoint position. The computer When it is determined that the amount of change exceeds the threshold value, fades out and erases the driver's viewpoint image. A display control method in which when the amount of change that exceeded the threshold value returns below the threshold value, the erased driver's viewpoint image is faded in and redisplayed in a time shorter than the time taken to fade out and erase the driver's viewpoint image.

5. A process of generating an image to be presented to the driver in a preset coordinate system, A process of detecting the position of the driver's eyes, A process of specifying a viewpoint position indicating the position of the driver's eyes in the vehicle interior based on the detected eye position, A process of specifying a display position for displaying the generated image in the driver's field of view based on the specified viewpoint position, A process of generating a driver's viewpoint image that is an image obtained by converting the generated image into a coordinate system based on the specified viewpoint position, A process of causing the driver's viewpoint image to be displayed so as to overlap the driver's field of view, A process of detecting the driver's line of sight, A process of determining whether or not a change amount of the detected line of sight exceeds a preset threshold value, When it is determined that the detected change amount of the line of sight exceeds the threshold value, a computer that executes a process of suppressing a change in the display position of the driver's viewpoint image accompanying the change in the viewpoint position is provided, The computer, When it is determined that the change amount exceeds the threshold value, the driver's viewpoint image is faded out and erased, A display control device that, when the change amount exceeding the threshold value returns to below the threshold value, fades in and redisplays the erased driver's viewpoint image in a time shorter than the time when the driver's viewpoint image was faded out and erased.

6. A process of generating an image to be presented to a driver in a preset coordinate system, A process of detecting the position of the driver's eyes, A process of specifying a viewpoint position indicating the position of the driver's eyes in the vehicle interior based on the detected eye position, A process of specifying a display position for displaying the generated image in the driver's field of view based on the specified viewpoint position, A process of generating a driver's viewpoint image that is an image obtained by converting the generated image into a coordinate system based on the specified viewpoint position, A process of causing the driver's viewpoint image to be displayed so as to overlap the driver's field of view, A process of detecting the driver's line of sight, A process of determining whether or not a change amount of the detected line of sight exceeds a preset threshold value, When it is determined that the detected change amount of the line of sight exceeds the threshold value, causing a computer to execute a process of suppressing a change in the display position of the driver's viewpoint image accompanying the change in the viewpoint position, The computer, When it is determined that the change amount exceeds the threshold value, reducing the degree of emphasis of the driver's viewpoint image, When the change amount exceeding the threshold value returns to below the threshold value, restoring the reduced degree of emphasis, The degree of emphasis is the brightness and chroma of the driver's viewpoint image. A display control method.

7. A process of generating an image to be presented to a driver in a preset coordinate system, A process of detecting the position of the driver's eyes, A process of specifying a viewpoint position indicating the position of the driver's eyes in the vehicle interior based on the detected eye position, A process of specifying a display position for displaying the generated image in the driver's field of view based on the specified viewpoint position, A process of generating a driver's viewpoint image, which is an image obtained by converting the generated image into a coordinate system based on the specified viewpoint position; A process of displaying the driver's viewpoint image so as to overlap with the driver's field of view; A process of detecting the driver's line of sight; A process of determining whether or not the detected amount of change in the line of sight exceeds a preset threshold; When it is determined that the detected amount of change in the line of sight exceeds the threshold, a process of suppressing a change in the display position of the driver's viewpoint image accompanying the change in the viewpoint position is executed, and a computer is provided; The computer: When it is determined that the amount of change exceeds the threshold, the degree of emphasis of the driver's viewpoint image is decreased; When the amount of change that exceeded the threshold returns below the threshold, the decreased degree of emphasis is restored; The degree of emphasis is the brightness and chroma of the driver's viewpoint image, and a display control device.

8. A process of generating an image to be presented to the driver in a preset coordinate system; A process of detecting the position of the driver's eyes; A process of specifying a viewpoint position indicating the position of the driver's eyes in the vehicle interior based on the detected position of the eyes; A process of specifying a display position for displaying the generated image in the driver's field of view based on the specified viewpoint position; A process of generating a driver's viewpoint image, which is an image obtained by converting the generated image into a coordinate system based on the specified viewpoint position; A process of displaying the driver's viewpoint image so as to overlap with the driver's field of view; A process of detecting the driver's line of sight; A process of determining whether or not the detected amount of change in the line of sight exceeds a preset threshold; When it is determined that the detected amount of change in the line of sight exceeds the threshold, a process of suppressing a change in the display position of the driver's viewpoint image accompanying the change in the viewpoint position is executed on the computer; The computer: When it is determined that the amount of change exceeds the threshold, the driver's viewpoint image is faded out and erased; When the amount of change that exceeded the threshold returns below the threshold, the erased driver's viewpoint image is immediately redisplayed, and a display control method.

9. A process of generating an image to be presented to the driver in a preset coordinate system; A process of detecting the position of the driver's eyes; A process of specifying a viewpoint position indicating the position of the driver's eyes in the vehicle interior based on the detected position of the eyes; A process of specifying a display position for displaying the generated image in the driver's field of view based on the specified viewpoint position; A process of generating a driver's viewpoint image which is an image obtained by converting the generated image into a coordinate system based on the specified viewpoint position; A process of causing the driver's viewpoint image to be displayed so as to overlap with the driver's field of view; A process of detecting the driver's line of sight; A process of determining whether or not the amount of change in the detected line of sight exceeds a preset threshold value; When it is determined that the detected amount of change in the line of sight exceeds the threshold value, a computer is provided that executes a process of suppressing a change in the display position of the driver's viewpoint image accompanying the change in the viewpoint position; The computer: When it is determined that the amount of change exceeds the threshold value, the driver's viewpoint image is faded out and erased; A display control device that immediately redisplays the erased driver's viewpoint image when the amount of change that exceeded the threshold value returns below the threshold value.

10. A process of generating an image to be presented to the driver in a preset coordinate system; A process of detecting the position of the driver's eyes; A process of specifying a viewpoint position indicating the position of the driver's eyes in the vehicle interior based on the detected position of the eyes; A process of specifying a display position for displaying the generated image in the driver's field of view based on the specified viewpoint position; A process of generating a driver's viewpoint image which is an image obtained by converting the generated image into a coordinate system based on the specified viewpoint position; A process of causing the driver's viewpoint image to be displayed so as to overlap with the driver's field of view; A process of detecting the driver's line of sight; A process of determining whether or not the amount of change in the detected line of sight exceeds a preset threshold value; When it is determined that the detected amount of change in the line of sight exceeds the threshold value, the computer is made to execute a process of suppressing a change in the display position of the driver's viewpoint image accompanying the change in the viewpoint position; The computer: When it is determined that the amount of change exceeds the threshold value, the degree of emphasis of the driver's viewpoint image is decreased as time elapses; When the amount of change that exceeded the threshold value returns below the threshold value, the decreased degree of emphasis is immediately restored to its original state. A display control method characterized by the above.

11. A process of generating an image to be presented to the driver in a preset coordinate system; A process of detecting the position of the driver's eyes; A process of specifying a viewpoint position indicating the position of the driver's eyes in the vehicle interior based on the detected eye positions; A process of specifying a display position for displaying the generated image in the driver's field of view based on the specified viewpoint position; A process of generating a driver's viewpoint image which is an image obtained by converting the generated image into a coordinate system based on the specified viewpoint position; A process of causing the driver's viewpoint image to be displayed so as to overlap with the driver's field of view; A process of detecting the driver's line of sight; A process of determining whether or not the detected amount of change in the line of sight exceeds a preset threshold value; When it is determined that the detected amount of change in the line of sight exceeds the threshold value, a process of suppressing a change in the position where the driver's viewpoint image is displayed accompanying the change in the viewpoint position; A computer that executes the above is provided. The computer is When it is determined that the amount of change exceeds the threshold value, the degree of emphasis of the driver's viewpoint image is decreased as time elapses. When the amount of change that has exceeded the threshold value returns below the threshold value, the decreased degree of emphasis is immediately restored to its original state. A display control device characterized by the above.

Images