Display control device, display control method, and computer program

Abstract

This improves the accuracy of making the information displayed on the HUD, superimposed on the object, track the object. [Solution] The display control unit 11, which performs display control for a display device capable of superimposing information in the driver's field of view, calculates the relative position of the vehicle with other vehicles traveling ahead of it based on detected values ​​that detect the environment around the vehicle, calculates display coordinates indicating the position of other vehicles in the field of view on the display device 20 based on the relative position, and displays a virtual display superimposed on the position of other vehicles in the field of view on the display device based on a first calculation process that limits the amount of display variation of the display coordinates per unit time to fall within predetermined conditions, and if it is determined that an error greater than predetermined occurs in the amount of display variation based on the detected values, displays a virtual display based on a second calculation process that reduces the display error of the virtual display compared to the first calculation process.

Description

Technical Field

[0001] The present invention relates to a display control device, a display control method, and a computer program for displaying information superimposed on an object included in a driver's field of view.

Background Art

[0002] A HUD (Head Up Display) that displays information superimposed on an object included in a driver's field of view is known. For example, Patent Document 1 proposes a technique for reducing the deviation between the information displayed superimposed on an object in a HUD and the object.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In recent years, HUDs have a function of displaying information recognized by millimeter-wave radars, cameras, etc. used in ADAS (Advanced Drive Assistance System) for supporting a driver's driving, superimposed on landscapes, other vehicles, etc. that are included in the driver's field of view. When configuring a HUD that is linked with a driving support function such as ADAS, it is desirable that the information displayed on the HUD superimposed on an object included in the driver's field of view accurately follows the object.

[0005] An object of the present invention is to provide a display control device, a display control method, and a computer program that can improve the accuracy of causing information displayed on a HUD superimposed on an object to follow the object.

Means for Solving the Problems

[0006] One aspect of the present invention is a display control device comprising a calculation unit that performs display control for a display device capable of displaying superimposed information in the driver's field of view, wherein the calculation unit calculates the relative position between the vehicle and other vehicles traveling ahead of the vehicle based on detected values ​​that detect the environment around the vehicle, calculates display coordinates in the display device that indicate the position of the other vehicles in the field of view based on the relative position, and displays a virtual display superimposed on the position of the other vehicles in the field of view in the display device based on a first calculation process that limits the amount of display variation per unit time of the display coordinates at the position to fall within predetermined conditions, and if it is determined that an error greater than a predetermined amount occurs in the amount of display variation based on the detected values, displays the virtual display in the display device based on a second calculation process that reduces the display error of the virtual display compared to the first calculation process. [Effects of the Invention]

[0007] According to the present invention, it is possible to improve the accuracy of making the information displayed on the HUD superimposed on the object track the object. [Brief explanation of the drawing]

[0008] [Figure 1] Block diagram showing the configuration of a vehicle according to the embodiment. [Figure 2] This figure shows an example of a virtual display superimposed on the HUD. [Figure 3] This figure shows an example of a virtual display that exhibits blurring. [Figure 4] This figure shows an example of a predetermined road environment in which display errors occur in virtual representations. [Figure 5] This figure shows an example of a method for calculating the amount of lateral movement when other vehicles move. [Figure 6] This figure shows an example of a method for calculating the amount of lateral movement when a vehicle is in motion. [Figure 7] This figure shows an example of a method for the second arithmetic operation. [Figure 8] This figure shows an example of a display error in the virtual representation based on the first calculation process. [Figure 9] This figure shows an example of a display error in the virtual representation based on the second calculation process. [Figure 10] This flowchart shows the processing flow of the display control method executed in the display control device. [Modes for carrying out the invention]

[0009] As shown in Figure 1, Vehicle 1 comprises a detection unit 2 that detects values ​​necessary for driving, a display control device 10 that controls the display of information, and a display device 20 capable of displaying information. Hereinafter, Vehicle 1 will also be referred to as "the vehicle itself" in relation to other vehicles. The detection unit 2 is composed of sensors that detect values ​​related to driving assistance and values ​​related to the driving state. The detection unit 2 is equipped with a camera 3 that images the area around Vehicle 1. The camera 3 is composed of a plurality of camera sensors that image a predetermined imaging range.

[0010] Multiple camera sensors are positioned on the vehicle 1 to capture images in areas such as the front, sides, and rear in the direction of travel of the vehicle 1. Camera 3 may be composed of camera sensors capable of capturing images in a 360-degree range. Camera 3 outputs the captured data as detected values ​​to the display control device 10. Based on the captured data, the display control device 10 recognizes the environment and objects present around the vehicle 1.

[0011] The detection unit 2 includes a radar device 4 that detects objects present around the vehicle 1. The radar device 4 is composed of, for example, a radar sensor that scans millimeter waves in the horizontal direction of the vehicle and receives reflected waves. Based on the transmission and reception of millimeter waves, the radar device 4 detects the relative position and relative direction of objects present around the vehicle 1. The radar device 4 outputs the detected object values ​​to the display control device 10. The calculation processing related to object detection may be performed in the radar device 4 or in the display control device 10.

[0012] The detection unit 2 includes a LiDAR (Light Detection and Ranging) device 5 that detects objects present around the vehicle 1. The LiDAR device 5 measures the distance to an object by transmitting and receiving laser light and creates three-dimensional point cloud data of the surrounding environment. The LiDAR device 5 outputs the detected object value to the display control device 10. The calculation processing related to object detection may be performed in the LiDAR device 5 or in the display control device 10.

[0013] The detection unit 2 includes a vehicle sensor 6 that acquires detection values ​​related to the movement of the vehicle 1. The vehicle sensor 6 is composed of a plurality of various sensors. For example, the vehicle sensor 6 includes a 6-axis acceleration sensor that detects acceleration in 6 axes. The 6-axis acceleration sensor detects acceleration occurring in the vehicle's longitudinal direction, vertical direction, and left-right direction, as well as angular acceleration occurring in the vehicle's yaw direction, pitching direction, and roll direction.

[0014] The vehicle sensor 6 detects values ​​related to vehicle operation, such as accelerator opening, brake operation amount, and steering operation amount. The detected values ​​related to vehicle operation may be control signals corresponding to the amount of operation performed by the driver on the vehicle 1's control system. In addition, the vehicle sensor 6 is equipped with a position sensor such as a GPS (Global Positioning System) to detect the current position of the vehicle 1. The vehicle sensor 6 outputs the detected values ​​to the display control device 10.

[0015] The display device 20 is, for example, a HUD (Head-Up Display). The display device 20 has a transparent display unit. The display device 20 is configured to superimpose and display information on the location of objects such as the environment and other vehicles that are included in the driver's field of view, by allowing information to pass through the display unit. The display device 20 is controlled by the display control device 10.

[0016] The display control device 10 includes an arithmetic unit 11 that performs display control of the information displayed on the display device 20. The arithmetic unit 11 is constituted by a hardware processor such as at least one CPU (Central Processing Unit). The display control device 10 includes a storage unit 12 that stores data. The storage unit 12 is constituted by a non-temporary storage medium such as a hard disk drive (HDD) or a solid state drive (SSD). The storage unit 12 stores computer programs and data necessary for control.

[0017] Based on the detection value detected by the detection unit 2, the arithmetic unit 11 executes display control to superimpose information on the position of an object included in the driver's field of view on the display device 20. Based on the detection value of the environment around the host vehicle (vehicle 1), the arithmetic unit 11 calculates the relative position between the other vehicle traveling ahead of the host vehicle and the host vehicle. The arithmetic unit 11 calculates the relative position from the host vehicle to the other vehicle and the relative angle of the other vehicle as seen from the host vehicle using at least one of the detection values of the radar device 4, the detection values of the lidar device 5, the imaging data by the camera 3, and the detection values of the vehicle sensor 6.

[0018] Based on the calculated relative position and relative angle, the arithmetic unit 11 calculates display coordinates indicating the position of the other vehicle present in the field of view on the display device 20. The arithmetic unit 11 causes a virtual display superimposed on the position of the other vehicle present in the field of view on the display device 20 to be displayed based on the display coordinates. The arithmetic unit 11 calculates a ratio so that the amount of change in the position of the other vehicle becomes the amount of change in the coordinates of the display device 20, and converts and follows the amount of change in the position of the other vehicle to the amount of change in the display coordinates for display. The arithmetic unit 11 calculates the display coordinates based on a known calculation method. If the display coordinates can be calculated, the arithmetic unit 11 may calculate the display coordinates based on another calculation method.

[0019] As shown in Figure 2, the calculation unit 11 superimposes a virtual display V1 onto the display coordinates C1 indicating the position of the other vehicle T on the display device 20. As shown in Figure 3, the virtual display V1 may flicker on the display device 20 due to the influence of noise in the detected values ​​and the amount of fluctuation per unit time of the other vehicle.

[0020] The calculation unit 11 performs a first calculation process to limit the amount of display fluctuation per unit time so that the display coordinates fall within predetermined conditions relative to the position of other vehicles. The first calculation process is, for example, a process to limit the amount of fluctuation of the display coordinates so that the amount of display fluctuation per unit time of the display coordinates falls within a predetermined range. The calculation unit 11 determines, for example, whether the amount of lateral fluctuation of the own vehicle relative to other vehicles falls within a predetermined range. If, for example, the other vehicle T is moving laterally within a predetermined range of lateral fluctuation amount equivalent to a lane change, the calculation unit 11 corrects the display coordinates C1 based on the first calculation process and suppresses the amount of fluctuation.

[0021] The first calculation process may not only limit the amount of horizontal display variation of the display coordinates, but may also limit the amount of vertical display variation of the display coordinates per unit time to within a predetermined range. Based on the corrected display coordinates, the calculation unit 11 displays a virtual display V1 superimposed on other vehicles in the driver's field of view on the display device 20. Through the above process, the calculation unit 11 can stably display the virtual display V1 on the display device 20.

[0022] Figure 4 shows the environmental conditions under which a display error in the virtual display V1 based on the first calculation process occurs. When the calculation unit 11 performs the first calculation process under conditions where there is a difference greater than a predetermined amount in the relative positional relationship between its own vehicle and other vehicles, such as on a curved section of a road, an error may occur between the position of other vehicles and the virtual display V1 in the display device 20.

[0023] As shown in the figure, according to the first calculation process of the calculation unit 11, the virtual display V1 is displayed following the display coordinates C1 when the relative lateral movement per unit time between the other vehicle T and the own vehicle is small, such as when the other vehicle T starts to curve. The error between the virtual display V1 and the display coordinates C1 increases when the relative lateral movement per unit time between the other vehicle T and the own vehicle increases, such as when the other vehicle T is traveling on a curve and the own vehicle starts to curve.

[0024] When the relative lateral movement per unit time between the other vehicle T and the own vehicle decreases, such as when the other vehicle T is traveling around a curve and the own vehicle is also traveling around a curve, the error between the virtual display V1 and the display coordinate C1 is small, and the virtual display V1 follows the display coordinate C1. When the relative lateral movement per unit time between the other vehicle T and the own vehicle increases, such as when the other vehicle T finishes traveling around a curve and travels in a straight line while the own vehicle is also traveling around a curve, the error between the virtual display V1 and the display coordinate C1 increases.

[0025] According to the first calculation process, if there is a difference greater than a predetermined amount in the relative positional relationship between the own vehicle and other vehicles, an error may occur between the display coordinate C1 and the virtual display V1. Therefore, if there is a difference greater than a predetermined amount in the relative positional relationship between the own vehicle and other vehicles, the calculation unit 11 executes a second calculation process that differs from the first calculation process. The second calculation process executes a calculation process that differs from the first calculation process based on the following considerations.

[0026] (A) Before the curved section of the road begins, the lateral movement of other vehicles T is dominant. (B) After the curved section ends, the lateral movement of the own vehicle is dominant. (C) Since the shape of the curve is not constant, constantly determining the road shape and switching between (A) and (B) to perform calculations would increase the processing load. Therefore, the maximum value of the movement in the environment of (A) or (B) is used to make the virtual display V1 follow the display coordinate C1. (D) In ​​order to determine the curved section, a determination threshold is needed for when the road transitions sharply from a nearly straight section to a curved section.

[0027] The threshold for determining whether a difference exceeding a predetermined level occurs in the relative positional relationship between one's own vehicle and other vehicles can be determined, for example, by determining the value of the curvature change rate at a point 100m before the start of a sharp curve on a highway, based on measured values ​​or calculated values ​​from computer simulations.

[0028] The calculation unit 11 determines, based on the detected values, whether there is a difference greater than a predetermined amount in the relative positional relationship between the vehicle and other vehicles. If the calculation unit 11 determines that there is a difference greater than a predetermined amount in the relative positional relationship between the vehicle and other vehicles, it executes a second calculation process. For example, the calculation unit 11 determines, based on the image data among the detected values, whether there are sharper curves than a predetermined amount in the road shape on which the vehicle is traveling. The calculation unit 11 is configured to recognize objects included in the captured image based on machine learning such as deep learning that is performed in advance.

[0029] The calculation unit 11 analyzes the captured data to extract lane markings and road boundaries such as sidewalks, and recognizes the road shape. The calculation unit 11 may also recognize the road shape based on the detection values ​​of the radar device 4 and the lidar device 5. If detailed map data is available, the calculation unit 11 may also recognize the road shape by comparing the vehicle's position data with the map data. The calculation unit 11 may also recognize the road shape by integrating multiple recognition results based on multiple detection values.

[0030] Based on the detected values, the calculation unit 11 determines that there are sharp curves exceeding a predetermined threshold within the display range of the display device 20, and executes a second calculation process. The calculation unit 11 executes the second calculation process to suppress errors exceeding a predetermined threshold in the amount of display variation of the display coordinates per unit time on the display device 20.

[0031] The calculation unit 11, for example, in the second calculation process, limits the display fluctuation amount of the virtual display V1 using one of the maximum values ​​among the first lateral fluctuation amount, the second lateral fluctuation amount based on the lateral fluctuation amount of other vehicles, and the third lateral fluctuation amount based on the lateral fluctuation amount of its own vehicle, and calculates the display coordinates. The calculation unit 11 uses the display coordinates, which have had their display fluctuation amount limited based on the second calculation process, to display the virtual display V1 on the display device 20.

[0032] The first lateral movement is calculated based on the first calculation process. The second and third lateral movement amounts are calculated based on the second calculation process when there is a difference of a predetermined amount or more in the relative positional relationship between the vehicle and other vehicles. The first lateral movement amount is, for example, the relative lateral movement between the vehicle and other vehicles when the lateral movement of both vehicles is small. The second lateral movement amount is, for example, the relative lateral movement between the vehicle and other vehicles when the lateral movement of the vehicle is small and mainly the movement is caused by other vehicles. The third lateral movement amount is, for example, the relative lateral movement between the vehicle and other vehicles when the lateral movement of both vehicles is small and mainly the movement is caused by the vehicle itself.

[0033] Figure 5 shows an example of a method for calculating the second lateral displacement based on the lateral displacement of other vehicles. The calculation unit 11 calculates the second lateral displacement of the other vehicle relative to the own vehicle by subtracting the lateral position of the other vehicle at time (t-1) from the lateral position of the other vehicle at time (t). The second horizontal displacement is calculated using the following equation (1). (Relative lateral position change due to other vehicles [deg]) = {(atan(lateral position of other vehicle [m] / depth of other vehicle [m] @ time (t)) - (atan(lateral position of other vehicle [m] / depth of other vehicle [m] @ time (t-1))} × 180 / π (1)

[0034] Figure 6 shows an example of a method for calculating the third fluctuation amount based on the fluctuation amount of the vehicle itself in the second calculation process. The calculation unit 11 calculates the third lateral fluctuation amount based on the motion of the vehicle by adding, for example, the lateral fluctuation amount based on the lateral movement of the vehicle and the lateral fluctuation amount based on the rotation of the vehicle. The calculation unit 11 calculates the lateral fluctuation amount based on the lateral movement of the vehicle itself based on the following equations (2) and (3). Lateral displacement due to the lateral movement of the vehicle [m] = v / ω(1-cos(ωΔt)) (2) Lateral displacement due to the lateral movement of the own vehicle [deg]) = atan((Lateral displacement due to the lateral movement of the own vehicle [m]) / Distance to other vehicles [m])) × 180 / π however, v: Vehicle speed [m / s] ω: Yaw rate [rad / s] Δt: Processing period [s] That is the case.

[0035] The calculation unit 11 calculates the amount of lateral movement based on the rotation of the vehicle based on the following equation (4). Lateral displacement due to the rotation of the vehicle [deg] = ωΔt × 180 / π (4) The calculation method described above is just one example; other calculation methods may be applied as long as they can calculate the lateral movement of the own vehicle and other vehicles.

[0036] Figure 7 shows the lateral displacement amount selected by the calculation unit 11 in the second calculation process. In a curved section, the second lateral displacement amount increases from the point Fa when another vehicle starts the curve until the point Fc when the curve ends. The third lateral displacement amount increases from the point Fb when the own vehicle starts the curve until the point Fd when the curve ends. As shown in the figure, the calculation unit 11 calculates the display coordinates using the lateral displacement amount that is the maximum value among the first lateral displacement amount, the second lateral displacement amount, and the third lateral displacement amount.

[0037] Figure 8 shows the error between the display coordinate C1 and the display fluctuation amount per unit time of the virtual display V1, calculated using only the first calculation process. As shown in the figure, when only the first calculation process is used, an error greater than a predetermined value occurs between the display coordinate C1 and the virtual display V1 in the curved section.

[0038] Figure 9 shows the error between the display coordinate C1 and the display fluctuation amount per unit time of the virtual display V1, calculated using the second calculation process.

[0039] Figure 10 shows the processing flow of the display control method executed in the display control device 10. The display control method is executed by a computer program installed in the computer mounted on the display control device 10. The computer program causes the processor that realizes the arithmetic unit 11 of the display control device 10 to perform the following processing.

[0040] The calculation unit 11 calculates the relative position of the vehicle and other vehicles traveling ahead of it based on detected values ​​that detect the environment around the vehicle. Based on the relative position, the calculation unit 11 calculates display coordinates that indicate the position of other vehicles in the field of view on the display device. The calculation unit 11 performs a first calculation process to limit the amount of display variation of the display coordinates per unit time to fall within predetermined conditions (S100). In the first calculation process, when the lateral movement of the vehicle and other vehicles is small, the calculation unit 11 calculates a first relative lateral variation amount between the vehicle and other vehicles (S102).

[0041] The calculation unit 11 determines, based on the detected value, whether or not there exists a predetermined road environment in which an error greater than a predetermined amount occurs in the display fluctuation amount (S104). If the calculation unit 11 makes a negative determination in S104 (S104: No), it displays a virtual display superimposed on the position of other vehicles in the field of view on the display device 20 based on the first lateral fluctuation amount (S114). If the calculation unit 11 makes a positive determination in S104 (S104: Yes), it executes a second calculation process that reduces the display error compared to the first calculation process (S106).

[0042] The calculation unit 11 calculates a second relative lateral movement between the vehicle and other vehicles when the vehicle's lateral movement is small and the other vehicles are the main ones moving laterally (S108). The calculation unit 11 calculates a third relative lateral movement between the vehicle and other vehicles when the lateral movement of both vehicles is small and the vehicle is the main one moving laterally (S110). The calculation unit 11 selects one of the first, second, and third lateral movement amounts that is the maximum value and limits the display movement amount (S112). Based on the second calculation process, the calculation unit 11 displays a virtual display on the display device (S114).

[0043] As described above, the display control device 10 can improve the accuracy of making the virtual display superimposed on the object and displayed on the display device 20 follow other vehicles. The display control device 10 can reduce the display error of the virtual display compared to the first calculation process by executing a second calculation process when a predetermined road environment such as a sharp curve exists. The display control device 10 can improve the accuracy of the virtual display following other vehicles by selecting one of the maximum values ​​among the first lateral movement amount, the second lateral movement amount, and the third movement amount in the second calculation process and limiting the display movement amount.

[0044] In the embodiments described above, the computer programs executed in each configuration of the display control device 10 may be provided in the form of being recorded on a computer-readable, portable, non-temporary recording medium such as a semiconductor memory, a magnetic recording medium, or an optical recording medium. The computer programs may also be provided as a computer product capable of realizing the display control device 10 and the display control method. [Explanation of Symbols]

[0045] 1 vehicle 2 Detection unit 3 cameras 4. Radar equipment 5. LiDAR device 6. Vehicle Sensors 10 Display control device 11 Arithmetic section 12 Storage section 20 Display device C1 Display coordinates T Other vehicles V1 Virtual Display

Claims

1. It includes a processing unit that performs display control for a display device capable of superimposing information within the driver's field of view, The aforementioned arithmetic unit, Based on the detected values ​​of the environment surrounding the vehicle, the relative position of the vehicle and other vehicles traveling ahead of it is calculated. Based on the relative position, the display device calculates display coordinates indicating the position of the other vehicle located within the field of view. Based on a first calculation process that limits the amount of display variation per unit time of the display coordinates at the aforementioned position to fall within predetermined conditions, the display device displays a virtual display superimposed on the aforementioned position of the other vehicle that is in the field of view. If it is determined that an error greater than a predetermined amount occurs in the display variation amount based on the detected value, the display device will display the virtual display based on a second calculation process that reduces the display error of the virtual display compared to the first calculation process. Display control device.

2. The aforementioned arithmetic unit, In the second calculation process, the display fluctuation amount is limited using one of the following: a first lateral fluctuation amount used in the first calculation process, a second lateral fluctuation amount based on the relative lateral fluctuation amount between the other vehicle and the own vehicle, and a third fluctuation amount based on the lateral fluctuation amount of the own vehicle, and the virtual display is displayed. The display control device according to claim 1.

3. The aforementioned arithmetic unit, If it is determined that there are sharper curves than a predetermined value in the road shape based on the detected values, the second calculation process is executed. The display control device according to claim 1.

4. A display control method performed in a computer mounted on a display control device that performs display control for a display device capable of displaying information superimposed in the driver's field of view, The aforementioned computer, Based on the detected values ​​of the environment surrounding the vehicle, the relative position of the vehicle and other vehicles traveling ahead of it is calculated. Based on the relative position, the display device calculates display coordinates indicating the position of the other vehicle located within the field of view. Based on a first calculation process that limits the amount of display variation per unit time of the display coordinates at the aforementioned position to fall within predetermined conditions, the display device displays a virtual display superimposed on the aforementioned position of the other vehicle that is in the field of view. If it is determined that an error greater than a predetermined amount occurs in the display variation amount based on the detected value, the display device will display the virtual display based on a second calculation process that reduces the display error of the virtual display compared to the first calculation process. Display control method.

5. A computer program installed on a computer mounted in a display control device that performs display control for a display device capable of superimposing information within the driver's field of view, Based on the detected values ​​of the environment surrounding the vehicle, the relative position of the vehicle and other vehicles traveling ahead of it is calculated. Based on the relative position, the display device calculates display coordinates indicating the position of the other vehicle located within the field of view. Based on a first calculation process that limits the amount of display variation per unit time of the display coordinates at the aforementioned position to fall within predetermined conditions, the display device displays a virtual display superimposed on the aforementioned position of the other vehicle that is in the field of view. If it is determined that an error greater than a predetermined amount occurs in the display variation amount based on the detected value, the computer is instructed to perform a process to display the virtual display on the display device based on a second calculation process that reduces the display error of the virtual display compared to the first calculation process. Computer program.

Images