In-vehicle display system

Abstract

To provide an on-vehicle display system capable of improving positional accuracy of an icon or the like on a windshield.SOLUTION: The onboard display system includes an angle estimation unit configured to estimate an angle α formed by a straight line connecting the own-vehicle V1 and the preceding-vehicle V2 with respect to the front-rear direction of the own-vehicle V1 based on an image captured by the camera 2, an inter-vehicle separation estimation unit configured to estimate an inter-vehicle separation D between the own-vehicle V1 and the preceding-vehicle V1 based on a detection result of the radar sensor on the own-vehicle V2, a height calculation unit configured to calculate a height H of the preceding-vehicle V2 based on the angle α and the inter-vehicle separation D, a gradient estimation unit configured to estimate a gradient of a road V2 ahead of the own-vehicle V1 based on a change over time in the height H of the preceding-vehicle, and a display control unit configured to display an icon or the like on the windshield so as to overlap a scene ahead of the own-vehicle. R2 V1. The display control unit adjusts the position of an icon or the like on the windshield based on the gradient of the forward road R2 estimated by the gradient estimation unit.SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] The present disclosure relates to an in-vehicle display system. [Background technology]

[0002] JP 2022-189118 A is a known technical document related to an in-vehicle display system. JP 2022-189118 A discloses a technology for displaying an image on the display screen of a head-up display of a vehicle so that the image is superimposed on the scenery ahead of the vehicle. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 2022-189118 Summary of the Invention [Problem to be solved by the invention]

[0004] In the conventional techniques described above, there are cases where it is required to improve the positional accuracy of the image on the display screen of the head-up display.

[0005] The present disclosure aims to provide an in-vehicle display system that can improve the positional accuracy of an image on the display screen of a head-up display. [Means for solving the problem]

[0006] The in-vehicle display system disclosed herein is an in-vehicle display system that displays an image on the display screen of a head-up display of the host vehicle, and includes: a preceding vehicle recognition unit that recognizes a preceding vehicle traveling in front of the host vehicle based on an image captured by a camera of the host vehicle; an angle estimation unit that estimates the angle that a straight line connecting the host vehicle and the preceding vehicle makes with respect to the longitudinal direction of the host vehicle based on the image captured by the camera; an inter-vehicle distance estimation unit that estimates the inter-vehicle distance between the host vehicle and the preceding vehicle based on the detection results of a sensor of the host vehicle; a height calculation unit that calculates the height of the preceding vehicle based on the angle estimated by the angle estimation unit and the inter-vehicle distance estimated by the inter-vehicle distance estimation unit; a gradient estimation unit that estimates the gradient of the road ahead of the host vehicle based on changes in the height of the preceding vehicle over time; and a display control unit that displays the image on the display screen so that it is superimposed on the scenery ahead of the host vehicle, and the display control unit adjusts the position of the image on the display screen based on the gradient estimated by the gradient estimation unit. [Effects of the Invention]

[0007] According to the present disclosure, it is possible to provide an in-vehicle display system that can improve the positional accuracy of an image on the display screen of a head-up display. [Brief explanation of the drawings]

[0008] [Figure 1] 1 is a block diagram of an in-vehicle display system according to an embodiment; [Figure 2] FIG. 2 is a schematic diagram showing a road ahead of the host vehicle. [Figure 3] FIG. 10 is a diagram illustrating a method for calculating the gradient of the road ahead of the host vehicle. [Figure 4] FIG. 10 is a diagram illustrating a method for calculating the gradient of the road ahead of the host vehicle. [Figure 5] 2 is a flowchart showing a process performed by the ECU shown in FIG. 1. [Figure 6] 2 is a flowchart showing a process performed by the ECU shown in FIG. 1. DETAILED DESCRIPTION OF THE INVENTION

[0009] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings.

[0010] Fig. 1 is a block diagram of an in-vehicle display system according to this embodiment. The in-vehicle display system 1 shown in Fig. 1 is used in, for example, an autonomous driving vehicle or a driving assistance vehicle. The in-vehicle display system 1 displays an image on a display screen of a head-up display 5 of the host vehicle V1 (see Fig. 2). As shown in Fig. 1, the in-vehicle display system 1 includes a camera 2, a radar sensor 3, a gyro sensor 4, a head-up display (HUD) 5, and an ECU (Electronic Control Unit) 10. The in-vehicle display system 1 is mounted on the host vehicle V1.

[0011] Camera 2 is an imaging device that captures images of the external situation of vehicle V1. Camera 2 is installed, for example, behind the windshield of vehicle V1. Camera 2 is a front camera that captures images of the area in front of vehicle V1. The optical axis of the lens of camera 2 is parallel to the longitudinal direction of vehicle V1. Camera 2 may be a monocular camera or a stereo camera. A stereo camera has two imaging units arranged to reproduce binocular parallax. The imaging information of the stereo camera also includes information in the depth direction. Camera 2 transmits captured images of the external situation of vehicle V1 to ECU 10.

[0012] The radar sensor 3 is, for example, a millimeter-wave radar or lidar provided at the front end of the host vehicle V1. The radar sensor 3 emits radio waves or light ahead of the host vehicle V1. The radar sensor 3 detects targets by receiving radio waves or light reflected by targets present ahead of the host vehicle V1. The radar sensor 3 detects the relative distance (inter-vehicle distance) from the host vehicle V1 to the leading vehicle V2 (see FIG. 3). The radar sensor 3 transmits the detection results to the ECU 10.

[0013] The gyro sensor 4 is a sensor for detecting changes in the attitude of the host vehicle V1. The gyro sensor 4 is provided, for example, in the center of the body of the host vehicle V1. The gyro sensor 4 measures angular velocity to detect the rotational motion of the host vehicle V1. The gyro sensor 4 can detect angular velocity along three axes (X-axis, Y-axis, and Z-axis). The X-axis corresponds to the longitudinal direction of the host vehicle V1, the Y-axis corresponds to the lateral direction of the host vehicle V1, and the Z-axis corresponds to the vertical direction of the host vehicle V1. The gyro sensor 4 may be one that uses, for example, MEMS (Micro-Electro-Mechanical Systems) technology. A MEMS gyro sensor is small and capable of detecting angular velocity with high accuracy, contributing to attitude control and improved stability of the host vehicle V1. The gyro sensor 4 transmits the detected angular velocity information to the ECU 10.

[0014] The head-up display 5 is a display device for the driver mounted on the host vehicle V1. The head-up display 5 projects and displays icons and the like (images) on the windshield (display screen of the head-up display 5) of the host vehicle V1 so that the images overlap with the scenery ahead of the host vehicle V1 as seen by the driver. The head-up display 5 has a projection unit that is provided, for example, under the dashboard behind the instrument panel of the host vehicle V1. The projection unit projects images by irradiating light onto the windshield through an opening provided on the dashboard.

[0015] When driving assistance control such as LKAS (Lane Keeping Assist System) or LDW (Lane Departure Warning) is executed on the host vehicle V1, the head-up display 5 displays icons or the like corresponding to the lane markings on the road ahead or the route on which the host vehicle V1 is traveling on the windshield of the host vehicle V1. The head-up display 5 displays icons or the like corresponding to the lane markings on the road ahead at an intersection (display position) between the windshield and the line of sight of the driver observing the lane markings on the road ahead (a virtual line connecting the lane markings and the driver's eyes).

[0016] The ECU 10 is an electronic control unit having a central processing unit (CPU) and a storage unit such as a read-only memory (ROM) or a random access memory (RAM). The ECU 10 performs various functions by, for example, executing programs stored in the storage unit. The ECU 10 is installed in, for example, a host vehicle V1.

[0017] FIG. 2 is a schematic diagram showing a road ahead of the host vehicle. As shown in FIG. 2, the road R2 ahead of the host vehicle V1 may have a gradient. In other words, the road R2 ahead may be inclined with respect to the road R1 on which the host vehicle V1 is traveling. In this embodiment, the road R2 ahead is a slope. The head-up display 5 displays, for example, an icon corresponding to a dividing line P of the road R2 ahead on the windshield. Even if the road R2 ahead has a gradient, if the icon corresponding to the dividing line P is displayed on the windshield assuming that the driver's line of sight F is on the extension line R11 of the road R1 (assuming that the dividing line P is located on the extension line R11), the icon may be displayed at a position offset from the dividing line P on the windshield. The in-vehicle display system 1 adjusts the position of the icon on the windshield based on the gradient of the road R2 ahead.

[0018] The ECU 10 has, as its functional configuration, a preceding vehicle recognition unit 11, an angle estimation unit 12, an inter-vehicle distance estimation unit 13, a height calculation unit 14, a gradient estimation unit 15, and a display control unit 16.

[0019] As shown in Fig. 3, the preceding vehicle recognition unit 11 recognizes a preceding vehicle V2 traveling ahead of the host vehicle V1 based on an image captured by the camera 2. The preceding vehicle recognition unit 11 recognizes the preceding vehicle V2 by using well-known image processing techniques such as edge extraction, noise removal, pattern matching, deep learning, etc. The preceding vehicle recognition unit 11 recognizes a feature point C2 of the preceding vehicle V2. The feature point C2 is, for example, the lower end of the back surface of the preceding vehicle V2.

[0020] The angle estimation unit 12 estimates the angle α, which is formed by a line connecting the host vehicle V1 and the preceding vehicle V2 with respect to the longitudinal direction of the host vehicle V1, based on the image captured by the camera 2. In this embodiment, the angle α is the angle formed by a line connecting the feature point C1 of the host vehicle V1 with the feature point C2 of the preceding vehicle V2 with respect to the longitudinal direction of the host vehicle V1. The feature point C1 is, for example, the lower end of the front of the host vehicle V1. In this embodiment, the angle estimation unit 12 estimates the angle α based on the distance (number of pixels) from a center line parallel to the left-right direction to the feature point C2 of the preceding vehicle V2 in the image captured by the camera 2.

[0021] The inter-vehicle distance estimation unit 13 estimates the inter-vehicle distance D between the host vehicle V1 and the preceding vehicle V2 based on the detection result of the radar sensor 3. The inter-vehicle distance D is the straight-line distance between the host vehicle V1 and the preceding vehicle V2. In this embodiment, the inter-vehicle distance D is the length of a straight line connecting the feature point C1 of the host vehicle V1 and the feature point C2 of the preceding vehicle V2.

[0022] The height calculation unit 14 calculates the height H of the preceding vehicle V2 based on the angle α estimated by the angle estimator 12, the angle β obtained by integrating the pitch change (angular velocity) over time by the gyro sensor 4, and the inter-vehicle distance D estimated by the inter-vehicle distance estimator 13, where α is the angle estimated by the angle estimator 12 and 0 is the angle at the start of height H sampling. The height H of the preceding vehicle V2 is the distance between the preceding vehicle V2 and the host vehicle V1 in the height direction of the host vehicle V1. For example, if the road R1 on which the host vehicle V1 is traveling is a flat road parallel to the horizontal plane, the height H of the preceding vehicle V2 is the distance between the preceding vehicle V2 and the host vehicle V1 in the vertical direction. In this embodiment, the height H of the preceding vehicle V2 is the distance in the vertical direction between the feature point C2 of the preceding vehicle V2 and the feature point C1 of the host vehicle V1. The height calculation unit 14 calculates the height H using the formula H=D×sin(α+β). where H is the height of the preceding vehicle V2, D is the inter-vehicle distance between the host vehicle V1 and the preceding vehicle V2, and α is the angle formed by the line connecting the host vehicle V1 and the preceding vehicle V2 with respect to the longitudinal direction of the host vehicle V1. β is the amount of change in pitch angle from the start of sampling the height H, assuming that the change in attitude of the host vehicle V1 is equivalent to the change in attitude of the camera 2. Also, because β is the integral value of the gyro sensor value (angular velocity information), there is a possibility that errors may accumulate. For this reason, if the gyro sensor value is sufficiently small, equal to or less than a predetermined value, and has occurred a predetermined number of times or more (i.e., if there is no change in the road gradient), β and the height H are also reset to 0.

[0023] The height calculation unit 14 calculates a change in the height H of the preceding vehicle V2 over time. The height calculation unit 14 continues to calculate the height H of the preceding vehicle V2 at predetermined intervals. The predetermined interval is approximately several hundred milliseconds. In this embodiment, the predetermined interval is, for example, approximately 100 milliseconds. FIG. 4 is a diagram showing the height of the preceding vehicle V2 at each time calculated by the height calculation unit 14. As shown in FIG. 4, the height calculation unit 14 calculates, as the height H of the preceding vehicle V2, a height H1 of the feature point C21 of the preceding vehicle V2 at a first time, a height H2 of the feature point C22 of the preceding vehicle V2 at a second time, a height H3 of the feature point C23 of the preceding vehicle V2 at a third time, a height H4 of the feature point C24 of the preceding vehicle V2 at a fourth time, and a height H5 of the feature point C25 of the preceding vehicle V2 at a fifth time. The height calculation unit 14 calculates the height H at at least three times.

[0024] The gradient estimation unit 15 estimates the gradient of the road R2 ahead of the host vehicle V1 based on the change over time in the height H of the preceding vehicle V2. The gradient estimation unit 15 calculates an approximate curve L of the height positions of the preceding vehicle V2 at multiple times (in this embodiment, feature points C21 to C25) as the gradient of the road R2 ahead. The approximate curve L is a straight line or a curve. In this embodiment, the gradient estimation unit 15 calculates the approximate curve L using at least three height positions of the preceding vehicle V2. When the inter-vehicle distance D is equal to or greater than a predetermined threshold, the gradient estimation unit 15 calculates the approximate curve L using a linear approximation formula. The predetermined threshold is, for example, approximately several tens of meters. In this embodiment, the predetermined threshold is approximately 20 meters. When the inter-vehicle distance D is smaller than the above threshold, the gradient estimation unit 15 calculates the approximate curve L using a quadratic approximation formula.

[0025] The display control unit 16 displays icons and the like corresponding to the dividing lines P and the like of the road R2 ahead on the windshield so that they overlap with the scenery ahead of the vehicle V1. The display control unit 16 adjusts the position of the icons and the like on the windshield based on the gradient estimated by the gradient estimation unit 15.

[0026] The display control unit 16 adjusts the position of icons, etc. on the windshield by a larger amount as the gradient of the road R2 ahead increases. When the road R2 ahead is a slope, the display control unit 16 adjusts the position of icons, etc. on the windshield by a larger amount as the gradient of the road R2 ahead increases. When the road R2 ahead is a downhill road, the display control unit 16 adjusts the position of icons, etc. on the windshield by a larger amount as the gradient of the road R2 ahead increases.

[0027] The display control unit 16 adjusts the position of icons, etc. on the windshield by rotating the projection unit of the head-up display 5. The rotation angle of the projection unit is an angle corresponding to the gradient of the road R2 ahead. The display control unit 16 may adjust the position of icons, etc. on the windshield by moving the projection unit of the head-up display 5. The amount of movement of the projection unit is an amount corresponding to the gradient of the road R2 ahead.

[0028] Next, a description will be given of the processing performed by the in-vehicle display system 1. Figures 5 and 6 are flowcharts showing the processing performed by the in-vehicle display system 1.

[0029] 5, in step S1, the ECU 10 determines whether or not a preceding vehicle V2 exists. If it is determined that a preceding vehicle V2 exists (step S1: YES), the ECU 10 calculates the height H of the preceding vehicle V2 in step S2. The ECU 10 continues to calculate the height H of the preceding vehicle V2 at predetermined intervals. If it is determined that a preceding vehicle V2 does not exist (step S1: NO), the ECU 10 ends the current process.

[0030] In step S3, the ECU 10 estimates the gradient of the road R2 ahead based on the change over time in the height H of the preceding vehicle V2. In step S4, the ECU 10 adjusts the positions of icons and the like on the windshield based on the gradient of the road R2 ahead.

[0031] Fig. 6 is a flowchart showing the details of step S3 shown in Fig. 5. As shown in Fig. 6, in step S31, ECU 10 determines whether the inter-vehicle distance D between the host vehicle V1 and the preceding vehicle V2 is equal to or greater than the threshold value. If the inter-vehicle distance D is equal to or greater than the threshold value (step S31: YES), ECU 10 calculates the approximate curve L as the gradient of the road R2 ahead using a linear approximation formula in step S32. If the inter-vehicle distance D is smaller than the threshold value (step S31: NO), ECU 10 calculates the approximate curve L as the gradient of the road R2 ahead using a quadratic approximation formula in step S33.

[0032] As described above, in the in-vehicle display system 1, the gradient estimation unit 15 estimates the gradient of the road R2 ahead of the host vehicle V1 based on the change over time in the height H of the leading vehicle V2, and the display control unit 16 adjusts the position of icons, etc. on the windshield based on the gradient estimated by the gradient estimation unit 15. This ensures the positional accuracy of icons, etc. on the windshield even if the road R2 ahead of the host vehicle V1 has a gradient. Therefore, the in-vehicle display system 1 can improve the positional accuracy of icons, etc. on the windshield.

[0033] The display control unit 16 adjusts the position of the icons, etc. on the windshield by a larger amount as the gradient of the road R2 ahead increases, thereby ensuring the accuracy of the position of the icons, etc. on the windshield.

[0034] The gradient estimator 15 calculates an approximate curve L of the height positions (feature points C21 to C25) of the preceding vehicle V2 at a plurality of times as a gradient, thereby enabling the gradient of the road R2 ahead to be estimated with high accuracy.

[0035] When the inter-vehicle distance D is equal to or greater than the threshold value, the gradient estimation unit 15 uses a linear approximation formula to calculate the approximate curve L. This makes it possible to estimate the gradient of the road R2 ahead with even greater accuracy.

[0036] When the inter-vehicle distance D is smaller than the threshold value, the gradient estimation unit 15 uses a quadratic approximation formula to calculate the approximate curve L. This makes it possible to estimate the gradient of the road R2 ahead with even greater accuracy.

[0037] Although the embodiments of the present disclosure have been described above, the present disclosure is not limited to the above-described embodiments. The present disclosure can be implemented in various forms including the above-described embodiments and various modifications and improvements based on the knowledge of those skilled in the art.

[0038] The gradient estimation unit 15 may calculate the approximate curve L using two height positions of the preceding vehicle V2. In this case, the height calculation unit 14 may calculate the height H at two times. The gradient estimation unit 15 may calculate the approximate curve L using multiple height positions of the preceding vehicle V2.

[0039] The gradient estimation unit 15 may calculate the approximate curve L using the least squares method, a robust estimation method, or the like.

[0040] The preceding vehicle recognition unit 11 may recognize the preceding vehicle V2 based on both the image captured by the camera 2 and the detection result of the radar sensor 3.

[0041] When the inter-vehicle distance D is smaller than the threshold value, the gradient estimation unit 15 may calculate the approximate curve L using a linear approximation formula. [Explanation of symbols]

[0042] 1...in-vehicle display system, 2...camera, 3...radar sensor, 4...gyro sensor, 5...head-up display, 11...preceding vehicle recognition unit, 12...angle estimation unit, 13...inter-vehicle distance estimation unit, 14...height calculation unit, 15...gradient estimation unit, 16...display control unit, V1...host vehicle, V2...preceding vehicle, R2...road ahead

Claims

1. An in-vehicle display system that displays an image on a display screen of a head-up display of a vehicle, a preceding vehicle recognition unit that recognizes a preceding vehicle traveling ahead of the host vehicle based on an image captured by a camera of the host vehicle; an angle estimation unit that estimates an angle formed by a line connecting the host vehicle and the preceding vehicle with respect to a longitudinal direction of the host vehicle based on an image captured by the camera; an inter-vehicle distance estimation unit that estimates an inter-vehicle distance between the host vehicle and the preceding vehicle based on a detection result of a sensor of the host vehicle; a height calculation unit that calculates a height of the preceding vehicle based on the angle estimated by the angle estimation unit and the inter-vehicle distance estimated by the inter-vehicle distance estimation unit; a gradient estimation unit that estimates a gradient of a road ahead of the host vehicle based on a change over time in the height of the leading vehicle; a display control unit that displays the image on the display screen so as to overlap with a view ahead of the vehicle; The display control unit adjusts the position of the image on the display screen based on the gradient estimated by the gradient estimation unit.

2. The in-vehicle display system according to claim 1 , wherein the display control unit adjusts the position of the image on the display screen by a larger amount as the gradient increases.

3. The in-vehicle display system according to claim 1 , wherein the gradient estimation unit calculates, as the gradient, an approximate curve of the height position of the leading vehicle at a plurality of times.

4. The in-vehicle display system according to claim 3 , wherein the gradient estimation unit calculates the approximate curve using a linear approximation formula when the inter-vehicle distance is equal to or greater than a predetermined threshold value.

5. The in-vehicle display system according to claim 3 , wherein the gradient estimation unit calculates the approximate curve using a quadratic approximation formula when the inter-vehicle distance is smaller than a predetermined threshold value.

Images