Display device

Abstract

To provide a display device capable of suppressing the influence of erroneous recognition that may occur at an image recognition position using an image captured by a camera on the position of a vehicle image representing another vehicle in front of an own vehicle in a virtual image for superimposition.SOLUTION: A display apparatus 100 includes a recognition generation unit 11 for recognizing vehicles on the basis of detection results of a camera 1a and radar sensors 1b, 1c and generating virtual images for superimposition including images of other vehicles, and a display control unit 12 for displaying the virtual images for superimposition on a display unit of an own vehicle. The recognition generation unit 11 calculates an image recognizing position using the captured image of the camera 1a, calculates a combined position using the detection results of both the camera 1a and the radar sensors 1b and 1c, and calculates a corrected combined position by correcting the combined position so as to be close to the image recognizing position. The display control unit 12 corrects the position of the car image in the virtual image for superimposition based on the distance measurement value to the other car using the radar sensors 1b and 1c, the combined position, and the corrected combined position.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a display device. [Background technology]

[0002] Patent Document 1 describes an image projection device that projects a projection image onto a windshield to display a virtual image, which is a forward display image for calling attention, for example. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 2023-136363 Summary of the Invention [Problem to be solved by the invention]

[0004] For example, it is possible to recognize a vehicle image representing another vehicle in front of the vehicle based on the detection results of cameras and radar sensors used for driving assistance or autonomous driving, and to superimpose a virtual image including the vehicle image on the display unit of the vehicle.

[0005] When recognizing a vehicle image representing another vehicle ahead of the vehicle, the detection results of both the camera and the radar sensor may be used (fused) to calculate an integrated position, which is the position of the vehicle image in the virtual image for superimposition that corresponds to a predetermined plane on which the vehicle is located. For example, if the road ahead of the vehicle has a slope, and the actual position of the other vehicle does not exist on the predetermined plane, the position of the vehicle image in the virtual image for superimposition will deviate from the image recognition position recognized using the image captured by the camera. Therefore, the integrated position may be corrected to approach the image recognition position.

[0006] However, when recognizing other vehicles near the vehicle, if there is a pattern that is confusingly similar to the characteristics of the other vehicle, there is a risk of erroneous recognition occurring in the image recognition position using the image captured by the camera. As a result, the position corrected to bring the integrated position closer to the image recognition position may be displaced from the actual position of the other vehicle. [Means for solving the problem]

[0007] A display device according to one aspect of the present invention comprises a recognition generation unit that performs vehicle recognition based on the detection results of a camera that captures an image ahead of the host vehicle and a radar sensor that detects objects around the host vehicle, and generates a virtual image for superimposition that includes a vehicle image representing another vehicle ahead of the host vehicle recognized by vehicle recognition, and a display control unit that displays the virtual image for superimposition on the display unit of the host vehicle.The recognition generation unit calculates an image recognition position, which is the position of the vehicle image in the virtual image for superimposition, using the position of the other vehicle in the image captured by the camera, and calculates an integrated position, which is the position of the vehicle image in the virtual image for superimposition that corresponds to the specific plane on which the host vehicle is located, using the detection results of both the camera and the radar sensor, and corrects the integrated position to approach the image recognition position, thereby calculating a corrected integrated position, which is the position of the vehicle image in the virtual image for superimposition that corresponds to the actual position of the other vehicle relative to the host vehicle.The display control unit corrects the position of the vehicle image in the virtual image for superimposition based on the distance measurement value to the other vehicle using the radar sensor, the integrated position, and the corrected integrated position.

[0008] In a display device according to one aspect of the present invention, the position of the vehicle image in the virtual image for superimposition is corrected based on the distance measurement value to the other vehicle using a radar sensor, the integrated position, and the corrected integrated position. Therefore, even when recognizing another vehicle near the host vehicle, the position of the vehicle image in the virtual image for superimposition can be corrected according to the distance from the host vehicle to the other vehicle. This reduces the influence of erroneous recognition that may occur in the image recognition position using the image captured by the camera, compared to, for example, a case where the corrected integrated position, which is corrected so that the integrated position approaches the image recognition position regardless of the distance from the host vehicle to the other vehicle, is uniformly used.

[0009] In one embodiment, the display control unit may correct the position of the vehicle image in the virtual image for superimposition so that the larger the distance measurement value, the closer the vehicle image is to the corrected integrated position, and may correct the position of the vehicle image in the virtual image for superimposition so that the smaller the distance measurement value, the closer the vehicle image is to the integrated position. In this case, if the image recognition position using the image captured by the camera is unlikely to be affected by erroneous recognition, the corrected integrated position corrected to be closer to the image recognition position can be used. Furthermore, if the image recognition position using the image captured by the camera is likely to be affected by erroneous recognition, the integrated position that has not been corrected to be closer to the image recognition position can be used to suppress the effect of erroneous recognition that may occur in the image recognition position.

[0010] In one embodiment, the display control unit may calculate a virtual road surface gradient along a virtual line based on an angle between a virtual line connecting the host vehicle and the actual position of the other vehicle relative to the host vehicle and a predetermined plane, and correct the position of the vehicle image in the virtual image to be superimposed when the virtual road surface gradient is equal to or less than a predetermined upper gradient threshold. In this case, when the other vehicle ahead of the host vehicle is located on a road within a range of gradients that are realistically possible, the position of the vehicle image in the virtual image to be superimposed can be corrected. [Effects of the Invention]

[0011] According to the present invention, it is possible to suppress the influence of erroneous recognition that may occur in the image recognition position using the image captured by the camera on the position of the vehicle image representing another vehicle in front of the vehicle in the virtual image for superimposition. [Brief explanation of the drawings]

[0012] [Figure 1] 1 is a block diagram showing an example of a configuration of a display device according to an embodiment. [Figure 2] 1A is a diagram showing an example of a virtual image obtained by superimposing an image recognition position and an integrated position, and FIG. 1B is a diagram showing an example of a virtual image obtained by superimposing a corrected integrated position. [Figure 3] 10A and 10B are diagrams illustrating an example of a deviation of a corrected integrated position. [Figure 4] 10A and 10B are diagrams illustrating another example of deviation of the integrated position after correction. [Figure 5] FIG. 10 is a diagram illustrating a correction ratio used in an example of correcting the position of a vehicle image. [Figure 6] 4 is a diagram illustrating an example in which the position of the vehicle image in FIG. 3 is corrected. FIG. [Figure 7] FIG. 10 is a diagram illustrating an example of a situation in which a virtual road surface gradient exceeds a predetermined upper gradient threshold. [Figure 8] 10 is a flowchart illustrating an example of processing performed by the display device according to the embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0013] Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

[0014] Fig. 1 is a block diagram showing a display device 100 according to an embodiment. The display device 100 shown in Fig. 1 is mounted on a vehicle such as a passenger car or a freight vehicle (hereinafter referred to as the host vehicle) and displays an image according to the surrounding conditions of the vehicle. The host vehicle may have a driving assistance function such as ACC (Adaptive Cruise Control) or an automatic driving function including vehicle speed control.

[0015] As shown in FIG. 1, the display device 100 includes a display control ECU (Electronic Control Unit) 10 that performs overall management of the device. The display control ECU 10 is an electronic control unit having a CPU (Central Processing Unit) and a storage unit. The storage unit is composed of, for example, a ROM (Read Only Memory), a RAM (Random Access Memory), and an EEPROM (Electrically Erasable Programmable Read-Only Memory). The display control ECU 10 realizes various functions by, for example, causing the CPU to execute programs stored in the storage unit. An external sensor 1 and a display 2 (display unit) are connected to the display control ECU 10. Note that the display control ECU 10 may be composed of multiple electronic units.

[0016] The external sensor 1 is a detection device that detects the situation around the vehicle, and includes a camera 1a that captures an image ahead of the vehicle, and a radar sensor.

[0017] The camera 1a is an imaging device that captures an image of the area in front of the vehicle. The camera 1a is provided, for example, behind the windshield of the vehicle and captures an image of the area in front of the vehicle. The types of the camera 1a include, for example, a telephoto camera, a wide-angle camera, a monocular camera, and a stereo camera. The stereo camera has two imaging units arranged to reproduce binocular parallax. The imaging information of the stereo camera also includes information in the depth direction. The camera 1a transmits information about the captured image to the display control ECU 10.

[0018] A radar sensor is a detection device that detects objects around the vehicle using radio waves (e.g., millimeter waves) or light. The radar sensor detects objects by transmitting radio waves or light to the vicinity of the vehicle and receiving the radio waves or light reflected by the objects. Examples of radar sensors include a LiDAR (Light Detection And Ranging) 1b and a millimeter-wave radar 1c. The radar sensor may acquire distance measurements to objects (other vehicles, etc.) around the vehicle. The radar sensor transmits information about the detected objects and the acquired distance measurements to the display control ECU 10. In the following description, radar sensors including at least one of the LiDAR 1b and the millimeter-wave radar 1c will be collectively referred to simply as "radar sensors 1b, 1c."

[0019] The display 2 is a display device provided in the cabin of the vehicle. An example of the display 2 is a head-up display (HUD). The display 2 may be configured as an AR-HUD that projects an image onto the display surface of the front windshield using, for example, augmented reality (AR) technology. The display area of ​​the head-up display is a pre-set area on the front windshield, and is the range into which a virtual image is projected and superimposed. The display 2 is controlled to display various information in response to control signals from the display control ECU 10. The display 2 displays a virtual image including a vehicle image representing another vehicle superimposed on an image representing the situation ahead of the vehicle.

[0020] Next, a description will be given of the functional configuration of the display control ECU 10. The display control ECU 10 includes a recognition generation unit 11 and a display control unit 12.

[0021] The recognition generation unit 11 performs vehicle recognition based on the detection results of the camera 1a that captures images ahead of the host vehicle and the radar sensor 1b that detects objects around the host vehicle. For vehicle recognition, the recognition generation unit 11 recognizes other vehicles ahead of the host vehicle, for example, by calculating an image recognition position, an integrated position, and a corrected integrated position. The other vehicles are, for example, vehicles traveling ahead of the host vehicle. The other vehicles are not limited to four-wheeled vehicles, but may also be two-wheeled vehicles.

[0022] The image recognition position is the position of the vehicle image in the virtual image for superimposition, calculated using the position of the other vehicle in the image captured by the camera 1a. The recognition generation unit 11 can calculate the image recognition position by performing well-known image recognition on the image captured by the camera 1a, for example, by deep learning.

[0023] The integrated position is the result of so-called FSN (fusion) recognition calculated using the detection results of both the camera 1a and the radar sensors 1b and 1c. The integrated position is, for example, the position of the vehicle image in the virtual image for superimposition, which corresponds to a predetermined plane on which the host vehicle exists. The predetermined plane may be assumed to be, for example, a horizontal plane (a plane with no gradient) on which the host vehicle exists. The recognition generation unit 11 can calculate the integrated position by performing known FSN recognition. Known FSN recognition refers to a known object recognition method that corrects the relative position of another vehicle calculated based on the image captured by the camera by also using the detection results of another sensor with higher ranging accuracy than the camera. For example, when there is a combination of another vehicle recognized from the image captured by the camera 1a and another vehicle recognized from the detection results of the radar sensors 1b and 1c that satisfies a predetermined positional relationship that determines that they belong to the same object, the recognition generation unit 11 combines (fuses) information representing the other vehicles in the combination to generate a fusion target as a combined target. In addition, the "relative position of another vehicle calculated based on the image captured by the camera" used for FSN recognition is not limited to "a position corresponding to a specified plane on which the vehicle itself is located" as in this embodiment, but may also be the relative position of another vehicle calculated with lower ranging accuracy than other sensors, for example, when the camera has a wide viewing angle.

[0024] The recognition generation unit 11 generates a virtual image for superimposition including a vehicle image representing another vehicle ahead of the host vehicle recognized by vehicle recognition. FIG. 2(a) is a diagram showing an example of a virtual image (virtual image for superimposition) in which an image recognition position and an integrated position are superimposed. In FIG. 2(a), a virtual image for superimposition 20 projected on the display 2 shows the situation ahead of the host vehicle as seen from the host vehicle. In FIG. 2(a), for a vehicle image 21 of another vehicle traveling ahead of the host vehicle, the image recognition position is indicated by a solid-line frame 22, and the integrated position is indicated by a solid-line marker 23. The marker 23 may be, for example, in the shape of a "^".

[0025] In the example of FIG. 2(a), the marker 23 is located below the bottom end of the solid-line frame 22. This positional deviation occurs because, for example, the road in FIG. 2(a) has an upward slope, and the position of the marker 23 in the virtual image for superimposition 20, which corresponds to a position on a predetermined plane without a slope, does not correspond to the actual position of another vehicle located on the sloped road. In the example of FIG. 2(a), the position of another vehicle located on the sloped road corresponds to the position of the solid-line frame 22 that surrounds the vehicle image 21 in the virtual image for superimposition 20. In the following description, the "position" in the virtual image for superimposition corresponds to the position in the depth direction ahead of the vehicle and refers to the vertical coordinate position in the virtual image for superimposition.

[0026] FIG. 2(b) is a diagram showing an example of a virtual image on which the corrected integrated position is superimposed. In the example of FIG. 2(b), the corrected integrated position is indicated by a marker 23 that has been moved from the solid-line marker 23 in FIG. 2(a). This corrected integrated position is calculated by correcting the integrated position (solid-line marker 23) in FIG. 2(a) to approach the image recognition position (solid-line frame 22) using a known correction method based on, for example, the difference in the vertical coordinates between the frame 22 and the marker 23 in the virtual image 20 for superimposition. The corrected integrated position is the position of the vehicle image in the virtual image 20 for superimposition (e.g., the bottom edge of the solid-line frame 22), and corresponds to the actual position of the other vehicle relative to the host vehicle. Such a corrected integrated position can be said to be a position corrected with high accuracy when the error in the superimposed image recognition position calculated using the position of the other vehicle in the image captured by camera 1a is small.

[0027] FIG. 3 is a diagram illustrating an example of a deviation in the corrected integrated position. In FIG. 3, another example of the superimposing virtual image 30 projected on the display 2 shows a situation ahead of the host vehicle as seen from the host vehicle in a scene different from that shown in FIG. 2. The superimposing virtual image 30 in FIG. 3 shows a vehicle image 31 representing a motorcycle, which is another vehicle traveling ahead of the host vehicle, in the lane immediately to the right of the lane in which the host vehicle is traveling. The other vehicle in FIG. 3 is closer to the host vehicle than the other vehicle in FIG. 2. However, for example, when recognizing another vehicle near the host vehicle, if there is a pattern that is confusingly similar to the characteristics of the other vehicle, there is a risk of erroneous recognition occurring at the image recognition position using the image captured by the camera 1a.

[0028] In the example of Figure 3, the pattern that is likely to be confused with the characteristics of another vehicle corresponds to a case where the other vehicle is a motorcycle. Therefore, the image recognition position (solid line frame 32) is different from the original position (the position that should be the bottom display position of the bottom edge of the frame) by dashed line 34, and is, for example, at the position of dashed line 35, which is further back than the original position. As a result, the corrected integrated position (marker 33) in Figure 3 is also at the position of dashed line 35, which is further back than the original position.

[0029] Fig. 4 is a diagram illustrating another example of a deviation in the corrected integrated position. Fig. 4 shows another example of a virtual image for superimposition 40 projected on the display 2, depicting a situation ahead of the host vehicle as seen from the host vehicle in a scene different from that shown in Figs. 2 and 3. The virtual image for superimposition 40 in Fig. 4 shows a vehicle image 41 representing a minivan, another vehicle traveling ahead of the host vehicle, in the lane in which the host vehicle is traveling, for example, at night when it is dark. The other vehicle in Fig. 4 has a pair of patterns 42 on both the left and right ends of its rear bumper.

[0030] In the example of Fig. 4, a pair of patterns 42 is mistakenly recognized as a pair of left and right tires, which is a pattern that may be confused with the characteristics of another vehicle. Therefore, another vehicle is mistakenly recognized as being present at the position of the pair of patterns 42, and the image recognition position is different from the original position (dashed line 43), for example, it is at the position of a dashed-dotted frame 44, which is higher than the original position. As a result, the corrected integrated position in Fig. 4 may also be higher than the original position by the height of the arrow 45.

[0031] Therefore, the display control unit 12 corrects the position of the vehicle image in the virtual image for superimposition based on the distance measurement value to the other vehicle using the radar sensors 1b and 1c, the integrated position, and the corrected integrated position. For example, the display control unit 12 corrects the position of the vehicle image in the virtual image for superimposition so that the larger the distance measurement value using the radar sensors 1b and 1c, the closer the position of the vehicle image in the virtual image for superimposition becomes to the corrected integrated position. Alternatively, the display control unit 12 may correct the position of the vehicle image in the virtual image for superimposition so that the smaller the distance measurement value using the radar sensors 1b and 1c becomes, the closer the position of the vehicle image in the virtual image for superimposition becomes to the integrated position.

[0032] FIG. 5 is a diagram showing a correction ratio used in an example of correcting the position of a vehicle image. The horizontal axis of FIG. 5 represents the distance measurement value to other vehicles using radar sensors 1b and 1c. The vertical axis of FIG. 5 represents the ratio α of the integrated positions. In the example of FIG. 5, an inverse proportional formula is used as the correction ratio α used in an example of correcting the position of the vehicle image, such that the ratio α becomes 1 when the distance measurement value is near 0. By applying such ratio α to the following formula (1), the display bottom position of the vehicle image in the virtual image for superimposition is corrected so that the larger the distance measurement value, the closer it is to the corrected integrated position, and the smaller the distance measurement value, the closer it is to the integrated position. Display bottom position = α × integrated position + (1-α) × integrated position after correction (1)

[0033] The display control unit 12 displays the superimposed virtual image, with the position of the vehicle image corrected, on the display 2 of the vehicle. FIG. 6 is a diagram illustrating an example in which the position of the vehicle image in FIG. 3 has been corrected. As shown in FIG. 6, the measured distance from the vehicle to the other vehicle, which is a motorcycle, is smaller than in the case shown in FIG. 2. Therefore, a ratio α close to 1 is applied to the above equation (1). As a result, the solid-line frame 62 that should surround the vehicle image 61 in the superimposed virtual image 60 is corrected to approach the integrated position (dashed-dotted line 63). Even if an error occurs in the image recognition position using the image captured by camera 1a, the marker 64 can be prevented from shifting due to the influence of the error.

[0034] FIG. 7 is a diagram showing an example of a situation in which the virtual road surface gradient exceeds a predetermined upper gradient threshold. In FIG. 7, a minivan 72 is loaded onto a loaded vehicle 71 traveling ahead of the host vehicle. In image recognition using images captured by camera 1a, both the loaded vehicle 71 and the minivan 72 may be recognized as "other vehicles ahead of the host vehicle." However, because the minivan 72 is not traveling on a road, the position of the vehicle image does not need to be corrected.

[0035] Therefore, the display control unit 12 may calculate a virtual road surface gradient along a virtual line based on the angle between a predetermined plane and a virtual line connecting the actual position of the other vehicle relative to the host vehicle and the host vehicle. The virtual road surface gradient is a value (for example, in units of "%)" that corresponds to the gradient of a road surface when it is assumed that the other vehicle ahead of the host vehicle is located on a sloped road surface. For example, for each of the loader vehicle 71 and the minivan 72, the display control unit 12 calculates the angle between the image recognition position acquired by vehicle recognition using the image captured by the camera 1a and a predetermined plane from the difference in the number of pixels in the captured image, and calculates the virtual road surface gradient from the angle.

[0036] When the virtual road surface gradient is equal to or less than a predetermined upper gradient threshold, the display control unit 12 may correct the position of the vehicle image in the superimposed virtual image 70. The upper gradient threshold is a predetermined gradient threshold set in advance for determining whether or not to correct the position of the vehicle image. The upper gradient threshold may be, for example, a value of the longitudinal gradient specified in the Road Structure Act.

[0037] If the virtual road surface gradient is equal to or less than the upper gradient threshold, it is highly likely that the vehicle ahead of the host vehicle is actually on a road surface with a gradient. Therefore, the position of the vehicle image in the superimposed virtual image 70 is corrected. On the other hand, if the virtual road surface gradient exceeds the upper gradient threshold, there is no road with such a large gradient in reality, so it is unlikely that the vehicle ahead of the host vehicle is actually on a road surface with a gradient. Therefore, it is not necessary to correct the position of the vehicle image in the superimposed virtual image 70.

[0038] In the example of Figure 7, the dashed line 73 surrounding the loaded vehicle 71 is almost entirely contained within the dashed line 74 surrounding the loaded vehicle 71 and the minivan 72. In addition, the dashed line 73 surrounding the loaded vehicle 71 is adjacent to the solid line 75 surrounding the minivan 72 on the top and bottom. If these conditions continue for a certain period of time or longer, the dashed line 73 surrounding the loaded vehicle 71 may be used as the integrated position.

[0039] [Display control ECU processing] Next, an example of processing by the display control ECU 10 will be described with reference to Fig. 8. Fig. 8 is a flowchart showing an example of processing by the display device according to the embodiment. The processing shown in Fig. 8 is performed repeatedly at predetermined intervals during operation of the display control ECU 10, for example.

[0040] 8, in S01, the display control ECU 10 calculates an image recognition position using the recognition generation unit 11. The recognition generation unit 11 calculates the image recognition position, which is the position of the vehicle image in the virtual image to be superimposed, using the position of another vehicle in the image captured by the camera 1a.

[0041] In S02, the display control ECU 10 calculates an integrated position using the recognition generation unit 11. The recognition generation unit 11 uses the detection results of both the camera 1a and the radar sensors 1b and 1c to calculate an integrated position, which is the position of the vehicle image in the virtual image for superimposition and corresponds to a predetermined plane on which the host vehicle exists.

[0042] In S03, the display control ECU 10 calculates the corrected integrated position using the recognition generation unit 11. The recognition generation unit 11 corrects the integrated position so that it approaches the image recognition position, thereby calculating the corrected integrated position, which is the position of the vehicle image in the virtual image for superimposition and corresponds to the actual position of the other vehicle relative to the host vehicle.

[0043] In S04, the display control ECU 10 acquires the distance measurement value to the other vehicle by the recognition generation unit 11. The recognition generation unit 11 acquires the distance measurement value to the other vehicle using the radar sensors 1b and 1c, using the detection results of the radar sensors 1b and 1c.

[0044] In S05, the display control ECU 10 calculates the virtual road surface gradient using the display control unit 12. The display control unit 12 calculates the virtual road surface gradient along the virtual line based on the angle between the virtual line connecting the actual position of the other vehicle relative to the host vehicle and a predetermined plane.

[0045] In S06, the display control ECU 10 causes the display control unit 12 to determine whether the virtual road surface gradient is equal to or less than a predetermined upper gradient threshold. If the display control unit 12 determines that the virtual road surface gradient is equal to or less than the upper gradient threshold (S06: YES), the display control ECU 10 proceeds to S07. If the display control unit 12 determines that the virtual road surface gradient exceeds the upper gradient threshold (S06: NO), the display control ECU 10 proceeds to S08.

[0046] In S07, the display control ECU 10 causes the display control unit 12 to correct the position of the vehicle image. For example, the display control unit 12 corrects the position of the vehicle image in the virtual image for superimposition so that the larger the distance measurement value, the closer the vehicle image approaches the corrected integrated position. Alternatively, for example, the display control unit 12 corrects the position of the vehicle image in the virtual image for superimposition so that the smaller the distance measurement value, the closer the vehicle image approaches the integrated position. Thereafter, the display control ECU 10 ends the processing of FIG. 3.

[0047] Meanwhile, in S08, the display control ECU 10 causes the display control unit 12 to end the correction of the position of the vehicle image. The correction of the position of the vehicle image may be ended by using an integrated position corresponding to a predetermined plane on which the host vehicle is located. Furthermore, the correction of the position of the vehicle image may be ended by holding the previous display for a certain period of time, or, if the certain period of time has passed and the display continues, by repeatedly multiplying the display by a predetermined forgetting factor such that the ratio α in the above equation (1) gradually approaches 0, thereby gradually bringing the display closer to the integrated position. Then, the display control ECU 10 ends the processing of FIG. 3.

[0048] As described above, the display device 100 corrects the position of the vehicle image in the virtual image for superimposition based on the distance measurement value to the other vehicle using the radar sensors 1b and 1c, the integrated position, and the corrected integrated position. Therefore, even when recognizing another vehicle near the subject vehicle, the position of the vehicle image in the virtual image for superimposition can be corrected according to the distance from the subject vehicle to the other vehicle. This reduces the influence of erroneous recognition that may occur in the image recognition position using the image captured by the camera 1a, compared to, for example, a case where the corrected integrated position, which is corrected so that the integrated position approaches the image recognition position regardless of the distance from the subject vehicle to the other vehicle, is uniformly used.

[0049] In the display device 100, the display control unit 12 corrects the position of the vehicle image in the virtual image for superimposition so that the larger the distance measurement value, the closer the vehicle image is to the corrected integrated position, and corrects the position of the vehicle image in the virtual image for superimposition so that the smaller the distance measurement value, the closer the vehicle image is to the integrated position. As a result, when the distance measurement value is large and the image recognition position using the image captured by camera 1a is unlikely to be affected by erroneous recognition, the corrected integrated position corrected to be closer to the image recognition position can be used. On the other hand, when the distance measurement value is small and the image recognition position using the image captured by camera 1a is likely to be affected by erroneous recognition, the integrated position that has not been corrected to be closer to the image recognition position can be used, thereby suppressing the impact of erroneous recognition that may occur in the image recognition position.

[0050] In the display device 100, the display control unit 12 calculates a virtual road surface gradient along a virtual line based on the angle between a predetermined plane and a virtual line connecting the actual position of the other vehicle relative to the host vehicle and the host vehicle, and corrects the position of the vehicle image in the virtual image to be superimposed when the virtual road surface gradient is equal to or less than a predetermined upper gradient threshold. This makes it possible to correct the position of the vehicle image in the virtual image to be superimposed when the other vehicle ahead of the host vehicle is on a road with a gradient within a realistic range.

[0051] Although the embodiments of the present invention have been described above, the present invention is not limited to the above-described embodiments.

[0052] In the above embodiment, the display control unit 12 corrects the position of the vehicle image in the virtual image for superimposition when the virtual road surface gradient is equal to or less than a predetermined upper gradient threshold. However, this example is not essential and may be omitted. If this example is omitted, the display control unit 12 does not need to calculate the virtual road surface gradient along the virtual line.

[0053] In the above embodiment, the position of the vehicle image is corrected using the ratio α given in inverse proportion as shown in FIG. 5 , but this example is not required. For example, the display control unit 12 may use another monotonically decreasing formula to correct the position of the vehicle image in the virtual image for superimposition so that the larger the distance measurement value, the closer the vehicle image approaches the corrected integrated position, and may correct the position of the vehicle image in the virtual image for superimposition so that the smaller the distance measurement value, the closer the vehicle image approaches the integrated position. The display control unit 12 may also use another formula that has a constant portion or an increasing section instead of a monotonically decreasing formula, or may switch whether or not to correct the position of the vehicle image depending on the result of comparing the distance measurement value to other vehicles using the radar sensors 1b and 1c with a predetermined threshold. In short, the display control unit 12 may correct the position of the vehicle image using the distance measurement value to other vehicles using the radar sensors 1b and 1c. [Explanation of symbols]

[0054] 1a... camera, 1b, 1c... radar sensor, 11... recognition generation unit, 12... display control unit, 20, 30, 40, 60... virtual image for superimposition, 21, 31, 41, 61... vehicle image, 100... display device.

Claims

1. a recognition and generation unit that performs vehicle recognition based on detection results from a camera that captures an image ahead of the host vehicle and a radar sensor that detects objects around the host vehicle, and generates a virtual image for superimposition that includes a vehicle image representing another vehicle ahead of the host vehicle that is recognized by the vehicle recognition; a display control unit that displays the superimposed virtual image on a display unit of the host vehicle, The recognition generation unit calculating an image recognition position, which is the position of the vehicle image in the virtual image for superimposition, using the position of the other vehicle in the image captured by the camera; calculating an integrated position, which is a position of the vehicle image in the superimposed virtual image and corresponds to a predetermined plane on which the host vehicle is located, using detection results from both the camera and the radar sensor; correcting the integrated position so as to approach the image recognition position, thereby calculating a corrected integrated position, which is a position of the vehicle image in the superimposed virtual image and corresponds to an actual position of the other vehicle relative to the host vehicle; The display control unit corrects the position of the vehicle image in the superimposed virtual image based on the distance measurement value to the other vehicle using the radar sensor, the integrated position, and the corrected integrated position.

2. The display control unit correcting the position of the vehicle image in the superimposed virtual image so that the position approaches the corrected integrated position as the distance measurement value increases; The display device according to claim 1 , wherein the position of the vehicle image in the superimposed virtual image is corrected so that the smaller the measured distance value, the closer the vehicle image is to the integrated position.

3. The display control unit calculating a virtual road surface gradient along a virtual line based on an angle formed between the actual position of the other vehicle relative to the host vehicle and the virtual line and the predetermined plane; The display device according to claim 1 , wherein the position of the vehicle image in the superimposed virtual image is corrected when the virtual road surface gradient is equal to or smaller than a predetermined upper gradient threshold.

Images