Display control device

Abstract

The display control device disclosed herein can reduce user discomfort when displaying targets whose type cannot be identified. [Solution] The display control device includes: an object detection unit that detects the presence of an object using a sensor other than a camera and identifies the type of object from an image captured by the camera; and a display control unit that, when an object is detected within the detection range of the other sensor but outside the object type identification range of the camera and the type of object cannot be identified, displays a general-purpose object in the shape of a roughly rectangular parallelepiped, with a height lower than the icon used to display according to the type, on the display unit.

Description

Technical Field

[0001] This disclosure relates to a display control device.

Background Art

[0002] Patent Document 1 discloses an object detection device that can more accurately determine the type of an object using a captured image. In Patent Document 1, a pedestrian or a two-wheeled vehicle as an object existing around the host vehicle is detected by performing image processing on a captured image of the traveling direction of the host vehicle. The type of the object is tentatively determined by analyzing the captured image, and the type of the tentatively determined object is finally determined using the moving speed of the object.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In the prior art, for example, there is a known technique of displaying a target as object presence information detected by a sensor or the like on a display device using an icon. However, when the target is outside the detection range of the camera and within the detection range of another sensor, even if the presence of the target can be identified, the type cannot be specified, and there is a problem that it cannot be displayed unless it enters the identification range of the camera. Therefore, when it enters the identification range of the camera, it will be switched to an appropriate type of icon, but if the icon is switched to a completely different type, it may cause a sense of discomfort to the user.

[0005] An object of this disclosure is to provide a display control device that can reduce the user's sense of discomfort in displaying a target whose type cannot be identified.

Means for Solving the Problems

[0006] The display control device according to claim 1 includes: an object detection unit that detects the presence of an object using other sensors provided by the vehicle other than a camera and identifies the type of object from an image captured by the camera provided by the vehicle; and a display control unit that, when an object is detected outside the object type identification range of the camera but within the detection range of the other sensors and the type of the object cannot be identified, causes the display unit to display a general-purpose object with a height lower than the icon used to display according to the type.

[0007] The display control device described in claim 1 displays a general-purpose target for which the type cannot be identified. This allows the user to easily understand that the type of target has not been identified, and to recognize the target without any sense of incongruity even when the target is identified and switched to.

[0008] The display control device according to claim 2 is the display control device according to claim 1, wherein the display control unit estimates the type of object that is most likely to be present around the vehicle and displays a general-purpose object of a size corresponding to the estimated type of object. According to the display control device according to claim 2, by displaying a general-purpose object corresponding to the most likely type, the user's sense of unease can be reduced.

[0009] The display control device according to claim 3 is the display control device according to claim 2, wherein the display control unit estimates the most likely type of object to be present around the vehicle from frequency information of object types identified around the vehicle, as a method for estimating the most likely type of object to be present around the vehicle. According to the display control device according to claim 3, the user's discomfort can be reduced by displaying general-purpose objects according to the frequency of objects in the surrounding area.

[0010] The display control device according to claim 4 is the display control device according to claim 2, wherein a high probability type is set in advance for a specific date and time and lane set, and the display control unit estimates the high probability type in the driving lane from the set set. According to the display control device according to claim 4, the user's discomfort can be reduced by displaying a general-purpose marker corresponding to a specific date and time and driving lane.

[0011] The display control device according to claim 5 is the display control device according to claim 4, wherein the target detection unit identifies at least one of the following types of targets: a passenger car, a commercial vehicle, a motorcycle, and a pedestrian, and the display control unit displays a general-purpose target of any of the sizes of the passenger car, commercial vehicle, motorcycle, and pedestrian that is estimated to be the most likely type of target. According to the display control device according to claim 5, by displaying general-purpose targets corresponding to user-friendly types such as passenger cars, commercial vehicles, motorcycles, and pedestrians, the user can easily recognize general-purpose targets. [Effects of the Invention]

[0012] The technology disclosed herein can reduce user discomfort when displaying objects whose type cannot be identified. [Brief explanation of the drawing]

[0013] [Figure 1] Figure 1 shows the configuration of the in-vehicle system according to this embodiment. [Figure 2A] Figure 2A is a schematic diagram illustrating an example of target detection. [Figure 2B] Figure 2B is a schematic diagram illustrating an example of target detection. [Figure 3] Figure 3 is a diagram illustrating general-purpose targets categorized by size and type. [Figure 4A] Figure 4A shows an example of estimating the most likely type of object based on frequency information of object types identified in the driving lane. [Figure 4B]Figure 4B shows an example of estimating the most likely type of object based on frequency information of object types identified in the driving lane. [Figure 5] Figure 5 is a flowchart illustrating the flow of the display control process as a display control method executed by the display control ECU of this embodiment. [Modes for carrying out the invention]

[0014] [First Embodiment] Hereinafter, an example of an embodiment of this disclosure will be described in detail with reference to the drawings. As shown in Figure 1, the in-vehicle system 10 according to this embodiment is equipped with a communication bus 12. The communication bus 12 is connected to a group of surrounding conditions acquisition devices 14, a group of vehicle driving state detection sensors 26, an ADAS (Advanced Driver-Assistance Systems) ECU 34, and a display control ECU 42, respectively. Note that Figure 1 shows only a part of the in-vehicle system 10. In the following, the vehicle on which the in-vehicle system 10 is installed will be referred to as "the vehicle."

[0015] The surrounding environment acquisition device group 14 includes a GNSS (Global Navigation Satellite System) device 16, an in-vehicle communication device 18, a navigation system 20, a camera device 24, and a radar device 22, as devices that acquire surrounding information representing the conditions of the environment around the vehicle.

[0016] The GNSS device 16 receives GNSS signals from multiple GNSS satellites to determine the position of the vehicle. The in-vehicle communication device 18 performs at least one of vehicle-to-vehicle communication with other vehicles and vehicle-to-infrastructure communication with roadside units. The navigation system 20 includes a map information storage unit 20A that stores map information, and based on the position information obtained from the GNSS device 16 and the map information stored in the map information storage unit 20A, it displays the position of the vehicle on a map and determines and guides the vehicle to a destination.

[0017] The radar device 22 is a group of radars that detect objects such as other vehicles and pedestrians existing around the host vehicle as point cloud information, and detects objects in front of, on the sides, and behind the host vehicle. Further, the radar device 22 acquires the relative position and relative speed between the detected object and the host vehicle. The radar device 22 is an example of another sensor of the present disclosure. In the display control ECU 42 described later, the presence of a target is detected from the detection result of the radar device 22.

[0018] Further, based on changes in the relative position and relative speed with respect to individual objects, the radar device 22 excludes roadside objects such as noise and guardrails from the monitoring targets, and outputs information such as the relative position and relative speed with respect to targets that are monitoring target objects such as other vehicles and pedestrians.

[0019] The camera device 24 is a plurality of cameras that photograph the periphery of the host vehicle and outputs the photographed images. In the present embodiment, the case where the camera device 24 photographs the front of the host vehicle will be described as an example, but a side camera that photographs the rear and a side camera that photographs the side closest to the host vehicle from the side mirror are also conceivable. In the display control ECU 42 described later, the type of the target is identified from the information of the object existing in the photographed image. The camera device 24 is an example of the camera of the present disclosure.

[0020] As described above, it is detected from the detection result of the radar device 22 that a target exists, but the type of the target is not identified, and the type is identified by analyzing the image of the camera device 24.

[0021] The vehicle running state detection sensor group 26 includes a steering angle sensor 28 that detects the steering angle of the host vehicle, a vehicle speed sensor 30 that detects the running speed of the host vehicle, and an acceleration sensor 32 that detects the acceleration applied to the host vehicle, as a plurality of sensors that acquire the running state of the host vehicle.

[0022] The ADAS-ECU 34 is connected to a throttle ACT 36 that changes the throttle opening degree of the host vehicle, a brake ACT 38 that changes the braking force generated by the braking device of the host vehicle, and a steering ACT 40 that changes the steering amount by the steering device of the host vehicle, respectively.

[0023] The ADAS-ECU34, although not shown in the diagram, includes a CPU, memory such as ROM and RAM, storage such as an HDD or SSD, and a communication interface (I / F). The storage contains a predetermined program that enables the ADAS-ECU34's CPU to function as a support unit 35. When the vehicle is in driving assistance mode, the support unit 35 controls the throttle ACT36, brake ACT38, and steering ACT40 based on information obtained from the surrounding situation acquisition device group 14 and the vehicle driving state detection sensor group 26, as well as the target detection results from the display control ECU42, to perform support processing that assists at least one of the vehicle's acceleration / deceleration and steering. Examples of driving assistance functions realized by this support processing include ACC (Adaptive Cruise Control), which drives the vehicle to follow a preceding vehicle in its lane, and LKA (Lane Keeping Assist), which assists steering to keep the vehicle in the center of its lane.

[0024] The display control ECU 42 includes a CPU 44, memory 46 such as ROM or RAM, storage 48 such as HDD or SSD, and a communication I / F 50. The CPU 44, memory 46, storage 48, and communication I / F 50 are connected to each other via an internal bus 52 so that they can communicate with one another. The display control program 54 is stored in the storage 48. The display control ECU 42 reads the display control program 54 from the storage 48 and expands it into the memory 46, and the display control program 54 expanded into the memory 46 is executed by the CPU 44. The CPU 44 functions as a target detection unit 62 and a display control unit 64, and performs the display control processing described later. Note that the function of the target detection unit 62 may be an external function of the display control ECU 42, and a separate target detection ECU may be provided.

[0025] The display control ECU42 is for AR (Augmented Reality) - Head-Up Display (and Below, an AR-HUD 56 (referred to as an AR-HUD) and a meter display 58 are connected. The AR-HUD 56 according to this embodiment is a small HUD that displays an image (forming an image in the lower foreground) on a portion of the forward field of view of the vehicle's occupants by reflection from the windshield glass, etc. The meter display 58 is a display provided on the instrument panel of the vehicle. The display control ECU 42 controls the display of information on the AR-HUD 56 and the meter display 58. Note that the display control ECU 42 is an example of a display control device according to this disclosure, and the AR-HUD 56 and meter display 58 are examples of display units in this disclosure.

[0026] The target detection unit 62 detects the presence of targets around the vehicle based on the detection results from the radar device 22. The target detection unit 62 also identifies the type of target from the image captured by the camera device 24. In this embodiment, the target detection unit 62 identifies at least passenger cars, large commercial vehicles such as trucks, motorcycles, and pedestrians as types of targets.

[0027] Figure 2 schematically illustrates an example of target detection. In the examples in Figures 2A and 2B, the vehicle image is 78, the targets around the vehicle are 80, the detection range of the radar device 22 is 90, and the identification range of the camera device 24 is 92. Note that the detection range 90 shown in Figures 2A and 2B is shown for illustrative purposes only as the detection range to the side of the vehicle that does not overlap with the identification range 92 of the camera device 24, and the front and rear areas that overlap with the identification range 92 of the camera device 24 may also be included in the detection range.

[0028] In the example in Figure 2A, the target 80 is located within the detection range 90 of the radar device 22 and within the identification range 92 of the camera device 24, and the type of target 80 can be identified. In the example in Figure 2B, the target 80 is located within the detection range 90 of the radar device 22, but outside the identification range 92 of the camera device 24, and the type of target 80 cannot be identified. As shown in Figure 2A, when the target 80 can be identified, an icon image 80A indicating the type of target 80, which is prepared according to the type, is displayed. As shown in Figure 2B, when the target 80 cannot be identified, a general-purpose target image 80B is displayed. The general-purpose target image 80B is a roughly rectangular parallelepiped with a lower height than the icon displayed according to the type. By displaying the general-purpose target in this way, it is possible to indicate that there are targets whose type has not yet been identified. The height of the general-purpose target is lower than the icon of the target type. By displaying general-purpose targets with different sizes and appearances from the type icons, it is possible to prevent users from mistaking them for type icons and making it easier to understand that the type is unidentified. Furthermore, by reducing the height of the general-purpose targets and displaying them at the minimum size necessary for identification, the estimated type can be more easily conveyed through size. The target shape can also be limited to a roughly rectangular prism, including trapezoids and truncated cones. Additionally, by temporarily displaying general-purpose targets that do not specify passenger cars, commercial vehicles, motorcycles, or pedestrians, it is possible to reduce user discomfort when the type is identified and the corresponding icon image is displayed.

[0029] Figure 3 is a diagram illustrating general-purpose targets of different sizes corresponding to different target types. Figures 3(1) to (4) show the respective icon images and their corresponding general-purpose target images. The general-purpose target images are set to approximately the same width and depth as the corresponding icon images of each type. (1) is the icon image (80A-1) representing a passenger car and the general-purpose target (80B-1) corresponding to the passenger car type. (2) is the icon image (80A-2) representing a commercial vehicle and the general-purpose target (80B-2) corresponding to the commercial vehicle type. The general-purpose target (80B-2) for commercial vehicles is set to be slightly larger in both width and depth than that for passenger cars. (3) is the icon image (80A-3) representing a motorcycle and the general-purpose target (80B-3) corresponding to the motorcycle type. The size of the general-purpose marker for motorcycles (80B-3) is set to be about half the width of a passenger car, with a depth about the same as the depth of a passenger car, resulting in a vertically elongated, roughly rectangular shape. (4) is an icon image (80A-4) representing a pedestrian and a general-purpose marker (80B-4) corresponding to the type of commercial vehicle. The size of the general-purpose marker for pedestrians (80B-4) is set to be about the same width as a motorcycle, with a depth about half the depth of a motorcycle. In this embodiment, the display control unit 64 controls which size of general-purpose marker to display.

[0030] The display control unit 64 displays a general-purpose target when it cannot identify the type of target detected by the radar device 22. Here, the display control unit 64 estimates the type of target that is most likely to be present around the vehicle and displays a general-purpose target of the size corresponding to the estimated type of target.

[0031] The following are examples of methods for estimating the most likely types of objects present around the display control unit 64: (1) In the first identification method, the display control unit 64 estimates the most likely type from the type of lane the vehicle is in and displays a general-purpose object of the size corresponding to the estimated type. For example, if the lane in which the vehicle is located, as measured by the GNSS device 16, is a general road, the most likely type is a passenger car or a commercial vehicle. If the lane in which the vehicle is located, as measured, is near an intersection, the most likely type is a pedestrian. (2) In the second identification method, the display control unit 64 estimates the most likely type from the frequency information of object types identified in the lane and displays a general-purpose object of the size corresponding to the estimated type. (3) In the third identification method, the display control unit 64 estimates the most likely type from the date and time information and displays a general-purpose object of the size corresponding to the estimated type. The third identification method will be described later in the second embodiment. In this embodiment, either method (1) or (2) is used.

[0032] Figures 4A and 4B illustrate examples of estimating the most likely type of object based on frequency information of object types identified in the driving lane. In both Figures 4A and 4B, the types of three objects in front have been identified, while the type of one object to the side has not been identified. In Figure 4A, a passenger car (80A-1) is identified to the front left, a vehicle in the front center (80A-1), and a commercial vehicle (80A-2) is identified to the front right, with an unidentified object (80n) detected to the side. Since two of the three identifiable objects are passenger cars (80A-1), the most likely type of the unidentified object (80n) is estimated to be a passenger car (80A-1). In Figure 4B, a commercial vehicle (80A-2) is identified to the front left, a vehicle in the front center (80A-1), and a commercial vehicle (80A-2) is identified to the front right, with an unidentified object (80n) detected to the side. Since two out of the three identifiable targets are commercial vehicles (80A-2), the most likely type of the unidentified target (80n) is estimated to be a commercial vehicle (80A-2).

[0033] (Control flow) Figure 5 is a flowchart illustrating the flow of the display control process as a display control method executed by the display control ECU 42 in this embodiment. The display control process is executed repeatedly at a predetermined cycle.

[0034] In step S100, the target detection unit 62 detects the presence of targets in the vicinity of the vehicle based on the detection results of the radar device 22.

[0035] In step S102, the target detection unit 62 identifies the type of target detected from the image captured by the camera device 24 and displays an icon image corresponding to the identified type of target on the display unit.

[0036] In step S104, the target detection unit 62 determines whether or not there are any targets among the detected targets whose type cannot be identified. Targets whose type cannot be identified are those detected outside the target type identification range of the camera device 24 but within the detection range of the radar device 22. If there are targets whose type cannot be identified, the process proceeds to step S106; if there are no targets whose type cannot be identified, the process ends.

[0037] In step S106, the display control unit 64 estimates the types of objects that are most likely to be present in the vicinity of the vehicle.

[0038] In step S108, the display control unit 64 displays a general-purpose target of the size corresponding to the estimated type of target. If the general-purpose target becomes an identifiable target through repeated execution of the display control process, in step S104, the display switches from the general-purpose target to an icon image corresponding to the type of target.

[0039] In summary, the display control ECU 42 of this embodiment can reduce user discomfort when displaying targets whose type cannot be identified.

[0040] [Second Embodiment] The second embodiment is a method for estimating the most likely type of vehicle in a driving lane based on date and time information.

[0041] The processing of the display control unit 64 in the second embodiment will now be described. The display control unit 64 acquires the position of the vehicle determined by the GNSS device 16 and map information from the map information storage unit 20A, and identifies the lane in which the vehicle is traveling by comparing the position of the vehicle with the lanes on the map. For estimating the most likely type, pre-prepared setting data is stored for a specific date and time and lane set. The setting data may be table data that stores the most likely types for a specific date and time and lane set, or it may be model data that has learned the most likely types according to the characteristics of the lane and the date and time. The display control unit 64 uses the setting data to estimate the most likely type corresponding to the lane at the current date and time, and displays a general-purpose marker of the corresponding size. For example, the setting data may be set or learned by distinguishing between "weekdays" and "holidays / weekends" or time periods as specific dates and times. If a lane tends to have many commercial vehicles on "weekdays" or at "nights," then commercial vehicles may be set or learned as the most likely type. If a driving lane tends to have a high concentration of passenger cars on "holidays / weekends" or "daytime," then passenger cars should be set or learned as the most likely vehicle type.

[0042] The processing flow is as follows: the process executed as the display control unit 64 in step S106 of Figure 5 above can be executed as a subroutine. In step S106-A, the position of the vehicle determined by the GNSS device 16 and map information from the map information storage unit 20A are acquired, and the lane in which the vehicle is traveling is identified by comparing the position of the vehicle with the lanes on the map. In step S106-B, the type that is most likely to correspond to the lane being traveled at the current date and time is estimated using the setting data. In step S106-C, a general-purpose marker of the size corresponding to the estimated type is displayed.

[0043] As described above, the display control ECU 42 of the second embodiment can reduce user discomfort by displaying a general-purpose object corresponding to a specific date and time and driving lane when displaying an object whose type cannot be identified.

[0044] Furthermore, in each of the above embodiments, a configuration was described in which a display control program 54 for causing the display control ECU 42 to function as a target detection unit 62 and a display control unit 64 is pre-stored (installed) in the storage 48. However, the display control program 54 can also be provided in a form recorded on a non-temporary recording medium such as an HDD, SSD, or DVD.

[0045] The processing flow described in each of the above embodiments is merely an example, and unnecessary steps may be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main point. [Explanation of Symbols]

[0046] 10 In-vehicle systems 22. Radar equipment (other sensors) 24 Camera equipment (camera) 42 Display Control ECU (Display Control Unit) 62 Target detection unit 64 Display Control Unit

Claims

1. A target detection unit that detects the presence of a target using other sensors on the vehicle besides the camera, and identifies the type of the target from the image captured by the camera on the vehicle, For objects detected outside the identification range of the camera but within the detection range of the other sensors, if the type of object cannot be identified, the display control unit displays a general-purpose object on the display unit, which is set to a lower height than the icon displayed according to the type. A display control device that includes a display control device.

2. The display control device according to claim 1, wherein the display control unit estimates the type of object that is most likely to be present around the vehicle, and displays a general-purpose object of a size corresponding to the estimated object of that type.

3. The display control device according to claim 2, wherein the display control unit estimates the most likely type of object present in the vicinity of the vehicle from frequency information of the types of objects identified in the vicinity of the vehicle.

4. Pre-set the most likely types for a specific date, time, and lane combination. The display control device according to claim 2, wherein the display control unit estimates the most likely type in the driving lane from the set.

5. The target detection unit identifies at least the types of targets: passenger cars, commercial vehicles, motorcycles, and pedestrians. The display control device according to claim 2, wherein the display control unit displays a general-purpose target of any of the sizes of a passenger car, commercial vehicle, motorcycle, or pedestrian, which is estimated to be the most likely type of target.

Images