Display control device, display control method, and display control program

Abstract

Provided is a display control device including: a determination unit that determines an object present in the surroundings of a host vehicle by performing image processing on a captured image obtained by capturing the surroundings of the host vehicle; and a display unit that, in a case where the object determined by the determination unit is a person, displays a host vehicle image to which the host vehicle is related on a bird's-eye view image that views the host vehicle from above, and displays a person image corresponding to the person and a shadow image representing a shadow of the person at a position corresponding to the location of the person.

Description

Technical Field

[0001] This disclosure relates to a display control device, a display control method, and a display control program. Background Technology

[0002] Techniques for using captured images to determine the type of an object with higher precision have been disclosed. For example, Japanese Patent Application Publication No. 2017-054311 discloses an object detection device comprising an object detection unit, a temporary determination unit, a speed acquisition unit, and a final determination unit. The object detection unit detects pedestrians or two-wheeled vehicles as objects existing around the vehicle by performing image processing on captured images obtained from images taken in the direction of travel of the vehicle. The temporary determination unit temporarily determines the type of the object detected by the object detection unit by analyzing the captured images. The speed acquisition unit obtains the moving speed of the object using a reflected wave of a carrier wave. The final determination unit uses the moving speed of the object obtained by the speed acquisition unit to make a final determination of the type of the object determined by the temporary determination unit. Summary of the Invention

[0003] However, when displaying the image corresponding to the identified object inside the vehicle, sometimes a top-down view from above the vehicle is used for ease of display. In this case, if the image of the person corresponding to the identified object is displayed exactly as it is on the top-down image at the location corresponding to the person's location, it may appear that the person is lying down on the road.

[0004] This disclosure was made in view of the foregoing points, and its object is to provide a display control device, display control method and display control program that, in an overhead view from above a vehicle, displays the state of a person more easily than displaying the image of a person as is.

[0005] The display control device involved in the first aspect of this disclosure includes: The determination unit determines objects existing around the vehicle by performing image processing on images captured of the area surrounding the vehicle; and The display unit, when the determination unit determines that the object is a person, displays an image of the vehicle involved in the vehicle on an overhead image viewed from above the vehicle, and displays an image of the person corresponding to the person and a shadow image representing the person's shadow at a position corresponding to the location of the person.

[0006] The display control device according to the second aspect of this disclosure is the same as the display control device according to the first aspect. When the object determined by the determination unit is a vehicle, the display unit displays a vehicle image corresponding to the vehicle on the overhead view in an overhead view state at a position corresponding to the location of the vehicle.

[0007] The display control device involved in the third aspect of this disclosure is the same as the display control device involved in the first aspect, and the top-view image is an image related to the operating status of the system that assists in driving the vehicle.

[0008] In the display control method of the fourth aspect of this disclosure, the processor performs the following processing: Objects existing around the vehicle are determined by image processing of images captured around the vehicle; and If the object is determined to be a person, an image of the vehicle involved is displayed on an overhead view from above the vehicle, and the image of the person and the shadow image representing the person's shadow are displayed at a position corresponding to the location of the person.

[0009] The display control program involved in the fifth aspect of this disclosure causes the computer to perform the following processes: Objects existing around the vehicle are determined by image processing of images captured around the vehicle; and If the object is determined to be a person, an image of the vehicle involved is displayed on an overhead view from above the vehicle, and the image of the person and the shadow image representing the person's shadow are displayed at a position corresponding to the location of the person.

[0010] According to this disclosure, a display control device, display control method, and display control program can be provided to display the state of a person in an overhead image viewed from above a vehicle in a way that is easier to understand than displaying the image of a person as is. Attached Figure Description

[0011] The features, advantages, and technical and industrial significance of exemplary embodiments of the present invention will be described with reference to the accompanying drawings, in which the same reference numerals denote the same parts, wherein: Figure 1 This is a diagram illustrating an example of the hardware configuration of a vehicle involved in an implementation of the disclosed technology. Figure 2 This is a block diagram illustrating an example of the functional structure of an ECU. Figure 3 This diagram shows an example of a display unit showing an overhead image on a monitor. Figure 4 This diagram shows an example of a display unit showing an overhead image on a monitor. Figure 5This is a flowchart illustrating the display control processing flow of the ECU. Detailed Implementation

[0012] Hereinafter, an example of an embodiment of the present disclosure will be described with reference to the accompanying drawings. Furthermore, in each drawing, the same or equivalent constituent elements and parts are labeled with the same reference numerals. Additionally, the scale of the drawings is exaggerated for ease of explanation and may sometimes differ from the actual scale.

[0013] Embodiments of this disclosure relate to a display control device for controlling the display of images shown inside a vehicle's passenger compartment. Specifically, it relates to a display control device that determines objects contained in images from cameras mounted on the vehicle, and controls the display of the determined objects on an overhead view taken from above the vehicle.

[0014] When displaying an image corresponding to a identified object inside a vehicle, sometimes a top-down view from above the vehicle is used for ease of display. In this case, if the image of the person corresponding to the identified object is displayed on the top-down image at the location corresponding to the person's location, it may appear as if the person is lying on the road. There are also methods that display only the upper body of the person on the top-down image, but in this case, the person appears to be buried in the road.

[0015] In this embodiment, when displaying such a top-down image, the image of the person corresponding to the object identified as a person is displayed at a predetermined angle. By displaying the image of the person corresponding to the object identified as a person at a predetermined angle, the state of the person can be more easily understood compared to displaying it as is.

[0016] Figure 1 This is a diagram illustrating an example of the hardware configuration of vehicle 20. (For example...) Figure 1 As shown, the vehicle 20 involved in this embodiment is configured to include an ECU 21. The ECU 21 is an example of the display control device of this disclosure. The vehicle 20 is also configured to include a door lock 25, an ignition switch (SW) 26, a monitor 27, a speaker 28, and a camera 29.

[0017] ECU 21 is configured to include a CPU (Central Processing Unit) 21A, a ROM (Read-Only Memory) 21B, a RAM (Random Access Memory) 21C, and an input / output (I / F) 21F. The CPU 21A, ROM 21B, RAM 21C, and I / F 21F are connected to each other via an internal bus 21H in a manner that enables them to communicate with each other.

[0018] CPU 21A is the central processing unit, which executes various programs or controls other parts. That is, CPU 21A reads programs from ROM 21B and uses RAM 21C as its working area to execute programs.

[0019] ROM 21B stores various programs and data. In this embodiment, ROM 21B stores a computer program for controlling ECU 21. In this embodiment, ROM 21B stores a display control program for controlling the display of images to monitor 27.

[0020] RAM 21C is used as a working area to temporarily store programs or data.

[0021] The input / output I / F 21F is an interface for communicating with the door lock 25, ignition device SW 26, monitor 27, and speaker 28 mounted on the vehicle 20.

[0022] Door lock 25 is the key to the door of vehicle 20. In addition to being opened and closed by the user of vehicle 20, it is also controlled by CPU 21A.

[0023] The ignition device SW 26 is a switch that starts the engine of vehicle 20. In addition to being turned on and off by the user of vehicle 20, it is also controlled by CPU 21A to be turned on and off.

[0024] Monitor 27, installed on the dashboard, instrument panel, etc., is an LCD monitor used to display images related to current location, driving path, and attention information. Monitor 27 may also be equipped with a touch panel that also serves as a switch for inputting the occupant's fingers.

[0025] Speaker 28 is installed in the dashboard, center console, front pillar, instrument panel, etc., and is a device for outputting sound.

[0026] Camera 29 captures images of the area around vehicle 20 and outputs the captured images to ECU 21. There can be one or more cameras 29.

[0027] exist Figure 1 In this embodiment, a single ECU 21 controls all devices of the vehicle 20, but the present invention is not limited to this example. For instance, the ECU controlling the door lock 25 and the ECU controlling the ignition device SW 26 may be different ECUs.

[0028] When executing the above-described display control program, ECU 21 uses the aforementioned hardware resources to implement various functions. The functional structure implemented by ECU 21 will be explained below.

[0029] Figure 2This is a block diagram illustrating an example of the functional configuration of ECU 21.

[0030] like Figure 2 As shown, the ECU 21 has a determination unit 101 and a display unit 102 as its functional components. Each functional component is implemented by the CPU 21A reading and executing the display control program stored in the ROM 21B.

[0031] The determination unit 101 determines objects existing around the vehicle 20 by performing image processing on images captured by the camera 29 of the area surrounding the vehicle 20. The timing of acquiring the images from the camera 29 is arbitrary. The objects determined by the determination unit 101 include people, cars, etc. The method by which the determination unit 101 determines objects is arbitrary; for example, it may determine objects existing around the vehicle 20 based on whether the captured images contain the object to be determined.

[0032] Display unit 102 displays an overhead image schematically showing the view from above vehicle 20 on monitor 27. When displaying the overhead image on monitor 27, display unit 102 arranges the image of the object determined by determination unit 101 at a position corresponding to the location of that object on the overhead image. Here, when display unit 102 arranges the object determined by determination unit 101 at a position corresponding to the location of that object on the overhead image, if the object is a person, then the image of the person corresponding to the person is displayed as the image of the object. The image of the person is, for example, an image imitating the shape of a person. Furthermore, when displaying the image of the person on the overhead image, display unit 102 displays the image of the person at a predetermined tilt, and in order to show that the person is standing on the road, a shadow image representing the person's shadow is attached to the image of the person. Furthermore, displaying the image of the person at a predetermined tilt means displaying the image of the person in an overhanging state (appearing to be lying down) on the overhead image.

[0033] If the determination unit 101 determines that the object is not a person, but a vehicle, then the vehicle image corresponding to the vehicle is displayed as the image of the object. Furthermore, when the display unit 102 displays the vehicle image on the overhead view, it displays the overhead view with the vehicle image configured as if the vehicle were being viewed from above.

[0034] Figure 3 This is a diagram showing an example of a display unit 102 displaying an overhead image on a monitor 27. Figure 3The overhead view 30 shown includes a vehicle image 40 showing the vehicle 20, which is the present vehicle, and a person image 50 showing a person. The vehicle image 40 is an example of the vehicle image of the present disclosure. Moreover, the display unit 102 displays the vehicle image 40 in an overhead view state on the overhead view 30, and displays the person image 50 in a tilted state on the overhead view 30. In addition, in order to show that the person is standing on the road, the display unit 102 displays the overhead view 30 by attaching a shadow image 70 to the person image 50, thereby visually showing that the person image 50 is in a tilted state.

[0035] Figure 4 This is a diagram showing an example of a display unit 102 displaying an overhead image on a monitor 27. Figure 4 The overhead view 30 includes a vehicle image 40 showing the vehicle 20, which is the present vehicle, and a vehicle image 60 showing the oncoming vehicle. The display unit 102 displays the vehicle images 40 and 60 in an overhead view on the overhead view 30.

[0036] In this way, when the overhead image is displayed on the monitor 27, the image of the person is displayed at a predetermined angle, and a shadow image representing the person's shadow is attached to the image of the person for display. As a result, the display unit 102 can display the state of the person more easily in the overhead image looking down from above the vehicle compared to displaying the image of the person as is.

[0037] Furthermore, when displaying an overhead view on the monitor 27, the display unit 102 can determine whether to display the image of the person at a predetermined tilt or at a fixed tilt, based on a setting indicating whether the image should be tilted. Additionally, when displaying an overhead view on the monitor 27, the display unit 102 can determine the degree of tilt of the image of the person based on a setting indicating the tilt angle when the image is tilted. Moreover, the display unit 102 can also change the size of the shadow image according to the tilt angle setting.

[0038] Furthermore, when the display unit 102 displays the overhead image on the monitor 27, if the determination unit 101 can also determine the direction of movement of the object identified as a person, it can also add arrows indicating the direction and speed of movement to the person image. By adding arrows indicating the direction and speed of movement to the person image, the driver can intuitively understand the person's actions.

[0039] Furthermore, the display unit 102 can dynamically adjust the tilt of the person image based on the distance and positional relationship between the vehicle 20 (which is the vehicle itself) and the person. Specifically, the display unit 102 can increase the tilt of the person image as the distance between the vehicle 20 (which is the vehicle itself) and the person, and decrease the tilt of the person image as the distance between them. Moreover, the display unit 102 can also change the size of the shadow image based on the tilt amount.

[0040] Furthermore, the display unit 102 can dynamically adjust the size of the person image based on the distance and positional relationship between the vehicle 20 (which is the vehicle itself) and the person. Specifically, the display unit 102 can increase the size of the person image as the vehicle 20 gets closer to the person, and decrease the size of the person image as the vehicle gets farther away. Moreover, the display unit 102 can also change the size of the shadow image based on the size of the person image.

[0041] Next, the function of ECU 21 will be explained.

[0042] Figure 5 This is a flowchart illustrating the display control processing flow of ECU 21. CPU 21A reads and executes the display control program stored in ROM 21B, thereby performing display control processing.

[0043] In step S101, CPU 21A acquires images taken by camera 29 of the area surrounding vehicle 20. The timing of acquiring the images is arbitrary.

[0044] Following S101, in S102, CPU 21A performs image processing on the captured image to determine objects existing around vehicle 20. The objects determined by CPU 21A include people, cars, etc. The method by which CPU 21A determines objects is arbitrary; for example, it can determine objects existing around vehicle 20 based on whether the captured image contains the object being determined.

[0045] Following S102, CPU 21A determines in S103 whether the object identified in S102 is a person or a vehicle.

[0046] As a result of the judgment in S103, if the object determined in S102 is a person, then in S104, the CPU 21A configures a person image corresponding to the person on an overhead image that schematically shows the view from above the vehicle 20, tilted by a predetermined amount. Furthermore, the CPU 21A adds a shadow image to the person image to represent the person standing on the road, thereby displaying the overhead image on the monitor 27.

[0047] As a result of the judgment in S103, if the object determined in S102 is a vehicle, then in S105, the CPU 21A configures the vehicle image corresponding to the vehicle in a top-down view of the vehicle 20 as if it were viewed from above, and displays the top-down image on the monitor 27.

[0048] Thus, when the overhead view is displayed on the monitor 27, by displaying the image of the person at a predetermined angle, the CPU 21A is able to display the state of the person in the overhead view from above the vehicle in a more understandable way compared to displaying the image of the person as is.

[0049] While embodiments of the present disclosure have been described in detail above with reference to the accompanying drawings, the technical scope of the present disclosure is not limited to these examples. The embodiments described above are illustrative and do not limit the technical scope of the present disclosure. Those skilled in the art will readily conceive of various modifications or alterations within the scope of the technical concept set forth in the claims. Furthermore, these modifications or alterations are naturally understood to fall within the technical scope of the present disclosure.

[0050] Furthermore, the effects described in the above embodiments are illustrative or exemplary and are not limited to the effects described in the above embodiments. That is to say, in addition to the effects described in the above embodiments, or in lieu of the effects described in the above embodiments, those skilled in the art can obtain other obvious effects from the descriptions in the above embodiments.

[0051] Furthermore, in the embodiments described above, the display control processing, which involves the CPU reading and executing software (programs), can also be performed by various processors other than the CPU. Examples of processors in this case include FPGAs (Field-Programmable Gate Arrays) which allow for post-manufacturing changes to their circuitry, and ASICs (Application Specific Integrated Circuits) which are dedicated circuits specifically designed for performing particular processing. Moreover, the display control processing can be performed by one of these various processors, or by a combination of two or more processors of the same or different types (e.g., multiple FPGAs or a combination of a CPU and an FPGA). More specifically, the hardware structure of these various processors is a circuit incorporating circuit elements such as semiconductor components.

[0052] Furthermore, while the above embodiments describe the method of pre-storing (installing) the display control processing program in a ROM or memory, this is not a limitation. The program may also be provided on a non-transitory recording medium such as a CD-ROM (Compact Disk Read Only Memory), DVD-ROM (Digital Versatile Disk Read Only Memory), or USB (Universal Serial Bus) memory. Additionally, the program may be downloaded from an external device via a network. Furthermore, this disclosure can also be applied to software products.

Claims

1. A display control device, comprising: The determination unit determines objects existing around the vehicle by performing image processing on images captured of the area surrounding the vehicle; and The display unit, when the determination unit determines that the object is a person, displays an image of the vehicle involved in the vehicle on an overhead image viewed from above the vehicle, and displays an image of the person corresponding to the person and a shadow image representing the person's shadow at a position corresponding to the location of the person.

2. The display control device according to claim 1, wherein, If the determination unit determines that the object is a vehicle, the display unit displays the vehicle image corresponding to the vehicle on the overhead view at the location corresponding to the vehicle's location.

3. The display control device according to claim 1, wherein, The overhead view is an image relating to the operational status of the system that assists in driving the vehicle.

4. A display control method, wherein a processor performs the following processing: Objects existing around the vehicle are determined by image processing of images captured around the vehicle; and If the object is determined to be a person, an image of the vehicle involved is displayed on an overhead view from above the vehicle, and the image of the person and the shadow image representing the person's shadow are displayed at a position corresponding to the location of the person.

5. A display control program that causes a computer to perform the following processes: Objects existing around the vehicle are determined by image processing of images captured around the vehicle; and If the object is determined to be a person, an image of the vehicle involved is displayed on an overhead view from above the vehicle, and the image of the person and the shadow image representing the person's shadow are displayed at a position corresponding to the location of the person.

Images