Display control device, display control method, and display control program

Abstract

The present invention provides a display control device that displays the state of a person more clearly in an overhead view image taken from above a vehicle, compared to displaying the person's image as is. [Solution] A display control device is provided, comprising: a determination unit that determines objects present around the vehicle by processing captured images taken around the vehicle; and a display unit that, if the object determined by the determination unit is a person, places and displays a person image corresponding to the person and a shadow image representing the person's shadow on an overhead view image taken from above the vehicle, together with an image of the vehicle itself, at a position corresponding to the person's location.

Description

Technical Field

[0001] The present disclosure relates to a display control device, a display control method, and a display control program.

Background Art

[0002] Techniques for more accurately determining the type of an object using a captured image have been disclosed. For example, Patent Document 1 discloses an object detection unit that detects a pedestrian or a two-wheeled vehicle as an object existing around the host vehicle by performing image processing on a captured image of the traveling direction of the host vehicle, a provisional determination unit that provisionally determines the type of the object detected by the object detection unit by analyzing the captured image, a speed acquisition unit that acquires the moving speed of the object using the reflected wave of a carrier wave, and a final determination unit that finally determines the type of the object determined by the provisional determination unit using the moving speed of the object acquired by the speed acquisition unit.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] By the way, when displaying an image corresponding to a determined object in a vehicle, in order to simplify the display, it may be displayed as an overhead image viewed from above the vehicle. In this case, if a person image corresponding to an object determined as a person is simply displayed at a position corresponding to the location of the person on the overhead image, there is a risk that the person may appear to be lying on the road.

[0005] This disclosure is made in view of the above points, and aims to provide a display control device, a display control method, and a display control program that display the state of a person more clearly in an overhead view image taken from above a vehicle, compared to displaying the person image as is. [Means for solving the problem]

[0006] A display control device according to a first aspect of the present disclosure includes: a determination unit that determines objects present around the vehicle by processing captured images taken around the vehicle; and a display unit that, if the object determined by the determination unit is a person, displays a person image corresponding to the person and a shadow image representing the person's shadow on an overhead view image taken from above the vehicle, together with an image of the vehicle itself, at a position corresponding to the person's location.

[0007] A display control device according to a second aspect of the present disclosure is a display control device according to a first aspect, wherein, if the object determined by the determination unit is a vehicle, the display unit displays a vehicle image corresponding to the vehicle on the overhead view image in an overhead view state at a position corresponding to the location of the vehicle.

[0008] A display control device according to a third aspect of this disclosure is a display control device according to a first aspect, wherein the overhead image is an image relating to the operating status of a system that assists in the driving operation of the vehicle.

[0009] A display control method according to a fourth aspect of this disclosure involves a processor performing image processing on captured images taken around the vehicle to determine objects present around the vehicle, and if the determined object is a person, it performs a process to place and display a person image corresponding to the person and a shadow image representing the person's shadow on an overhead view image taken from above the vehicle, together with an image of the vehicle itself, at a position corresponding to the person's location.

[0010] The display control program according to the fifth aspect of this disclosure causes a computer to perform image processing on captured images taken around the vehicle to determine objects present around the vehicle, and if the determined object is a person, to place and display an image of the person and a shadow image representing the person on an overhead view image taken from above the vehicle, together with an image of the vehicle itself, at a position corresponding to the location of the person. [Effects of the Invention]

[0011] According to this disclosure, a display control device, a display control method, and a display control program can be provided that display the state of a person more clearly in an overhead view image taken from above a vehicle, compared to displaying the person's image as is. [Brief explanation of the drawing]

[0012] [Figure 1] This figure shows an example of the hardware configuration of a vehicle according to an embodiment of the disclosed technology. [Figure 2] This block shows an example of the ECU's functional configuration. [Figure 3] This figure shows an example of displaying an overhead view image on a monitor using the display unit. [Figure 4] This figure shows an example of displaying an overhead view image on a monitor using the display unit. [Figure 5] This flowchart shows the flow of display control processing by the ECU. [Modes for carrying out the invention]

[0013] Hereinafter, an example of an embodiment of this disclosure will be described with reference to the drawings. In each drawing, identical or equivalent components and parts are given the same reference numerals. Also, the dimensional ratios in the drawings are exaggerated for illustrative purposes and may differ from the actual ratios.

[0014] Embodiments of this disclosure relate to a display control device that controls the display of images shown inside a vehicle. Specifically, the invention relates to a display control device that determines objects contained in an image from an image taken by a camera installed in the vehicle, and controls the display of the determined objects in an overhead view image taken from above the vehicle.

[0015] When displaying images corresponding to detected objects inside a vehicle, the images are sometimes displayed as an overhead view from above the vehicle to simplify the display. In this case, if a person image corresponding to an object detected as a person is displayed on the overhead view at the position corresponding to the person's location, displaying it as is may make it appear as if the person is lying on the road. There is also a method of displaying only the upper body of the person image on the overhead view, but in that case, the person may appear to be embedded in the road.

[0016] In this embodiment, when displaying such an overhead image, the person image corresponding to an object identified as a person is displayed at a predetermined angle. By displaying the person image corresponding to an object identified as a person at a predetermined angle, the state of the person can be displayed more clearly compared to displaying it as is.

[0017] Figure 1 is a diagram showing an example of the hardware configuration of a vehicle 20. As shown in Figure 1, the vehicle 20 according to this embodiment is configured to include an ECU 21. The ECU 21 is an example of a display control device of the present disclosure. The vehicle 20 is further configured to include a door lock 25, an ignition switch (SW) 26, a monitor 27, a speaker 28, and a camera 29.

[0018] The ECU21 consists of a CPU (Central Processing Unit) 21A, a ROM (Read Only Memory) 21B, a RAM (Random Access Memory) 21C, and an I / F 21F. The CPU 21A, ROM 21B, RAM 21C, and I / F 21F are interconnected via an internal bus 21H so that they can communicate with each other.

[0019] The CPU 21A is a central processing unit that executes various programs and controls each part. That is, the CPU 21A reads a program from the ROM 21B and executes the program using the RAM 21C as a work area.

[0020] The ROM 21B stores various programs and various data. In the ROM 21B of the present embodiment, a computer program for controlling the ECU 21 is stored. In the present embodiment, a display control program for controlling the display of an image on the monitor 27 is stored in the ROM 21B.

[0021] The RAM 21C temporarily stores a program or data as a work area.

[0022] The input / output I / F 21F is an interface for communicating with the door lock 25, ignition SW 26, monitor 27, and speaker 28 mounted on the vehicle 20.

[0023] The door lock 25 is a key for the door of the vehicle 20. It is opened and closed by a user getting into the vehicle 20 and is also controlled to be opened and closed by the CPU 21A.

[0024] The ignition SW 26 is a switch for starting the engine of the vehicle 20. It is turned on and off by a user getting into the vehicle 20 and is also controlled to be turned on and off by the CPU 21A.

[0025] The monitor 27 is a liquid crystal monitor provided on the instrument panel, meter panel, etc. and displays images related to the current location, driving route, and caution information. The monitor 27 may be provided with a touch panel that also serves as a switch for inputting operations by the fingers of the passengers.

[0026] The speaker 28 is a device provided on the instrument panel, center console, front pillar, dashboard, etc. and outputs sound.

[0027] Camera 29 captures images of the area around the vehicle 20 and outputs the captured images to the ECU 21. There may be one or more cameras 29.

[0028] In Figure 1, a single ECU 21 controls each device of the vehicle 20, but the present invention is not limited to this example. For example, the ECU that controls the door lock 25 and the ECU that controls the ignition switch 26 may be different ECUs.

[0029] When executing the above display control program, the ECU21 uses the above hardware resources to implement various functions. The functional configuration implemented by the ECU21 will be described below.

[0030] Figure 2 is a block diagram showing an example of the functional configuration of ECU21.

[0031] As shown in Figure 2, the ECU 21 has a determination unit 101 and a display unit 102 as its functional configuration. Each functional configuration is realized when the CPU 21A reads and executes a display control program stored in the ROM 21B.

[0032] The determination unit 101 determines objects present around the vehicle 20 by processing the images captured by the camera 29. The timing of acquiring the images from the camera 29 is arbitrary. The objects that the determination unit 101 determines include people, automobiles, etc. The method used by the determination unit 101 to determine objects is arbitrary, but for example, it determines objects present around the vehicle 20 by checking whether the captured image contains the object to be determined.

[0033] The display unit 102 displays an overhead image on the monitor 27 that schematically shows the vehicle 20 as viewed from above. When the display unit 102 displays the overhead image on the monitor 27, it places and displays the image of the object determined by the determination unit 101 on the overhead image at the position corresponding to the location of the object. Here, when the display unit 102 places and displays the object determined by the determination unit 101 on the overhead image at the position corresponding to the location of the object, if the object is a person, it displays a person image corresponding to the person as the image of the object. The person image is, for example, an image that mimics the shape of a human being. When the display unit 102 displays the person image on the overhead image, it displays the person image tilted by a predetermined amount, and also displays a shadow image representing the person's shadow to indicate that the person is standing on the road. Note that displaying the person image tilted by a predetermined amount means displaying the person image on the overhead image in a tilted state from its original state (which appears as if lying down).

[0034] If the object determined by the determination unit 101 is not a person, for example, a vehicle, then the vehicle image corresponding to the vehicle is displayed as the object image. When the display unit 102 displays the vehicle image on the overhead view image, it positions the vehicle image as if it were viewed from above and displays the overhead view image.

[0035] Figure 3 shows an example of the display unit 102 displaying an overhead view image on the monitor 27. The overhead view image 30 shown in Figure 3 includes a vehicle image 40 representing the vehicle 20, which is the vehicle itself, and a person image 50 representing a person. The vehicle image 40 is an example of the vehicle image of the vehicle itself as disclosed herein. The display unit 102 displays the vehicle image 40 on the overhead view image 30 in an overhead view, and also displays the person image 50 on the overhead view image 30 at an angle. Furthermore, to indicate that the person is standing on the road, the display unit 102 displays the person image 50 with a shadow image 60 attached to it, visually indicating that the person image 50 is at an angle.

[0036] Figure 4 shows an example of the display unit 102 displaying an overhead view image on the monitor 27. The overhead view image 30 shown in Figure 3 includes a vehicle image 40 representing the vehicle 20, which is the vehicle itself, and a vehicle image 60 representing an oncoming vehicle. The display unit 102 displays the vehicle image 40 and the vehicle image 60 on the overhead view image 30 in an overhead view.

[0037] In this way, when displaying the overhead view image on the monitor 27, the person image is displayed at a predetermined angle, and a shadow image representing the person's shadow is added to the person image. As a result, the display unit 102 can display the person's state more clearly in the overhead view image taken from above the vehicle, compared to when the person image is displayed as is.

[0038] Furthermore, when displaying the overhead image on the monitor 27, the display unit 102 may decide whether to display the person image tilted by a predetermined amount or not, based on the setting for whether or not to display the person image tilted. Also, when displaying the overhead image on the monitor 27, the display unit 102 may decide the degree of tilt of the person image based on the setting for the tilt angle when displaying the person image tilted. The display unit 102 may also change the size of the shadow image according to the setting for the tilt angle.

[0039] Furthermore, when the display unit 102 displays the overhead image on the monitor 27, if the determination unit 101 has also determined the direction of movement of an object identified as a person, it may add arrows or effects indicating the direction and speed of movement to the person image. By adding arrows or effects indicating the direction and speed of movement to the person image, the driver can intuitively understand the person's movement.

[0040] Furthermore, the display unit 102 may dynamically adjust the amount of tilt of the person image according to the distance and positional relationship between the vehicle 20 (the vehicle itself) and the person. Specifically, the display unit 102 may increase the amount of tilt of the person image as the distance between the vehicle 20 and the person decreases as the distance increases. The display unit 102 may also change the size of the shadow image according to the amount of tilt.

[0041] Furthermore, the display unit 102 may dynamically adjust the size of the person image according to the distance and positional relationship between the vehicle 20 (the vehicle itself) and the person. Specifically, the display unit 102 may enlarge the person image as the distance between the vehicle 20 and the person decreases, and decrease the person image as the distance increases. The display unit 102 may also change the size of the shadow image according to the size of the person image.

[0042] Next, I will explain the function of ECU21.

[0043] Figure 5 is a flowchart showing the flow of display control processing by the ECU21. The CPU21A reads the display control program stored in the ROM21B and executes it, thereby performing the display control processing.

[0044] In step S101, the CPU 21A acquires the image captured by the camera 29 around the vehicle 20. The timing of image acquisition is arbitrary.

[0045] Following step S101, in step S102, the CPU 21A performs image processing on the captured image to determine objects present around the vehicle 20. The objects that the CPU 21A determines include people, automobiles, etc. The method used by the CPU 21A to determine objects is arbitrary, but for example, it can determine objects present around the vehicle 20 by checking whether the captured image contains the object to be determined.

[0046] Following step S102, in step S103, the CPU 21A determines whether the object determined in step S102 is a person or a vehicle.

[0047] If, as a result of the determination in step S103, the object determined in step S102 is a person, then in step S104, the CPU 21A places a person image corresponding to the person on an overhead image schematically showing a view of the vehicle 20 from above, tilted by a predetermined amount, and adds a shadow image to the person image to indicate that the person is standing on the road, and displays the overhead image on the monitor 27.

[0048] If, as a result of the determination in step S103, the object determined in step S102 is a vehicle, then in step S105, the CPU 21A places a vehicle image corresponding to the vehicle on an overhead view image schematically showing the vehicle 20 as viewed from above, and displays the overhead view image on the monitor 27.

[0049] In this way, when displaying the overhead view image on the monitor 27, the CPU 21A can display the state of the person more clearly in the overhead view image taken from above the vehicle, compared to when the person image is displayed as is.

[0050] While embodiments of the present disclosure have been described in detail above with reference to the attached drawings, the technical scope of the present disclosure is not limited to these examples. The embodiments described above are illustrative and do not limit the technical scope of the present disclosure. It is clear to any person with ordinary skill in the art of the present disclosure that various modifications or alterations may occur within the scope of the technical idea set forth in the claims, and these modifications or alterations will naturally also fall within the technical scope of the present disclosure.

[0051] Furthermore, the effects described in the above embodiments are descriptive or illustrative, and are not limited to those described in the above embodiments. In other words, the technology relating to this disclosure may produce other effects that would be obvious to a person of ordinary skill in the art of this disclosure from the descriptions in the above embodiments, in addition to or in lieu of the effects described in the above embodiments.

[0052] In addition, the display control processing that the CPU reads and executes in each of the above embodiments may be executed by various processors other than the CPU. Examples of such processors include PLDs (Programmable Logic Devices) such as FPGAs (Field-Programmable Gate Arrays) whose circuit configuration can be changed after manufacturing, and dedicated electrical circuits that are processors with circuit configurations specifically designed to execute specific processing, such as ASICs (Application Specific Integrated Circuits). Furthermore, the display control processing may be executed by one of these various processors, or by a combination of two or more processors of the same or different types (for example, multiple FPGAs, and a combination of a CPU and an FPGA). More specifically, the hardware structure of these various processors is an electrical circuit that combines circuit elements such as semiconductor elements.

[0053] Furthermore, while the above embodiments describe a configuration in which the display control processing program is pre-stored (installed) in ROM or storage, the invention is not limited thereto. The program may be provided in a form recorded on a non-transitory recording medium such as a CD-ROM (Compact Disk Read Only Memory), DVD-ROM (Digital Versatile Disk Read Only Memory), or USB (Universal Serial Bus) memory. The program may also be provided in a form that can be downloaded from an external device via a network. Moreover, this disclosure may also be applied to program products. [Explanation of symbols]

[0054] 20 vehicles 21 ECU 21A CPU 21B ROM 21C RAM 21F Input / output I / F 25 Door lock 26. Ignition switch (SW) 27 monitors 28 speakers 29 Cameras

Claims

1. A determination unit that determines objects present around the vehicle by processing captured images of the area surrounding the vehicle, If the object determined by the determination unit is a person, the display unit displays a person image corresponding to the person and a shadow image representing the person's shadow, placed on an overhead view image taken from above the vehicle, together with an image of the vehicle itself, at a position corresponding to the person's location. A display control device comprising:

2. The display control device according to claim 1, wherein, if the object determined by the determination unit is a vehicle, the display unit displays a vehicle image corresponding to the vehicle on the overhead view image in an overhead view state at a position corresponding to the location of the vehicle.

3. The display control device according to claim 1, wherein the overhead view image is an image relating to the operating status of a system that assists in the driving operation of the vehicle.

4. The processor, By processing the captured images of the area around the vehicle, objects present in the area around the vehicle are determined. If the identified object is a person, the person's image and the shadow image representing the person are displayed on the overhead view image, taken from above the vehicle, along with the vehicle's image, at a position corresponding to the person's location. A method for executing processing and controlling the display.

5. On the computer, By processing the captured images of the area around the vehicle, objects present in the area around the vehicle are determined. If the identified object is a person, the person's image and the shadow image representing the person are displayed on the overhead view image, taken from above the vehicle, along with the vehicle's image, at a position corresponding to the person's location. A display control program that executes a process.

Images