Method

Abstract

To improve a technique for displaying an image to a driver of a vehicle.SOLUTION: A method is a method executed by an in-vehicle device 10 including a display unit 14 provided in a driver's seat of a vehicle and a control unit 11, the method including the control unit 11 detecting a change in at least one of a situation inside the vehicle and a situation outside the vehicle on the basis of at least one of vehicle sensor information, route information, vehicle-to-vehicle communication information, and road-to-vehicle communication information, acquiring a three dimensional image representing the change, determining transparency of the acquired three dimensional image on the basis of a speed of the vehicle, and causing the display unit 14 to display the three dimensional image with the determined transparency.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present disclosure relates to a method. [Background technology]

[0002] BACKGROUND ART Conventionally, there are known techniques for displaying images to a driver of a vehicle. For example, Patent Document 1 discloses a technique for outputting images to both the driver's seat and the rear seats. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2010-245950 Summary of the Invention [Problem to be solved by the invention]

[0004] To view the screen displaying the image, the driver is required to look away from the view ahead while driving the vehicle, which may lead to dangerous driving. Also, if the displayed information is a two-dimensional image, it may be difficult to grasp the situation. As such, there is room for improvement in the technology for displaying images to the vehicle driver. [Means for solving the problem]

[0005] The method disclosed herein is a method executed by an on-board device having a display unit provided in the driver's seat of a vehicle and a control unit, and includes the control unit detecting a change in at least one of the interior and exterior conditions of the vehicle based on at least one of vehicle sensor information, route information, vehicle-to-vehicle communication information, and road-to-vehicle communication information, acquiring a three-dimensional image representing the change, determining the transparency of the acquired three-dimensional image based on the speed of the vehicle, and displaying the three-dimensional image on the display unit at the determined transparency. [Effects of the Invention]

[0006] The present disclosure enables improvements to be made to the technology used to display images to a vehicle driver. [Brief explanation of the drawings]

[0007] [Figure 1] 1 is a diagram showing a schematic configuration of an in-vehicle device according to an embodiment of the present invention; [Figure 2] 4 is a flowchart showing the operation of the in-vehicle device according to the present embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0008] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings. In each drawing, identical or corresponding parts are designated by the same reference numerals. In the description of the present embodiments, the description of identical or corresponding parts will be omitted or simplified as appropriate.

[0009] An overview of an in-vehicle device 10 according to this embodiment will be described with reference to FIG. 1. The in-vehicle device 10 is equipment mounted on a vehicle. The vehicle includes, for example, a passenger car, a small truck, and a large truck. The vehicle is any type of automobile, such as a gasoline vehicle, a diesel vehicle, a hydrogen vehicle, an HEV, a PHEV, a BEV, or an FCEV. "HEV" is an abbreviation for Hybrid Electric Vehicle. "PHEV" is an abbreviation for Plug-in Hybrid Electric Vehicle. "BEV" is an abbreviation for Battery Electric Vehicle. "FCEV" is an abbreviation for Fuel Cell Electric Vehicle. The vehicle is equipped with a control unit, and various information is output from the control unit to the in-vehicle device 10.

[0010] First, an overview of this embodiment will be described, and details will be provided later. The in-vehicle device 10 detects changes in at least one of the interior and exterior conditions of the vehicle based on at least one of vehicle sensor information, route information, vehicle-to-vehicle communication information, and road-to-vehicle communication information. The in-vehicle device 10 acquires a three-dimensional image representing the changes, and determines the transparency of the acquired three-dimensional image based on the vehicle speed. The in-vehicle device 10 displays the three-dimensional image at the determined transparency.

[0011] According to this embodiment, a three-dimensional image showing changes inside or outside the vehicle is displayed in a transparent manner in a predetermined area in front of the driver's seat of the vehicle. The image is displayed in a transparent manner so as not to distract the driver while driving and not to interfere with the driver's driving, allowing the driver to more safely and accurately grasp changes inside and outside the vehicle. This makes it possible to improve the technology for displaying images to the driver of a vehicle.

[0012] The configuration of an in-vehicle device 10 according to this embodiment will be described with reference to Fig. 1. The in-vehicle device 10 includes a control unit 11, a storage unit 12, a communication unit 13, and a display unit 14.

[0013] The control unit 11 includes at least one processor, at least one programmable circuit, at least one dedicated circuit, or any combination thereof. The processor is a general-purpose processor such as a CPU or GPU, or a dedicated processor specialized for a specific process. "CPU" is an abbreviation for central processing unit. "GPU" is an abbreviation for graphics processing unit. An example of the programmable circuit is an FPGA. "FPGA" is an abbreviation for field-programmable gate array. An example of the dedicated circuit is an ASIC. "ASIC" is an abbreviation for application specific integrated circuit. The control unit 11 controls each part of the in-vehicle device 10 and executes processes related to the operation of the in-vehicle device 10.

[0014] The storage unit 12 includes at least one semiconductor memory, at least one magnetic memory, at least one optical memory, or any combination thereof. The semiconductor memory is, for example, a RAM, a ROM, or a flash memory. "RAM" is an abbreviation for random access memory. "ROM" is an abbreviation for read only memory. RAM is, for example, an SRAM or a DRAM. "SRAM" is an abbreviation for static random access memory. "DRAM" is an abbreviation for dynamic random access memory. ROM is, for example, an EEPROM. "EEPROM" is an abbreviation for electrically erasable programmable read only memory. Flash memory is, for example, an SSD. "SSD" is an abbreviation for solid-state drive. Magnetic memory is, for example, an HDD. "HDD" is an abbreviation for hard disk drive. The storage unit 12 functions, for example, as a main storage device, an auxiliary storage device, or a cache memory. The storage unit 12 stores information used for the operation of the in-vehicle device 10 and information obtained by the operation of the in-vehicle device 10.

[0015] The communication unit 13 includes at least one communication module. The communication module is, for example, a module that complies with a wired LAN communication standard such as Ethernet (registered trademark) or a wireless LAN communication standard such as IEEE802.11. "IEEE" is an abbreviation for Institute of Electrical and Electronics Engineers. The communication unit 13 communicates with devices other than the in-vehicle device 10. The communication unit 13 receives information used for the operation of the in-vehicle device 10 via a network, and transmits information obtained by the operation of the in-vehicle device 10.

[0016] The display unit 14 is a partially or completely transparent display. The display is, for example, an LCD or an organic EL display. "LCD" is an abbreviation for liquid crystal display. "EL" is an abbreviation for electroluminescent. The display unit 14 is transparent enough to allow the driver to see what is ahead. The display unit 14 is provided in front of the driver's seat of the vehicle in which the in-vehicle device 10 is installed. A three-dimensional image, described below, is output to a predetermined area of ​​the display unit 14. This allows the driver to see the three-dimensional image that appears in a predetermined area in front of the driver's seat of the vehicle.

[0017] The network may include the Internet, at least one WAN, at least one MAN, or any combination thereof. "WAN" is an abbreviation for wide area network. "MAN" is an abbreviation for metropolitan area network. The network may include at least one wireless network, at least one optical network, or any combination thereof. The wireless network may be, for example, an ad hoc network, a cellular network, a wireless LAN, a satellite communication network, or a terrestrial microwave network. "LAN" is an abbreviation for local area network.

[0018] The functions of the in-vehicle device 10 are realized by executing a program according to this embodiment on a processor serving as the control unit 11. That is, the functions of the in-vehicle device 10 are realized by software. The program causes a computer to execute the operations of the in-vehicle device 10, thereby causing the computer to function as the in-vehicle device 10. That is, the computer functions as the in-vehicle device 10 by executing the operations of the in-vehicle device 10 in accordance with the program.

[0019] The program can be stored on a non-transitory computer-readable medium. Examples of non-transitory computer-readable media include flash memory, magnetic recording devices, optical disks, magneto-optical recording media, and ROMs. The program can be distributed by selling, transferring, or lending portable media such as SD cards, DVDs, or CD-ROMs that store the program. "SD" is an abbreviation for Secure Digital. "DVD" is an abbreviation for digital versatile disc. "CD-ROM" is an abbreviation for compact disc read only memory. The program can also be distributed by storing it in the storage of a server and transferring it from the server to another computer. The program can also be provided as a program product.

[0020] A computer temporarily stores a program stored on a portable medium or transferred from a server in its main storage device. The computer then reads the program stored in the main storage device using a processor and executes processing in accordance with the read program. The computer may also read the program directly from a portable medium and execute processing in accordance with the program. The computer may also execute processing in accordance with the received program each time a program is transferred from a server to the computer. Processing may also be executed through a so-called ASP-type service that achieves its functions by issuing execution instructions and obtaining results without transferring the program from the server to the computer. "ASP" is an abbreviation for application service provider. A program is information used for processing by a computer and includes something equivalent to a program. For example, data that is not a direct instruction to a computer but has properties that specify computer processing falls under the category of "something equivalent to a program."

[0021] Some or all of the functions of the in-vehicle device 10 may be realized by a programmable circuit or a dedicated circuit as the control unit 11. In other words, some or all of the functions of the in-vehicle device 10 may be realized by hardware.

[0022] The operation of the in-vehicle device 10 according to this embodiment will be described with reference to Fig. 2. The operation in Fig. 2 corresponds to the method according to this embodiment. In the following, it is assumed that communication between the in-vehicle device 10 and an external device is performed via the communication unit 13 and a network.

[0023] In S1 of FIG. 2, the control unit 11 of the in-vehicle device 10 acquires at least one of vehicle sensor information, route information, vehicle-to-vehicle communication information, and road-to-vehicle communication information.

[0024] The vehicle sensor information is information output from a sensor unit provided in the vehicle in which the in-vehicle device 10 is installed. The sensor unit of the vehicle includes one or more sensors that detect information about the vehicle. The sensor unit includes any sensors such as an acceleration sensor, an angular velocity sensor, an in-vehicle microphone, an in-vehicle camera, an exterior camera, and a floor sensor that detects an impact to the floor of the vehicle's feet or the floor of the trunk. The in-vehicle camera can capture images of each seat in the vehicle as well as the inside of the trunk. The exterior camera is a camera installed in the vehicle that can capture images outside the vehicle. The sensor unit outputs the detected information to the control unit 11 as vehicle sensor information. For example, the vehicle sensor information includes speed information, acceleration information, angular velocity information, information indicating whether or not an impact has occurred to the floor of the trunk, images captured by the in-vehicle camera and the exterior camera, audio information indicating audio detected by the in-vehicle microphone, and vehicle position information.

[0025] The route information includes a route from the departure point to the destination of the vehicle. The control unit 11 acquires the route information from a navigation device installed in the vehicle. The navigation device may be configured integrally with the in-vehicle device 10. Alternatively, the control unit 11 may acquire the route information from an external server device via the communication unit 13.

[0026] The inter-vehicle communication information is information that the vehicle equipped with the in-vehicle device 10 receives from other vehicles in the vicinity. A control unit of the vehicle outputs the received inter-vehicle communication information to the control unit 11 of the in-vehicle device 10. The inter-vehicle communication information includes the presence or absence of obstacles around the other vehicle detected by the other vehicle, the traveling conditions of the other vehicle such as the speed of the other vehicle, or images captured by a camera of the other vehicle.

[0027] The road-to-vehicle communication information is information that a vehicle equipped with the in-vehicle device 10 receives from a roadside device installed on a road. A control unit of the vehicle outputs the received road-to-vehicle communication information to the control unit 11 of the in-vehicle device 10. The road-to-vehicle communication information includes images captured by a camera of the roadside device, traffic information indicating the presence of congestion or the like on the road, and information indicating the presence of people or other vehicles or the like detected by the roadside device.

[0028] In S2, the control unit 11 detects a change in at least one of the interior and exterior conditions of the vehicle based on at least one of the vehicle sensor information, route information, vehicle-to-vehicle communication information, and road-to-vehicle communication information acquired in S1.

[0029] For example, the control unit 11 detects a change in the situation inside the vehicle when acceleration information as vehicle sensor information indicates that the acceleration is equal to or greater than a predetermined value and information indicating the presence or absence of an impact indicates that an impact has occurred on the floor surface at the foot of the vehicle seat or on the floor surface of the trunk. Specifically, the control unit 11 detects that luggage inside the trunk or on a seat has fallen to the floor when the absolute value of the acceleration is equal to or greater than a predetermined value, i.e., when the vehicle is suddenly accelerating or decelerating. The detection of luggage falling to the floor may be performed by analyzing images captured by an in-vehicle camera. For example, the control unit 11 constantly acquires images of the inside of the vehicle seat or trunk captured by the in-vehicle camera, analyzes the acquired images using any image analysis technology, and detects that luggage has fallen.

[0030] For example, when an image captured by an in-vehicle camera as vehicle sensor information indicates that a child sitting in the back seat is not sitting in a normal position, or when audio information detected by an in-vehicle microphone as vehicle sensor information indicates that a child is crying, the control unit 11 detects that there is a change in the situation inside the vehicle. The control unit 11 may analyze the captured image using any image analysis technology and detect that the child is not sitting in a normal position or that the child's posture is not normal. The control unit 11 may analyze the sound indicated by the audio information using any audio analysis technology and detect the child's crying.

[0031] For example, when the vehicle-to-vehicle communication information or the road-to-vehicle communication information indicates that there is a traffic jam ahead in the traveling direction of the vehicle, the control unit 11 detects that there is a change in the situation outside the vehicle.

[0032] For example, the control unit 11 detects a change in the situation outside the vehicle when the position information and route information as vehicle sensor information indicate that the distance from the current position of the vehicle to the destination is less than a predetermined value. The predetermined value may be set in advance and stored in the storage unit 12.

[0033] For example, when the position information and route information as vehicle sensor information indicate that the vehicle is approaching an intersection such as a T-junction, the control unit 11 detects that there is a change in the situation outside the vehicle.

[0034] In S3, the control unit 11 acquires a three-dimensional image that represents the change detected in S2.

[0035] For example, the control unit 11 acquires a three-dimensional image by processing a two-dimensional image of the inside of the back seat or trunk taken by an in-vehicle camera into a three-dimensional image. By outputting the three-dimensional image in S5 below, the driver can easily grasp changes without having to look into the back seat or get out of the vehicle to check the inside of the trunk.

[0036] For example, the control unit 11 communicates with a roadside device installed on the road on which the vehicle is traveling, and receives road images of the road captured by a camera of the roadside device. The road images are two-dimensional images captured from the sky showing the state of road congestion. The control unit 11 processes the two-dimensional images into three dimensions to obtain three-dimensional images. The three-dimensional images are output in the following step S5, allowing the driver to easily grasp the degree of road congestion.

[0037] For example, the control unit 11 receives a three-dimensional image representing the vehicle's destination indicated by the route information in three dimensions from an external server device. The control unit 11 may also receive a three-dimensional image representing landmark buildings near the destination in three dimensions from the external server device. The control unit 11 may also obtain the three-dimensional image by reading it from the storage unit 12. The three-dimensional image is output in S5 below, allowing the driver to grasp the three-dimensional appearance characteristics of the destination.

[0038] For example, the control unit 11 communicates with a roadside device installed at an intersection and receives an intersection image captured by a camera of the roadside device. The intersection image is a two-dimensional image of pedestrians, vehicles, etc. at the intersection captured from above. The control unit 11 processes the two-dimensional image into a three-dimensional image to obtain a three-dimensional image. By outputting the three-dimensional image in S5 below, the driver can recognize the presence of pedestrians, etc. in a blind spot as viewed from the driver's seat without having to take any action such as leaning forward.

[0039] In S4, the control unit 11 determines the transparency of the three-dimensional image acquired in S3. Any method may be used to determine the transparency. For example, the control unit 11 determines a higher transparency as the speed indicated by the speed information as vehicle sensor information increases. This allows the control unit 11 to determine a transparency that allows a clear view of the road ahead through the display unit 14 and does not interfere with driving when the vehicle is moving. For example, the control unit 11 may determine a low transparency (e.g., 10%) only when the speed is 0, and a high transparency (e.g., 60%) when the speed is greater than 0. This allows the control unit 11 to determine a low transparency only when the vehicle is completely stopped, and a high transparency when the vehicle is moving.

[0040] Without being limited to this, the control unit 11 may determine the transparency depending on the type of the 3D image acquired in S3. For example, the control unit 11 may determine the transparency to be a high value for a 3D image showing a road or an intersection, or a 3D image of a destination building, and may determine the transparency to be a low value for a 3D image showing the inside of a back seat or a trunk.

[0041] In S5, the control unit 11 displays the 3D image acquired in S3 on the display unit 14 with the transparency determined in S4. The transparency of the 3D image allows the driver's vision to be unobstructed, allowing the driver to see the road through the image. Furthermore, the display of the 3D image makes it easier for the user to intuitively grasp changes inside or outside the vehicle compared to a 2D image. Thereafter, the operation of the in-vehicle device 10 ends.

[0042] The present disclosure is not limited to the above-described embodiments. For example, two or more blocks shown in the block diagrams may be integrated, or one block may be divided. Two or more steps shown in the flowcharts may be executed in parallel or in a different order, instead of being executed in chronological order as described, depending on the processing capabilities of the device executing each step, or as needed. Other modifications are possible within the scope of the present disclosure. [Explanation of symbols]

[0043] 10 Onboard equipment 11 Control section 12 Storage section 13 Communications Department 14 Display section

Claims

[Claim 1] A method executed by an in-vehicle device including a display unit provided at a driver's seat of a vehicle and a control unit, The control unit Detecting a change in at least one of an interior situation and an exterior situation of the vehicle based on at least one of vehicle sensor information, route information, vehicle-to-vehicle communication information, and road-to-vehicle communication information; obtaining a three-dimensional image representative of said change; determining a transparency of the acquired three-dimensional image based on a speed of the vehicle; displaying the three-dimensional image on the display unit at the determined transparency; Including, method.

Images