Information processing device

Abstract

To provide technology for a vehicle to safely avoid an obstacle.SOLUTION: An information processing device comprises a control unit 110 which executes the steps of: acquiring a front image as an image of a road extending in front of a second vehicle 20, which is imaged by a front camera of the second vehicle 20 traveling in front of a first vehicle 10; detecting an obstacle 300 existing on the road from the front image; superposing a graphic of the second vehicle 20 a part of which is transparent with a graphic representing the obstacle 300 so as to be displayed on a head-up display of the first vehicle 10 when the obstacle 300 is detected; and displaying support information to the first vehicle 10 to avoid the obstacle 300 on the head-up display.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present disclosure relates to driving assistance.

Background Art

[0002] Techniques for detecting obstacles at positions that are blind spots from the host vehicle are known. In this regard, for example, Patent Document 1 discloses an in-vehicle device that receives an image captured by a front camera of a preceding vehicle via V2V (Vehicle to Vehicle) communication, detects a falling object or the like at a position that is a blind spot from the host vehicle due to the presence of the preceding vehicle based on the received image, and displays information regarding the falling object or the like.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] An object of the present disclosure is to provide a technique for a vehicle to safely avoid an obstacle.

Means for Solving the Problems

[0005] <管理番号 One aspect of an embodiment of the present disclosure is <管理番号 obtaining a front image, which is an image of a road extending in front of a second vehicle captured by a front camera of the second vehicle traveling in front of a first vehicle, <管理番号 detecting an obstacle existing on the road from the front image, <管理番号 when the obstacle is detected, displaying, on a head-up display of the first vehicle, a graphic of the second vehicle with a part thereof being transparent and a graphic representing the obstacle superimposed thereon, <管理番号 On the head-up display, the first vehicle displays information to help it avoid the obstacle, This is an information processing device having a control unit that performs the following actions.

[0006] Other embodiments include a method performed by the above-mentioned device, a program for causing a computer to perform the method, or a computer-readable storage medium that non-temporarily stores the program. [Effects of the Invention]

[0007] This disclosure provides technology for vehicles to safely avoid obstacles. [Brief explanation of the drawing]

[0008] [Figure 1] A diagram illustrating the components of an information processing device according to an embodiment. [Figure 2] A flowchart of the processes executed by the control unit. [Figure 3] An example of outputting an image generated by the display unit to a HUD (Head-Up Display). [Modes for carrying out the invention]

[0009] An in-vehicle device is known that receives images from a front camera mounted on a vehicle traveling ahead, detects obstacles on the road that are in the blind spot from the vehicle's perspective, and displays information about such obstacles.

[0010] However, conventional in-vehicle devices do not prevent drivers from shifting their gaze away from driving when checking information.

[0011] When an in-vehicle device detects an obstacle or the like, it is preferable to display information about the obstacle or the like on a display device in a manner that enables the driver to ensure safe driving.

[0012] For example, when providing information via a display device of a car navigation system, the driver has to move their line of sight forward and gaze at the display device. Thus, conventionally, there has been a problem that it is difficult to ensure sufficient safety when checking information.

[0013] The information processing apparatus of the present disclosure solves such problems.

[0014] An information processing apparatus according to an aspect of the present disclosure includes: acquiring a front image, which is an image of a road extending in front of a second vehicle captured by a front camera of the second vehicle running in front of a first vehicle; detecting an obstacle existing on the road from the front image; when the obstacle is detected, superimposing and displaying, on a head-up display of the first vehicle, a graphic of the second vehicle with a part thereof being transparent and a graphic representing the obstacle; and displaying, on the head-up display, assistance information for the first vehicle to avoid the obstacle. The information processing apparatus is characterized by having a control unit that executes the above operations.

[0015] According to such a configuration, it is possible to provide information regarding obstacles on the road or the like without causing the driver to move their line of sight, and it is possible to more safely avoid the obstacles.

[0016] (Embodiment) The outline of the information processing apparatus according to the embodiment will be described with reference to FIG. 1.

[0017] The information processing apparatus 100 is a device mounted on a first vehicle 10 and a second vehicle 20. The second vehicle 20 is a vehicle that runs ahead of the first vehicle 10.

[0018] The information processing apparatus 100 mounted on the second vehicle 20 transmits, by wireless communication, an image captured by the front camera of the second vehicle 20 to the information processing apparatus 100 mounted on the first vehicle 10.

[0019] The information processing device 100 mounted on the first vehicle 10 detects obstacles on the road from the acquired images and displays the detected obstacles on the head-up display (HUD) of the first vehicle 10. In addition, the information processing device 100 mounted on the first vehicle 10 also displays route guidance and the like for the first vehicle 10 to avoid obstacles on the HUD of the first vehicle 10.

[0020] Thereby, the driver of the first vehicle 10 can obtain information about the obstacles shielded by the second vehicle 20.

[0021] The information processing device 100 includes a control unit 110, a communication unit 120, a storage unit 130, and a display 140.

[0022] Each element constituting the system will be described in detail.

[0023] First, the control unit 110 will be described.

[0024] The control unit 110 is realized by a processor such as a CPU (Central Processing Unit) or a GPU (Graphics Processing Unit) and a memory. As functional modules, the control unit 110 includes an acquisition unit 111, a detection unit 112, a display unit 113, a determination unit 114, an information transmission unit 115, and an image transmission unit 116. These functional modules may be realized by executing a program by the control unit 110.

[0025] The acquisition unit 111 acquires a front image, which is an image of the road extending in front of the second vehicle 20, captured by the front camera of the second vehicle 20 traveling in front of the first vehicle 10. Here, the front image may be video data or still image data. In addition, the acquisition unit 111 may repeatedly acquire the front image at a predetermined time interval. In the present embodiment, the front image is transmitted from the information processing device 100 mounted on the second vehicle 20.

[0026] The detection unit 112 detects obstacles on the road from the forward image acquired by the acquisition unit 111. Specifically, the detection unit 112 performs image analysis on the forward image to detect the presence of obstacles on the road. In addition, the detection unit 112 calculates the distance from the first vehicle 10 to the obstacle along with the detection of the obstacle. Machine learning, such as deep learning, may be used for the image analysis performed by the detection unit 112.

[0027] When the detection unit 112 detects an obstacle, the display unit 113 generates graphics representing the second vehicle 20 and the obstacle, as well as support information, and displays them on the display 140 of the first vehicle 10. In this embodiment, the support information is information to assist in changing lanes to avoid the obstacle.

[0028] When the detection unit 112 detects an obstacle, the determination unit 114 determines whether the first vehicle 10 should change lanes to avoid the obstacle. If the determination unit 114 determines that the first vehicle 10 should change lanes to avoid the obstacle, the determination unit 114 generates support information for the first vehicle 10 to change lanes.

[0029] Furthermore, if the determination unit 114 determines that the first vehicle 10 should not change lanes to avoid an obstacle, it will turn on the hazard lights of the first vehicle 10 and activate the brakes of the first vehicle 10.

[0030] The information transmission unit 115 notifies the third vehicle, which is the vehicle behind the first vehicle 10, and an external server of the presence of an obstacle detected by the detection unit 112 and the location of the obstacle.

[0031] The image transmission unit 116 acquires a forward image via the front camera of its own vehicle and transmits the forward image to the information processing device 100 installed in the following vehicle. The forward image is received by the information processing device 100 (acquisition unit 111) installed in the following vehicle. The image transmission unit 116 may broadcast the forward image or transmit it to the information processing device 100 installed in the following vehicle via peer-to-peer transmission.

[0032] Next, we will describe the components of the information processing device 100 other than the control unit 110.

[0033] The communication unit 120 is a communication module that communicates with another vehicle equipped with an information processing device 100, which is different from the first vehicle. The communication unit 120 consists of a communication circuit. The communication unit 120 may perform wireless communication using 5G (5th Generation), or IT Wireless communication using a dedicated frequency for S (Intelligent Transport Systems) is also permitted.

[0034] The memory unit 130 is a main memory device such as RAM or ROM, an EPROM, a hard disk drive, and an auxiliary memory device such as removable media. The auxiliary memory device stores the operating system (OS), various programs, various tables, etc., and by executing the programs stored therein, each function that matches the predetermined purpose of each part of the control unit 110 can be realized. However, some or all of the functions may be realized by hardware circuits such as ASICs or FPGAs.

[0035] The storage unit 130 stores data used or generated in processing performed by the control unit 110. The storage unit 130 may also store data received from a vehicle other than the first vehicle 10 (for example, a forward image received from an information processing device 100 mounted on a preceding vehicle, the second vehicle 20).

[0036] The display 140 is a display for showing images (e.g., graphics representing obstacles) and information (e.g., numerical values ​​representing the distance to obstacles, or support information such as lines indicating a route for lane changes) generated by the display unit 113. The display 140 is, for example, a HUD (Head-Up Display). The display 140 may also be the display of a glasses-type wearable device such as AR goggles.

[0037] Note that the configuration shown in Figure 1 is just one example, and all or part of the illustrated functions may be performed using specially designed circuits. Furthermore, program storage and execution may be performed using combinations of main memory and auxiliary memory other than those shown.

[0038] Next, we will explain the specific details of the processing performed by the information processing device 100. Figure 2 is a flowchart of the processing performed by the control unit 110. The illustrated processing is repeatedly performed while the first vehicle 10 is in motion.

[0039] First, in step S11, the acquisition unit 111 acquires an image (forward image) from the front camera of the second vehicle 20, which is the vehicle preceding the first vehicle 10. The acquisition unit 111 performs V2V communication with the information processing device 100 mounted on the second vehicle 20 via the communication unit 120 and acquires the forward image. The acquisition unit 111 may select the information processing device 100 to communicate with via V2V based on the relative positions of the vehicles.

[0040] Next, in step S12, the detection unit 112 detects obstacles present on the road included in the forward image acquired by the acquisition unit 111. The detection unit 112 performs image processing on the forward image and extracts obstacles included in the forward image. The detection unit 112 may also determine the obstacles included in the forward image using deep learning or the like.

[0041] Next, in step S13, the determination unit 114 determines whether or not an obstacle has been detected in the forward image. For example, if the detection unit 112 extracts at least one object other than a moving vehicle on the road in the direction of travel in the forward image, this step results in a positive determination.

[0042] If the result in this step is positive, the process proceeds to step S14.

[0043] If the result in this step is negative, the process proceeds to step S11.

[0044] Next, in step S14, the display unit 113 generates an image in which both a graphic representing the second vehicle 20 and a graphic representing an obstacle are superimposed.

[0045] The graphic corresponding to the second vehicle 20 may be, for example, a graphic representing the outline of the second vehicle 20, and may be transparent inside. By superimposing such a graphic with a graphic corresponding to an obstacle, it is possible to intuitively convey that an obstacle exists beyond the second vehicle 20.

[0046] The display unit 113 displays an image on the display 140 in which both graphics are superimposed. In addition, the display unit 113 may also display on the display 140 a numerical value representing the distance from the first vehicle to the obstacle, and a line connecting the first vehicle to the obstacle.

[0047] Next, in step S15, the determination unit 114 determines whether the first vehicle should change lanes to avoid an obstacle. Specifically, the determination unit 114 determines whether the first vehicle 10 can safely change lanes to the adjacent lane based on the positional relationship between the second vehicle 20 and the first vehicle 10, the positional relationship between the third vehicle, which is traveling behind the first vehicle 10, and the first vehicle 10, as well as the speeds of those vehicles, the positions and speeds of vehicles traveling in adjacent lanes, etc. If the first vehicle 10 can safely change lanes to the adjacent lane, this step results in an affirmative determination.

[0048] If the result in this step is positive, the process proceeds to step S16.

[0049] If the result in this step is negative, the process proceeds to step S17.

[0050] Next, in step S16, the display unit 113 displays route guidance on the display 140. Route guidance is a graphic that guides the vehicle along the route to be taken when changing lanes. In this step, the display unit 113 displays a line on the display 140 superimposed on the road, indicating the route for the first vehicle 10 to change lanes into an adjacent lane. In this example, route guidance for changing lanes is shown as support information, but in this step, information other than the route for changing lanes may also be displayed as support information for avoiding obstacles.

[0051] If the process proceeds to step S17, the determination unit 114 turns on the hazard lights of the first vehicle 10 and activates the brakes of the first vehicle 10. Specifically, the determination unit 114 uses automatic braking to decelerate and stop the first vehicle 10 so that it can stop at the stopping position of the second vehicle 20, in accordance with the speed of the second vehicle 20.

[0052] Next, in step S18, the information transmission unit 115 notifies the third vehicle, which is following the first vehicle 10, and an external server of the location of the obstacle. At this time, the information transmission unit 115 may also transmit a forward image to the third vehicle and the external server.

[0053] Next, an example of an image generated by the display unit 113 will be described. Figure 3 shows an example of when the image generated by the display unit 113 is output to the HUD. By outputting the image to a HUD that allows the view of the scenery in front, information can be provided using AR (augmented reality).

[0054] For example, the display unit 113 displays an image in the HUD that includes a graphic representing the second vehicle 20 and a graphic representing the obstacle 300, superimposed on the driver's field of view. In this case, the second vehicle 20 may be displayed as a graphic with a portion of it being transparent. Specifically, the second vehicle 20 may be displayed as a graphic with parts other than its outline being transparent. The graphic representing the obstacle 300 is then displayed superimposed on the graphic of the second vehicle 20 with a portion of it being transparent.

[0055] The graphic representing the obstacle 300 may be highlighted by a border 200. The display unit 113 may also display the distance from the first vehicle 10 to the obstacle 300 as a numerical value on the HUD.

[0056] Furthermore, if the determination unit 114 determines that the first vehicle 10 is capable of changing lanes to an adjacent lane, the display unit 113 displays support information for changing lanes on the HUD of the first vehicle 10. Specifically, the display unit 113 displays a line 400 representing the lane change route superimposed on the road on the HUD.

[0057] As described above, the information processing device according to this embodiment displays a graphic of the vehicle ahead, partially transparent, and a graphic of an obstacle obscured by the vehicle ahead, superimposed on the HUD, and also displays support information such as routes for avoiding the obstacle. With this configuration, it becomes possible to provide the driver with information about obstacles on the road without requiring them to shift their gaze, and it becomes possible to avoid obstacles more safely.

[0058] Furthermore, a process described as being performed by a single device may be divided and executed by multiple devices. Conversely, a process described as being performed by different devices may be executed by a single device. In a computer system, the hardware configuration (server configuration) used to implement each function can be flexibly changed.

[0059] The present disclosure can also be realized by supplying a computer program implementing the functions described in the embodiments above to a computer, and having one or more processors in the computer read and execute the program. Such a computer program may be provided to the computer by a non-temporary computer-readable storage medium that can be connected to the computer's system bus, or it may be provided to the computer via a network. Non-temporary computer-readable storage mediums include, for example, any type of disk such as magnetic disks (floppy disks, hard disk drives (HDDs), etc.), optical disks (CD-ROMs, DVDs, Blu-ray discs, etc.), read-only memory (ROM), random access memory (RAM), EPROM, EEPROM, magnetic cards, flash memory, optical cards, and any type of medium suitable for storing electronic instructions. [Explanation of Symbols]

[0060] 10. First vehicle 20...Second vehicle 100. Information Processing Device 110... Control Unit 111...Acquisition part 112...Detection unit 113...Display section 114...judgment department 115... Information Transmission Department 116...Image transmission section 120... Communications Department 130...Storage section 140... Display 200... border lines 300... Obstacles 400...line

Claims

[Claim 1] An information processing device mounted on a first vehicle, To acquire a forward image, which is an image of the road extending in front of the second vehicle, captured by the front camera of the second vehicle traveling in front of the first vehicle, From the aforementioned forward image, detect obstacles present on the road, When the aforementioned obstacle is detected, the head-up display of the first vehicle superimposes a graphic of the second vehicle, partially transparent, with a graphic representing the obstacle. Based on the relative positions of the first vehicle and the second vehicle, the relative positions of the first vehicle and the third vehicle traveling behind the first vehicle, the speed of the first vehicle, the speed of the second vehicle, the speed of the third vehicle, the position of a vehicle traveling in the second lane adjacent to the first lane in which the first vehicle is traveling, and the speed of the vehicle traveling in the second lane, it is determined whether the first vehicle should change lanes from the first lane to the second lane in order to avoid the obstacle. When the first vehicle determines that it should change lanes from the first lane to the second lane, On the head-up display, as support information for the first vehicle to avoid the obstacle, lines indicating a route for the first vehicle to change lanes into the second lane are displayed. To notify the third vehicle of the location of the aforementioned obstacle, If it is determined that the first vehicle should not change lanes from the first lane to the second lane, The hazard lights of the first vehicle are turned on, and the brakes of the first vehicle are applied. To notify the third vehicle of the location of the aforementioned obstacle, An information processing device having a control unit that performs the following.

Images