Driving assistance device, vehicle, and recording medium having computer program recorded thereon

Abstract

The present invention detects an object existing in a blind area and enables determination of collision between the object and a subject vehicle. This driving assistance device for assisting in driving so as to avoid collision with an object is configured to: detect the shadow of an object existing in a blind area from a vehicle on the basis of image data obtained by photographing the surrounding area of the vehicle; estimate the position of a light source forming the shadow; and calculate the position, movement direction, and movement speed of the object on the basis of a temporal change in shadow and position of the light source, and determine the possibility of collision between the subject vehicle and the object.

Description

[Technical field]

[0001] The present disclosure relates to a driving assistance device that assists in driving a vehicle so as to avoid a collision with an object that appears from a blind spot, a vehicle, and a recording medium having a computer program recorded thereon. [Background technology]

[0002] In recent years, vehicles equipped with driving assistance functions and automatic driving functions have been put into practical use, mainly for the purpose of reducing traffic accidents and reducing driving load. For example, a device is known that detects objects around the vehicle based on information detected by various sensors such as an exterior camera and LiDAR (Light Detection and Ranging) installed in the vehicle, and assists the driving of the vehicle to avoid collisions between the vehicle and the objects. However, some traffic accidents are difficult to avoid unless preliminary actions such as deceleration are taken in anticipation of an accident, such as a sudden jump from a blind spot.

[0003] In response to this, for example, Patent Document 1 proposes a vehicle control device that can appropriately control the vehicle depending on the situation when the host vehicle is traveling in a situation with a blind spot. Specifically, Patent Document 1 discloses a vehicle control device that detects a blind spot area that is a blind spot for the host vehicle, determines the relative priority between the path of a moving object that may appear from the blind spot area and the path of the host vehicle, and outputs a control signal for the host vehicle based on the determined priority. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2016-122308 A Summary of the Invention [Problem to be solved by the invention]

[0005] However, the driving support device disclosed in Patent Document 1 does not actually detect objects jumping out from the blind spot area, so even if there is no object actually jumping out when passing through the blind spot area, control such as deceleration is executed. If deceleration is repeatedly performed even when there is no object jumping out, the trust and acceptability of the driving support device will decrease, and in some cases, the driving support function may not be utilized.

[0006] The present disclosure has been made in consideration of the above problems, and an object of the present disclosure is to provide a driving assistance device and vehicle, as well as a recording medium having a computer program recorded thereon, that are capable of detecting an object present in a blind spot area and determining whether the object has collided with the vehicle. [Means for solving the problem]

[0007] In order to solve the above problems, according to one aspect of the present disclosure, a driving assistance device that assists driving to avoid a collision with an object is provided, which includes one or more processors and one or more memories communicatively connected to the one or more processors, wherein the processor detects the shadow of an object present in a blind spot area as seen from the vehicle based on image data captured around the vehicle, estimates the position of a light source that forms the shadow, calculates the position, direction of movement, and speed of movement of the object based on the position of the light source and changes in the shadow over time, and determines the possibility of a collision between the vehicle and the object.

[0008] Furthermore, in order to solve the above problem, according to another aspect of the present disclosure, there is provided a driving assistance device that assists driving to avoid a collision with an object, the driving assistance device including a shadow area detection unit that detects the shadow of an object present in a blind spot area as seen from the vehicle based on image data captured around the vehicle, a light source position estimation unit that estimates the position of a light source that forms the shadow, and a collision determination unit that calculates the position, movement direction, and movement speed of the object based on the position of the light source and changes in the shadow over time, and determines the possibility of a collision between the vehicle and the object.

[0009] In addition, in order to solve the above problem, according to another aspect of the present disclosure, in a vehicle equipped with a driving assistance device that assists driving to avoid a collision with an object, the driving assistance device includes one or more processors and one or more memories communicatively connected to the one or more processors, and the processor detects the shadow of an object present in a blind spot area as seen from the vehicle based on image data captured around the vehicle, estimates the position of a light source that forms the shadow, calculates the position, direction of movement and speed of the object based on the position of the light source and changes in the shadow over time, and determines the possibility of a collision between the vehicle and the object.

[0010] In addition, in order to solve the above problem, according to another aspect of the present disclosure, there is provided a recording medium having recorded thereon a computer program applied to a driving assistance device that assists in driving to avoid a collision with an object, the computer program causing one or more processors to execute processing including detecting the shadow of an object present in a blind spot area as seen from the vehicle based on image data captured around the vehicle, estimating the position of a light source that forms the shadow, calculating the position, direction of movement and speed of the object based on the position of the light source and changes in the shadow over time, and determining the possibility of a collision between the vehicle and the object. Effect of the Invention

[0011] As described above, according to the present disclosure, it is possible to detect an object present in a blind spot area and determine whether or not there is a collision between the object and the vehicle. [Brief description of the drawings]

[0012] [Figure 1] 1 is a schematic diagram illustrating a configuration example of a vehicle equipped with a driving assistance device according to an embodiment of the present disclosure. [Diagram 2] 1 is a block diagram showing a configuration example of a driving assistance device according to a first embodiment. [Diagram 3] 4 is a flowchart showing a processing operation of the driving assistance device according to the embodiment. [Figure 4] 4 is a flowchart showing a processing operation of the driving assistance device according to the embodiment. [Diagram 5] 4 is a flowchart showing a light source position estimation process performed by the driving assistance device of the embodiment. [Figure 6] FIG. 11 is a diagram showing a driving scene for explaining an example in which the light source is the sun. [Figure 7] FIG. 7 is a diagram showing the driving scene in FIG. 6 as viewed from the traveling direction of the host vehicle. [Figure 8] FIG. 7 is a diagram showing the driving scene in FIG. 6 as viewed from the traveling direction of the host vehicle. [Figure 9] FIG. 11 is a diagram showing a driving scene for explaining an example in which a light source is a street lamp. [Figure 10] 10 is a diagram showing the driving scene in FIG. 9 as viewed from the traveling direction of the host vehicle. [Figure 11] 10 is a diagram showing the driving scene in FIG. 9 as viewed from the traveling direction of the host vehicle. [Figure 12] FIG. 11 is a diagram showing a driving scene for explaining an example in which the light source is a light of a moving body. [Figure 13] 13 is a diagram showing the driving scene in FIG. 12 as viewed from the traveling direction of the host vehicle. [Figure 14] 13 is a diagram showing the driving scene in FIG. 12 as viewed from the traveling direction of the host vehicle. [Figure 15] 10 is a flowchart showing a processing operation of a driving assistance device according to a second embodiment. [Figure 16] 4 is a flowchart showing a processing operation of the driving assistance device according to the embodiment. [Figure 17] 1A to 1C are diagrams illustrating driving scenes for explaining a method for estimating a light source position according to the embodiment; [Figure 18] FIG. 18 is a diagram showing the driving scene in FIG. 17 as viewed from the traveling direction of the host vehicle. [Figure 19] FIG. 18 is a diagram showing the driving scene in FIG. 17 as viewed from the traveling direction of the host vehicle. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0013] Hereinafter, preferred embodiments of the present disclosure will be described in detail with reference to the accompanying drawings. In this specification and the drawings, components having substantially the same functional configurations are denoted by the same reference numerals, and redundant description will be omitted.

[0014] <<1. First embodiment>> <1-1. Overall configuration of the vehicle> First, an example of the overall configuration of a vehicle to which a driving assistance device according to an embodiment of the present disclosure can be applied will be described.

[0015] Fig. 1 is a schematic diagram showing an example of the configuration of a vehicle 1 equipped with a driving support device 50 according to this embodiment. The vehicle 1 shown in Fig. 1 is configured as a four-wheel drive vehicle in which driving torque output from a driving force source 9 that generates driving torque for the vehicle 1 is transmitted to a left front wheel 3LF, a right front wheel 3RF, a left rear wheel 3LR, and a right rear wheel 3RR (hereinafter collectively referred to as "wheels 3" unless a distinction is particularly required). The driving force source 9 may be an internal combustion engine such as a gasoline engine or a diesel engine, may be a driving motor, or may include both an internal combustion engine and a driving motor.

[0016] The vehicle 1 may be an electric vehicle equipped with two drive motors, for example a front-wheel drive motor and a rear-wheel drive motor, or an electric vehicle equipped with drive motors corresponding to the respective wheels 3. If the vehicle 1 is an electric vehicle or a hybrid electric vehicle, the vehicle 1 is equipped with a secondary battery that stores power supplied to the drive motors, and a generator such as a motor or fuel cell that generates power to be charged into the battery. Furthermore, the vehicle 1 may be a two-wheel drive four-wheel vehicle, or may be another type of vehicle, such as a two-wheel vehicle.

[0017] The vehicle 1 is equipped with a driving force source 9, an electric steering device 15, and a brake fluid pressure control unit 20 as devices used for driving control of the vehicle 1. The driving force source 9 outputs driving torque that is transmitted to the front wheel drive shaft 5F and the rear wheel drive shaft 5R via a transmission, a front wheel differential mechanism 7F, and a rear wheel differential mechanism 7R (not shown). The driving of the driving force source 9 and the transmission is controlled by a vehicle control device 41 that includes one or more electronic control devices (ECU: Electronic Control Unit).

[0018] The front-wheel drive shaft 5F is provided with an electric steering device 15. The electric steering device 15 includes an electric motor and a gear mechanism (not shown). The electric steering device 15 adjusts the steering angles of the left front wheel 3LF and the right front wheel 3RF by being controlled by a vehicle control device 41. During manual driving, the vehicle control device 41 controls the electric steering device 15 based on the steering angle of the steering wheel 13 by the driver. During automatic driving, the vehicle control device 41 controls the electric steering device 15 based on a target steering angle set by the driving assistance device 50 or an automatic driving control device (not shown).

[0019] The brake system of the vehicle 1 is configured as a hydraulic brake system. A brake fluid pressure control unit 20 adjusts the hydraulic pressure supplied to brake calipers 17LF, 17RF, 17LR, and 17RR (hereinafter collectively referred to as "brake calipers 17" unless a distinction is required) provided on the front, rear, left, and right drive wheels 3LF, 3RF, 3LR, and 3RR, respectively, to generate braking force. The drive of the brake fluid pressure control unit 20 is controlled by a vehicle control device 41. When the vehicle 1 is an electric vehicle or a hybrid electric vehicle, the brake fluid pressure control unit 20 is used in conjunction with regenerative braking by a drive motor.

[0020] The vehicle control device 41 includes one or more electronic control devices that control the driving of the driving force source 9 that outputs the driving torque of the vehicle 1, the electric steering device 15 that controls the steering angle of the steering wheel 13 or the steering wheels, and the brake fluid pressure control unit 20 that controls the braking force of the vehicle 1. The vehicle control device 41 may have a function of controlling the driving of a transmission that changes the speed of the output output from the driving force source 9 and transmits it to the wheels 3. The vehicle control device 41 is configured to be able to acquire information transmitted from the driving assistance device 50 or an automatic driving control device not shown, and is configured to be able to execute automatic driving control of the vehicle 1. In addition, during manual driving of the vehicle 1, the vehicle control device 41 acquires information on the amount of operation by the driver's driving, and controls the driving of the driving force source 9 that outputs the driving torque of the vehicle 1, the electric steering device 15 that controls the steering angle of the steering wheel 13 or the steering wheels, and the brake fluid pressure control unit 20 that controls the braking force of the vehicle 1.

[0021] The vehicle 1 also includes forward photographing cameras 31LF, 31RF, a vehicle state sensor 35, and a Global Navigation Satellite System (GNSS) sensor 37. The forward photographing cameras 31LF, 31RF constitute an ambient environment sensor 31 for acquiring information on the ambient environment of the vehicle 1. The forward photographing cameras 31LF, 31RF photograph the area in front of the vehicle 1 and generate image data. The forward photographing cameras 31LF, 31RF include imaging elements such as Charged-Coupled Devices (CCD) or Complementary Metal-Oxide-Semiconductor (CMOS), and transmit the generated image data to the driving assistance device 50.

[0022] 1, the front photographing cameras 31LF, 31RF are configured as stereo cameras including a pair of left and right cameras, but may be monocular cameras. In addition to the front photographing cameras 31LF, 31RF, the vehicle 1 may be equipped with, for example, a rear photographing camera provided at the rear of the vehicle 1 to photograph the rearward view, or a camera provided in a side mirror to photograph the left rear or right rear view.

[0023] The vehicle 1 may be equipped with, in addition to the forward-facing cameras 31LF, 31RF, one or more sensors selected from the group consisting of LiADR, radar sensors such as millimeter-wave radar, and ultrasonic sensors, as surrounding environment sensors for acquiring information about the surrounding environment.

[0024] The vehicle state sensor 35 is composed of one or more sensors that detect the operation state and behavior of the vehicle 1. The vehicle state sensor 35 includes at least one of, for example, a steering angle sensor, an accelerator position sensor, a brake stroke sensor, a brake pressure sensor, or an engine speed sensor. These sensors detect the operation state of the vehicle 1, such as the steering angle of the steering wheel 13 or the steering wheels, the accelerator opening, the brake operation amount, or the engine speed. The vehicle state sensor 35 also includes at least one of, for example, a vehicle speed sensor, an acceleration sensor, or an angular velocity sensor. These sensors detect the behavior of the vehicle, such as the vehicle speed, the longitudinal acceleration, the lateral acceleration, and the yaw rate. The vehicle state sensor 35 transmits a sensor signal including the detected information to the driving assistance device 50.

[0025] The GNSS sensor 37 receives satellite signals transmitted from satellites such as GPS (Global Positioning System) satellites. The GNSS sensor 37 transmits position information of the host vehicle 1 on the world coordinate system contained in the received satellite signals to the driving assistance device 50. The GNSS sensor 37 may include an antenna for receiving satellite signals from other satellite systems that identify the position of the host vehicle 1, in addition to GPS satellites.

[0026] The notification device 43 presents various information to the driver by means of image display, audio output, etc., based on the information transmitted from the driving assistance device 50. The notification device 43 includes, for example, a display device provided in an instrument panel and a speaker provided in the vehicle. The display device may be a display device of a navigation system. The notification device 43 may also include a HUD (head-up display) that displays information on the front window by superimposing it on the scenery around the vehicle 1.

[0027] <1-2. Driving support devices> Next, the driving support device 50 according to this embodiment will be specifically described. Note that, hereinafter, the vehicle 1 equipped with the driving support device 50 will be referred to as the host vehicle.

[0028] (1-2-1. Overall composition) FIG. 2 is a block diagram showing an example of the configuration of the driving support device 50 according to this embodiment. The surrounding environment sensor 31, the vehicle state sensor 35, and the GNSS sensor 37 are connected to the driving assistance device 50 via a dedicated line or a communication means such as a controller area network (CAN) or a local inter net (LIN). The vehicle control device 41 and the notification device 43 are also connected to the driving assistance device 50 via a dedicated line or a communication means such as a CAN or a LIN. The driving assistance device 50 is not limited to an electronic control device mounted on the vehicle 1, and may be a terminal device such as a smartphone or a wearable device.

[0029] The driving assistance device 50 functions as a device that assists in driving the vehicle 1 by executing a computer program by a processor such as one or more CPUs (Central Processing Units). The computer program is a computer program for causing the processor to execute operations to be performed by the driving assistance device 50, which will be described later. The computer program executed by the processor may be recorded on a recording medium that functions as a storage unit (memory) 53 provided in the driving assistance device 50, or may be recorded on a recording medium built into the driving assistance device 50 or any recording medium that can be externally attached to the driving assistance device 50.

[0030] Examples of recording media for recording computer programs include magnetic media such as hard disks, floppy disks, and magnetic tapes; optical recording media such as CD-ROMs (Compact Disk Read Only Memory), DVDs (Digital Versatile Disks), and Blu-ray (registered trademark); magneto-optical media such as floptical disks; memory elements such as RAMs (Random Access Memory) and ROMs (Read Only Memory); and flash memories such as USB (Universal Serial Bus) memories and SSDs (Solid State Drives), as well as other media capable of storing programs.

[0031] The driving support device 50 includes a processing unit 51 and a storage unit 53. The processing unit 51 is configured with one or more processors. A part or all of the processing unit 51 may be configured with an updatable firmware or the like, or may be a program module or the like executed by a command from a CPU or the like. The storage unit 53 is configured with a memory such as a RAM or a ROM, and is communicatively connected to the processing unit 51. However, the number and type of the storage units 53 are not particularly limited. The storage unit 53 stores information such as computer programs executed by the processing unit 51, various parameters used in the calculation process, detection data, and calculation results.

[0032] The driving assistance device 50 also includes a high-precision map data storage unit 55 and a solar position data storage unit 57. The high-precision map data storage unit 55 and the solar position data storage unit 57 are each composed of a recording medium communicably connected to the processing unit 51. The high-precision map data storage unit 55 stores map data including position information and three-dimensional shape data of roads and structures outside the roads. The high-precision map data includes position information and three-dimensional shape data of structures such as buildings, side walls, hedges, bridges, etc., or installations such as traffic signals and street lights. In particular, the high-precision map data includes data on the height of the light source of the street light. The solar position data storage unit 57 also stores the altitude and direction of the sun. The solar position data includes data on the altitude and direction of the sun according to the time, season, and region.

[0033] Any recording medium may be used as the recording medium constituting the high-precision map data storage unit 55 and the solar position data storage unit 57. In addition, the high-precision map data storage unit 55 and the solar position data storage unit 57 may be provided in a server or the like outside the vehicle connected via wireless communication means such as mobile communication.

[0034] The driving assistance device 50 also includes a first communication unit 58 and a second communication unit 59. The first communication unit 58 and the second communication unit 59 are interfaces for external communication that are communicatively connected to the processing unit 51. The first communication unit 58 is an interface for wireless communication such as mobile communication, and is used for transmitting and receiving data to and from a telematics service. The second communication unit 59 is an interface for vehicle-to-vehicle communication, and is used for communication with other vehicles around the vehicle 1.

[0035] (1-2-2. Functional configuration) The processing unit 51 of the driving support device 50 includes a driving state detection unit 61, a surrounding environment detection unit 63, a blind spot area detection unit 65, a shadow area detection unit 67, a light source position estimation unit 69, a collision determination unit 71, a driving condition setting unit 73, and a notification unit 75. Each of these units is a function realized by the execution of a computer program by a processor, but a part of each unit may be configured with an analog circuit. Below, the function of each unit will be briefly described, and then the processing operation of the driving support device 50 will be described in detail.

[0036] (Driving condition detection unit) The driving condition detection unit 61 detects information on the operation state and behavior of the vehicle 1 based on the detection data transmitted from the vehicle condition sensor 35. The driving condition detection unit 61 obtains information on the operation state of the vehicle 1, such as the steering wheel or steering wheel steering angle, accelerator opening, brake operation amount, or engine speed, and information on the behavior of the vehicle 1, such as the vehicle speed, longitudinal acceleration, lateral acceleration, and yaw rate, at predetermined calculation intervals, and stores this information in the storage unit 53.

[0037] (Ambient environment detection section) The surrounding environment detection unit 63 detects the surrounding environment of the vehicle 1 based on image data transmitted from the surrounding environment sensor 31. Specifically, the surrounding environment detection unit 63 calculates the type, size (width, height, and depth), and position of an object present in front of the vehicle 1, the distance from the vehicle 1 to the object, and the relative speed between the vehicle 1 and the object. The detected objects include other moving vehicles, parked vehicles, pedestrians, bicycles, side walls, curbs, buildings, utility poles, traffic signs, traffic signals, natural objects, and all other moving and stationary objects present around the vehicle 1.

[0038] (Blind spot detection section) The blind spot area detection unit 65 detects a blind spot area seen from the vehicle 1 based on the information of the surrounding environment detected by the surrounding environment detection unit 63 and the high-precision map data storage unit 55. For example, the blind spot area detection unit 65 detects, based on the high-precision map data storage unit 55, an area beyond a corner ahead of the vehicle 1, where a structure such as a side wall exists before the corner and cannot be detected by the surrounding environment sensor 31, as a blind spot area. Also, based on the information of the surrounding environment detected by the surrounding environment detection unit 63, the blind spot area detection unit 65 detects, based on the information of the surrounding environment detected by the surrounding environment detection unit 63, an area behind the parked vehicle, which cannot be detected by the surrounding environment sensor 31, as a blind spot area. Note that the blind spot area detection unit 65 may detect a blind spot area seen from the vehicle 1 based on either the information of the surrounding environment detected by the surrounding environment detection unit 63 or the information of the high-precision map data storage unit 55. However, the method of detecting the blind spot area is not particularly limited.

[0039] (Shadow area detection section) The shadow area detection unit 67 detects a shadow present in the image data based on the image data transmitted from the surrounding environment sensor 31. In this embodiment, the "shadow" refers to a dark area that occurs on a wall, the ground, etc. as a result of an object such as a moving body or a stationary object blocking the progression of light. For example, the shadow area detection unit 67 can identify a shadow by determining a contour line where the brightness or hue changes significantly within a predetermined distance based on the brightness information and hue information of each pixel of the image data. However, the method of detecting a shadow is not particularly limited, and any known shadow detection technology can be used. The shadow detected by the shadow area detection unit 67 includes not only the shadow of an object detected by the surrounding environment detection unit 63, but also the shadow of an object (hereinafter also referred to as a "latent object") that exists in a blind spot area as seen from the vehicle 1 and is not directly detected by the surrounding environment detection unit 63.

[0040] (Light source position estimation part) The light source position estimation unit 69 estimates the position of the light source emitting light that creates the shadow detected by the shadow area detection unit 67. In this embodiment, the light source position estimation unit 69 estimates the light source and its position using image data, time information, high-precision map data, sun position data, information acquired from a telematics service, and information acquired from other vehicles via vehicle-to-vehicle communication transmitted from the surrounding environment sensor 31. Specifically, the light source position estimation unit 69 determines whether the light source creating the shadow is the sun, a lighting device installed in a non-movable manner, or a light of a mobile body equipped with a light source, and estimates the position of the sun, the lighting device installed in a non-movable manner, or the mobile body that is the light source.

[0041] (Collision determination unit) The collision determination unit 71 determines the possibility of a collision between the host vehicle 1 and a latent object, based on information on the shadow of the latent object detected by the shadow area detection unit 67 and information on the position of the light source estimated by the light source position estimation unit 69. Specifically, the collision determination unit 71 calculates the position, moving direction, and moving speed of the latent object in the blind spot area based on the position of the light source and the change over time in the shadow of the latent object, and determines the possibility of a collision between the host vehicle 1 and the latent object.

[0042] (Operation condition setting section) When the collision determination unit 71 determines that there is a possibility of collision between the host vehicle 1 and the potential object, the driving condition setting unit 73 sets driving conditions for the host vehicle 1 to avoid the collision. Specifically, the driving condition setting unit 73 sets a target acceleration or a target steering angle for decelerating the host vehicle 1 or correcting the traveling trajectory, and transmits this information to the vehicle control device 41. The vehicle control device 41, which has received the information on the driving conditions, controls the driving of each control device based on the information on the set driving conditions.

[0043] (Notification Department) When the collision determination unit 71 determines that there is a possibility of a collision between the vehicle 1 and the latent object, the notification unit 75 outputs a command signal to the notification device 43 to notify the driver of the possibility of a collision. The notification unit 75 may not only notify the driver of the possibility of a collision, but also notify the driver of information on the jumping-out latent object, such as the jumping-out position and the jumping-out speed.

[0044] <1-3. Operation of driving support device> Next, an example of the operation of the driving support device 50 according to this embodiment will be specifically described with reference to a flowchart.

[0045] 3 and 4 are flowcharts showing an example of the processing operation of the driving support device 50 of this embodiment. First, when the in-vehicle system including the driving support device 50 is started (step S11), the surrounding environment detection unit 63 of the processing unit 51 acquires surrounding environment information of the host vehicle 1 (step S13). Specifically, the surrounding environment detection unit 63 detects objects present around the host vehicle 1 based on detection data transmitted from the surrounding environment sensor 31. In addition, the surrounding environment detection unit 63 calculates the position, size, and speed of the detected object, the distance from the host vehicle 1 to the object, and the relative speed between the host vehicle 1 and the object.

[0046] For example, the surrounding environment detection unit 63 detects an object in front of the vehicle 1 by performing image processing on the image data transmitted from the front photographing cameras 31LF, 31RF, using a pattern matching technique or the like. The surrounding environment detection unit 63 also calculates the position, size, and distance to the object as seen from the vehicle 1 based on the position of the object in the image data, the size of the object in the image data, and the parallax information of the left and right front photographing cameras 31LF, 31RF. Furthermore, the surrounding environment detection unit 63 calculates the relative speed between the vehicle 1 and the object by time-differentiating the change in distance, and calculates the speed of the object based on the information on the relative speed and the vehicle speed of the vehicle 1.

[0047] The surrounding environment detection unit 63 labels the detected objects whose speed is equal to or lower than a predetermined value, for example, a slow speed (10 km / h or lower), as stationary objects, and labels the detected objects whose speed exceeds the predetermined value as moving objects. The predetermined value may be 0 km / h, but by setting the slow speed to a predetermined value, a pedestrian who is at risk of jumping out of a blind spot between vehicles moving slowly during a traffic jam can be detected as an object present in the blind spot in the technology of the present disclosure. The surrounding environment detection unit 63 records information about the detected objects in the storage unit 53. However, the method of detecting objects is not limited to the above example.

[0048] Next, the blind spot area detection unit 65 of the processing unit 51 refers to the high precision map data stored in the high precision map data storage unit 55, and acquires data on roads, structures, and the like around the vehicle 1 based on the information on the current position of the vehicle 1 transmitted from the GNSS sensor 37 (step S15). For example, the blind spot area detection unit 65 acquires data on the position and shape of the road on which the vehicle 1 is traveling, and the position, shape, and size (height) of structures at the edge of the road on which the vehicle 1 is traveling.

[0049] Next, the blind spot area detection unit 65 determines whether or not there is a blind spot area ahead of the host vehicle 1 (step S17). Specifically, the blind spot area detection unit 65 determines whether or not there is a blind spot area ahead of the host vehicle 1 as seen from the host vehicle 1, based on the information on the surrounding environment detected in step S13 and the data around the host vehicle 1 acquired in step S15. For example, when there is a parked vehicle ahead of the host vehicle 1 or when the rear of a corner ahead of the host vehicle 1 cannot be seen, the blind spot area detection unit 65 determines that there is a blind spot area.

[0050] If it is not determined that there is a blind spot area (S17 / No), the processing unit 51 returns to step S13 and repeats the process until a blind spot area is detected, unless it is determined in step S21 that the in-vehicle system has stopped. On the other hand, if it is determined that there is a blind spot area (S17 / Yes), the shadow area detection unit 67 determines whether there is a shadow of a latent object existing in the blind spot area (step S19). Specifically, the shadow area detection unit 67 detects a shadow existing in the image data based on the image data transmitted from the surrounding environment sensor 31. Furthermore, the shadow area detection unit 67 determines that there is a shadow of a latent object existing in the blind spot area when there is a shadow of an object not detected by the surrounding environment detection unit 63 and extending outward from the blind spot area among the detected shadows.

[0051] If it is not determined that there is a shadow of a latent object existing in the blind spot area (S19 / No), the processing unit 51 returns to step S13 and repeats the processes described up to this point unless it is determined in step S21 that the in-vehicle system has stopped. On the other hand, if it is determined that there is a shadow of a latent object existing in the blind spot area (S19 / Yes), the light source position estimation unit 69 of the processing unit 51 executes a process of estimating the position of the light source generating the shadow (step S23).

[0052] FIG. 5 is a flowchart showing an example of a process for estimating the position of a light source. First, the light source position estimation unit 69 acquires various data for estimating the light source and the position of the light source (step S41). In this embodiment, the light source position estimation unit 69 acquires image data transmitted from the surrounding environment sensor 31, time information, information on the position and traveling direction of the vehicle 1 transmitted from the GNSS sensor 37, high-precision map data stored in the high-precision map data storage unit 55, sun position data stored in the sun position data storage unit 57, weather information transmitted from the telematics service, and information on other vehicles acquired via vehicle-to-vehicle communication.

[0053] Next, the light source position estimation unit 69 determines whether the light source generating the shadow is the sun (step S43). For example, when the weather is fine and the current time is after sunrise and before sunset, the light source position estimation unit 69 determines that the light source is the sun. The time may be limited, such as from a predetermined time after sunrise to a predetermined time before sunset. Furthermore, the light source position estimation unit 69 may also determine whether the sun is shining around a shadow determined to be the shadow of a potential object existing in a blind spot area in the image data transmitted from the surrounding environment sensor 31, based on the luminance and hue around the shadow.

[0054] If it is determined that the light source is the sun (S43 / Yes), the light source position estimation unit 69 estimates the position of the sun as seen from the vehicle 1 or the blind spot area (step S45). Specifically, the light source position estimation unit 69 identifies the altitude and direction of the sun at the current time based on the time information and the sun position data. Furthermore, the light source position estimation unit 69 estimates the direction of the sun as seen from the vehicle 1 based on the information on the altitude and direction of the sun and the information on the traveling direction of the vehicle 1. The altitude and direction of the sun as seen from the blind spot area can be considered to be the same as the altitude and direction of the sun as seen from the vehicle 1. The light source position estimation unit 69 stores the estimated information on the altitude and direction of the sun in the memory unit 53.

[0055] On the other hand, if the light source is not determined to be the sun (S43 / No), the light source position estimation unit 69 determines whether the light source is a street light or not (step S47). For example, the light source position estimation unit 69 determines that the light source is a street light when the current time is after sunset and there is data of a street light that corresponds to the data of the installation position of the street light included in the high-precision map data. Specifically, when the current time is after sunset and there is data of a street light that is installed in the opposite direction to the direction in which the shadow extending from the blind spot extends, the light source position estimation unit 69 determines that the light source is a street light.

[0056] In this embodiment, a street lamp is taken as an example of a lighting device that is installed so as to be immovable, but the lighting device that is installed so as to be immovable is not limited to a street lamp. Any lighting device may be used as long as its installation position can be identified by the processing unit 51, for example, the lighting device is stored in advance in the high-precision map data storage unit 55. The lighting device that is installed so as to be immovable may be, for example, a lighting device installed in a commercial facility or a leisure facility.

[0057] If it is determined that the light source is a street light (S47 / Yes), the light source position estimation unit 69 refers to the installation position and height data of the street light included in the high-precision map data, and estimates the position of the street light as seen from the blind spot area (step S49). In this case, the position of the street light means the height and direction of the light source of the street light. The light source position estimation unit 69 stores the estimated height and direction information of the street light in the storage unit 53.

[0058] On the other hand, if the light source is not determined to be a street light (S47 / No), the light source position estimation unit 69 determines whether or not the light source is a light of a moving body such as a vehicle (step S51). For example, when the current time is after sunset and there is another vehicle moving toward the blind spot area from the opposite direction to the direction of the shadow extending from the blind spot area based on information on the position and traveling direction of the other vehicle acquired via vehicle-to-vehicle communication, the light source position estimation unit 69 determines that the light source is a light of a moving body.

[0059] When it is determined that the light source is the light of the moving body (S51 / Yes), the light source position estimation unit 69 estimates the position of the light of the other vehicle as seen from the blind spot area based on the information of the other vehicle (step S53). When the information of the other vehicle acquired through the vehicle-to-vehicle communication includes data on the type of the other vehicle, the light source position estimation unit 69 estimates the height and direction of the light of the other vehicle as seen from the blind spot area according to the distance from the position coordinate of the other vehicle on the GNSS system to the front of the other vehicle and the vehicle height of the other vehicle. When the information of the other vehicle does not include data on the type of the other vehicle, the light source position estimation unit 69 estimates the height and direction of the light of the other vehicle as seen from the blind spot area using information on the distance and height set in advance. The preset distance and height may be, for example, the average distance from the position coordinate of the passenger vehicle on the GNSS system to the front of the vehicle body and the average vehicle height of the passenger vehicle, but are not particularly limited. The light source position estimation unit 69 stores the estimated information on the height and direction of the light of the moving body in the storage unit 53.

[0060] On the other hand, if the light source is not determined to be the light of a moving object (S51 / No), the light source position estimation unit 69 determines that the position of the light source cannot be specified (step S55).

[0061] Returning to Fig. 4, after the light source estimation process is completed, the collision determination unit 71 determines whether or not the light source has been identified (step S25). If the light source has not been identified by the light source estimation process (S25 / No), the shadow region detection unit 67 determines whether or not the shadow of a latent object existing in the blind spot area is moving (step S27). For example, when the position of the shadow of a stationary object whose position of the light source can be estimated is changing, the shadow region detection unit 67 determines that the shadow of a latent object existing in the blind spot area is moving.

[0062] If the shadow of the latent object existing in the blind spot area is not moving (S27 / No), the process returns to step S13 and repeats the process described up to this point unless it is determined in step S39 that the in-vehicle system has stopped. If the shadow of the latent object existing in the blind spot area is moving (S27 / Yes), the notification unit 75 performs a warning process to notify the driver of the presence of a latent object in the blind spot area (step S29), although it cannot be determined that the latent object existing in the blind spot area will jump out in front of the vehicle 1. For example, the notification unit 75 notifies the driver of the presence of a latent object in the blind spot area by at least one of voice, warning sound, and warning display.

[0063] On the other hand, if the light source is identified by the light source estimation process (S25 / Yes), the collision determination unit 71 estimates the position, moving direction, and moving speed of the latent object based on the position of the light source and the change over time of the shadow of the latent object existing in the blind spot area (step S31). Below, a method of estimating the position, moving direction, and moving speed of the latent object will be described for each of the cases where the light source is the sun, a street lamp, and a light of a moving body.

[0064] (When the light source is the sun) 6 to 8 are diagrams for explaining a method for estimating the position and moving speed of a latent object based on the position of the sun S estimated in step S53 and the time change of the shadow 84 of the latent object when the light source is the sun S. Here, as shown in FIG. 6, a driving scene will be explained as an example in which the blind spot is beyond a parked vehicle 80 on the left side ahead of the road on which the host vehicle 1 is traveling at a time when the sun S is shining in the daytime. A pedestrian exists as a latent object in the blind spot area, and the pedestrian is moving toward the path on which the host vehicle 1 is traveling. A shadow 81 of the parked vehicle 80 and a shadow 84 of the pedestrian are visually recognized from the host vehicle 1.

[0065] 7 and 8 are diagrams showing the driving scene of FIG. 6 as viewed from the traveling direction of the vehicle 1, with FIG. 7 showing the situation at time t and FIG. 8 showing the situation at time t+Δt. The height of the pedestrian 83 is h1, the distance from the pedestrian 83 to the outer edge of the parked vehicle 80 (the tire position in the illustrated example) is x1, the distance from the parked vehicle 80 to the tip of the shadow 84 of the pedestrian 83 is L1, and the angle between the line connecting the sun S and the tip of the shadow 84 and the ground is θ1. The height h1 of the pedestrian 83 may be set to any value, such as the average height of a child or adult. From trigonometry, the following formulas (1) and (2) hold at time t and time t+Δt, respectively.

[0066]

number

[0067]

number

[0068] When the light source is the sun S, the position of the light source can be considered not to change in a short period of time, so cotθ1(t) and cotθ1(t+Δt) can be said to be approximately equal. Therefore, the movement speed v of the pedestrian 83 can be expressed by the following equation (3).

[0069]

number

[0070] That is, the position of the pedestrian 83 at each time t and time t+Δt is obtained by the above formulas (1) and (2), and the moving speed v(t) of the pedestrian 83 between time t and time t+Δt is obtained by the above formula (3). Therefore, the collision determination unit 71 can obtain the position and moving speed of the pedestrian 83 by obtaining the distance L1 from the parked vehicle 80 to the tip of the shadow 84 of the pedestrian 83 and the angle θ1 between the line connecting the sun S and the tip of the shadow 84 and the ground based on the image data transmitted from the surrounding environment sensor 31. Furthermore, the collision determination unit 71 can obtain the moving direction of the pedestrian 83 from the change in the position of the tip of the shadow 84 of the pedestrian 83. The reference point for obtaining the moving direction of the pedestrian 83 is not limited to the position of the tip of the shadow 84, and may be any position.

[0071] (When the light source is a street light) 9 to 11 are diagrams for explaining a method for estimating the position and moving speed of a latent object based on the position of the street light 93 estimated in step S53 and the time change of the shadow 91 of the latent object when the light source is a street light 93. Here, as shown in FIG. 9, a driving scene will be explained as an example in which the host vehicle 1 is driving in an alley with side walls 94, 95 on both the left and right sides at night, and the corner ahead on the right side is in a blind spot. A pedestrian 90 exists as a latent object in the blind spot area, and the pedestrian 90 is moving toward the route on which the host vehicle 1 is driving. A shadow 91 of the pedestrian 90 is visually recognized from the host vehicle 1.

[0072] 10 and 11 are diagrams showing the traveling scene of FIG. 9 as viewed from the traveling direction of the vehicle 1, with FIG. 10 showing the situation at time t and FIG. 11 showing the situation at time t+Δt. The height of the pedestrian 90 is h2, the height of the light source of the street lamp 93 is H2, the distance from the pedestrian 90 to the end of the side wall 94 is x2, the distance from the installation position of the street lamp 93 to the end of the side wall 94 is X2, the distance from the end of the side wall 94 to the tip of the shadow 91 of the pedestrian 90 is L2, and the angle between the line connecting the light source of the street lamp 93 and the tip of the shadow 91 and the ground is θ2. The height h2 of the pedestrian 90 may be set to any value, such as the average height of a child or an adult. From trigonometry, the following formulas (4) and (5) are respectively established at time t and time t+Δt.

[0073]

number

[0074]

number

[0075] Since the movement speed v of the pedestrian 90 can be obtained by dividing the movement distance from time t to time t+Δt by the time Δt, from the above equations (4) and (5), the movement speed v of the pedestrian 90 can be expressed by the following equation (6).

[0076]

number

[0077] That is, the position of the pedestrian 90 at each time t and time t+Δt is obtained by the above formulas (4) and (5), and the moving speed v(t) of the pedestrian 90 between time t and time t+Δt is obtained by the above formula (6). Therefore, the collision determination unit 71 can obtain the position and moving speed of the pedestrian 90 by obtaining the horizontal distance X2 from the installation position of the street light 93 to the end of the side wall 94 based on the high-precision map data, and obtaining the distance L2 from the end of the side wall 94 to the tip of the shadow 91 of the pedestrian 90 based on the image data transmitted from the surrounding environment sensor 31, and the angle θ2 between the line connecting the light source of the street light 93 and the tip of the shadow 91 and the ground. Furthermore, the collision determination unit 71 can obtain the moving direction of the pedestrian 90 from the change in the position of the tip of the shadow 91 of the pedestrian 90. The reference point for obtaining the moving direction of the pedestrian 90 is not limited to the position of the tip of the shadow 91, and may be any position.

[0078] (When the light source is a moving object) 12 to 14 are diagrams for explaining a method for estimating the position and moving speed of a latent object based on the position of the light estimated in step S53 and the time change of the shadow 97 of the latent object when the light source is the light of a moving body. Here, as shown in FIG. 12, a driving scene will be explained as an example in which the host vehicle 1 is traveling in an alley with side walls 94, 95 on both the left and right sides at night, and the blind spot is beyond a corner on the right side ahead. A pedestrian 96 exists as a latent object in the blind spot area, and the pedestrian 96 is moving toward the route on which the host vehicle 1 is traveling. A shadow 97 of the pedestrian 96 is visually recognized from the host vehicle 1.

[0079] 13 and 14 are diagrams showing the driving scene of FIG. 12 as viewed from the traveling direction of the vehicle 1, with FIG. 13 showing the situation at time t and FIG. 14 showing the situation at time t+Δt. The height of the pedestrian 96 is h3, the height of the light of the other vehicle 98, which is a moving body, is H3, the distance from the pedestrian 96 to the edge of the side wall 95 on the left side of the road is x3, the distance from the position of the light of the other vehicle 98 to the edge of the side wall 95 is X3, the height of the shadow 97 of the pedestrian 96 projected on the side wall 95 is L3, and the angle between the line connecting the light of the other vehicle 98 and the upper end of the shadow 97 projected on the side wall 95 and a line parallel to the ground is θ3. The height h3 of the pedestrian 96 may be set to any value, such as the average height of a child or an adult. From trigonometry, the following formulas (7) and (8) are respectively established at time t and time t+Δt.

[0080]

number

[0081]

number

[0082] Since the movement speed v of the pedestrian 96 can be obtained by dividing the movement distance from time t to time t+Δt by the time Δt, from the above equations (7) and (8), the movement speed v of the pedestrian 96 can be expressed by the following equation (9).

[0083]

number

[0084] That is, the position of the pedestrian 96 at each time t and time t+Δt is obtained by the above formulas (7) and (8), and the moving speed v(t) of the pedestrian 96 between time t and time t+Δt is obtained by the above formula (9). Therefore, the collision determination unit 71 can obtain the position and moving speed of the pedestrian 96 by obtaining the distance X3 from the light of the other vehicle 98 to the side wall 95 based on the data of the other vehicle 98 acquired by vehicle-to-vehicle communication and the high-precision map data, and obtaining the height L3 of the shadow 91 of the pedestrian 90 projected on the side wall 95 based on the image data transmitted from the surrounding environment sensor 31, and the angle θ3 between the line connecting the light of the other vehicle 98 and the upper end of the shadow 97 and a line parallel to the ground. Furthermore, the collision determination unit 71 can obtain the moving direction of the pedestrian 96 from the change in the position of the tip of the shadow 97 of the pedestrian 96. The reference point for obtaining the moving direction of the pedestrian 96 is not limited to the position of the tip of the shadow 97, and may be any position.

[0085] Although the example of estimating the position, moving direction, and moving speed of a latent object based on the position of a light source and the change over time of a shadow has been described in which the latent object is a pedestrian, the object assumed as a latent object is not limited to a pedestrian. The collision determination unit 71 may estimate the position, moving direction, and moving speed of a latent object by assuming other moving objects such as bicycles and passenger cars other than pedestrians. In this case, it is preferable to change the value of the height h used in each of the above formulas according to the assumed latent object. In addition, the processing unit 51 may detect the outline of the detected shadow area and assume the type of the latent object by a matching process or the like. By assuming the type of the latent object based on the outline of the shadow area, the load of the calculation process on the processor can be reduced compared to the case where multiple types of latent objects are assumed.

[0086] Returning to FIG. 4, the collision determination unit 71 estimates the position, moving direction, and moving speed of the latent object, and then determines the possibility of collision between the latent object and the host vehicle 1 (step S33). For example, the collision determination unit 71 estimates the running trajectory of the host vehicle 1 based on information on the running state of the host vehicle 1 detected by the running state detection unit 61, and assumes a running range according to the size of the host vehicle 1. Then, the collision determination unit 71 estimates the trajectory of the latent object based on the position, moving direction, and moving speed of the latent object, and determines that the latent object will collide with the host vehicle 1 when the trajectory of the latent object overlaps with the running range of the host vehicle 1. However, the method of determining the possibility of collision between the latent object and the host vehicle 1 is not limited to the above example, and may be any determination method. For example, the possibility of collision between the latent object and the host vehicle 1 may be determined by a known risk calculation process that sets risk values ​​for the host vehicle 1 and the latent object and calculates a collision risk.

[0087] Next, the collision determination unit 71 determines whether or not there is a possibility of a collision between the potential object and the vehicle 1 based on the result of the collision possibility determination (step S35). If there is no possibility of a collision (S35 / No), the processing unit 51 returns to step S13 and repeats the processing described up to this point unless it is determined in step S39 that the in-vehicle system has stopped. On the other hand, if there is a possibility of a collision (S35 / Yes), the processing unit 51 executes a driving assistance process to avoid a collision between the potential object and the vehicle 1 (step S37).

[0088] For example, the driving condition setting unit 73 of the processing unit 51 transmits a command signal to the vehicle control device 41 to decelerate the host vehicle 1, and decelerates the host vehicle 1. In this case, the driving condition setting unit 73 may set a minimum target acceleration that can avoid a collision within a range that does not exceed the maximum value of the deceleration. This can reduce the risk of an excessively rapid deceleration of the host vehicle 1. In addition, the driving condition setting unit 73 may set a target steering angle for correcting the traveling trajectory of the host vehicle 1 when a collision cannot be avoided only by decelerating the host vehicle 1, or in conjunction with the deceleration operation of the host vehicle 1. In this case, the target steering angle is set so as not to cause the host vehicle 1 to spin out or collide with a side wall or the like.

[0089] Furthermore, the notification unit 75 of the processing unit 51 may output a command signal to the notification device 43 to notify that there is a possibility of a collision between the host vehicle 1 and the latent object. In this case, the notification unit 75 may also notify that there is a possibility that the latent object will jump out from the blind spot area, and information on the latent object such as the jumping out position and the jumping out speed of the latent object. Furthermore, the notification unit 75 may notify that an intervention has been made in the driving operation to avoid a collision between the host vehicle 1 and the latent object.

[0090] Next, the processing unit 51 determines whether the in-vehicle system has stopped (step S39). If the in-vehicle system has not stopped (S39 / No), the processing unit 51 returns to step S13 and repeats the processing of each step described above. On the other hand, if the in-vehicle system has stopped (S39 / Yes), the processing unit 51 ends the processing of this routine.

[0091] As described above, the driving support device 50 according to this embodiment detects the shadow of a latent object present in a blind spot area seen from the vehicle 1 based on image data generated by the forward-facing cameras 31LF and 31RF, and estimates the position of the light source forming the shadow. The driving support device 50 also calculates the position, moving direction, and moving speed of the latent object based on the estimated light source position and the time change of the shadow of the latent object, and determines the possibility of a collision between the vehicle 1 and the latent object. As a result, when a blind spot area is detected, driving support such as slowing down the vehicle 1 is not performed even when it is not known whether an object will actually jump out, but the presence of a latent object in the blind spot area is recognized, and driving support is performed when there is a risk of the latent object jumping out and colliding with the vehicle 1. Therefore, it is possible to reduce the possibility of a collision between the vehicle 1 and the latent object, and to prevent a decrease in trust and acceptability of the driving support device.

[0092] Furthermore, the driving support device 50 according to this embodiment can estimate the position of the sun, calculate the position, moving direction, and moving speed of a potential object, and determine the possibility of a collision between the host vehicle 1 and the potential object. As a result, if the condition that sunlight is irradiated onto the potential object during the daytime and a shadow is created is met, the possibility of a collision between the host vehicle 1 and the potential object can be determined based on the position of the potential object present in the blind spot area as seen from the host vehicle 1, etc.

[0093] Furthermore, the driving support device 50 according to this embodiment can estimate the position of a lighting device that is installed so as not to be movable, calculate the position, moving direction, and moving speed of a latent object, and determine the possibility of a collision between the host vehicle 1 and the latent object. As a result, even in the time period after sunset, if there is a lighting device that illuminates the latent object, it is possible to determine the possibility of a collision between the host vehicle 1 and the latent object based on the position of the latent object that exists in the blind spot area as seen from the host vehicle 1.

[0094] Furthermore, the driving support device 50 according to this embodiment can estimate the position of a light source provided on a moving object, calculate the position, moving direction, and moving speed of a latent object, and determine the possibility of a collision between the host vehicle 1 and the latent object. As a result, even if there is no lighting device illuminating the latent object during the time period after sunset, when there is another vehicle or the like illuminating the latent object, it is possible to determine the possibility of a collision between the host vehicle 1 and the latent object based on the position of the latent object present in the blind spot area as seen from the host vehicle 1.

[0095] In the above embodiment, the driving support device 50 is configured to be able to distinguish between the sun, a non-movable lighting device, and a light source of a moving object. However, the driving support device 50 may be configured to be able to distinguish between only one of the above or a combination of any two of the above. In this case, may be configured not to acquire data that is not necessary for the determination, depending on the type of light source to be determined.

[0096] <<2. Second embodiment>> Next, a driving assistance device according to a second embodiment of the present disclosure will be described. In the first embodiment, an embodiment is described in which the driving assistance device 50 can acquire information regarding the location or installation position of a light source. In the second embodiment, an embodiment is described in which the driving assistance device cannot acquire information regarding the location of a light source.

[0097] The functions of the driving support device according to this embodiment may be used in combination with the functions of the driving support device according to the first embodiment.

[0098] <2-1. Overall configuration of the vehicle and driving support device> The overall configuration of the driving assistance device according to this embodiment and the vehicle equipped with the driving assistance device may be the same as the driving assistance device 50 and vehicle 1 according to the first embodiment. However, when the driving assistance device has only the functions of the driving assistance device according to the second embodiment, the driving assistance device does not need to have the sun position data storage unit 57, the first communication unit 58, and the second communication unit 59.

[0099] Hereinafter, the driving support device according to the second embodiment will be described with reference to the configuration example of the driving support device 50 shown in FIG. 2, using the same reference numerals.

[0100] <2-2. Functional configuration of driving support device> Of the functional configuration of the processing unit 51 of the driving assistance device 50, the driving condition detection unit 61, the surrounding environment detection unit 63, the blind spot area detection unit 65, the shadow area detection unit 67, the collision judgment unit 71, the driving condition setting unit 73 and the notification unit 75 have functions similar to those of each unit of the processing unit 51 of the driving assistance device 50 of the first embodiment.

[0101] In this embodiment, the light source position estimation unit 69 traces the shadow of a moving object that has moved from inside the blind spot area to outside the blind spot area and is made manifest, and estimates the position of the light source based on the change over time of the shadow of the moving object. If the position of the light source is estimated based on the change over time of the shadow of the made manifest moving object, the position, moving direction, and moving speed of the latent object can be estimated based on the change over time of the shadow of another latent object created by the same light source and the position of the light source. Therefore, even in a driving scene where it is not possible to obtain information on the position of the light source that is creating the shadow, it is possible to determine the possibility of a collision between the host vehicle 1 and a latent object present in the blind spot area.

[0102] <2-3. Operation of driving support device> Next, an example of the operation of the driving support device 50 according to this embodiment will be specifically described with reference to a flowchart.

[0103] 15 and 16 are flowcharts showing an example of the processing operation of the driving support device 50 of this embodiment. In the flowcharts shown in Figs. 15 and 16, the processing of steps S11 to S21 and steps S27 to S39 is executed similarly to the processing of steps S11 to S21 and steps S27 to S39 in the processing operation of the driving support device 50 of the first embodiment shown in Figs. 3 and 4. On the other hand, the driving support device 50 of this embodiment executes the processing of steps S61 to S65 instead of steps S23 to S25 in the processing operation of the driving support device 50 of the first embodiment.

[0104] When the in-vehicle system including the driving assistance device 50 is started (step S11), the processing unit 51 executes the processes of steps S11 to S21 in accordance with the procedure explained in the first embodiment.

[0105] If it is determined in step S19 that there is a shadow of an object existing in the blind spot area (S19 / Yes), the shadow area detection unit 67 determines whether or not an object that has moved from the blind spot area to outside the blind spot area has been detected (step S61). Specifically, the shadow area detection unit 67 determines that an object that has moved from the blind spot area to outside the blind spot area has been detected when the shadow that was detected as the shadow of a potential object existing in the blind spot area is changed to the shadow of an object detected by the surrounding environment detection unit 63.

[0106] If an object that has moved from the blind spot area to outside the blind spot area is not detected (S61 / No), the shadow area detection unit 67 proceeds to step S27 to determine whether or not the shadow of a latent object existing in the blind spot area is moving. On the other hand, if an object that has moved from the blind spot area to outside the blind spot area is detected (S61 / Yes), the light source position estimation unit 69 executes a process of estimating the position of the light source that is creating the shadow (step S63). Hereinafter, a method of estimating the position of the light source will be described using a case where the light source is a street lamp as an example.

[0107] 17 to 19 are diagrams for explaining a method for estimating the position of a light source (street light) 109 based on a time change in a shadow 105 of another vehicle 103 that has moved from a blind spot area to outside the blind spot area when the light source is a street light 109. Here, as shown in FIG. 17, a driving scene will be explained as an example in which a building 107 is present on the right side of a road on which the host vehicle 1 is traveling at night, and the blind spot is beyond a corner behind the building 107. A pedestrian 100 is present in the blind spot area as a latent object, and the pedestrian 100 is moving toward the path on which the host vehicle 1 is traveling. In addition, the other vehicle 103 that was present in the blind spot area as a latent object has moved out of the blind spot area and become an apparent object. A shadow 101 of the pedestrian 100, the other vehicle 103, and the shadow 105 of the other vehicle 103 are visually recognized from the host vehicle 1.

[0108] 18 and 19 are diagrams showing the traveling scene of FIG. 17 as viewed from the traveling direction of the vehicle 1, with FIG. 18 showing the situation at time t and FIG. 19 showing the situation at time t+Δt. The height of the light source of the street lamp 109 is H4, the height from the ground to the part 104 of the other vehicle 103 where the line connecting the light source of the street lamp 109 and the tip of the shadow 105 of the other vehicle 103 meets is h4, the horizontal distance from the part 104 to the tip of the shadow 105 is x4, the horizontal distance from the light source of the street lamp 109 to the edge of the building 107 is X4, the horizontal distance from the edge of the building 107 to the tip of the shadow 105 of the other vehicle 103 is L4, and the angle between the line connecting the light source of the street lamp 109 and the tip of the shadow 105 of the other vehicle 103 and the ground is θ4. The tip of the shadow 105 is the shadow of the part 104 of the other vehicle 103. In this embodiment, the height H4 of the light source of the street lamp 109 and the horizontal distance X4 from the light source of the street lamp 109 to the edge of the building 107 are unknown values. From trigonometry, the following equations (10) and (11) hold at time t and time t+Δt, respectively.

[0109]

number

[0110]

number

[0111] The height H4 of the light source of the street lamp 109 and the height h4 and distances x4 and L4 other than the horizontal distance X4 from the light source of the street lamp 109 to the end of the building 107 can be calculated based on image data transmitted from the surrounding environment sensor 31. Therefore, the light source position estimation unit 69 can calculate the height H4 of the light source of the street lamp 109 and the horizontal distance X4 from the light source of the street lamp 109 to the end of the building 107 using the above formulas (10) and (11) by determining the respective heights h4 and distances x4 and L4 based on the image data.

[0112] Returning to FIG. 16, after the light source position estimation unit 69 estimates the position of the light source, the shadow region detection unit 67 judges whether or not the shadow of a latent object existing in the blind spot area exists in the same procedure as the process of step S19 (step S65). If the shadow of a latent object existing in the blind spot area does not exist (S65 / No), the processing unit 51 returns to step S13 and repeats the process described up to this point, unless it is determined in step S39 that the in-vehicle system has stopped. On the other hand, if the shadow of a latent object existing in the blind spot area exists (S65 / Yes), the process proceeds to step S31, and the processing unit 51 executes the process of steps S31 to S39 according to the procedure described in the first embodiment. As a result, even if the driving support device 50 cannot acquire information regarding the position of the light source, it can estimate the position of the light source by tracing the shadow of a moving object that has moved from the blind spot area to outside the blind spot area, and estimate the position, moving direction, and moving speed of a latent object existing in the blind spot area.

[0113] As described above, the driving support device 50 according to this embodiment can estimate the position of the light source by tracing the shadow of a moving object that moves from the blind spot area to outside the blind spot area, even if information about the position of the light source cannot be obtained. Therefore, even if information about the position of the light source cannot be obtained, the driving support device 50 according to the first embodiment can calculate the position, moving direction, and moving speed of the latent object based on the estimated position of the light source and the time change of the shadow of the latent object, and determine the possibility of a collision between the host vehicle 1 and the latent object. As a result, when a blind spot area is detected, driving support such as slowing down the host vehicle 1 is not performed even when it is not known whether the object will actually jump out, but the presence of a latent object in the blind spot area is recognized, and driving support is performed when there is a risk of the latent object jumping out and colliding with the host vehicle 1. Therefore, it is possible to reduce the possibility of a collision between the host vehicle 1 and the latent object, and to prevent a decrease in trust and acceptability of the driving support device.

[0114] In this embodiment, the light source position estimation unit 69 may record data on the estimated light source position on the high-precision map data. This allows the light source position data to be accumulated, and similarly to the case where the light source is a street lamp in the first embodiment, it becomes easier to identify the light source that is creating the shadow of the potential object, and the load of the process of estimating the light source position can be reduced.

[0115] Although the preferred embodiment of the present disclosure has been described in detail above with reference to the accompanying drawings, the present disclosure is not limited to such examples. It is clear that a person having ordinary knowledge in the technical field to which the present disclosure belongs can conceive of various modified or amended examples within the scope of the technical ideas described in the claims, and it is understood that these also naturally belong to the technical scope of the present disclosure.

[0116] For example, in the above embodiment, an example in which a single light source creates a shadow has been described, but the technology of the present disclosure can be applied even when a plurality of light sources create a shadow, that is, when a plurality of shadows are formed for one latent object. In this case, the driving assistance device may calculate the position, moving direction, and moving speed of the latent object based on the positions of the light sources that form each shadow and the time change of the shadow, and then determine the possibility of a collision between the host vehicle and the latent object using the average value of each. Alternatively, the driving assistance device may calculate the position, moving direction, and moving speed of the latent object by weighting based on the density of the shadow, the sharpness of the outline of the shadow, and the like, and then determine the possibility of a collision between the host vehicle and the latent object. By estimating the position, etc. of the latent object based on the time change of the shadow formed by multiple light sources, the accuracy of determining the possibility of a collision between the host vehicle and the latent object can be improved.

[0117] In addition, a computer program that causes a processor constituting the processing unit 51 to execute processes including detecting the shadow of an object present in a blind spot area as seen from the vehicle based on image data captured around the vehicle, estimating the position of the light source that forms the shadow, calculating the position, direction of movement and speed of the object based on the position of the light source and changes in the shadow over time, and determining the possibility of a collision between the vehicle and the object also falls within the technical scope of the technology disclosed herein. [Explanation of symbols]

[0118] 1: vehicle (own vehicle), 31: surrounding environment sensor, 31LF·31RF: forward-facing camera, 35: vehicle state sensor, 37: GNSS sensor, 41: vehicle control device, 43: notification device, 50: driving support device, 51: processing unit, 53: memory unit, 55: high-precision map data memory unit, 57: sun position data memory unit, 61: driving state detection unit, 63: surrounding environment detection unit, 65: blind spot area detection unit, 67: shadow area detection unit, 69: light source position estimation unit, 71: collision determination unit, 73: driving condition setting unit, 75: notification unit, 80: parked vehicle, 81: shadow, 83: pedestrian, 84: shadow, 90: pedestrian, 91: shadow, 93: street light, 94: side wall, 95: side wall, 96: pedestrian, 97: shadow, 98: other vehicle

Claims

1. A driving assistance device that assists a driver in avoiding a collision with an object, one or more processors; and one or more memories communicatively coupled to the one or more processors; The processor, Detecting the shadow of an object existing in a blind spot area as seen from the vehicle based on image data obtained by photographing the surroundings of the vehicle; Estimating a position of a light source that is present at a position different from the vehicle and that is forming the shadow; A driving assistance device that calculates the position, movement direction and movement speed of the object based on the position of the light source and the change over time of the shadow, and determines the possibility of a collision between the vehicle and the object.

2. The processor, The driving support device according to claim 1 , wherein the position of the sun as the light source is estimated based on information on sunrise and sunset times and a current time.

3. The processor, The driving support device according to claim 1 , wherein the position of the lighting device serving as the light source is estimated based on data on an installation position of the lighting device that is installed so as not to be movable.

4. The processor, The driving support device according to claim 1 , further comprising: a driver for estimating a position of the moving object having the light source based on data of the moving object received from outside the vehicle.

5. The processor, Detecting a moving object that has moved from the blind spot area to outside the blind spot area and a shadow of the moving object based on the image data; The driving assistance device according to claim 1 , wherein the position of the light source is estimated based on the moving object and a shadow of the moving object.

6. A driving assistance device that assists a driver in avoiding a collision with an object, a shadow area detection unit that detects a shadow of an object existing in a blind spot area as seen from the vehicle based on image data obtained by photographing the surroundings of the vehicle; a light source position estimation unit that estimates a position of a light source that is present at a position different from the vehicle and that is forming the shadow; a collision determination unit that calculates a position, a moving direction, and a moving speed of the object based on the position of the light source and a change over time of the shadow, and determines a possibility of a collision between the vehicle and the object; A driving assistance device equipped with the above.

7. A vehicle equipped with a driving assistance device that assists driving to avoid collisions with objects, The driving assistance device includes one or more processors and one or more memories communicatively connected to the one or more processors; The processor, Detecting the shadow of an object existing in a blind spot area as seen from the vehicle based on image data obtained by photographing the surroundings of the vehicle; Estimating a position of a light source that is present at a position different from the vehicle and that is forming the shadow; The vehicle calculates the position, movement direction and movement speed of the object based on the position of the light source and the change over time of the shadow, and determines the possibility of a collision between the vehicle and the object.

8. A recording medium having a computer program applied to a driving assistance device that assists in avoiding a collision with an object, One or more processors, Detecting a shadow of an object existing in a blind spot area as seen from the vehicle based on image data obtained by photographing the surroundings of the vehicle; estimating a position of a light source that is present at a position different from the vehicle and that is forming the shadow; A recording medium having a computer program recorded thereon for executing a process including: calculating the position, direction of movement, and speed of movement of the object based on the position of the light source and the change in the shadow over time, and determining the possibility of a collision between the vehicle and the object.

Images