Travel support method, travel support device, and program

Abstract

A processor 10 acquires a captured image of a right side region RS and / or a left side region LS along a travel path L1 of a host vehicle V1 using a camera 3 mounted on the host vehicle V1, detects a stationary object SO existing on the side of the travel path L1 from the captured image, calculates the degree of shielding occupied by an image of the detected stationary object SO with respect to a predefined setting region S, determines that visibility of the travel path L1 is poor when the degree of shielding is equal to or greater than a predetermined value, and outputs a determination result to the outside.

Description

Travel Support Method, Travel Support Device, and Program

[0001] The present invention relates to a travel support method, a travel support device, and a program for a vehicle.

[0002] When it is determined that there is an intersection with poor visibility in the traveling direction of the vehicle, a technique is known in which the vehicle is repeatedly stopped and started to pass through the intersection.

[0003] Japanese Patent Application Laid-Open No. 2019-067295

[0004] However, according to the technique of the patent document, there is a problem that it is impossible to determine whether the visibility of the intersection is poor unless the vehicle actually reaches the intersection.

[0005] The problem to be solved by the present invention is to determine whether the visibility of the intersection is poor before the vehicle actually reaches the intersection.

[0006] The present invention detects a stationary object from an imaging image on the right side and / or the left side along the traveling path of the host vehicle, calculates the degree of occlusion occupied by the image of the detected stationary object with respect to a preset setting area, and determines that the visibility of the traveling path is poor when the degree of occlusion is equal to or greater than a predetermined value, and outputs the determination result to solve the above problem.

[0007] According to the present invention, it is possible to determine whether the visibility of the intersection is poor before the vehicle actually reaches the intersection.

[0008] It is a block diagram showing the hardware configuration of a travel support system. It is a flowchart showing the processing procedure of travel support control. FIGS. 3(A) and (B) are the first diagrams for explaining the control content. FIGS. 4(A) and (B) are diagrams showing an example of travel support information. FIG. 5 is the second diagram for explaining the control content. FIGS. 6(A) and (B) are the third diagrams for explaining the control content.

[0009] <First Embodiment> Figure 1 shows the hardware configuration of a driving support system 100 equipped with a driving support device 1 according to this embodiment. This driving support method is implemented by the processor 10 of the driving support device 1 using the hardware of the driving support system 100. The driving support system 100 includes one or more sensors 2, a camera 3 for imaging the outside of the vehicle, a vehicle information acquisition device 4, and a navigation device 5. Multiple sensors 2 are provided on the vehicle and form a sensor group that cooperates with each other. The sensors 2 detect the presence or absence of objects around the vehicle, the distance to the objects, the relative velocity of the objects, and the relative acceleration of the objects. Based on the relative velocity and relative acceleration of the objects, the sensors 2 can recognize whether an object is stationary or moving. The sensors 2 include a radar device 21 that detects (measures distance) the presence of objects around the vehicle, the position of objects, and changes in their position. The radar device 21 is a device that measures the distance and direction from the vehicle to the object, and the distance between objects, by emitting electromagnetic waves toward an object and measuring the reflected waves. The radar system 21 includes a laser radar, a millimeter-wave radar (LRF), a LiDAR (light detection and ranging) unit, an ultrasonic radar, and sonar. Sensor 2 includes a GNSS (Global Navigation Satellite System) signal receiver, a gyro sensor, and a vehicle speed sensor, and includes a position detection device 22 that uses these to detect the position of the vehicle. The position detection device 22 detects the position of an object (relative position to the vehicle) and the area where the object exists based on the detection information from Sensor 2. The detection information acquired by Sensor 2 is provided to the processor 10. Camera 3 is provided at least in front of the vehicle and captures images of the area in front of the road on which the vehicle is traveling. Camera 3 includes one or more cameras 3 positioned on the vehicle. Camera 3 includes an image sensor with a CCD (Charge-Coupled Device) or CMOS (Complementary Metal-Oxide-Semiconductor) image sensor, an ultrasonic camera, and an infrared camera. Camera 3 provides the captured images to the processor 10. Camera 3 is equipped with a wide-angle lens or a fisheye lens and captures images with a field of view (angle of view) of 60° to 250°.Camera 3 is equipped with multiple wide-angle or fisheye lenses and can acquire panoramic images of the vehicle's surroundings as captured images. Camera 3 includes a cabin camera 31. The cabin camera 31 captures images of the driver present in the cabin. The cabin camera 31 captures at least the driver's head. The processor 10 calculates the position of the driver's head in the captured image. Based on the position of the cabin camera 31, the processor 10 can calculate the absolute position of the driver's head (height coordinate relative to the road surface). Camera 3 includes a camera 3 installed as a roadside device. The camera 3 as a roadside device provides captured images to the external server 6. The images captured by the roadside camera 3 can be used as driving assistance information for intersections with poor visibility.

[0010] The vehicle information acquisition device 4 calculates the behavior of the vehicle, including at least one of the following: current position, direction of movement, speed, and acceleration, and their changes, based on detection information acquired from the sensor 2 or the vehicle controller 200, and provides this information to the processor 10. The navigation device 5 includes map information 51, a display 52, and a speaker 53. The navigation device 5 refers to the map information 51 and calculates a route to a set destination. This route includes a target trajectory in which the lane the vehicle is traveling is identified. The route and target trajectory calculated by the navigation device 5 are provided to the vehicle controller 200 and used for driving assistance or autonomous driving control. The map information 51 includes intersection information 511. The intersection information 511 includes the location information of the intersection, identification information of the intersecting roads at the intersection, and the boundaries of the intersecting roads. The map information 51 includes information on the road width and attributes (one-way, one lane in each direction, etc.) of each road identified by the identification information. Map information 51 is recorded in one or more of the following: ROM 12, RAM 13, the storage device of the navigation device 5, and the storage device (not shown) of an external server accessible to the processor 10 via the communication device 20. The navigation device 5 identifies intersections that the vehicle is approaching on the route to the destination. The navigation device 5 calculates the distance between the vehicle and the intersection. Based on the vehicle speed, the navigation device 5 calculates the time it will take for the vehicle to reach the intersection. These calculation results (distance / time) are provided to the processor 10. The display 52 displays and outputs route guidance information, driving support information, and other information. The speaker 53 outputs route guidance information, driving support information, and other information as voice. Driving support information is information output according to the judgment result of whether the visibility ahead of the road the vehicle is traveling on is good or bad.

[0011] The driving support system 100 has a vehicle controller 200. The vehicle controller 200 controls the vehicle actuators. The driving support system 100 includes a steering control device 210, a drive control device 220, and a braking control device 230 as vehicle actuators. The processor 10 of the driving support device 1 executes driving support control to assist driving control / driving control in accordance with the planned driving plan. The steering control device 210, the drive control device 220, and / or the braking control device 230 execute control content according to the judgment result of whether the visibility ahead of the road on which the vehicle is traveling is good or bad.

[0012] The driving support system 100 includes an external server 6. The external server 6 is a computer comprising a communication device 61, a storage device 62, and an arithmetic unit 63. The storage device 62 stores programs for driving support. The external server 6 provides driving support programs to the driving support device 1 via the communication device 61 in response to requests from the driving support device 1. The external server 6 sequentially receives imaging information captured by the roadside camera 3, stores it at least temporarily, and provides the imaging information as driving support information in response to request commands received from the processor 10.

[0013] The driving support device 1 of the driving support system 100 performs driving support control to assist the driving of the vehicle. The processor 10 of the driving support device 1 includes a ROM (Read Only Memory) 12 that stores a program for controlling driving support, a CPU (Central Processing Unit) 11 that executes the program stored in the ROM 12, and a RAM (Random Access Memory) 13 that functions as an accessible storage device. The processor 10 implements this driving support method using each piece of hardware of the driving support system 100. The processor 10 may use an MPU (Micro Processing Unit), a GPU (Graphics Processing Unit) instead of a CPU, or may be composed of an integrated circuit such as an ASIC (Application Specific Integrated Circuit) or FPGA (Field Programmable Gate Array). The processor 10 implements this driving support control method using each piece of hardware of the driving support system 100 according to the program for driving support control. The program is stored at least temporarily in a readable storage medium. The processor 10 can access the readable storage medium in which the program is stored at least temporarily. The driving support device 1 includes a readable storage medium on which a program is stored at least temporarily.

[0014] The driving support system 1 includes a communication device 20. The communication device 20 communicates with each device of the driving support system 100 using a CAN (Controller Area Network) communication system or the like. The communication device 20 has a communication function with in-vehicle devices and a wireless communication function with an external server 6. The ROM 12 / RAM 13 and / or the storage device 62 of the external server 6 are non-transitory storage media in which a program for executing driving support control is stored (remembered) in a state that can be read by the processor 10 (computer). The program is an electronically encoded command (or code) that causes the processor 10 to execute each step of the driving support control.

[0015] The processor 10 executes a driving assistance method to assist the driving of its own vehicle according to a program for driving assistance control. The control procedure of this embodiment will be explained based on the flowchart in Figure 2. An example of a scene in which the driving assistance control of this embodiment is executed is shown in Figure 3(A). The processor 10 uses the camera 3 mounted on the vehicle V1 to acquire images of the right RS and / or left LS along the road L1 of the vehicle V1 (S1). The road L1 on which the vehicle V1 is traveling intersects with intersections CL1 and CL2 at intersection IS. Intersection CL1 is a lane whose direction of travel is opposite to that of intersection CL2. The vehicle V1 approaches intersection IS. The range from which images are acquired for this driving assistance may be 10m or 30m from intersection IS. Another vehicle V3 is traveling on intersection CL1 which intersects with road L1 and is approaching intersection IS. Another vehicle V4 is traveling on intersection CL2 which intersects with road L1 and is approaching intersection IS. Furthermore, another vehicle V2 travels on the opposing track L2, which is in the opposite direction of travel to track L1. This situation is also common in Figures 3 and 6. Since the vehicle V1 is moving, the images acquired while it is moving include images of objects in the area RS to the right front and / or the area LS to the left front, which are along the track. In this example, stationary objects such as walls or fences, SOR and SOL (hereinafter sometimes collectively referred to as SO), are provided continuously between areas RS and LS and track L1 and the opposing track L2. In such a case, an image like the one shown in (B1) of Figure 3(B) is captured by camera 3. The images shown as KL and KR in the figure (hereinafter sometimes collectively referred to as K) correspond to the stationary objects SOR and SOL. If there are no stationary objects such as walls, images K1 to K5 (hereinafter sometimes collectively referred to as K) of individual stationary objects, as shown in (B2) of Figure 3(B), are captured in one of the areas RS and LS. The processor 10 detects stationary objects SO located in the right-hand region RS and / or left-hand region LS of the road L1 from the captured image (S2). The processor 10 detects each stationary object SO from the digital data of the captured image using image segmentation techniques. The processor 10 divides the digital data of the captured image into objects on a pixel-by-pixel basis, identifies the background (including the road surface) and the objects, and extracts the regions of the objects.The processor 10 determines one or more of the extracted objects: position, size, velocity, relative velocity, number, and type. Based on the change in the movement speed of each object, the processor 10 detects stationary objects OS from each object. The method for extracting images of each object, including stationary objects, from the captured image is not particularly limited, and any method known at the time of filing can be used. The processor 10 recognizes the area enclosed by the thick lines shown in Figure 3 (B1) and (B2) as the image K of stationary objects. From the captured image shown in Figure 3 (B1), the processor 10 recognizes continuous images KL and KR of stationary objects. From the captured image shown in Figure 3 (B2), the processor 10 recognizes discontinuous images K1 to K5 of stationary objects (independent objects lined up). The continuous images KL and KR of stationary objects shown in Figure 3 (B1) obstruct the driver's field of view of the vehicle V1 and / or the detection signal of sensor 2 over a wide area. The processor 10 calculates the degree of occlusion (occlusion rate) to which a stationary object obstructs the driver's field of view of the vehicle V1 and / or the range in which the detection signal of sensor 2 is acquired. The occlusion rate (occlusion degree) based on continuous images KL and KR of a stationary object tends to be higher than the occlusion rate (occlusion degree) based on discontinuous images K1 to K5 of a stationary object shown in Figure 3 (B2).

[0016] The processor 10 sets a predefined setting area S (S3). The setting area S becomes the denominator in the calculation of the occlusion rate (degree of occlusion) described later. The "degree of occlusion" includes the "occlusion rate". The occlusion rate is the ratio of the area (area or length of line segment) occupied by the image K of a stationary object to the setting area S (area or length of line segment). The setting area S may be defined as a two-dimensional figure or as a line segment. When the setting area S is defined as a two-dimensional figure, the denominator of the occlusion rate (degree of occlusion) is the area or the number of pixels, and when the setting area is defined as a line segment, the denominator of the occlusion rate is the length or the number of pixels. The setting area S can be set at a predetermined height from the road surface of the road L1. The "predetermined height" is defined based on the area to be checked in order to determine whether the visibility of the intersection CL1 is good or bad. The "predetermined height" for defining the "setting area S" includes a first predetermined height that defines the height of the upper edge of the setting area S and a second predetermined height that defines the height of the lower edge. The "setting region S," defined by a two-dimensional figure, is the region between a first predetermined height and a second predetermined height. In this embodiment, the "setting region S" is a strip-shaped region located at a predetermined height along the extending direction of the road L1. The "predetermined height" that defines the height of the "setting region S" can be defined according to the height position of the vehicle's side windows, the height position of the vehicle's side mirrors, the height position of the vehicle's seats, and the height position of the vehicle's headrests. The predetermined height can be defined according to the height position of the detection area of ​​the sensor 2. Furthermore, the predetermined height can be defined based on the position of the driver's head in the vehicle V1. Specifically, the processor 10 uses the cabin camera 31 to image the driver's head in the vehicle V1 and calculates the driver's head height based on the captured head image. The processor 10 extracts the image corresponding to the head from the captured image based on the driver's features such as eyes, ears, and shoulders. Since the information of the installation position of the cabin camera 31 in the real space axis is stored in advance, the processor 10 can calculate the driver's head height in the real space axis based on the position of the head image in the captured image and the position of the camera 3. A predetermined height is defined to define the position of the setting area S based on the driver's head height. An example of setting the setting area S is shown in (B3) of Figure 3(B), although this is not particularly limited.In this example, the defined area S is a plane perpendicular to the road surface, extending along the left and right edges of the road L1, and is a band-shaped area with its lower end at a height of 1.2m above the road surface and its upper end at a height of 2.5m. The "defined area S" is the information necessary to determine the visibility of the intersection CL1 that intersects with the road L1. Information below the defined area S (below the second predetermined height) and above the defined area S (above the first predetermined height) is not essential for determining the visibility of the intersection CL1. Since the defined area S at a predetermined height is defined as the area for calculating the occlusion rate (degree of occlusion) described later, the processing load for calculating the occlusion rate (degree of occlusion) can be reduced. Furthermore, by calculating the occlusion rate (degree of occlusion) for the "defined area S" corresponding to the height of the driver's head, the visibility of the intersection CL1 can be determined with high accuracy. In addition, by limiting the height of the "defined area S", the processing load for image processing of the captured images can be reduced.

[0017] The processor 10 calculates the ratio of the area (area occupied by the image), length, or number of pixels of the detected stationary object's image to the area, length, or number of pixels of the set region S (S4). The same applies when the set region S is defined at a predetermined height from the road surface of the track 1. The processor 10 divides the area, length, or number of pixels occupied by the detected stationary object's image by the area, length, or number of pixels of the set region S to calculate the occlusion rate (degree of occlusion) (S5). The occlusion rate (degree of occlusion) may be calculated separately for the right and left sides of the track L1, or it may be calculated comprehensively for both the right and left sides of the track L1. Figure 3(B) shows two examples where the ratio occupied by the stationary object's image K differs on the left and right sides of (B4). In the example of the first information shown on the left, the occlusion rate (degree of occlusion) is calculated as the ratio of the area and number of pixels occupied by the images K11, K12, and K13 of the three stationary objects within the set region S. In the example of the second information shown on the right, the occlusion rate (degree of occlusion) is calculated as the ratio of the area and number of pixels occupied by the images K14, K15, and K16 of the three stationary objects within the set region S. The area and number of pixels of the set region S on the right and left sides of (B4) are the same. The total area of ​​the images K (K11 to K13) of the three stationary objects in the first information shown on the left side of (B4) is smaller than the total area of ​​the images K (K14 to K16) of the three stationary objects in the second information shown on the right side. Therefore, the occlusion rate (degree of occlusion) of the first information on the left side of (B4) is lower than the occlusion rate (degree of occlusion) of the second information on the right side. The visibility of the road L1 in the presence of stationary object SO, as indicated by the first information on the left, is predicted to be better than the visibility of the road L1 in the presence of stationary object SO, as indicated by the second information on the right. The processor 10 determines whether the calculated occlusion rate (degree of occlusion) is greater than or equal to a predetermined value (S6). The predetermined value of the occlusion rate (degree of occlusion) can be set to 70-90%. In this example, the predetermined value of the occlusion rate (degree of occlusion) is set to 80%. In Figure 3 (B4), the occlusion rate (degree of occlusion) of the first information is less than the predetermined value, and the occlusion rate (degree of occlusion) of the second information is greater than or equal to the predetermined value. If the occlusion rate (degree of occlusion) is greater than or equal to the predetermined value (YES in S6), it is determined that visibility ahead of the road L1 is poor (S7). Poor visibility ahead of the road L1 means that it is difficult to perceive the traffic situation (presence of moving objects) on the intersecting road CL1 that intersects with the road L1 at intersection IS ahead of the road L1.Poor visibility on road L1 means that when road L1 intersects with intersection CL1 in the future, it will be difficult to perceive the traffic conditions on intersection CL1 at that intersection IS. If the processor 10 determines that visibility ahead on road L1 is poor if either the occlusion rate (degree of occlusion) based on the image K of a stationary object on the right side of road L1 or the occlusion rate (degree of occlusion) based on the image K of a stationary object on the left side of road L1 is greater than or equal to a predetermined value, then appropriate driving assistance is required. If the calculated occlusion rate (degree of occlusion) is less than the predetermined value (NO in S6), the processing from S1 onwards is repeated. If the processor 10 determines that visibility on road L1 leading to intersection IS is poor, it outputs the result of that determination (S6). The processor 10 outputs the judgment result to one or more of the navigation device 5, vehicle controller 200, and external server 6. Along with the judgment result, the processor 10 outputs a command to output driving support information to the navigation device 5, vehicle controller 200, and external server 6. This allows the system to determine in advance whether the visibility at intersection IS will be good or bad on the road L1 that the vehicle V1 will be traveling on before arriving at intersection IS, and to output the judgment result. By outputting the judgment result, control processing based on the judgment result can be executed quickly. If it is determined that the visibility on road L1 is poor, the processor 10 may output a command to the navigation device 5 via the speaker 53 to output a warning message saying, "Visibility is poor ahead. Drive carefully," or it may output a command to display the warning message via the display 52. ​​If it is determined that the visibility on road L1 is poor, the processor 10 may output a command to reduce the set speed or a deceleration command to the vehicle controller 200. If it is determined that visibility on road L1 is poor, the processor 10 may output a request to the external server 6 that includes identification information for the intersection IS and road L1, and obtain traffic information for the intersection CL1 that intersects road L1 at the intersection IS as driving support information.Traffic information for intersection CL1 includes images of intersection CL1 captured by camera 3, which is installed as a road facility near intersection IS.

[0018] Furthermore, the processor 10 causes the display 52 to display driving support information including an extended image of the intersection CL1 where the road L1 intersects. If the processor 10 determines that visibility of the road L1 is poor, it may cause the camera 3 to acquire a wide-angle image and provide it to the navigation device 5, and output a command to the controller of the display 52 of the navigation device 5 to output driving support information. The processor 10 displays an extended image on the display 52 of the right side, the left side, or both sides of the road L1 where visibility is determined to be poor (S8). The extended image includes video of the intersection CL1 as seen from the intersection IS to the right and / or left side. Figure 4(A) shows an example of driving support information including an extended image showing the state of the intersection CL1 on both the left and right sides, as displayed on the display surface of the vehicle's display 52. ​​The extended image shown in Figure 4(A) shows another vehicle V3 approaching the vehicle V1 from the right side of the intersection CL1, and another vehicle V4 approaching the vehicle V1 from the left side of the intersection CL2. Figure 4(B) shows an example of driving support information, including an extended image showing the traffic conditions at the right-hand intersection CL1, as displayed on the display surface of the display 52. ​​The extended image shown in Figure 4(B) shows another vehicle (motorcycle) V3 approaching the vehicle V1 from the right at the intersection CL1. Of course, an extended image showing the conditions at the left-hand intersection CL1 can also be displayed on the display 52 in the same way. The extended image is captured by a camera 3 equipped with a wide-angle lens or a fisheye lens. The processor 10 sends a command to the camera 3 to acquire the extended image when it determines that the occlusion rate (degree of occlusion) is above a predetermined value. The extended image on the right is acquired using a camera 3 located on the right front of the vehicle. The extended image on the left is acquired using a camera 3 located on the left front of the vehicle. Extended images on both the left and right sides are acquired using a camera 3 located at the front of the vehicle or a camera 3 located on the roof of the vehicle. Extended images on both the left and right sides may be generated by acquiring images from two or more cameras 3 located on the right front and left front of the vehicle, and combining multiple images in a panoramic manner.In this way, when it is determined that visibility ahead of the road L1 is poor, driving support information including an extended image of the intersection CL1 that intersects the road L1 is automatically displayed on the display 52. ​​This allows the driver to see the extended image of the intersection CL1 at the moment they reach the intersection IS with poor visibility without any special operation. It is difficult for a driver to predict whether visibility to the intersection CL1 at the intersection IS they will pass through in the future will be good or bad while driving on the road L1. In this embodiment, the possibility that images K of stationary objects to the left and right in front of the driver will obstruct their view is determined based on the occlusion rate (degree of occlusion), and the visibility at the intersection IS is determined before reaching the intersection IS. Then, driving support information including an extended image of the intersection CL1 can be displayed in advance and automatically. As a result, the driver can understand the condition of the intersection CL1 that intersects the road at the intersection IS in advance.

[0019] The processor 10 refers to the location information of the intersection IS in the map information 51 and presents driving support information including an extended image when the distance between the vehicle V1 and the intersection IS ahead becomes less than a first predetermined distance, or when the time until the vehicle V1 reaches the intersection IS becomes less than a first predetermined time. Preferably, the driving support information including the extended image is presented before passing through the intersection IS or before turning left or right. According to this process, the driving support information can be presented when the vehicle V1 approaches the intersection IS and before entering the intersection IS. The first predetermined distance and the first predetermined time are set so that the driving support information is presented when the vehicle V1 approaches the intersection IS. The first predetermined distance may be 40m, 30m, 20m, 10m, 5m, 3m, 1m, or 0.5m. The first predetermined time may be 10 seconds, 5 seconds, 3 seconds, 2 seconds, or 1 second. Furthermore, the first predetermined time may be 10 seconds, 5 seconds, 3 seconds, 2 seconds, or 1 second, set based on the time to reach the intersection calculated based on the speed of the vehicle V1 and the distance from the vehicle's position to the intersection. Although not particularly limited, the first predetermined distance and first predetermined time can be set longer when the vehicle speed is high than when the vehicle speed is low. For the vehicle V1 with a relatively high speed, driving support information including augmented images can be presented earlier. The first predetermined distance and / or first predetermined time may be defined according to the vehicle speed, the attributes of the road L1, or the width of the road L1. The first predetermined distance and / or first predetermined time may be set arbitrarily by the driver. The designation "first" is formal, and "second" may also be added.

[0020] <Second Embodiment> In this embodiment, it is preferable that the driving support control is performed before the intersection IS. The processor 10 refers to the map information 51 and performs a process to calculate the occlusion rate (degree of occlusion) when the distance between the vehicle V1 and the intersection IS where the road L1 intersects with the intersection CL1 becomes less than a second predetermined distance, or when the time until the vehicle V1 reaches the intersection IS becomes less than a second predetermined time. Specifically, as shown in process (A) of Figure 2, the processor 10 acquires vehicle information via the vehicle information acquisition device 4 (S11). The processor 10 acquires the current position of the vehicle V1 (S12). The processor 10 refers to the map information 51 (S13), identifies the road L1 on which the vehicle V1 is traveling (S14), and identifies the intersection CL1 that intersects with the road L1 (S15). The processor 10 identifies the intersection IS where the road L1 and the intersection CL1 intersect (S16). The position of the intersection IS is stored in the map information 51. The processor 10 may calculate the distance to the intersection IS based on the image captured by the camera 3. If the distance between the current position of the vehicle V1 and the intersection IS is less than the second predetermined distance, or if the time it takes for the vehicle V1 to reach the intersection IS from its current position is less than the second predetermined time (YES in S17), the processes from S1 onwards are executed. Otherwise (NO in S17), the processes from S11 to S17 are repeated. The second predetermined distance may be 50m, 40m, 30m, 20m, 10m, or 5m, although this is not particularly limited. The second predetermined time may be 30 seconds, 20 seconds, 10 seconds, 5 seconds, or 3 seconds. The second predetermined distance and / or the second predetermined time may be defined according to the vehicle speed, the attributes of the road L1, or the width of the road L1. The second predetermined distance and / or the second predetermined time may be arbitrarily set by the driver. The designation "second" is formal, and "first" may be added instead. Incidentally, it is preferable that the threshold (second predetermined distance, second predetermined time) for calculating the occlusion rate (degree of occlusion) be higher than the threshold (first predetermined distance, first predetermined time) for presenting driving support information. By starting this process when the vehicle V1 is traveling on the road L1 upstream of the intersection IS, it is possible to determine in advance whether the visibility to the intersection IS that the vehicle V1 will pass through in the future is good or not.This allows the system to determine whether the road L1 is one where the intersecting road CL1 is difficult to perceive before the vehicle V1 arrives at the intersection IS. If visibility is poor, the system can provide driving assistance information, including augmented images, at an optimal timing before the vehicle passes through the intersection IS.

[0021] Furthermore, the processor 10 may refer to the road attributes of the map information 51 and calculate the occlusion ratio (degree of occlusion) when the road width of the road L1 is less than a predetermined value, when the road L1 is a single lane, or when the road L1 is a one-way street. When driving on a narrow road L1 with a road width less than a predetermined value (for example, a road width of 4.5 m or less), when driving on a single-lane road L1, or when driving on a one-way road L1, it tends to be difficult to grasp the situation of the intersecting road CL1 at an intersection IS. For this reason, by executing this process when the vehicle V1 is driving on a road L1 with these road attributes, it is possible to determine in advance whether the visibility to the intersection IS to be passed in the future is good or not. If the visibility is poor, driving support information including an extended image can be presented at an appropriate timing before passing the intersection IS. In addition, by referring to the road attributes of the map information 51, the process of calculating the road width based on the captured image can be omitted.

[0022] <Third Embodiment> An example of the processing method for calculating the occlusion rate (degree of occlusion) will be explained based on Figure 5. This example corresponds to processing (B) in Figure 2. When the processor 10 detects an intersection road CL1 that intersects with the road L1 on which the vehicle V1 is traveling, it calculates the intersection point PX of road L1 and intersection road CL1 (S21). As shown in Figure 5, the processor 10 sets two intersection points PX: intersection point PXR on the right side of the direction of travel VX of the vehicle V1 and intersection point PXL on the right side. The processor 10 calculates the intersection angle DR of road L1 and intersection road CL1 that intersect at intersection point PXR, and the intersection angle DL of road L1 and intersection road CL1 that intersect at intersection point PXL (S22). In order to calculate the area of ​​the image of a stationary object SO for determining the occlusion rate (degree of occlusion), the processor 10 sets a common projection plane onto which the image of each stationary object SO is projected. The processor 10 sets a bisector of the intersection angle DR (or intersection angle DL) between the track L1 and the intersection CL1 (S23), and sets a projection plane DGR (or projection plane DGL) that passes through the bisector HA and is perpendicular to the road surface of track L1 (S24). The angles DR1 and DR2 obtained by the bisector HA are equal. Preferably it is the bisector HA, but it may also be a line that bisects the intersection angle DR.

[0023] Figure 5 is a plan view from the sky, and for convenience the projection plane DGR (or projection plane DGL) is shown as a straight line, but the projection plane DGR (or projection plane DGL) is a plane that extends vertically upward from the road surface of the track L1. The length from intersection point PX can be set in advance. It is not particularly limited, but it may be set based on the length Q from intersection point PX, from which the calculation of the occlusion ratio (degree of occlusion) begins. The processor 10 sets the set area S on the projection plane DGR (or projection plane DGL) (S25). The processor 10 projects the image of the detected stationary object SOR onto the projection plane DGR (or projection plane DGL) from a predetermined reference viewpoint (S26). The predetermined reference viewpoint may be one or more. The processor 10 transforms the coordinates of the captured image based on the position of the camera 3 and projects it onto the projection plane DGR (or projection plane DGL). The projection method is not particularly limited and may be a perspective projection method in which an image is projected onto the projection plane DGR from a single reference viewpoint, or a parallel projection method in which an image is projected perpendicularly onto the projection plane DGR from multiple reference viewpoints. In this example, a parallel projection method is used in which the image of a stationary object SOR is projected perpendicularly onto the projection plane DGR. The processor 10 calculates the area (or number of pixels) occupied by the image of the stationary object SOR projected onto the projection plane DGR (S4). The processor 10 calculates the occupancy rate (degree of occlusion) of the image of the stationary object SOR with respect to a set area S on the projection plane DGR (or projection plane DGL) (S5). After that, the processing of S6-S9 described in the first embodiment is executed. In this way, by setting a common projection plane DGR (or projection plane DGL), the area of ​​the image of a stationary object SO in the captured images sequentially captured at different points by the camera 3 mounted on the vehicle V1 can be calculated with high accuracy. By accurately calculating the area of ​​the image of the stationary object SO, the occupancy rate (degree of occlusion) with respect to the set area S can also be calculated with high accuracy. This allows for a highly accurate determination of whether or not the visibility of the road L1 ahead of the vehicle V1 is good.

[0024] <Fourth Embodiment> Based on Figure 5, another example of the method for setting the projection plane DGR will be described. This example corresponds to the process (C) in Figure 2. When the processor 10 detects an intersection CL1 that intersects the road L1, it sets a rectangular target area SQ1 (or SQ2) containing the first side of the first distance Q1 along the road L1 and the second side of the second distance Q2 along the intersection CL1 to the right side of the direction of travel of the vehicle V1 (S31). In this example, the target area SQ1 is a square with a side length of 20m for the first distance Q1. The first distance Q1 may be set higher the higher the vehicle speed. The first distance Q1 may be set shorter the shorter the distance from the intersection PXR to the vehicle V1. The first distance Q1 and the second distance Q2 may be the same distance or different distances. The shape of the target area SQ is not particularly limited as long as it is rectangular, and may be a square, rectangle, parallelogram, or rhombus. The target area SQ1 on the right is a square, and the target area SQ2 is a parallelogram or a rhombus. The processor 10 sets a rectangular target area SQ3 in the area to the left of the vehicle V1. The shapes of the target areas SQ on the right and left may be the same or different. The processor 10 sets the diagonal DA of the target area SQ3 (S32). The processor 10 identifies the intersection point PXR of the track L1 and the crossing road CL1, and sets a projection plane DGR1 that passes from the intersection point PXR along the diagonal DA of the target area SQ1 and is perpendicular to the road surface of track L1 (S33). The processor 10 sets the set area S on the projection plane DGR1 (S34). Figure 5 is a plan view from a virtual viewpoint above, and for convenience the projection plane DGR1 is shown as a straight line, but in this example the projection plane DGR1 is a plane that extends vertically upward from the road surface of track L1. The processor 10 projects the image of the detected stationary object SOR onto the projection plane DGR1 from a predetermined reference viewpoint, similar to the third embodiment (S35). The processor 10 calculates the area (number of pixels) occupied by the image of the stationary object SOR projected onto the projection plane DGR1 (S4). The processor 10 calculates the occlusion rate (degree of occlusion) of the image of the stationary object SO with respect to a set region S on the projection plane DGR1 (S5). The projection method for the image of the stationary object SOR, the method for calculating the area (number of pixels) of the image of the stationary object SOR, and the method for calculating the occlusion rate (degree of occlusion) will be described herein by reference.Once the occlusion rate (degree of occlusion) is calculated, the processes S6-S9 described in the first embodiment are executed. In this way, by setting a rectangular target area SQ that includes the first side of the first distance Q1 along the road L1 and the second side of the second distance Q2 along the intersection CL1, the projection plane DGR1 can be set on the diagonal of the rectangular target area SQ. By defining the first distance Q1 as the distance at which the occlusion rate (degree of occlusion) calculation process should begin, the rectangular target area SQ can be set with a low processing load. Similar to the third embodiment, by setting a common projection plane DGR or projection plane DGL, the sum of the area of ​​the images of stationary objects SO in the images captured sequentially at different points by the camera 3 mounted on the vehicle V1 can be calculated with high accuracy. By accurately calculating the sum of the area of ​​the images of stationary objects SO, the occlusion rate (degree of occlusion) can also be calculated with high accuracy. This makes it possible to determine with high accuracy whether or not the visibility of the road L1 in front of the vehicle V1 is good.

[0025] <Fifth Embodiment> Based on Figure 6, an example of a method for calculating the occlusion rate (degree of occlusion) by setting an evaluation line DA and providing a setting area S on the evaluation line DA will be described. This processing example corresponds to processing (D) in Figure 2. As shown in Figure 6(A), when the processor 10 detects an intersection CL1 that intersects the road L1, it sets a rectangular target area SQ4 containing the first side of the first distance Q1 along the road L1 and the second side of the second distance Q2 along the intersection CL1 in the area to the right in the direction of travel of the vehicle V1 (S41). The target area SQ4 is a square. The first distance Q1 and the second distance Q2 may be the same distance or different distances. Although not shown in Figure 6(A), a target area SQ5 may also be set in the area to the left in the direction of travel of the vehicle V1. In the example shown in Figure 6(A), the target area SQ4 contains objects including stationary objects SO1 to SO4, and the camera 3 captures images of the stationary objects SO1 to SO4.

[0026] An example of an captured image is shown in (B1) of Figure 6(B). The processor 10 detects an image of a stationary object from the captured image. The processor 10 transforms the detected image of the stationary object from an aerial viewpoint to the coordinates of the target area SQ4 (S42). The image of the stationary object as seen from the aerial viewpoint is projected onto the target area SQ4. In this process, the bird's-eye view transformation method known at the time of filing can be used. The projected image onto the target area SQ4 shows the width and depth (shape of the plane or cross section) of the stationary object along the road surface. Figure 6(B)(B2) shows an example of the image of a stationary object as seen from an aerial viewpoint after coordinate transformation to the coordinates of the target area SQ4. In this example, the image of the stationary object is shown as a circular figure (symbol figure) with a diameter corresponding to its width.

[0027] The processor 10 identifies the intersection point PXR of the road L1 and the crossing road CL1, and sets the diagonal line DA of the target area SQ4 from the intersection point PXR (S43). The processor 10 sets the evaluation line DLR passing through the diagonal line DA, as shown in (B3) of Figure 6(B) (S44). In this example, the evaluation line DLR corresponds to the set area S, and the length of the evaluation line DLR becomes the denominator of the occlusion ratio (degree of occlusion). The processor 10 moves each of the coordinate-transformed images of the stationary objects so that the reference point of each image coincides with the evaluation line DLR. The process of moving is shown in (B3) of Figure 6(B). In this example, the processor 10 uses the center of the circular figure (symbol) corresponding to the image of the stationary object as the reference point, and translates each circular figure so that its center coincides with the evaluation line DLR (S45). The center of the figure corresponding to the image of the stationary object may also be used as the reference point. The processor 10 calculates the sum of the lengths of the images of one or more stationary objects that overlap with the evaluation line DLR (S46). The sum of the lengths of the images of one or more stationary objects that overlap with the evaluation line DLR is the sum of the line segments in which the image of any one of the stationary objects lies on the evaluation line DLR. If the images of two or more stationary objects overlap, the sum is calculated using the length in which the image of one of the stationary objects overlaps with the evaluation line DLR. In the example shown in (B3) of Figure 6(B), the images of six stationary objects SOa to SOf are detected, and their reference points are moved onto the evaluation line DLR. SOa and SOf do not overlap with the images of the other stationary objects, but the images of the four stationary objects SOb, SOc, SOd, and SOe overlap on the evaluation line DLR in part or in whole. The length over which the image SOa of a stationary object overlaps with the evaluation line DLR is 1m, the length over which the image SOf of a stationary object overlaps with the evaluation line DLR is 0.5m, and the total length over which any one of the four images of stationary objects SOb, SOc, SOd, and Soe overlaps with the evaluation line DLR is 2.5m. As shown in (B4) of Figure 6(B), the total length over which the images SOa to Sof of stationary objects overlap with the evaluation line DLR is 1m + 2.5m + 0.5m = 4m. The length of the evaluation line DLR is 14m, which is the diagonal of a square with sides of 10m. The processor 10 calculates the occlusion rate (degree of occlusion) as the ratio of the total length of the images of stationary objects to the length of the evaluation line DLR (S47). In the example in Figure 6(B), as shown in (B4), the occlusion rate (degree of occlusion) = 4m / 14m = 28.6%.

[0028] The processor 10 determines whether the calculated occlusion rate (degree of occlusion) is greater than or equal to a predetermined value (S6). The predetermined value of the occlusion rate (degree of occlusion) can be set to 70-90%, although this is not particularly limited. In this example, the predetermined value of the occlusion rate (degree of occlusion) is set to 80%. If the occlusion rate (degree of occlusion) is greater than or equal to the predetermined value (YES in S6), the processor determines that visibility ahead of the road L1 is poor (S7). Poor visibility ahead of the road L1 means that it is difficult to perceive the state of the intersection CL1 that intersects the road L1 ahead (the presence of moving objects traveling on the intersection CL1). Poor visibility ahead of the road L1 means that when the road L1 intersects with the intersection CL1 in the future, it will be difficult to perceive the traffic situation at that intersection IS. If the processor 10 determines that visibility ahead of the road L1 is poor (S7), it outputs the result of that determination (S8). On the other hand, if the shielding rate (degree of shielding) is less than a predetermined value (NO in S6), the process from S1 onwards is repeated.

[0029] In this way, by setting a rectangular target area SQ that includes the first side of the first distance Q1 along the road L1 and the second side of the second distance Q2 along the intersection CL1, the evaluation line DLR can be set using the diagonal DA of the rectangular target area SQ. The image of a stationary object detected from the captured image is coordinate-transformed from the aerial viewpoint to the coordinates of the target area SQ, and the density of stationary objects as seen from the vehicle V1 can be predicted by the distance (length) at which the circular figure corresponding to the plan view of the coordinate-transformed stationary object overlaps with the evaluation line DLR. A high density of stationary objects in the driver's field of view of the vehicle V1 and the detection area of ​​sensor 2 will hinder the driver's visibility or the detection of sensor 2, worsening the visibility of the road L1. By predicting the density of stationary objects visible from the vehicle V1, the visibility of the road L1 at the intersection IS to be passed can be determined with high accuracy. In this embodiment, the area occupied by a stationary object is converted into a planar figure (circle) viewed from above, and the length of the line segment where the planar figure intersects with the evaluation line DLR is calculated. This allows for the determination of the density of stationary object images included in the video footage visible from the vehicle V1 traveling along the road L1. The length over which the planar figure and the evaluation line DLR overlap corresponds to the width of the stationary object along the evaluation line DLR (the area that is obscured). By positioning the image of the stationary object as an obstruction that blocks the driver's view of the vehicle V1, and considering the length over which the extracted image of the stationary object and the evaluation line DLR overlap as the width obscured by the stationary object, the area of ​​the image of the stationary object SO in the captured images sequentially captured by camera 3 at different points can be calculated with high accuracy. By accurately calculating the area of ​​the image of the stationary object SO, the occupancy rate (degree of occlusion) can also be calculated with high accuracy, allowing for a highly accurate determination of whether or not the visibility of the road L1 ahead of the vehicle V1 is good. Furthermore, this method reduces the processing load compared to calculating the density of stationary objects by sequentially processing images captured by the camera 3 mounted on the vehicle V1 traveling along a track L1 of stationary objects.

[0030] The degree of occlusion is a value that indicates the extent to which a stationary object obstructs the field of view of the driver of the vehicle V1 and / or the range from which the detection signal of sensor 2 is acquired. The degree of occlusion is a value that indicates the degree of poor visibility of the intersecting road CL1 at the intersection IS. Of course, the degree of occlusion can be determined based on the ratio (occlusion rate) of the image K of a stationary object to a predetermined set area S (area or line segment), as in the first to fifth embodiments. The processor 10 divides the predetermined set area S into multiple areas based on priority, and if there is at least one object determined to be a stationary object in a high-priority area, it can determine that the degree of occlusion is high. Furthermore, the processor 10 divides the predetermined set area S into multiple areas based on priority, defines the number of stationary objects that constitute a threshold TH according to the priority, and determines that the degree of occlusion is high if the number of stationary objects in each area is equal to or greater than the threshold TH. Here, priority can be defined according to the distance from the intersection IS. Areas that are close to the intersection (IS) are given higher priority than areas that are farther from the intersection (IS).

[0031] 100...Driving support system, 1...Driving support device, 10...Processor, 11...CPU, 12...ROM, 13...RAM, 20...Communication device, 2...Sensor, 21...Radar device, 22...Position detection device, 3...Camera, 31...In-cabin camera, 4...Vehicle information acquisition device, 5...Navigation device, 51...Map information, 511...Intersection information, 52...Display, 53...Speaker, 6...External server, 61...Communication device, 62...Storage device, 63...Calculation unit, 200...Vehicle controller, 210...Steering control device, 220...Drive control device, 230...Brake control device

Claims

1. A driving support method used in a processor to assist the driving of the vehicle, wherein the processor uses a camera mounted on the vehicle to acquire images of the right and / or left side along the vehicle's path, detects stationary objects present to the side of the path from the captured images, calculates the degree of occlusion of the detected stationary object's image over a predetermined set area, and determines that the visibility of the path is poor if the degree of occlusion is greater than or equal to a predetermined value, and outputs the determination result.

2. The driving support method according to claim 1, wherein, when an intersection road intersects the driving path, the processor sets a projection plane that passes through the bisector of the intersection angle between the driving path and the intersection road and is perpendicular to the road surface of the driving path, sets the set area on the projection plane, projects the image of the detected stationary object from a predetermined reference viewpoint onto the projection plane, and calculates the degree of occlusion of the image of the stationary object with respect to the set area on the projection plane.

3. The driving support method according to claim 1, wherein, when an intersection road intersecting the driving path is detected, the processor sets a rectangular target area including a first side of a first distance along the driving path and a second side of a second distance along the intersection road, identifies the intersection point of the driving path and the intersection road, sets a projection plane perpendicular to the road surface of the driving path, passes through the diagonal of the target area from the intersection point, sets the set area on the projection plane, projects the image of the detected stationary object onto the projection plane from a predetermined reference viewpoint, and calculates the degree of occlusion of the image of the stationary object with respect to the set area on the projection plane.

4. The driving support method according to claim 1, wherein, when an intersection road intersecting the driving path is detected, the processor sets a rectangular target area including a first side of a first distance along the driving path and a second side of a second distance along the intersection road, transforms the image of the detected stationary object from an aerial viewpoint to the coordinates of the target area, identifies the intersection point of the driving path and the intersection road, sets an evaluation line passing from the intersection point through the diagonal of the target area, moves each of the coordinate-transformed images of the stationary object so that the reference point of the image of the stationary object coincides with the evaluation line, calculates the sum of the lengths of one or more images of the stationary object that coincide with the evaluation line, and calculates the degree of occlusion as the ratio of the sum of the lengths of the images of the stationary objects to the length of the evaluation line.

5. The driving assistance method according to any one of claims 1 to 4, wherein the processor refers to map information and calculates the degree of occlusion when the distance between the vehicle and the intersection becomes less than a predetermined distance, or when the time until the vehicle reaches the intersection becomes less than a predetermined time.

6. The driving assistance method according to any one of claims 1 to 5, wherein the processor refers to map information and calculates the degree of occlusion when the road width of the road is less than a predetermined value, the road is a single lane, or the road is one-way.

7. The driving assistance method according to any one of claims 1 to 6, wherein the processor determines that visibility of the road is poor, and then presents driving assistance information, including an extended image of an intersection where the road intersects, using a display.

8. The driving support method according to claim 7, wherein the processor refers to map information and presents the driving support information when the distance between the vehicle and the intersection ahead becomes less than a predetermined distance, or when the time until the vehicle reaches the intersection becomes less than a predetermined time.

9. The driving support method according to any one of claims 1 to 8, wherein the setting area is defined at a predetermined height from the road surface of the running path.

10. The driving assistance method according to any one of claims 1 to 9, wherein the processor uses the vehicle's interior camera to capture an image of the driver's head, calculates the height of the driver's head based on the captured image of the head, and sets a predetermined height that defines the position of the height of the set area based on the height of the head.

11. A driving support device equipped with a processor that assists the driving of the vehicle, wherein the processor uses a camera mounted on the vehicle to acquire images of the right and / or left side along the vehicle's path, detects stationary objects present to the side of the path from the captured images, calculates the degree of occlusion of the detected stationary object's image relative to a predetermined set area, and determines that visibility ahead of the path is poor if the degree of occlusion is greater than or equal to a predetermined value, and outputs the determination result.

12. A program that causes a processor in a driving assistance system's computer to perform driving assistance control of the vehicle, the program causing the processor to perform the following processes: acquiring images of the right and / or left side along the vehicle's road using a camera mounted on the vehicle; detecting stationary objects located to the side of the road from the acquired images; calculating the degree of occlusion of the detected images of the stationary objects relative to a predetermined set area; determining that visibility ahead of the road is poor if the degree of occlusion is greater than or equal to a predetermined value; and outputting the determination result.

Images