Driving support device and driving support method

Abstract

The present invention provides a driver assistance system that is less susceptible to the influence of the type and presence or absence of road markings and installations at intersections on the accuracy of its entry and exit determination. [Solution] The driving support device 1 includes an image recognition unit 15 that acquires information indicating the positions of stop lines, pedestrian crossings, lane lines, and traffic lights in the vehicle's lane and the opposing lane from an image captured in front of the vehicle; a first determination unit 21 that determines from the position of the stop line or pedestrian crossing in the vehicle's lane whether it can be estimated that the vehicle has entered an intersection; a second determination unit 23 that determines from the position of traffic lights or lane lines whether it can be estimated that the vehicle has entered an intersection if the first determination unit 21 cannot make a determination; a third determination unit 25 that determines from the position of the stop line or pedestrian crossing in the opposing lane whether it can be estimated that the vehicle has exited the intersection; and a fourth determination unit 27 that determines from the position of traffic lights or lane lines whether it can be estimated that the vehicle has exited the intersection if the third determination unit 25 cannot determine that the vehicle has exited the intersection.

Description

Technical Field

[0001] The present invention relates to a driving support device and a driving support method.

Background Art

[0002] Conventionally, there has been proposed a driving support device that determines, based on the color of a traffic signal when the host vehicle enters an intersection, whether the host vehicle has traveled safely through the intersection, specifically whether the host vehicle has forcedly entered the intersection during a yellow signal or a red signal, and stores the determination result. In such a driving support device, the position of a road marking drawn on the road surface of an intersection such as a stop line, or the position of an installation that regulates the traffic of the host vehicle such as a traffic signal, is identified from an image captured by an in-vehicle camera or the like, so as to determine whether the host vehicle has entered the intersection (Patent Document 1).

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of Invention

Problem to be Solved by the Invention

[0004] However, when determining entry into an intersection from a road marking as in Patent Document 1, there is a problem that the determination is difficult to make when the road marking is faded and invisible, or when the road marking is hidden by another vehicle and invisible. In addition, when determining entry into an intersection from the position of an installation that regulates the traffic of the host vehicle, there is a problem that it is difficult to determine entry into an intersection where no installation is provided, such as an intersection without a traffic signal.

[0005] The present invention has been made to solve such problems, and an object thereof is to provide a driving support device and a driving support method in which the accuracy of entry and exit determination for an intersection is less likely to be affected by the type or presence of road markings and installations at the intersection.

Means for Solving the Problem

[0006] The driver assistance device of the present invention includes a position acquisition unit that acquires information from an image captured in front of the vehicle, including the position of the stop line for the vehicle's lane, the stop line for the oncoming lane, and the pedestrian crossing drawn on the road surface, the position of lane lines drawn on the road surface adjacent to the left and right of the vehicle's lane, and the position of traffic signals that manage the vehicle's traffic; a first determination unit that determines from the information indicating the position of the stop line for the vehicle's lane or the pedestrian crossing whether it can be estimated that the vehicle has entered an intersection; a second determination unit that, if the first determination unit cannot determine whether it can be estimated that the vehicle has entered an intersection, determines from the information indicating the position of the traffic signals or the lane lines whether it can be estimated that the vehicle has entered an intersection; and if it can be estimated that the vehicle has entered an intersection The system includes: a third determination unit that, when the first determination unit or the second determination unit determines that it can be estimated that the vehicle has exited the intersection, determines from information indicating the position of the stop line of the opposing lane or the pedestrian crossing, and if it determines that it can be estimated that it can be estimated, it deems that the vehicle has entered and exited the intersection; and a fourth determination unit that, when the first determination unit or the second determination unit determines that it can be estimated that the vehicle has entered the intersection, and the third determination unit cannot determine whether it can be estimated that the vehicle has exited the intersection, determines from information indicating the position of the traffic light or the position of the lane lines, and if it determines that it can be estimated that it can be estimated, it deems that the vehicle has entered and exited the intersection.

[0007] Furthermore, the driving assistance method of the present invention includes a position acquisition step of acquiring information from an image taken in front of the vehicle, which includes the position of the stop line for the vehicle's lane, the stop line for the oncoming lane, and the pedestrian crossing drawn on the road surface, the position of the lane lines drawn on the road surface adjacent to the left and right of the vehicle's lane, and the position of a traffic signal that manages the vehicle's traffic; a first determination step of determining whether it can be estimated that the vehicle has entered an intersection based on the information indicating the position of the stop line for the vehicle's lane or the pedestrian crossing; a second determination step of determining whether it can be estimated that the vehicle has entered an intersection based on the information indicating the position of the traffic signal or the lane lines if it cannot be determined in the first determination step; and if it can be estimated that the vehicle has entered an intersection, The system includes a third determination step in which, if a determination is made in the first determination step or the second determination step, it is determined from information indicating the position of the stop line of the opposing lane or the pedestrian crossing whether it can be estimated that the vehicle has exited the intersection, and if it is determined that it can be estimated, the system deems that the vehicle has entered and exited the intersection; and a fourth determination step in which, if a determination is made in the first determination step or the second determination step that it can be estimated that the vehicle has entered the intersection, and if it is not determined in the third determination step whether it can be estimated that the vehicle has exited the intersection, it is determined from information indicating the position of the traffic light or the position of the lane lines whether it can be estimated that the vehicle has exited the intersection, and if it is determined that it can be estimated, the system deems that the vehicle has entered and exited the intersection. [Effects of the Invention]

[0008] According to the present invention, it is possible to provide a driving assistance device and a driving assistance method in which the accuracy of determining entry into and exit from an intersection is less affected by the type and presence or absence of road markings and installations at the intersection. [Brief explanation of the drawing]

[0009] [Figure 1] This is a functional block diagram of a driver assistance device according to an embodiment of the present invention. [Figure 2] This figure shows an example of an image captured by the front camera in Figure 1. [Figure 3] This figure schematically shows a plan view of the intersection in Figure 2. [Figure 4] This is a flowchart illustrating a driver assistance method using a driver assistance device, specifically showing the entry determination. [Figure 5] This is a flowchart showing the details of S7 in Figure 4. [Figure 6] This is a flowchart illustrating a driver assistance method using a driver assistance device, specifically showing the exit determination. [Modes for carrying out the invention]

[0010] The present invention will be described below in accordance with preferred embodiments. It should be noted that the present invention is not limited to the embodiments shown below, and can be modified as appropriate without departing from the spirit of the invention. Furthermore, in the embodiments shown below, some illustrations and descriptions of certain components are omitted. It goes without saying that, regarding the details of the omitted technologies, publicly known or well-known technologies are applied as appropriate, to the extent that they do not contradict the content described below.

[0011] First, the schematic configuration of the driver assistance device according to an embodiment of the present invention will be described with reference to Figures 1 to 3. Figure 1 is a functional block diagram of the driver assistance device according to an embodiment of the present invention. Figure 2 is a diagram showing an example of an image captured by the forward camera in Figure 1. Figure 3 is a schematic diagram showing the plan view of the intersection in Figure 2.

[0012] The driver assistance device 1 shown in Figure 1 is mounted on a vehicle (the vehicle itself) and is a device that assists in driving the vehicle. It comprises a control unit 3, a storage unit 5, and a communication unit 7. The control unit 3 is the part of the driver assistance device 1 that performs calculations and decisions, and comprises an image recognition unit 15 (position acquisition unit), a calculation unit 17, and a driving decision unit 19. The image recognition unit 15 is the part that acquires information indicating the position of road markings drawn on the road surface (for example, the road surface of road 40 in Figure 2) while the vehicle is traveling, and installations that manage the vehicle's traffic (for example, traffic lights). An image 30 of the area in front of the vehicle, as shown in Figure 2 and captured by the front camera 9, is input to the image recognition unit 15. Image 30 is an example of an image taken by the front camera 9 of the area in front of the vehicle just before the vehicle enters the intersection 31. The front camera 9 is an imaging device installed at the front end of the vehicle that captures images of the area in front of the vehicle. The image recognition unit 15 acquires information from image 30 indicating the positions of the stop line 45 of the vehicle's own lane 41 and the stop line 47 of the opposing lane 43, as well as information indicating the positions of pedestrian crossings 48 and 49, using known image analysis techniques such as an AI (Artificial Intelligence) engine. The image recognition unit 15 also acquires information from image 30 indicating the positions of lane lines 55 and 57 drawn on the road surface adjacent to the left and right of the vehicle's own lane 41, as well as information indicating the positions of traffic signals 51 and 53 that manage the vehicle's traffic, using known image analysis techniques. In the following description, this information indicating positions will be collectively referred to as position information.

[0013] The calculation unit 17 shown in Figure 1 is responsible for determining whether the vehicle has entered or exited the intersection 31 based on the information acquired by the image recognition unit 15, and receives the position information acquired by the image recognition unit 15. The calculation unit 17 includes a first determination unit 21, a second determination unit 23, a third determination unit 25, a fourth determination unit 27, a fifth determination unit 28, a first acquisition unit 29, a second acquisition unit 34, and a sixth determination unit 32.

[0014] The first determination unit 21 is the part that determines whether it can be estimated that the vehicle has entered the intersection 31 based on information indicating the position of the vehicle's stop line 45 or the position of the pedestrian crossing 48, as shown in Figure 2. Specifically, the first determination unit 21 first determines that the vehicle has crossed the stop line 45 of its own lane 41 when the stop line 45 that was visible in the image 30 disappears from the image 30 (goes out of frame), and estimates that the vehicle has entered the intersection 31. More precisely, at the moment the vehicle crosses the stop line 45 of its own lane 41, the vehicle is traveling in the area 63 in front of the stop line 45, between the intersection area 61 where the road 40 the vehicle is traveling on intersects with another road 59, as shown in Figure 3. However, the first determination unit 21 estimates that the vehicle has entered the intersection 31 even when the vehicle is traveling in the area 63 in front of the stop line 45. The reason is that if your vehicle crosses stop line 45, there is a very high probability that you will pass through intersection 31.

[0015] If the first determination unit 21 cannot determine from the information indicating the position of the vehicle's stop line 45 whether it can be estimated that the vehicle has entered the intersection 31, it determines from the position of the pedestrian crossing 48 whether it can be estimated that the vehicle has entered the intersection 31. Specifically, if the pedestrian crossing 48, the closer of the pedestrian crossings 48 and 49 installed at the intersection 31, disappears from the image 30 (goes out of frame), it is estimated that the stop line 45 was located a predetermined distance in front of the pedestrian crossing 48 in the direction of travel. In this case, it is also estimated that the vehicle has crossed the stop line 45 and entered the intersection 31. This is because, normally, the pedestrian crossing 48 is installed closer to the intersection area 61 than the stop line 45, so if the vehicle passes the pedestrian crossing 48, it can be estimated that it has also crossed the stop line 45, and therefore it can be estimated that the vehicle has entered the intersection 31. Furthermore, if it is not possible to determine from the information indicating the position of the vehicle's stop line 45 whether the vehicle has entered intersection 31, this may be because, for example, the stop line 45 is blurred and the image recognition unit 15 cannot recognize it. Alternatively, another vehicle traveling in front of the vehicle is moving between the front camera 9 and the stop line 45, and the stop line 45 is hidden by the other vehicle and not visible in image 30.

[0016] The second determination unit 23 is the part that determines whether it can be estimated that the vehicle has entered the intersection 31 when the first determination unit 21 cannot determine whether it has entered the intersection 31, based on information indicating the positions of the traffic lights 51 and 53 or the lane lines 55 and 57. The first determination unit 21 cannot determine whether it can be estimated that the vehicle has entered the intersection 31 when, for example, the stop line 45 or the pedestrian crossing 48 is faded and the image recognition unit 15 cannot recognize the stop line 45 or the pedestrian crossing 48. Alternatively, the stop line 45 or the pedestrian crossing 48 may be hidden by other vehicles and not appear in the image 30. Furthermore, the stop line 45 or the pedestrian crossing 48 may not be drawn on the road surface at the entrance to the intersection 31. In such cases, the second determination unit 23 determines whether it can be estimated that the vehicle has entered the intersection 31 based on information indicating the positions of traffic lights 51 and 53 or information indicating the positions of lane lines 55 and 57. If it determines that it can be estimated, it estimates that the vehicle has entered the intersection 31. More specifically, the second determination unit 23 first estimates that the vehicle has entered the intersection 31 if traffic light 51, which is installed closer to the viewer than traffic lights 51 and 53, moves out of frame from the image 30.

[0017] It should be noted that, near the intersection 31, there may be installed traffic signals that illuminate in the same colors as the traffic signals 51 and 53 installed at the intersection 31, such as advance notice traffic signals. The intersection 31 is also provided with traffic signals that regulate the traffic of vehicles traveling on another road 59 intersecting with the own lane 41. In this case, it is preferable that the second determination unit 23 does not estimate that the own vehicle has entered the intersection 31 when the advance notice traffic signals or the traffic signals for regulating vehicles traveling on the other road 59 are out of the frame from the image 30. On the other hand, the traffic signals 51 and 53 for the own lane 41 installed at the intersection 31 are illuminated in the same color. Further, as shown in FIG. 2, regarding the traffic signals 51 and 53 for the own lane 41, in the image 30, the traffic signal 51 on the near side is located on the right side of the FOE (Focus of Expansion, also referred to as the vanishing point), and the traffic signal 53 on the far side is located near the FOE. Therefore, when a plurality of traffic signals 51 and 53 illuminated in the same color are captured in the image 30, the second determination unit 23 preferably identifies the traffic signal 51 on the right side of the FOE as the near-side traffic signal 51, and identifies the traffic signal 53 near the FOE as the far-side traffic signal 53. It should be noted that the vicinity of the FOE specifically refers to a position where the XY coordinates are within a predetermined threshold value from the coordinates of the FOE.

[0018] It should be noted that, as in the case where the traffic signals 51 and 53 are not installed at the intersection 31, the second determination unit 23 may not be able to determine whether it is possible to estimate that the own vehicle has entered the intersection 31 from the information indicating the positions of the traffic signals 51 and 53. In this case, the second determination unit 23 determines whether it is possible to estimate that the own vehicle has entered the intersection 31 from the information indicating the positions of the lane lines 55 and 57. Specifically, since the lane lines 55 and 57 are not drawn within the intersection region 61 of the intersection 31 shown in FIG. 3, when the lane lines 55 and 57 go out of the frame from the image 30 (when they are interrupted), it is estimated that the own vehicle has entered the intersection 31. The lane lines 55 and 57 referred to herein mean boundary lines drawn on the road surface at both left and right ends of the own lane 41.

[0019] The third determination unit 25 is responsible for determining whether the vehicle entered and exited the intersection 31. Specifically, if the first determination unit 21 or the second determination unit 23 determines that it can be estimated that the vehicle entered the intersection 31, the third determination unit 25 determines whether it can be estimated that the vehicle entered and exited the intersection 31. More specifically, the third determination unit 25 determines whether it can be estimated that the vehicle exited the intersection 31 based on information indicating the position of the stop line 47 of the opposing lane 43 or the pedestrian crossing 49. If it determines that it can be estimated that the vehicle exited the intersection 31, the third determination unit 25 considers that the vehicle entered and exited the intersection 31. Specifically, first, if the stop line 47 goes out of frame from the image 30, the third determination unit 25 estimates that the vehicle has crossed the stop line 47 and exited the intersection 31, and considers that the vehicle entered and exited the intersection 31. On the other hand, if the stop line 47 is faded or otherwise indistinct, the image recognition unit 15 cannot recognize the position of the stop line 47, it cannot determine from the stop line 47 of the opposing lane 43 whether it can be estimated that the vehicle has exited the intersection 31. In this case, the third determination unit 25 determines that it can be estimated that the vehicle has passed the traffic light 51 and exited the intersection 31 when the far-side pedestrian crossing 49 goes out of frame from the image 30, and considers that the vehicle has entered and exited the intersection 31. At the time the stop line 47 and pedestrian crossing 49 go out of frame, the vehicle is traveling in the far-side area 66, which is the area between the intersection area 61 and the stop line 47. However, the third determination unit 25 determines that it can be estimated that the vehicle has exited the intersection 31 even when the vehicle is traveling in the far-side area 66. The reason is that when the vehicle is traveling in the far-side area 66, it has already exited the intersection area 61, and it can be estimated that the vehicle has exited the intersection 31.

[0020] The fourth determination unit 27 is a section that estimates whether the host vehicle has exited the intersection 31 when the third determination unit 25 cannot estimate whether the host vehicle has exited the intersection 31. Specifically, the fourth determination unit 27 determines whether it can be estimated that the host vehicle has exited the intersection 31 when the following conditions are satisfied. First, the first condition is a case where it is determined that the first determination unit 21 or the second determination unit 23 can estimate that the host vehicle has entered the intersection 31. The second condition is a case where the third determination unit 25 cannot determine whether it can be estimated that the host vehicle has exited the intersection 31. The case where the third determination unit 25 cannot determine whether it can be estimated that the host vehicle has exited the intersection 31 is, for example, a case where the stop line 47 or the crosswalk 49 on the oncoming lane 43 is blurred, and the image recognition unit 15 cannot identify it from the image 30. Alternatively, it is a case where the stop line 47 and the crosswalk 49 are not drawn on the road surface at the exit of the intersection 31. When such conditions are satisfied, first, the fourth determination unit 27 estimates that the host vehicle has exited the intersection 31 when the traffic signal 51 installed on the far side among the traffic signals 51 and 53 is out of frame, and regards that the host vehicle has entered and exited the intersection 31.

[0021] On the other hand, as in the case where the traffic signal 53 is not installed at the intersection 31, there is a case where the fourth determination unit 27 cannot determine whether it can be estimated that the host vehicle has exited the intersection 31 from the information indicating the position of the traffic signal 53. In this case, the fourth determination unit 27 determines whether it can be estimated that the host vehicle has exited the intersection 31 from the information indicating the positions of the lane lines 55 and 57. Specifically, when the host vehicle passes the position where the lane lines 55 and 57 that were interrupted at the intersection 31 start to be drawn again, it is estimated that the host vehicle has exited the intersection 31, and it is regarded that the host vehicle has entered and exited the intersection 31.

[0022] Thus, the calculation unit 17 of the control unit 3 includes a first determination unit 21, a second determination unit 23, a third determination unit 25, and a fourth determination unit 27. In this configuration, if it is estimated from the positional information of the stop lines 45, 47, the pedestrian crossings 48, 49, the lane lines 55, 57, and the traffic lights 51, 53 that the vehicle has entered and exited the intersection 31, then the vehicle is considered to have entered and exited the intersection 31. Therefore, the accuracy of the entry / exit determination is less affected by the type or presence of road markings such as the stop lines 45, 47, the pedestrian crossings 48, 49, and the lane lines 55, 57, or installed objects such as the traffic lights 51, 53 at the intersection 31. Furthermore, even if some information of the road markings such as the stop lines 45, 47, the pedestrian crossings 48, 49, and the lane lines 55, 57, or the traffic lights 51, 53 cannot be obtained, the entry / exit determination can still be made if other information can be obtained.

[0023] The first acquisition unit 29 is the part that acquires the first time from information indicating the position of the stop line 45 of the vehicle's lane 41 or the position of the pedestrian crossing 48 when the first determination unit 21 determines that it can be estimated that the vehicle has entered the intersection 31. The first time is the time when it is estimated that the vehicle has entered the intersection 31. Specifically, based on the position of the stop line 45, the first acquisition unit 29 acquires the time when the vehicle has crossed a predetermined distance beyond the position of the stop line 45 as the first time when the first determination unit 21 determines that it can be estimated that the vehicle has entered the intersection 31. Alternatively, based on the position of the pedestrian crossing 48, the first acquisition unit 29 acquires the time when the vehicle has crossed a predetermined distance beyond the position of the pedestrian crossing 48 as the first time when the vehicle has crossed a predetermined distance beyond the position of the pedestrian crossing 48 as the first time when the first determination unit 21 determines that it can be estimated that the vehicle has entered the intersection 31.

[0024] The second acquisition unit 34 is the part that acquires a second time from information indicating the position of the stop line 47 on the opposing lane 43 or the position of the pedestrian crossing 49 when the third determination unit 25 determines that it can be estimated that the vehicle has exited the intersection 31. The second time is the time when it is estimated that the vehicle has exited the intersection 31. Specifically, based on the position of the stop line 47, the second acquisition unit 34 acquires the time when the vehicle has crossed a predetermined distance in front of the stop line 47 as the second time when the third determination unit 25 determines that it can be estimated that the vehicle has exited the intersection 31. Alternatively, based on the position of the pedestrian crossing 49, the second acquisition unit 34 acquires the time when the vehicle has crossed a predetermined distance in front of the pedestrian crossing 49 as the second time when the third determination unit 25 determines that it can be estimated that the vehicle has exited the intersection 31.

[0025] As described above, the control unit 3 includes a first acquisition unit 29 and a second acquisition unit 34. In this configuration, the estimated time when the vehicle entered the intersection 31 and the estimated time when the vehicle exited the intersection 31 can be obtained. Therefore, not only can the presence or absence of entry into and exit from the intersection 31 be determined, but the timing of entry into and exit from the intersection 31 can also be determined. Furthermore, knowing the timing of entry into and exit from the intersection 31 allows for the estimation of the length of the intersection 31 in the direction of travel of the vehicle. If the length of the intersection 31 can be estimated, for example, if the vehicle entered the intersection 31 before or after the traffic lights 51 and 53 turned red, the distance the vehicle traveled within the intersection 31 under red light conditions can also be estimated. The longer the distance the vehicle traveled within the intersection 31 under red light conditions, the higher the probability of running a red light. Therefore, estimating the length of the intersection 31 can improve the accuracy of determining whether or not a red light was run.

[0026] The fifth determination unit 28 is the part that determines whether the vehicle has turned right or left within the intersection 31 and exited the intersection 31. Specifically, the fifth determination unit 28 acquires right / left turn information, which is information regarding whether the vehicle has turned right or left, from the G sensor 11, which is a known in-vehicle acceleration sensor. Next, the fifth determination unit 28 estimates that the vehicle has exited the intersection 31 and considers that the vehicle has entered and exited the intersection 31 if the following conditions are met. First, the first condition is that the first determination unit 21 or the second determination unit 23 determines that it can be estimated that the vehicle has entered the intersection 31. The second condition is that the right / left turn information indicates that the vehicle is in the process of turning right or left. In this configuration, if the vehicle turns right or left within the intersection 31 after it has been determined that it can be estimated that the vehicle has entered the intersection 31, it is considered that the vehicle has exited the intersection 31. Therefore, even if the road 40 into which the vehicle entered and the road 59 from which it exited intersect at intersection 31, the exit can be estimated.

[0027] The sixth determination unit 32 is responsible for processing cases where none of the third determination unit 25, the fourth determination unit 27, or the fifth determination unit 28 can determine whether the vehicle entered or exited the intersection 31. Specifically, the sixth determination unit 32 considers that the vehicle did not enter the intersection 31 if the following conditions are met. The first condition is when the first determination unit 21 or the second determination unit 23 determines that it can be estimated that the vehicle entered the intersection 31. The second condition is when, after the first condition is met, none of the third determination unit 25, the fourth determination unit 27, or the fifth determination unit 28 can estimate that the vehicle exited within a predetermined time. A case where entry can be estimated but exit cannot be estimated is, for example, when the image recognition unit 15 mistakenly recognizes a stop line or pedestrian crossing drawn on a road other than the intersection 31 as being drawn on the intersection 31, and an entry is incorrectly estimated. Alternatively, the image recognition unit 15 may mistakenly recognize a traffic light installed on a road other than intersection 31 as a traffic light installed at intersection 31, leading to an incorrect estimation of entry. Even in such cases of incorrect estimation of entry, the sixth determination unit 32 will consider that the vehicle did not enter intersection 31 if it cannot estimate that the vehicle has exited intersection 31. Therefore, the possibility of the system incorrectly determining that the vehicle entered and exited intersection 31 when the vehicle passes a stop line, pedestrian crossing, or traffic light drawn on a road other than intersection 31 is reduced.

[0028] The driving determination unit 19 is responsible for determining whether the driver of the vehicle ran a red light or entered the intersection 31 recklessly, when the calculation unit 17 determines that the vehicle has entered or exited the intersection 31. Specifically, the driving determination unit 19 determines whether a red light was run using known means, but as an example, it makes the following determination. First, when the calculation unit 17 determines that the vehicle has entered or exited the intersection 31, if the traffic lights 51 and 53 in image 30 were red, the driving determination unit 19 can determine that there is a possibility that the vehicle ran a red light and entered the intersection 31. Also, if the first time has been acquired, the driving determination unit 19 can acquire the vehicle's speed and other information as vehicle information 13 from an in-vehicle device (not shown) or a monitoring server that monitors the vehicle's movement, and can determine whether the vehicle was at a speed that allowed it to stop at the stop line 45. If the vehicle was traveling at a speed that would allow it to stop at stop line 45, the driving determination unit 19 can determine that the vehicle entered intersection 31 ignoring the signal and failing to stop at stop line 45, even though it could have. On the other hand, if the vehicle was traveling at a speed that would not allow it to stop at stop line 45, the driving determination unit 19 can also determine that the vehicle entered intersection 31 on a red light because it was unable to stop at stop line 45.

[0029] Furthermore, if the calculation unit 17 determines that the vehicle has entered or exited intersection 31, and the traffic lights 51 and 53 in image 30 were yellow, the driving determination unit 19 can determine that the vehicle may have entered intersection 31 recklessly. In particular, if the first time was acquired, and the vehicle was at a speed that would have allowed it to stop at stop line 45, the driving determination unit 19 can determine that the vehicle could have stopped at stop line 45 but instead entered intersection 31 recklessly on a yellow light. If the driving determination unit 19 determines that the vehicle ran a red light or entered intersection 31 recklessly when entering intersection 31, it may issue a warning to the driver of the vehicle using an on-board speaker (not shown).

[0030] The memory unit 5 stores information from the image 30 captured by the front camera 9, information indicating the location of road markings and installed objects acquired by the image recognition unit 15 based on the image 30, and information indicating the judgment results of the calculation unit 17 and the driving judgment unit 19.

[0031] The communication unit 7 is the part of the driver assistance device 1 that communicates with external communication equipment. For example, the communication unit 7 transmits and notifies the monitoring server that monitors the vehicle's movement of the driving judgment unit 19, specifically, information indicating whether or not the vehicle ran a red light or entered a certain intersection 31 illegally.

[0032] The driver assistance device 1 may be implemented as a function provided by a known computer, a known drive recorder, or a digital tachometer, or it may be implemented as a dedicated device. For example, if the driver assistance device 1 is implemented by a known computer, the control unit 3 is a central processing unit, and the storage unit 5 is a device that stores programs for the control unit 3 to function as an image recognition unit 15, a calculation unit 17, and a driving judgment unit 19. The above is a description of the configuration of the driver assistance device 1.

[0033] Next, the driving assistance method using the driving assistance device 1, specifically the method for determining entry and exit to intersection 31, will be explained with reference to Figures 4 to 6. Figure 4 is a flowchart of the driving assistance method using the driving assistance device 1, showing the entry determination. Figure 5 is a flowchart showing the details of S7 in Figure 4. Figure 6 is a flowchart of the driving assistance method using the driving assistance device 1, showing the exit determination.

[0034] First, the procedure for determining the entrance (estimating whether the vehicle has entered intersection 31) will be explained with reference to Figures 4 and 5. First, it is assumed that the front camera 9 is capturing images of the area in front of the vehicle at a predetermined imaging interval (frame rate). In this case, the image recognition unit 15 acquires an image 30 from the front camera 9 and obtains position information from the acquired image 30 (S1 in Figure 4, position acquisition process). Specifically, the image recognition unit 15 acquires information indicating the positions of the stop line 45 on the vehicle's own lane 41 and the stop line 47 on the opposing lane 43, the positions of pedestrian crossings 48 and 49, the positions of lane lines 55 and 57, and the positions of traffic lights 51 and 53.

[0035] Next, the first determination unit 21 determines whether it can be estimated that the vehicle has entered the intersection 31 based on information indicating the position of the stop line 45 or pedestrian crossings 48 and 49 in the vehicle's lane 41. Specifically, first, the first determination unit 21 determines from the information acquired by the image recognition unit 15 whether there is a stop line 45 ahead of the vehicle's lane 41. If it determines that there is a stop line 45, it proceeds to S3; otherwise, it proceeds to S4 (S2 in Figure 4, first determination step). If it determines in S2 that there is a stop line 45, the first determination unit 21 determines that it can be estimated that the vehicle has entered the intersection 31 when the stop line 45 leaves the frame of the image 30. Specifically, it causes the first acquisition unit 29 to acquire the first time based on the position of the stop line 45 (S3 in Figure 4). Furthermore, the first determination unit 21 turns on the intersection flag and proceeds to the exit determination (S5 in Figure 4). Here, the intersection flag indicates whether or not it is presumed that the vehicle has entered intersection 31. If it is "ON", it is presumed that the vehicle has entered intersection 31, and if it is "OFF", it is presumed that the vehicle has not entered intersection 31.

[0036] If it cannot be determined in S2 that there is a stop line 45 ahead of the vehicle's lane 41, the first determination unit 21 determines whether there is a pedestrian crossing 48 ahead of the vehicle. If it determines that there is, it proceeds to S6; otherwise, it proceeds to S7 (S4 in Figure 4, first determination step). If the positions of two pedestrian crossings 48 and 49 are identified as shown in Figure 2, it determines whether there is a pedestrian crossing 48 drawn on the near side of the road 40 the vehicle is traveling on. If it is determined in S4 that there is a pedestrian crossing 48, the first determination unit 21 determines that it can be estimated that the vehicle entered the intersection 31 when the intersection 31 went out of frame from the image 30. In this case, the first determination unit 21 causes the first acquisition unit 29 to acquire the first time based on the position of the pedestrian crossing 48 (S6 in Figure 4), proceeds to S5, turns on the intersection flag, and proceeds to the exit determination.

[0037] If the first determination unit 21 cannot determine in S4 that there is a pedestrian crossing 48 in front of the vehicle, the second determination unit 23 determines whether it can be estimated that the vehicle has entered the intersection 31 based on information indicating the positions of the traffic lights 51 and 53 or the lane lines 55 and 57. Specifically, the second determination unit 23 first determines whether the traffic lights 51 and 53 installed at the intersection 31 are on the image 30. If it determines that they are there, it proceeds to S5; otherwise, it proceeds to S8 (S7 in Figure 4, second determination step). Specifically, as shown in Figure 5, the second determination unit 23 first determines whether there are two traffic lights 51 and 53 of the same color that are lit on the image 30. If it determines that they are there, it proceeds to S7-2; otherwise, it proceeds to S8 (S7-1 in Figure 5). If the second determination unit 23 determines in S7-1 that there are two traffic lights 51 and 53 on the image 30, it determines whether the condition is met that one traffic light 51 is to the right of the FOE and the other traffic light 53 is near the FOE. If the condition is met, the process proceeds to S7-3; otherwise, it proceeds to S8 (S7-2 in Figure 5). If the condition is met in S7-2, the second determination unit 23 assigns an identification ID to the traffic light 53 near the FOE, in this case the traffic light 53 further back, and stores the identification ID and the information indicating the position of the traffic light 53 in the storage unit 5 (S7-3 in Figure 5). The identification ID is information that indicates that the traffic light 53 is the traffic light 53 further back, that is, information to distinguish the traffic light 53 from other traffic lights. Furthermore, the second determination unit 23 determines that it can estimate that the vehicle entered the intersection 31 when the traffic light 53 moves out of frame from the image 30, and proceeds to S5 as shown in Figure 4, turning on the intersection flag.

[0038] If it cannot be determined in S7 that traffic lights 51 and 53 are on image 30, the second determination unit 23 determines whether lane lines 55 and 57 have changed from "present" to "absent," specifically whether lane lines 55 and 57 are interrupted at intersection 31 (S8 in Figure 4, second determination step). If it is determined that the lines are interrupted, it is determined that the vehicle has entered intersection 31, and the process proceeds to S5, turning on the intersection flag. If it is determined in S5 that lane lines 55 and 57 are not interrupted, the second determination unit 23 returns, indicating that the vehicle has not entered intersection 31. This completes the entrance determination procedure.

[0039] Next, the procedure for exit determination (determination of whether the vehicle has entered or exited intersection 31), which follows the entry determination, will be explained with reference to Figure 6. First, as a premise, assume that the first determination unit 21 or the second determination unit 23 has estimated that the vehicle has entered intersection 31. Also, assume that the front camera 9 is capturing images in front of the vehicle at a predetermined imaging interval (frame rate). In this case, the image recognition unit 15 acquires an image 30 from the front camera 9 and obtains position information from the acquired image 30 (S11 in Figure 6, position acquisition step).

[0040] Next, the third determination unit 25 determines whether it can be estimated that the vehicle has exited the intersection 31 based on information indicating the location of the stop line 47 or pedestrian crossings 48 and 49 on the opposing lane 43. Specifically, the third determination unit 25 first determines in image 30 whether there is a stop line 47 on the opposing lane 43. If it determines that there is, it proceeds to S13; otherwise, it proceeds to S14 (S12 in Figure 6, third determination step). If it determines in S12 that there is a stop line 47, the third determination unit 25 determines whether the first determination unit 21 determined that there was a stop line 45 or pedestrian crossing 48 when determining the entrance (S13 in Figure 6). If it determined that there was, it proceeds to S15; otherwise, it proceeds to S16. If the first determination unit 21 determines that there is a stop line 45 or a pedestrian crossing 48 when determining the entrance, the third determination unit 25 determines that it can be estimated that the vehicle has exited the intersection 31 when the stop line 47 goes out of frame from the image 30. Furthermore, the third determination unit 25 considers that the vehicle has entered and exited the intersection 31. Specifically, the third determination unit 25 causes the second acquisition unit 34 to acquire the second time based on the position of the stop line 47 (S15 in Figure 6), turns off the intersection flag, and returns to S1 (S17 in Figure 6). If the first determination unit 21 does not determine that there is a stop line 45 or a pedestrian crossing 48 when determining the entrance, the third determination unit 25 determines that it can be estimated that the vehicle has exited the intersection 31 when the stop line 47 goes out of frame from the image 30. In this case, the third determination unit 25 considers that the vehicle entered and exited the intersection 31 and stores the exit from the intersection 31 in the storage unit 5 (S16 in Figure 6). Furthermore, the third determination unit 25 turns off the in-intersection flag and returns to S1 (S17 in Figure 6).

[0041] If it cannot be determined in S12 whether the vehicle has crossed the stop line 47 of the oncoming lane 43, the third determination unit 25 determines whether there is a pedestrian crossing 49 in image 30. If it determines that there is a pedestrian crossing 49, it proceeds to S18; otherwise, it proceeds to S19 (S14 in Figure 6, third determination step). If it is determined in S14 that there is a pedestrian crossing 49, the third determination unit 25 determines whether the first determination unit 21 determined that there was a stop line 45 or a pedestrian crossing 48 when determining the entrance (S18 in Figure 6). If it was determined that there was a pedestrian crossing 48, it proceeds to S20; otherwise, it proceeds to S21. If the first determination unit 21 determined that there was a stop line 45 or a pedestrian crossing 48 when determining the entrance, the third determination unit 25 determines that it can be estimated that the vehicle has exited the intersection 31 when the pedestrian crossing 48 is out of frame in image 30. Furthermore, the third determination unit 25 considers that the vehicle has entered and exited the intersection 31. Specifically, the third determination unit 25 instructs the second acquisition unit 34 to acquire the second time based on the position of the pedestrian crossing 48 (S20 in Figure 6), turns off the intersection flag, and returns to S1 (S17 in Figure 6). Even if the first determination unit 21 does not determine that there is a stop line 45 or a pedestrian crossing 48 when determining the entrance, the third determination unit 25 determines that it can be estimated that the vehicle has exited the intersection 31, and considers that the vehicle has entered and exited the intersection 31. In this case, the third determination unit 25 stores the exit from the intersection 31 in the storage unit 5 (S21 in Figure 6), turns off the intersection flag, and returns to S1 (S17 in Figure 6).

[0042] If it is not possible to determine in S14 whether there is a pedestrian crossing 49, that is, if the third determination unit 25 cannot estimate that the vehicle has exited the intersection 31, the fourth determination unit 27 determines whether it can estimate that the vehicle has exited the intersection 31 based on information indicating the position of the traffic light 53 or the positions of the lane lines 55 and 57. Specifically, the third determination unit 25 first determines whether the vehicle has passed the traffic light 53 to which it has been assigned an identification ID (see S7-3 in Figure 5) (whether it has gone out of frame in image 30). If it is determined that the vehicle has passed the traffic light 53 to which it has been assigned an identification ID, the process proceeds to S22; otherwise, the process proceeds to S23 (S19 in Figure 6, fourth determination step). If it is determined in S19 that the vehicle has passed the traffic light 53 to which it has been assigned an identification ID, the fourth determination unit 27 determines that it can estimate that the vehicle has exited the intersection 31 and considers that the vehicle entered and exited the intersection 31. Specifically, the fourth determination unit 27 stores the exit from intersection 31 in the memory unit 5 (S22 in Figure 6), turns off the intersection flag, and returns to S1 (S24 in Figure 6). If it cannot be determined in S19 that the vehicle has passed through the traffic light 53 to which an identification ID has been assigned, the fourth determination unit 27 determines whether the lane lines 55 and 57 have changed from "absent" to "present" (S23 in Figure 6, fourth determination step). Specifically, it determines whether the lane lines 55 and 57 that were interrupted at intersection 31 have been redrawn. If it is determined that the lane lines 55 and 57 have changed from "absent" to "present", it is presumed that the vehicle has entered and exited intersection 31 and proceeds to S22. If it cannot be determined that the lane lines have changed from "absent" to "present", it proceeds to S25.

[0043] If the fourth determination unit 27 cannot determine in S23 that lane lines 55 and 57 have changed from "absent" to "present", the fifth determination unit 28 determines whether the vehicle has turned right or left based on the acceleration values ​​obtained from the G sensor 11 (S25, fifth determination step in Figure 6). If it determines that the vehicle has turned right or left, it determines that it can be estimated that the vehicle has exited the intersection 31, and proceeds to S22, assuming that the vehicle entered and exited the intersection 31. If it cannot determine in S25 that the vehicle has turned right or left, it corresponds to a case where none of the third determination unit 25, fourth determination unit 27, and fifth determination unit 28 can determine whether the vehicle has exited the intersection 31. In this case, the sixth determination unit 32 returns to S11 if a certain amount of time has not elapsed since the start of the exit determination, and if a certain amount of time has elapsed, it assumes that the vehicle did not enter the intersection 31. Specifically, the sixth determination unit 32 does not store the vehicle's exit from intersection 31 in the memory unit 5 (S26 in Figure 6), turns off the intersection flag (S27 in Figure 6), and returns to S1. This completes the exit determination procedure.

[0044] As described above, the driver assistance device 1 of this embodiment includes an image recognition unit 15, a first determination unit 21, a second determination unit 23, a third determination unit 25, and a fourth determination unit 27. In this configuration, if it is estimated that the vehicle has entered or exited the intersection 31 based on the positional information of the stop lines 45, 47, pedestrian crossings 48, 49, lane lines 55, 57, and traffic lights 51, 53, the vehicle is considered to have entered and exited the intersection 31. Therefore, the accuracy of the entry / exit determination is less affected by the type or presence of road markings and installations at the intersection 31.

[0045] Furthermore, the driving support device 1 of this embodiment includes a first acquisition unit 29 that acquires a first time, which is the estimated time when the vehicle entered the intersection 31, and a second acquisition unit 34 that acquires a second time, which is the estimated time when the vehicle exited the intersection 31. With this configuration, the time when the vehicle entered and exited the intersection 31 can be obtained. Therefore, not only can it be determined whether or not the vehicle entered or exited the intersection 31, but also the timing of the entry and exit.

[0046] Furthermore, the driver assistance device 1 of this embodiment includes a fifth determination unit 28 that, when it is estimated that the vehicle has entered the intersection 31, and the vehicle then turns right or left, it is determined that the vehicle has entered and exited the intersection 31. In this configuration, if the vehicle turns right or left within the intersection 31 after it is estimated that the vehicle has entered the intersection 31, it is determined that the vehicle has entered and exited the intersection 31. Therefore, even if the road the vehicle entered and the road it exited intersect, the exit determination can be made.

[0047] On the other hand, the driver assistance device 1 of this embodiment includes a sixth determination unit 32 that, if none of the third determination unit 25, fourth determination unit 27, and fifth determination unit 28 can determine that the vehicle has exited the intersection 31 within a predetermined time, then the device considers that the vehicle has not entered the intersection 31. In this configuration, if entry can be estimated but exit cannot be estimated, the device considers that the vehicle has not entered the intersection 31. Therefore, the possibility of incorrectly determining that the vehicle has entered the intersection 31 when stop lines, pedestrian crossings, or traffic lights are installed on roads other than the intersection 31 can be reduced.

[0048] Furthermore, the driving assistance method of this embodiment includes a position acquisition step, a first determination step, a second determination step, a third determination step, and a fourth determination step. In this configuration, if it is estimated that the vehicle has entered or exited the intersection 31 based on the position information of the stop lines 45, 47, pedestrian crossings 48, 49, lane lines 55, 57, and traffic lights 51, 53, the vehicle is deemed to have entered and exited the intersection 31. Therefore, the accuracy of the entry / exit determination is less affected by the type or presence of road markings and installations at the intersection 31.

[0049] Although the present invention has been described above based on embodiments, the present invention is not limited to the above embodiments, and modifications may be made without departing from the spirit of the invention, and other technologies may be combined as appropriate to the extent possible. Furthermore, publicly known or well-known technologies may be combined as appropriate to the extent possible. [Explanation of Symbols]

[0050] 1: Driving assistance system 15: Image recognition section (position acquisition section) 21: 1st judgment part 23:Second judgment part 25:Third Judgment Section 27: 4th judgment part 28: 5th judgment part 29:1st acquisition part 30: Image 31: Intersection 32: 6th judgment part 34:Second acquisition part 40: Road 41: Own lane 43: Opposing lane 45, 47: Stop line 48, 49: Pedestrian crossing 51, 53: Traffic lights 55, 57: Lane lines S1, S11: Position acquisition process S2, S4: 1st judgment step S7, S8: Second judgment step S12, S14: Third judgment step S19, S23: 4th judgment step

Claims

1. A position acquisition unit acquires information from an image taken of the area in front of the vehicle, including the location of the stop line for the vehicle's lane, the stop line for the oncoming lane, and the pedestrian crossing drawn on the road surface, the location of lane lines drawn on the road surface adjacent to the left and right of the vehicle's lane, and the location of traffic signals that manage the vehicle's traffic. A first determination unit determines whether it can be estimated that the vehicle has entered the intersection based on information indicating the position of the stop line in the vehicle's lane or the position of the pedestrian crossing, If the first determination unit cannot determine whether it can be estimated that the vehicle has entered the intersection, the second determination unit determines whether it can be estimated that the vehicle has entered the intersection based on information indicating the position of the traffic light or the position of the lane lines. If the first or second determination unit determines that it can be estimated that the vehicle entered the intersection, the third determination unit determines from information indicating the position of the stop line on the opposing lane or the pedestrian crossing whether it can be estimated that the vehicle exited the intersection, and if it determines that it can be estimated, it considers that the vehicle entered and exited the intersection. If the first or second determination unit determines that it can be estimated that the vehicle entered the intersection, and the third determination unit cannot determine whether it can be estimated that the vehicle exited the intersection, the fourth determination unit determines whether it can be estimated that the vehicle exited the intersection based on information indicating the position of the traffic light or the position of the lane lines, and if it determines that it can be estimated, it considers that the vehicle entered and exited the intersection. Equipped with A driver assistance device mounted on a vehicle, characterized by the following features.

2. When the first determination unit determines that it can be estimated that the vehicle has entered the intersection, the first acquisition unit obtains a first time, which is the estimated time when the vehicle entered the intersection, from information indicating the position of the stop line in the vehicle's lane or the position of the pedestrian crossing. The system includes a second acquisition unit that, when the third determination unit determines that it can be estimated that the vehicle has exited the intersection, acquires a second time, which is the estimated time when the vehicle exited the intersection, from information indicating the position of the stop line on the opposing lane or the position of the pedestrian crossing. The driving support device according to claim 1, characterized in that

3. A fifth determination unit acquires right / left turn information, which is information regarding whether the vehicle is making a right or left turn, and if the first determination unit or the second determination unit determines that it can be estimated that the vehicle has entered the intersection, and the right / left turn information indicates that the vehicle is making a right or left turn, then the fifth determination unit determines that the vehicle has entered and exited the intersection. The driving support device according to claim 1 or 2, characterized in that

4. If the first or second determination unit determines that it can be estimated that the vehicle entered the intersection, and none of the third, fourth, or fifth determination units can determine whether it can be estimated that the vehicle exited the intersection within a predetermined time, the sixth determination unit determines that the vehicle did not enter the intersection. The driving support device according to claim 3, characterized in that

5. A position acquisition step involves acquiring information from an image taken of the area in front of the vehicle, including the location of the stop line for the vehicle's own lane, the stop line for the oncoming lane, and the pedestrian crossing drawn on the road surface, the location of lane lines drawn on the road surface adjacent to the left and right of the vehicle's own lane, and the location of traffic signals that manage the vehicle's traffic. A first determination step involves determining whether it can be estimated that the vehicle entered the intersection based on information indicating the position of the stop line in the vehicle's lane or the position of the pedestrian crossing, If the first determination step cannot determine whether it can be estimated that the vehicle entered the intersection, a second determination step is performed to determine whether it can be estimated that the vehicle entered the intersection based on information indicating the position of the traffic light or the position of the lane lines. If the first or second determination step determines that it can be estimated that the vehicle entered the intersection, the third determination step determines whether it can be estimated that the vehicle exited the intersection based on information indicating the position of the stop line on the opposing lane or the pedestrian crossing, and if it is determined that it can be estimated, the vehicle is deemed to have entered and exited the intersection. If the first or second determination step determines that it can be estimated that the vehicle entered the intersection, and the third determination step does not determine whether it can be estimated that the vehicle exited the intersection, then the fourth determination step determines whether it can be estimated that the vehicle exited the intersection based on information indicating the position of the traffic light or the position of the lane lines, and if it is determined that it can be estimated, then the vehicle is deemed to have entered and exited the intersection. Equipped with A driving assistance method characterized by the following features.

Images