Vehicle control device, vehicle control system, vehicle control method, and program

Abstract

To provide a vehicle control device, a vehicle control system, a vehicle control method, and a program capable of preventing an erroneous entry of a vehicle into an intersection when a blue signal is erroneously detected.SOLUTION: A vehicle control device 10 includes an imaging part 220 that images the surroundings of a vehicle 1, a position information acquisition part 210 that acquires position information of the vehicle 1, a map database 230 that stores map information including traffic light information, and a controller 100 that controls the vehicle 1. The controller 100 determines a traffic light signal based on a position of the traffic light estimated from the position information and the map information and a forward image captured by the imaging part 220 in front of the vehicle, and when the determination result of the traffic light signal is an enterable signal, detects a moving object from a surrounding image captured by the imaging part 220 around the intersection in front of the vehicle 1, calculates a signal certainty indicating the certainty of the traffic light signal based on the movement of the moving object, and controls the vehicle 1 to stop at a stop line when the signal certainty is at or below a predetermined threshold.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a vehicle control device, a vehicle control system, a vehicle control method, and a program. [Background technology]

[0002] In automated driving of a vehicle, there is a technology that determines the signal of a traffic light ahead based on an image captured in front of the vehicle, and controls the vehicle's travel based on the determination result (for example, Patent Document 1). When the vehicle control device described in Patent Document 1 cannot determine the signal of a traffic light ahead, it determines the signal of the traffic light ahead based on the detection result of information about the vehicle's surroundings. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 2017-182297 Summary of the Invention [Problem to be solved by the invention]

[0004] Depending on surrounding conditions such as weather conditions, time of day, and congestion while traveling, when determining the signal of a traffic light ahead based on a captured image, a no-entry signal (such as a red light) may be erroneously determined to be an entry-permitted signal (such as a green light). In the technology described in Patent Document 1, if the no-entry signal of the traffic light ahead is erroneously determined to be an entry-permitted signal, there is a possibility that the vehicle will mistakenly enter the intersection.

[0005] The present invention has been made in view of the above-mentioned circumstances, and has an object to provide a vehicle control device, a vehicle control system, a vehicle control method, and a program that can prevent a vehicle from mistakenly entering an intersection. [Means for solving the problem]

[0006] In order to achieve the above object, a vehicle control device according to the present invention includes an imaging unit that captures images of the surroundings of the vehicle, a position information acquisition unit that acquires position information of the vehicle, a map database that stores map information including information about traffic lights, and a controller that controls the vehicle. The controller determines a traffic light signal based on the position of the traffic light estimated based on the position information and the map information and a front image captured by the imaging unit of the area ahead of the vehicle, and when the determined result of the traffic light signal is an entry OK signal, detects a moving object from a surrounding image captured by the imaging unit of the area around the intersection ahead of the vehicle, and calculates a signal accuracy that indicates the likelihood of the traffic light signal based on the movement of the moving object. If the calculated signal accuracy is equal to or less than a predetermined threshold, the controller controls the vehicle to stop at the stop line. [Effects of the Invention]

[0007] According to the present invention, when the result of the traffic light signal determination is an entry-permit signal, the vehicle is controlled in accordance with the signal accuracy, thereby making it possible to prevent the vehicle from erroneously entering the intersection. [Brief explanation of the drawings]

[0008] [Figure 1] 1 is a block diagram showing an example of a functional configuration of a vehicle control device according to a first embodiment of the present invention. [Figure 2] FIG. 2 is a diagram illustrating an example of a hardware configuration of a controller according to the first embodiment. [Figure 3] FIG. 1 is a diagram showing moving objects around an intersection. [Figure 4] 1 is a table showing the movement of moving objects around an intersection. [Figure 5] 4 is a flowchart of a vehicle control process according to the first embodiment. [Figure 6] FIG. 10 is a diagram showing a flow of a part of a vehicle control process according to Modification 1. [Figure 7] FIG. 10 is a schematic diagram showing a vehicle control system according to a second embodiment of the present invention. [Figure 8] FIG. 10 is a block diagram showing an example of a functional configuration of a vehicle control device according to a second embodiment. [Figure 9]FIG. 10 is a diagram showing a flow of a part of a vehicle control process according to the second embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0009] A vehicle control device, a vehicle control system, a vehicle control method, and a program according to an embodiment of the present invention will be described with reference to the drawings. In the drawings, the same or equivalent parts are designated by the same reference numerals.

[0010] (Embodiment 1) A vehicle control device 10 according to a first embodiment of the present invention is a device that controls the traveling of a vehicle 1. For example, the vehicle control device 10 realizes autonomous driving of the vehicle 1 by controlling an actuator 240 that drives each part of the vehicle 1. In this embodiment, the vehicle control device 10 is assumed to control level 4 autonomous driving, but other levels may also be used. Note that the vehicle control device 10 is not limited to controlling autonomous driving, and may also control driving assistance such as brake assistance and steering assistance. FIG. 1 is a diagram showing an example of the functional configuration of the vehicle control device 10 according to this embodiment, and illustrates a portion related to control based on traffic light signal determination and signal accuracy according to this embodiment.

[0011] 1, the vehicle control device 10 includes a controller 100 that judges traffic light signals and performs calculation processing based on the signal accuracy, a location information acquisition unit 210 that acquires location information of the vehicle 1, an imaging unit 220 that captures images of the surroundings of the vehicle 1, and a map database 230 that stores map information including information on roads on which the vehicle 1 is traveling. Note that the location information acquisition unit 210, the imaging unit 220, and the map database 230 may be configured to be shared with other systems or devices such as a navigation system.

[0012] The position information acquisition unit 210 is any device, such as a GNSS (Global Navigation Satellite System) receiver, that can acquire the position of the vehicle 1. The GNSS receiver receives orbit information and time information from a plurality of positioning satellites, and outputs position information indicating the position of the vehicle 1 calculated based on the received signals to the controller 100.

[0013] The imaging unit 220 is, for example, one or more cameras that capture images of the surroundings including the area in front of the vehicle 1, and each camera is installed in a position and direction that allows it to capture images of traffic lights in front of the vehicle 1 and moving objects located around the intersection in front of the vehicle 1.

[0014] The map database 230 stores map information including road information, intersection information, and traffic light information, and is a database used in, for example, a navigation system. In this embodiment, the map information stored in the map database 230 includes the number and shape of lanes around intersections, as well as the types of signals displayed by traffic lights and the locations of the traffic lights.

[0015] The controller 100 determines the traffic light signal based on the position of the traffic light estimated based on the position information acquired by the position information acquisition unit 210 and the map information in the map database 230, and on a forward image captured by the imaging unit 220 of the area ahead of the vehicle 1. The controller 100 also detects a moving object from a peripheral image captured by the imaging unit 220 of the area around the intersection ahead of the vehicle 1, calculates a signal accuracy indicating the accuracy of the traffic light signal based on the movement of the moving object, and controls the vehicle based on the signal accuracy.

[0016] Fig. 2 is a diagram showing an example of the hardware configuration of the controller 100. In the example of Fig. 2, the controller 100 includes a processor 1011, a memory 1012, a storage 1013, and a communication interface (indicated as communication I / F in the figure) 1014, which are connected to each other via a bus 1010.

[0017] The processor 1011 includes, for example, one or more CPUs (Central Processing Units) and their peripheral circuits, and executes various types of arithmetic processing. The processor 1011 loads a control program stored in a storage 1013 into a memory 1012 and executes the program. The processor 1011 may further include arithmetic circuits such as a logical operation unit and a numerical operation unit.

[0018] The memory 1012 includes, for example, a volatile semiconductor memory such as a RAM (Random Access Memory), and functions as a work memory for the processor 1011. The memory 1012 also temporarily stores the control program that the processor 1011 reads from the storage 1013 and various data used in the processor 1011's arithmetic processing.

[0019] The storage 1013 includes, for example, a nonvolatile semiconductor memory such as an EEPROM (Electrically Erasable and Programmable Read Only Memory), a flash memory, etc. The storage 1013 stores the control program executed by the processor 1011 and various data used in the arithmetic processing of the processor 1011.

[0020] The communication interface 1014 includes an interface circuit for connecting the controller 100 to an in-vehicle network that complies with standards such as CAN (Controller Area Network). The communication interface 1014 receives signals from the position information acquisition unit 210, the imaging unit 220, the map database 230, and other in-vehicle components, and passes the signals to the processor 1011.

[0021] Furthermore, the communication interface 1014 transmits a vehicle control signal generated by the processor 1011 based on the traffic light signal and the signal accuracy to an actuator 240 that operates the vehicle 1. In other words, the actuator 240 is communicatively connected to the vehicle control device 10 via an in-vehicle network, and the vehicle control device 10 controls the actuator 240. The actuator 240 includes, for example, a drive device (at least one of an engine and a motor) for accelerating the vehicle 1, a brake actuator for braking the vehicle 1, a steering motor for steering the vehicle 1, and the like. In this way, the vehicle control device 10 controls the actuator 240 to realize automatic driving or driving assistance of the vehicle 1.

[0022] 1 in the controller 100, for example, a processor 1011 and a memory 1012 work together to realize the functions shown in Fig. 1. That is, the controller 100 functions as a host vehicle position estimation unit 101 that estimates the position of the vehicle 1, a map information acquisition unit 102 that acquires map information about the area around the intersection, a traffic light position estimation unit 103 that estimates the position of a traffic light based on the position of the vehicle 1 and the map information, a signal determination unit 104 that determines a traffic light signal from an image captured by the imaging unit 220, a moving object detection unit 105 that detects a moving object from the captured image, a signal accuracy calculation unit 106 that calculates a signal accuracy that indicates the accuracy of a traffic light signal, and a vehicle control unit 107 that controls the vehicle 1 based on the signal accuracy.

[0023] The vehicle position estimation unit 101 estimates the position of the vehicle 1 (the vehicle's position) on the map in the map database 230 based on an output signal from the position information acquisition unit 210 installed in the vehicle 1. Here, the estimated vehicle position also estimates which lane out of multiple lanes on one side of the road the vehicle is traveling in. The map information acquisition unit 102 acquires map information from the map database 230, including the position of traffic lights, the type of traffic light, the number of lanes, the presence or absence of dedicated right and left turn lanes, etc., for intersections ahead in the traveling direction of the vehicle 1 position estimated by the vehicle position estimation unit 101.

[0024] The traffic light position estimation unit 103 estimates the position where the image of a traffic light appears in the image captured by the imaging unit 220, based on the position of the vehicle 1 estimated by the host vehicle position estimation unit 101 and map information. The signal determination unit 104 determines the traffic light signal located at the estimated position from the forward image captured by the imaging unit 220 in front of the vehicle 1.

[0025] The signal determination unit 104 checks the type of signal included in the map information and determines, for example, a red, green, or yellow solid signal, a red or yellow flashing signal, a right, left, diagonal, or straight arrow signal, or a combination of these, as the traffic light signal. Conventional color identification processing, pattern recognition processing, etc. are used to determine the traffic light signal from the forward image. In the following description, a signal with only a red or yellow solid signal indicating that entry into the intersection is prohibited will be referred to as an "entry prohibited signal," and a green solid signal indicating that entry into the intersection is permitted, as well as a combination of a red solid signal and an arrow signal corresponding to the traveling direction of the vehicle 1, will be referred to as an "entry permitted signal."

[0026] The moving object detection unit 105 detects moving objects from a surrounding image captured by the imaging unit 220 around the intersection ahead of the vehicle 1. The moving objects are other vehicles, pedestrians, etc. moving around the intersection. The moving object detection unit 105 refers to lane information and intersection information included in map information acquired from a map database, and divides the area around the intersection ahead of the vehicle 1 in the captured image into multiple detection areas. The moving object detection unit 105 sequentially selects the divided detection areas, searches for moving objects in each detection area, and detects movements of the detected moving objects, such as acceleration / deceleration, stopping, walking, etc.

[0027] Specific examples of the detection area detected by the moving object detection unit 105 and the movement of the moving object will be described using Figures 3 and 4. Figure 3 is a diagram showing an example of a moving object around an intersection on a left-hand traffic road, in which a vehicle 1 is traveling straight in the left lane of two lanes toward the intersection. Figure 4 is a table showing the movement patterns of moving objects around the intersection when the actual traffic light is red, indicating that entry is prohibited.

[0028] 3, when an adjacent lane 301 in the same direction as the own lane 300 in which the vehicle 1 is traveling is selected as the first detection area in which the moving object detection unit 105 detects a moving object, the moving object detection unit 105 searches for another vehicle 311 located in the adjacent lane 301. When the traffic light in the own lane 300 is actually red, the other vehicle 311 detected in the adjacent lane 301 will stop at the stop line or decelerate toward the stop line as shown in FIG.

[0029] 3, when an oncoming lane 302 opposite to the own lane 300 is selected as the second detection area, the moving object detection unit 105 searches for another vehicle 312 located in the oncoming lane 302. When the traffic light for the own lane 300 is actually red, the other vehicle 312 detected in the oncoming lane 302 stops at the stop line or decelerates toward the stop line as shown in FIG.

[0030] 3, when a right intersecting lane 303, which is an intersecting lane on the right side intersecting with the own lane 300, is selected as the third detection area, the moving object detection unit 105 searches for another vehicle 313 located in the right intersecting lane 303. When the traffic light for the own lane 300 is actually red, the other vehicle 313 detected in the right intersecting lane 303 moves to pass the stop line at a constant speed or while accelerating, as shown in FIG.

[0031] 3, when the left intersecting lane 304, which is the intersecting lane on the left side intersecting the own lane 300, is selected as the fourth detection area, the moving object detection unit 105 searches for another vehicle 314 located in the left intersecting lane 304. When the traffic light for the own lane 300 is actually red, the other vehicle 314 detected in the left intersecting lane 304 moves to pass the stop line at a constant speed or while accelerating, as shown in FIG.

[0032] When a right turn waiting area 305 in an intersecting lane is selected as the fifth detection area as shown in Fig. 3, the moving object detection unit 105 searches for another vehicle 315 located in the right turn waiting area 305. In the right turn waiting area 305, when the traffic light in the current lane 300 is actually red, another vehicle 315 waiting to turn right in the intersecting direction can be detected as shown in Fig. 4. Note that Fig. 3 shows a diagram for left-hand traffic, but in the case of right-hand traffic, the moving object detection unit 105 may similarly detect another vehicle in the left turn waiting area.

[0033] 3, when a crosswalk 306 that intersects with the current vehicle lane 300 is selected as the sixth detection area, the moving object detection unit 105 searches for pedestrians 316 located at the crosswalks 306 in front and behind the current vehicle lane. When the traffic light for the current vehicle lane 300 is actually red, pedestrians 316 crossing the crosswalk 306 can be detected as shown in FIG.

[0034] 3, when a same-direction crosswalk 307 in the same direction as the current vehicle lane 300 is selected as the seventh detection area, the moving object detection unit 105 searches for pedestrians 316 located on the left and right of the same-direction crosswalks 307. When the traffic light for the current vehicle lane 300 is actually red, pedestrians 316 crossing the same-direction crosswalks 307 are not detected as shown in FIG.

[0035] Although not shown in FIG. 3 , a dedicated right-left turn lane for turning right or left from the same direction as the current lane 300 may be selected as the eighth detection area. The moving object detection unit 105 acquires lane information indicating the presence or shape of a dedicated right-left turn lane and lane connection information from the map information in the map database 230 to the lane into which the current lane will be traveled after turning right or left, and searches for the dedicated right-left turn lane if one is available. As shown in FIG. 4 , when the traffic light for the current lane 300 is actually red, the moving object detection unit 105 detects the movement of another vehicle in the dedicated right-left turn lane stopping at the stop line or slowing down toward the stop line. Furthermore, when the traffic light for the current lane 300 is actually a combination of a red light and a right-left turn arrow signal, the moving object detection unit 105 detects the movement of another vehicle in the dedicated right-left turn lane stopping according to the pattern of the arrow signal or turning right or left according to the lane connection information.

[0036] Although not shown in FIG. 3, a tram line may be selected as the ninth detection area. The moving object detection unit 105 acquires information on the presence or absence of a tram line and tram signal information from the map information in the map database 230, and searches for the tram line if one is present. As shown in FIG. 4, when the signal for the current lane 300 is actually red, the moving object detection unit 105 detects the movement of a tram on the tram line stopping or passing according to the pattern of the tram signal. Note that the detection areas are not limited to the first to ninth detection areas described above, and any other detection area may be selected.

[0037] When the traffic light signal determined by the signal determination unit 104 is an entry-permit signal such as a green light, the signal accuracy calculation unit 106 calculates a signal accuracy indicating the likelihood that the traffic light signal determined by the signal determination unit 104 is an entry-permit signal based on the movement of the moving object detected by the moving object detection unit 105.

[0038] Any method may be used to calculate the signal accuracy, but an example will be described with reference to Fig. 4. First, the index value S corresponding to each detection area listed in Fig. 4 n (n is a natural number that identifies the detection area) is derived. n is an index indicating the likelihood that the signal is entry-permitted (green light), with 1 being the highest (1: green light) and 0 being the lowest (0: red light). When the movement of a moving object detected by the moving object detection unit 105 matches each pattern shown in FIG. 4, there is a high likelihood that the actual traffic light signal is an entry-prohibited signal (red light). Therefore, the index value S corresponding to that detection area n will be 0.

[0039] For example, when the moving object detection unit 105 detects that the other vehicle 311 in the adjacent lane 301 is slowing down or stopping at the stop line, the index value S1 is 0. On the other hand, when the moving object detection unit 105 detects that the other vehicle 311 in the adjacent lane 301 is passing the stop line at a constant speed or accelerating, the index value S1 is 1. nmay be a binary value of 0 or 1, or may be any value between 0 and 1. For example, the index value S1 may be determined to be 0 for the movement of the other vehicle 311 stopping at a stop line, 0.75 for the movement of decelerating a certain distance or more before the stop line, and 0.1 for the movement of suddenly decelerating just before the stop line.

[0040] The index value S determined in this way n Using the following formula (1), the signal accuracy F c where a n is a weighting coefficient set in advance for each index value. The weighting coefficient may be set based on the shape of the lane or intersection, safety considerations, etc. For example, the coefficient for a detection area that is close to the vehicle 1 may be set higher than the weighting coefficient for a detection area that is farther away. Also, the weighting coefficient for a detection area of ​​a crosswalk may be set higher than the weighting coefficient for a detection area of ​​a lane.

[0041]

number

[0042] The vehicle control unit 107 generates and outputs a control signal for controlling each actuator of the vehicle 1 based on the signal accuracy calculated by the signal accuracy calculation unit 106. Specifically, even if the signal determination unit 104 determines that the signal is an entry OK signal, if the signal accuracy calculated by the signal accuracy calculation unit 106 is equal to or less than a predetermined threshold, the vehicle control unit 107 determines that the signal is not likely to be an entry OK signal and performs control to stop the vehicle at the stop line.

[0043] The operation of the vehicle control device 10 configured as above will be described in detail with reference to the flowchart in Fig. 5. Fig. 5 is a flowchart of the vehicle control process when the vehicle 1 travels before an intersection. Note that the vehicle control process shown in the flowchart in Fig. 5 is executed in parallel with other vehicle controls such as follow-up control that automatically follows a vehicle ahead, lane keeping control that uses lane markings, etc. Also, for the sake of simplicity, the flowchart in Fig. 5 is a flowchart for the case where the vehicle travels straight through an intersection.

[0044] The vehicle control process shown in the flowchart of Fig. 5 starts when the vehicle 1 arrives a predetermined distance before an intersection. First, the vehicle position estimation unit 101 of the controller 100 estimates the position of the vehicle 1 (the vehicle's position) on the map in the map database 230 based on the output signal of the position information acquisition unit 210 mounted on the vehicle 1 (step S101). At this time, the estimated vehicle position also estimates which lane the vehicle is traveling in among multiple lanes on one side of the road.

[0045] Next, based on the position of vehicle 1 estimated in step S101, the map information acquisition unit 102 acquires map information about the intersection ahead of vehicle 1 from the map database 230, including the position of traffic lights, the type of traffic light, the number of lanes, whether or not there are lanes for turning right or left, etc. (step S102).

[0046] Next, the traffic light position estimation unit 103 estimates the position where the image of the traffic light appears in the image captured by the imaging unit 220 based on the position of the vehicle 1 estimated in step S101 and the map information acquired in step S102 (step S103). After that, a timer (time t) is started (step S104).

[0047] Next, the signal determination unit 104 determines the traffic light signal from the forward image captured by the imaging unit 220 of the area ahead of the vehicle 1 (step S105). At this time, the signal determination unit 104 checks the type of traffic light included in the map information and determines that the traffic light signal is a solid red, green, or yellow signal, a flashing red or yellow signal, a right, left, diagonal, or straight arrow signal, or a combination of these. For the sake of simplicity, this embodiment will describe a case where the traffic light signal is either a green signal indicating entry permission, or a red or yellow signal indicating entry prohibition.

[0048] If the traffic light signal determined in step S105 is a red light or a yellow light, but not a green light (step S106: No), the vehicle control unit 107 outputs a control signal to the actuator 240 to stop the vehicle 1 at the stop line (step S107). Also, if the traffic light signal cannot be detected or determined in step S105, the vehicle control unit 107 determines that the traffic light signal is not a green light (step S106: No), and performs control to stop the vehicle 1 at the stop line (step S107). At this time, the vehicle control unit 107 may optimize the deceleration change depending on the distance to the stop line so as to avoid sudden braking. Thereafter, the timer is reset (t=0, step S108), and the process returns to step S105.

[0049] On the other hand, if the traffic light signal determined in step S105 is green (step S106: Yes), the moving object detection unit 105 searches for a moving object from a surrounding image captured by the imaging unit 220 around the intersection ahead of the vehicle 1 (step S109). More specifically, the moving object detection unit 105 refers to lane information, intersection information, etc. included in the map information, sets multiple detection areas in the surrounding image as exemplified in Figures 3 and 4, and searches for a moving object located in each detection area.

[0050] If the moving object detection unit 105 fails to detect a moving object in each detection area as a result of the search in step S109 (step S110: No), it outputs a control signal to the actuator 240 to decelerate by a predetermined speed range (step S111). After that, the timer is reset (t=0, step S112), and the process returns to step S105, where the determination of the traffic light signal and the search for the moving object are repeated. By decelerating in this way when a moving object cannot be detected, the probability of detecting a moving object can be increased by driving slowly to the stop line. Note that a lower limit speed may be set for the deceleration in step S111. If the moving object detection unit 105 does not detect a moving object because there is no moving object, the vehicle enters the intersection at the lower limit speed based on the traffic light signal determined in step S105, thereby ensuring safety.

[0051] On the other hand, if the moving object detection unit 105 detects a moving object in each detection area through the search in step S109 (step S110: Yes), the signal accuracy calculation unit 106 calculates the signal accuracy of the green light based on the movement of the moving object (step S113). For example, the signal accuracy calculation unit 106 determines whether the movement of the moving object searched and detected in each detection area in step S109 matches the pattern shown in FIG. 4, and calculates the signal accuracy F c Calculate.

[0052] The signal accuracy F calculated in step S113 c If is equal to or less than the threshold (step S114: Yes), there is a high possibility that the red light has been mistakenly determined to be a green light, so the vehicle control unit 107 outputs a control signal to the actuator 240 to stop the vehicle 1 at the stop line (step S107).

[0053] The signal accuracy F calculated in step S113 c When exceeds the threshold value (step S114: No), the signal accuracy F c The duration of the state in which the signal accuracy F exceeds the threshold is measured. That is, the timer time t is measured, and while the time t is equal to or less than the preset determination time t0 (step S115: Yes), the process returns to step S105, and the determination of the traffic light signal and the detection of the moving object are repeated. While the time t is equal to or less than the determination time t0, the signal accuracy F c When the value of the traffic signal fluctuates and becomes equal to or smaller than the threshold value (step S114: Yes), control is performed to stop the vehicle at the stop line from that point on (step S107). Note that the determination of the traffic signal and the detection of the moving object may be repeatedly performed at a predetermined cycle.

[0054] Signal Accuracy F c exceeds the threshold value (step S114: No), and if the timer time t exceeds the judgment time t0 (step S115: No), the vehicle control unit 107 outputs a control signal to the actuator 240 so that the vehicle 1 passes through the intersection at a constant speed (step S116), and ends the processing.

[0055] 5, the vehicle control device 10 detects a moving object from a peripheral image captured around the intersection when the green light is determined, and performs control to stop the moving object at the stop line when the signal accuracy calculated based on the movement of the moving object is equal to or less than a threshold. On the other hand, if the signal accuracy calculated based on the movement of the moving object exceeds the threshold and remains sufficiently high beyond the determination time t0, the vehicle control device 10 performs control to pass the stop line. Note that if the signal accuracy calculated in step S113 is reliable to a certain level, for example, because it is an average value calculated by performing multiple detections of moving objects, step S115 may be omitted, and passage control may be performed (step S116) when the signal accuracy exceeds the threshold (step S114: No).

[0056] As described above, in the vehicle control device 10 according to this embodiment, the traffic light position estimation unit 103 estimates the position of a traffic light ahead of the vehicle 1 based on the position information acquired from the position information acquisition unit 210 and the map information acquired from the map database 230. The signal determination unit 104 determines the traffic light signal based on the estimated traffic light position and the forward image captured by the imaging unit 220. The moving object detection unit 105 detects a moving object from the peripheral image captured by the imaging unit 220 around the intersection, and the signal accuracy calculation unit 106 calculates a signal accuracy indicating the accuracy of the traffic light signal based on the movement of the moving object. When the signal determination unit 104 determines that the traffic light signal is an entry OK signal, the vehicle control unit 107 controls the vehicle to stop at the stop line if the signal accuracy is equal to or less than a predetermined threshold. This controls the vehicle according to the signal accuracy of the determined traffic light signal, thereby preventing erroneous entry into the intersection even if the signal accuracy is erroneously determined to be an entry OK signal.

[0057] In addition, in a conventional configuration that does not calculate the signal accuracy, if an entry-permissible signal is erroneously determined several tens of meters before the stop line and then the vehicle heads towards the stop line and an entry-prohibited signal is determined immediately before the stop line, the vehicle will decelerate rapidly. In contrast, according to the vehicle control device 10 according to the present embodiment, when the determination result of the traffic signal by the signal determination unit 104 is an entry-permissible signal several tens of meters before the stop line and the signal accuracy is below a predetermined threshold, vehicle control is performed to stop at the stop line. Therefore, the vehicle can decelerate gently towards the stop line, and the deceleration G felt by the occupant can be reduced.

[0058] (Modification Example 1) The above Embodiment 1 can be variously modified. FIG. 6 is a diagram showing a part of the flow of the vehicle control process according to Modification Example 1. Modification Example 1 changes the determination time required for determining whether or not to pass the stop line according to the signal accuracy. In Modification Example 1, as the threshold values of the signal accuracy, a first threshold value T1 and a second threshold value T2 (T1 < T2) are used, and as the determination times required for determining whether or not to pass the stop line, t1 and t2 (t1 > t2) are used.

[0059] In the vehicle control process according to this Modification Example 1, the processes of steps S101 to S113 shown in FIG. 5 are executed in the same manner as in Embodiment 1. The signal accuracy F of the green signal calculated in step S113 c When it is below the first threshold value T1 (step S211: Yes), since there is a high possibility of erroneously determining a red signal as a green signal, the vehicle control unit 107 outputs a control signal to the actuator 240 so that the vehicle 1 stops at the stop line (step S107). Then, the timer is reset (t = 0, step S108), and the process returns to step S105.

[0060] The signal accuracy F calculated in step S113 c When it exceeds the first threshold value T1 (step S211: No) but is below the second threshold value T2 (T1 < F c ≦ T 2、 Step S212: Yes), when the time of the timer is less than or equal to the determination time t1 (step S213: Yes), the process returns to step S105, and the determination of the traffic signal and the detection of the moving object are repeated.

[0061] Signal Accuracy F c is T1 <F c If the state of ≦T2 (step S212: Yes) continues beyond the judgment time t1 (step S213: No), the vehicle control unit 107 outputs a control signal to the actuator 240 so that the vehicle 1 passes through the intersection at a constant speed (step S116), and ends the processing.

[0062] The signal accuracy F calculated in step S113 c exceeds the second threshold T2, (F c >T2, step S212: No), if the timer time is equal to or less than the determination time t2 (step S214: Yes), the process returns to step S105, and the determination of the traffic light signal and the detection of the moving object are repeated.

[0063] Signal Accuracy F c F c If the state where the time t2 is greater than T2 (step S212: No) continues beyond the determination time t2 (step S214: No), the vehicle control unit 107 outputs a control signal to the actuator 240 so that the vehicle 1 passes through the intersection at a constant speed (step S116), and ends the process. At this time, since t2 is shorter than t1, the signal accuracy F c When this is sufficiently high, the decision to cross the stop line can be made in a short time.

[0064] This reduces the signal accuracy F c If is clearly low (F c ≦T1), it is possible to determine at an early stage that the vehicle should stop at the stop line. Here, the longer the distance to the stop line, the more gradually the vehicle can be decelerated and stopped in the stop control of step S107, which improves the riding comfort of the passengers. On the other hand, when the signal accuracy F c If is sufficiently high (F c >T2), the time required to determine whether to cross the stop line can be shortened, enabling stable constant speed driving.

[0065] In this modification, the case where two signal accuracy thresholds and two judgment times are used has been described, but three or more signal accuracy thresholds and three or more judgment times may also be used. By using three or more signal accuracy thresholds and three or more judgment times, the judgment time can be shortened in stages as the signal accuracy increases. In other words, when the signal accuracy remains higher than the threshold associated with the judgment time for a judgment time that is set shorter as the signal accuracy increases, the vehicle control unit 107 performs control to pass the stop line.

[0066] As explained above, in this first modification, a plurality of thresholds for signal accuracy and a plurality of determination times required to determine whether or not to cross the stop line are provided, the determination time is shortened in stages as the signal accuracy of the green light increases, and the vehicle control unit 107 performs control to cross the stop line in a short determination time when the signal accuracy is sufficiently high. This makes it possible to determine to stop at an early stage when the signal accuracy of the green light is clearly low, and to determine to cross the stop line at an early stage when the signal accuracy is sufficiently high.

[0067] (Embodiment 2) A vehicle control system 2 according to a second embodiment of the present invention is a system including a plurality of vehicles 3 having wireless communication capabilities and a server 6 communicatively connected to the plurality of vehicles 3 via a wireless base station 4 and the Internet 5. The vehicle control system 2 is, for example, a mobility service system that controls the automatic driving and driving assistance of the plurality of vehicles 3. FIG. 7 is a schematic diagram showing the vehicle control system 2 according to this embodiment, and FIG. 8 is a functional configuration diagram of a vehicle control device 20 according to this embodiment. The vehicle control device 20 is mounted in each vehicle 3 and controls the automatic driving or driving assistance of the vehicles 3, and FIG. 8 shows a portion according to the second embodiment that relates to control based on traffic light signal judgment and signal accuracy.

[0068] The server 6 is a computer that is connected to a plurality of vehicles 3 via the wireless base station 4 and the Internet 5, receives information indicating the position and driving status of each vehicle 3, and transmits road traffic information including traffic signal information to each vehicle 3. The server 6 is, for example, a server at a facility such as a traffic control center or a traffic signal information center that manages and provides traffic signal information such as the timing of switching traffic signals, or a server that aggregates and provides information obtained from such a facility.

[0069] As in the first embodiment, the vehicle control device 20 includes a controller 100 that judges traffic light signals and performs calculation processing based on the signal accuracy, a position information acquisition unit 210 that acquires position information of the vehicle 1, an imaging unit 220 that captures images of the surroundings of the vehicle 1, and a map database 230 that stores map information including information on roads on which the vehicle 1 is traveling. The vehicle control device 20 further includes a wireless communication unit 250. The wireless communication unit 250 includes a communication module that performs wireless communication according to any standard, such as mobile phone communication or wireless LAN (Local Area Network).

[0070] The controller 100 functions as a vehicle position estimation unit 101, a map information acquisition unit 102, a traffic light position estimation unit 103, a signal determination unit 104, a moving object detection unit 105, a signal accuracy calculation unit 116, and a vehicle control unit 107, as in the first embodiment, but the processing of the signal accuracy calculation unit 116 differs from that in the first embodiment.

[0071] The signal accuracy calculation unit 116 according to this embodiment calculates the signal accuracy of the traffic light signal determined by the signal determination unit 104, and uses signal information including the traffic light signal and the timing of the signal change received by the wireless communication unit 250 from the server 6 to calculate the signal accuracy. More specifically, the signal accuracy calculation unit 116 acquires the traffic light information received by the wireless communication unit 250 from the server 6, and selects a detection area to be detected by the moving object detection unit 105 according to the time from the current time until the light changes to red. The moving object detection unit 105 detects a moving object within the selected detection area from the surrounding image captured by the imaging unit 220, and the signal accuracy calculation unit 116 calculates the signal accuracy based on the movement of the detected moving object.

[0072] The operation of the vehicle control device 20 according to this embodiment will be described with reference to the flow diagram of Fig. 9. Fig. 9 is a diagram showing the flow of part of the vehicle control process according to this embodiment. The processes of steps S101 to S108 shown in Fig. 5 are executed in the same manner as in embodiment 1. In Fig. 9, if the signal determined by the signal determination unit 104 in step S105 is a green signal (step S106: Yes), the wireless communication unit 250 acquires signal information received from the server 6 (step S311). Here, the signal information includes information on the current traffic light signal at each intersection and the timing at which the signal changes, and is, for example, information acquired by the wireless communication unit 250 by making a request to the server 6 at predetermined time intervals.

[0073] Based on the traffic light information acquired in step S311, the signal accuracy calculation unit 116 calculates the time from the current time until the traffic light changes to red, and selects a detection area for the moving object detection unit 105 to detect based on the calculated time (step S312). The moving object detection unit 105 searches for a moving object within the selected detection area from the surrounding image captured by the imaging unit 220 (step S109).

[0074] For example, if the traffic light information acquired in step S311 indicates that the time until the light changes to red is within a certain time and indicates the timing when the pedestrian light for the same direction will change to red, the same-direction crosswalk 307, adjacent lane 301, and oncoming lane 302 shown in Fig. 3 are selected as the detection area. The moving object detection unit 105 searches the selected same-direction crosswalk 307, adjacent lane 301, and oncoming lane 302 in the surrounding image to detect a moving object.

[0075] Furthermore, if the signal information acquired in step S311 indicates that the right-turn arrow is on and will be turned off within a certain time, the same-direction crosswalk 307, adjacent lane 301, oncoming lane 302, and right-turn lane are selected as the detection area. The moving object detection unit 105 searches the selected same-direction crosswalk 307, adjacent lane 301, oncoming lane 302, and right-turn lane from the surrounding image to detect a moving object. By limiting the detection area for a moving object in this way, it is possible to reduce the amount of processing or processing time compared to searching the entire area, and also to improve the accuracy of signal accuracy.

[0076] The subsequent processing of steps S110 to S116 is the same as in embodiment 1, but taking into consideration that the reliability of signal accuracy is improved by using signal information, the signal accuracy threshold or judgment time may be set to a value different from that in embodiment 1. For example, the signal accuracy threshold used in step S114 may be set lower than when searching the entire area. Also, the judgment time t0 used in step S115 may be set shorter than when searching the entire area. By lowering the signal accuracy threshold or shortening the judgment time, it is possible to increase the judgment speed.

[0077] As described above, in the vehicle control device 20 according to this embodiment, the wireless communication unit 250 receives traffic light information including information on traffic light signals and the timing at which the signals change from the server 6, and the signal accuracy calculation unit 116 selects a detection area in which the moving object detection unit 105 searches for a moving object according to the time until the traffic light changes, and calculates signal accuracy based on the movement of the moving object detected by the moving object detection unit 105 in the selected detection area. This makes it possible to reduce the amount of processing or processing time for moving object detection and signal accuracy calculation, and improve the accuracy of signal accuracy.

[0078] The hardware configurations and flowcharts shown in the above-described first and second embodiments and modification 1 are merely examples, and can be modified or applied as desired. For example, in the above-described first embodiment, the position of the vehicle 1 is estimated based on GNSS signals, and the distance to the intersection is measured. However, the method for estimating the position of the vehicle 1 is not limited to this, and any other method may be used. For example, the position may be estimated using a wireless signal from a wireless communication network, or based on the output of a motion sensor mounted on the vehicle 1.

[0079] Furthermore, in the above-described first embodiment, moving objects around an intersection are detected from images captured by the imaging unit 220, but this is not limiting. For example, instead of the imaging unit 220, any sensor such as LiDAR (Light Detection and Ranging), RADAR (Radio Detection and Ranging), LRF (Laser Range Finder), or SONAR (Sound Navigation and Ranging) may be used to detect the movement of the moving object. Furthermore, the movement of the moving object may be detected using position information of a mobile terminal carried by a pedestrian or another vehicle. Alternatively, the movement of the moving object may be detected based on the reception status of a short-range wireless signal, such as Bluetooth (registered trademark), transmitted by a mobile terminal carried by a pedestrian or another vehicle. Alternatively, the movement of the moving object may be detected by a sensor installed in infrastructure facilities such as traffic lights and roads.

[0080] Furthermore, in the above-described first embodiment, a case where the vehicle 1 goes straight through an intersection has been described, but the present invention can also be applied to a case where the vehicle 1 turns right or left at an intersection. When an entry OK signal is determined as the traffic light signal in the direction in which the vehicle 1 is traveling, the signal accuracy calculation unit 106 may calculate the signal accuracy by consulting traffic light information acquired from the map database 230 and detecting the movement of the moving object corresponding to the traffic light signal. Based on the calculated traffic light signal accuracy, the vehicle control device 10 controls the vehicle to turn right or left, or to stop at a stop line.

[0081] Furthermore, in the above-mentioned first embodiment, the case where the traffic light signal is a steady red, green, or yellow signal has been described, but the present invention can also be applied to cases where the traffic light signal is a flashing red or yellow signal, or a signal indicating going straight, turning right or left, or a diagonal arrow. When the signal determination unit 104 references the traffic light information acquired from the map database 230 and determines from the forward image that the traffic light is a flashing red or yellow signal, or a signal indicating going straight, turning right or left, or a diagonal arrow, the signal accuracy calculation unit 106 may calculate the signal accuracy by detecting the movement of a moving object corresponding to the determined traffic light signal.

[0082] In the first embodiment, the vehicle control device 10 calculates the signal accuracy when the signal determination unit 104 determines that the signal is green, performs control to stop the vehicle when the signal accuracy is equal to or less than a threshold, and performs control to allow the vehicle to pass when the determination time has elapsed with the signal accuracy exceeding the threshold, but this is not limited to this. The vehicle control device 10 may at least perform control to stop the vehicle 1 at a stop line when the signal is determined to be green and the signal accuracy is equal to or less than the threshold.

[0083] Furthermore, in the above-described first embodiment, the vehicle control device 10 calculates the signal accuracy based on the movement of moving objects around the intersection ahead of the vehicle 1, but this is not limited to this. In calculating the signal accuracy, the vehicle control device 10 may determine the traffic light signal for vehicles or pedestrians in the intersecting lane and further use an index value based on the determination result. In calculating the signal accuracy, the vehicle control device 10 may also detect road signs indicating the speed limit, whether or not a right or left turn is permitted, etc., and detect the movement of the vehicle in accordance with the detected signs. Note that information such as the speed limit and whether or not a right or left turn is permitted may be acquired from map information in the map database 230.

[0084] Furthermore, in the above-mentioned first embodiment, the configuration is described assuming driving on the left side, but the first and second embodiments and the modified examples can be similarly applied to the case of driving on the right side. Specifically, by defining a pattern of the movement of a moving object in a detection area when an entry prohibition signal is issued for driving on the right side, the signal accuracy can be calculated based on whether or not the movement of a moving object that matches that pattern is detected.

[0085] In the second embodiment, the wireless communication unit 250 of the vehicle 3 acquires traffic light information including traffic light signals and the timing at which the signals change from the server 6 via the wireless base station 4 and the Internet 5, but this is not limiting. The wireless communication unit 250 of the vehicle 3 may receive traffic light information transmitted by a pedestrian traffic light or a vehicular traffic light, and use the received traffic light information to calculate the signal accuracy.

[0086] Furthermore, in the above-described first and second embodiments and variant example 1, examples have been described in which the processor 1011 executes a control program to realize each function, but the controller 100 may also be configured using dedicated hardware that realizes each function.

[0087] Furthermore, a control program for executing the operations of the above-described first and second embodiments and modification 1 may be stored and distributed on a computer-readable recording medium such as a CD-ROM (Compact Disc Read-Only Memory), a DVD (Digital Versatile Disc), an MO (Magneto Optical Disc), or a memory card, and the program may be installed on a computer to configure controller 100 that can realize each function. When each function is realized by sharing the work between an OS (Operating System) and an application, or by cooperation between the OS and an application, only the parts other than the OS may be stored on the recording medium.

[0088] The present invention allows various embodiments and modifications without departing from the broad spirit and scope of the present invention. Furthermore, the above-described embodiments are intended to explain the present invention and do not limit the scope of the present invention. In other words, the scope of the present invention is defined by the claims, not by the embodiments. Various modifications made within the scope of the claims and the meaning of the disclosure equivalent thereto are considered to be within the scope of the present invention. [Explanation of symbols]

[0089] 1,3 Vehicle, 2 Vehicle control system, 4 Wireless base station, 5 Internet, 6 Server, 10,20 Vehicle control device, 100 Controller, 101 Vehicle position estimation unit, 102 Map information acquisition unit, 103 Traffic light position estimation unit, 104 Signal judgment unit, 105 Moving object detection unit, 106,116 Signal accuracy calculation unit, 107 Vehicle control unit, 210 Position information acquisition unit, 220 Imaging unit, 230 Map database, 240 Actuator, 250 Wireless communication unit, 300 Vehicle lane, 301 Adjacent lane, 302 Oncoming lane, 303 Right intersecting lane, 304 Left intersecting lane, 305 Right turn waiting area, 306 Intersecting crosswalk, 307 Same direction crosswalk, 311,312,313,314,315 Other vehicles, 316 Pedestrian, 1010 Bus, 1011 processor, 1012 memory, 1013 storage, 1014 communication interface.

Claims

1. an imaging unit that captures images of the surroundings of the vehicle; a location information acquisition unit that acquires location information of the vehicle; a map database that stores map information including traffic light information; a controller that determines a traffic light signal based on the position of the traffic light estimated based on the position information and the map information and a front image of the area ahead of the vehicle captured by the imaging unit, and when the result of the determination of the traffic light signal is an entry OK signal, detects a moving object from a peripheral image of the intersection area ahead of the vehicle captured by the imaging unit, calculates a signal accuracy indicating the accuracy of the traffic light signal based on the movement of the moving object, and performs control to stop the vehicle at a stop line when the signal accuracy is equal to or less than a predetermined threshold. Vehicle control device.

2. the controller performs control to pass the stop line when the determination result of the traffic light signal is the entry permitted signal and the signal accuracy continues to be higher than the threshold value for a predetermined determination time. The vehicle control device according to claim 1 .

3. When the determination result of the traffic light signal is the entry permitted signal, the controller performs control to pass the stop line when the signal certainty continues to be higher than the threshold value associated with the determination time, the higher the signal certainty is, and the shorter the determination time is set. The vehicle control device according to claim 1 .

4. the controller performs control to decelerate the moving object to a speed equal to or lower than a predetermined speed when the moving object cannot be detected within a predetermined detection area. The vehicle control device according to claim 1 .

5. The area in which the controller detects the moving object is at least one of an adjacent lane in the same direction as the own lane in which the vehicle is traveling, an oncoming lane opposite to the own lane, an intersecting lane intersecting the own lane, a waiting area for turning right or left in the intersecting lane, an intersecting crosswalk intersecting the own lane, and a same-direction crosswalk in the same direction as the own lane, The moving object is another vehicle present in at least one of the detection areas of the adjacent lane, the oncoming lane, the intersecting lane, and the right / left turn waiting area, or a pedestrian present in at least one of the detection areas of the intersecting crosswalk and the same-direction crosswalk. The vehicle control device according to claim 1 .

6. the controller acquires, from the map information, lane information including information on dedicated right-and-left turn lanes or lane connection information from the dedicated right-and-left turn lanes to lanes in which the vehicle will travel after turning right or left, and calculates the signal accuracy based on the movement of the moving object including the movement of other vehicles in the dedicated right-and-left turn lanes. The vehicle control device according to claim 1 .

7. the controller calculates the signal accuracy based on signal information received from a server, the signal information including the traffic light signal or a timing at which the signal changes; The vehicle control device according to claim 1 .

8. the controller calculates the signal accuracy based on signal information received from a pedestrian traffic light or a vehicular traffic light, the signal information including the traffic light signal or a signal change timing; The vehicle control device according to claim 1 .

9. the controller detects the moving object in a detection area selected from the surrounding image based on the received signal information, and calculates the signal accuracy based on the movement of the detected moving object; The vehicle control device according to claim 7 or 8.

10. the controller acquires the traffic light information indicating that the traffic light signal will change within a certain period of time, and the threshold value when detecting the moving object in the detection area selected based on the traffic light information is set to a lower value than when detecting the moving object in all detection areas; The vehicle control device according to claim 9.

11. A vehicle control device according to any one of claims 1 to 8; a server that is communicatively connected to the vehicle control device and receives information indicating the position or driving state of the vehicle from the vehicle control device, or transmits road traffic information including traffic signal information to the vehicle control device; A vehicle control system comprising:

12. determining a traffic light signal based on a traffic light position estimated based on vehicle position information and map information and a forward image captured in front of the vehicle; When the determination result of the traffic light signal is an entry OK signal, a moving object is detected from a surrounding image captured around the intersection ahead of the vehicle; calculating a signal accuracy indicating the accuracy of the traffic light signal based on the movement of the moving object; When the signal accuracy is equal to or less than a predetermined threshold, control is performed to stop the vehicle at a stop line. Vehicle control method.

13. Computer, a signal determination unit that determines a traffic signal based on a position of a traffic signal estimated based on vehicle position information and map information, and a forward image captured in front of the vehicle; a moving object detection unit that detects a moving object from a surrounding image captured around the intersection ahead of the vehicle when the result of the determination of the traffic light signal by the signal determination unit is an entry OK signal; a signal accuracy calculation unit that calculates a signal accuracy indicating the accuracy of the traffic light signal based on the movement of the moving object; a vehicle control unit that performs control to stop the vehicle at a stop line when the signal accuracy is equal to or less than a predetermined threshold; A program to function as a

Images