Vehicle Control Systems

Abstract

The present invention provides a vehicle control system that, when the lighting state of a traffic light is erroneously determined, prevents controlling of a host vehicle by execution of a driving assistance function on the basis of that erroneous lighting state. The vehicle control system comprises: a lighting state determination unit 123 that determines the lighting state of a traffic light installed on a road on which a host vehicle is traveling, and calculates the reliability level of the determined lighting state of the traffic light; a dynamic object recognition unit 122 that recognizes information relating to dynamic objects that exist around the host vehicle (behavior of other vehicles, pedestrians, etc.); and a lighting state evaluation unit 131 that updates the reliability level of the light state calculated by the lighting state determination unit 123 on the basis of the information recognized by the dynamic object recognition unit 122 (information relating to dynamic objects that exist around the host vehicle).

Description

[Technical field]

[0001] The present invention relates to a vehicle control system that controls the running of a vehicle based on the illumination state of a traffic light. [Background technology]

[0002] In recent years, for automobiles and other vehicles, technologies for driving assistance, including automatic driving, have been developed that control the vehicle to drive by following a target course or a preceding vehicle. When this driving control is performed automatically, at points where traffic lights are installed, such as intersections, it is necessary to accurately determine the lighting status of the traffic lights and control the vehicle so that it can stop, start, and pass safely.

[0003] In response, a technology has been developed that can determine the lighting status of traffic lights with high accuracy based on captured images obtained by an in-vehicle camera or the like (see, for example, Patent Document 1). Also, a vehicle driving control system has been developed that can ensure safety even when the lighting status of traffic lights cannot be determined due to weather conditions such as a snowstorm, thick fog, or heavy rain (see, for example, Patent Document 2). [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2017-130163 A [Patent Document 2] JP 2018-173723 A Summary of the Invention [Problem to be solved by the invention]

[0005] However, since the method of determining the lighting state of a traffic light described in Patent Document 1 only uses a captured image as input, if the captured image contains a lot of noise, for example, due to the presence of other light sources, it is possible that the lighting state of the traffic light cannot be determined correctly and may even be erroneously determined. In this case, even if it is an erroneous determination, the lighting state of the traffic light is detected, so in the cruise control system described in Patent Document 2, for example, the vehicle will be controlled according to the erroneously determined lighting state, and it is considered that safety cannot be ensured.

[0006] On the other hand, there are many situations where dynamic objects such as other vehicles and pedestrians are present around the vehicle. These dynamic objects can be detected by sensors such as radar and lidar in addition to on-board cameras, and their behavior can be observed without relying on captured images. In addition, the behavior of dynamic objects around the vehicle changes depending on the lighting state of the same traffic light as the traffic light for the vehicle. Therefore, by observing the behavior of dynamic objects around the vehicle, it is possible to predict the lighting state of the traffic light for the vehicle and evaluate whether the lighting state of the traffic light determined from the captured image is reliable.

[0007] In view of the above, the present invention aims to provide a vehicle control system that, when controlling the vehicle, does not directly use the judgment result of the traffic light status obtained from an captured image, but rather observes dynamic objects around the vehicle and evaluates the judgment result of the traffic light status based on their behavior, thereby preventing the vehicle from being controlled based on an erroneous judgment result. [Means for solving the problem]

[0008] In order to achieve the above-mentioned object, the present invention is characterized in comprising a lighting state determination unit that determines the lighting state of a traffic light installed on a road on which the vehicle is traveling and calculates a reliability of the determined lighting state of the traffic light, a dynamic object recognition unit that recognizes information regarding dynamic objects present around the vehicle, and a lighting state evaluation unit that updates the reliability of the lighting state calculated by the lighting state determination unit based on the information recognized by the dynamic object recognition unit. Effect of the Invention

[0009] According to the present invention, the determination result of the illumination state of a traffic light obtained from a captured image can be evaluated based on the behavior of a dynamic object existing around the vehicle. This makes it possible to provide a vehicle control system that can prevent the vehicle from being controlled based on the erroneous determination result and ensure safety even when the captured image contains a lot of noise and an erroneous illumination state is determined.

[0010] Problems, configurations and effects other than those described above will become apparent from the following description of the embodiments. [Brief description of the drawings]

[0011] [Figure 1] 1 is a functional block diagram of a vehicle control system to which the present invention is applied; [Diagram 2] 4 is a flowchart showing a characteristic process flow when determining the illumination state of a traffic light. [Diagram 3] 3 shows an image of the processing content in step S105 of the flowchart shown in FIG. 2. [Figure 4] 11 is a diagram showing an example of a result of determining the lighting state of a traffic light. [Diagram 5] A diagram showing an intersection with traffic lights. [Figure 6] 4 is a flowchart showing a characteristic process flow when evaluating the illumination state of a traffic light. [Figure 7A] An example of a probability distribution table summarizing the behavior of parallel vehicles depending on the status of traffic lights. [Figure 7B] An example of a probability distribution table summarizing the behavior of crossing vehicles depending on the status of traffic lights. [Figure 8A] An example of the results of updating the reliability of a traffic light's lighting state based on the behavior of vehicles traveling alongside. [Figure 8B] An example of the results of updating the reliability of a traffic light's lighting state based on the behavior of parallel and intersecting vehicles. [Figure 9]13 is an example of a result of calculating the deviation between reliability before an update and reliability after an update. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

[0013] Fig. 1 is a functional block diagram of a vehicle control system to which the present invention is applied. The vehicle control system 1 shown in Fig. 1 is mounted on a vehicle (host vehicle) V1 and performs driving control of the vehicle V1. The vehicle control system 1 shown in Fig. 1 can automatically drive the vehicle V1 to a preset target point and can assist the driver in driving.

[0014] As shown in FIG. 1, the vehicle control system 1 includes an input device 11, a recognition device 12, a control device 13, and an output device 14.

[0015] The input device 11 includes, for example, a host vehicle behavior acquisition unit 111, a vehicle exterior environment acquisition unit 112, and a road information acquisition unit 113. The input device 11 is connected to the recognition device 12, and outputs information acquired by the host vehicle behavior acquisition unit 111, the vehicle exterior environment acquisition unit 112, and the road information acquisition unit 113 to the recognition device 12.

[0016] The host vehicle behavior acquisition unit 111 acquires information on the physical behavior of the host vehicle V1, such as the position, speed, and acceleration. The host vehicle behavior acquisition unit 111 uses, for example, a GNSS (Global Navigation Satellite System) receiving device, and internal sensors such as a gyro sensor, an acceleration sensor, and a wheel speed sensor (none of which are shown).

[0017] The vehicle exterior environment acquisition unit 112 acquires information about objects around the vehicle V1, road markings (road paint) such as white lines and stop lines on the road, traffic lights and speed signs around the road, etc. The vehicle exterior environment acquisition unit 112 uses external sensors such as a camera, a 77 GHz radar, a 24 GHz radar, a short-range lidar, a long-range lidar, and a sonar sensor (none of which are shown).

[0018] The road information acquisition unit 113 acquires, for example, road network information consisting of nodes and links, traffic regulation information, and traffic safety facility information. The road network information includes road structure information such as node details (crossroads, T-junctions, etc.) and link details (number of lanes, shape, etc.). The traffic regulation information refers to a concept including not only traffic laws but also traffic manners generally shared in society. The traffic safety facility information refers to facilities such as traffic lights and road signs that are intended to be visually recognized by drivers for traffic safety. The road information acquisition unit 113 may acquire such road information from a storage medium that stores it as needed, or may acquire such information from a server on the network as needed.

[0019] The recognition device 12 is connected to the input device 11, and creates information necessary to control the traveling of the vehicle based on various information input from the input device 11. The recognition device 12 is mainly configured as a computer including a CPU 12A. The CPU 12A includes, for example, a traveling situation recognition unit 121, a dynamic object recognition unit 122, and a lighting state determination unit 123. The recognition device 12 is connected to the control device 13, and outputs information created by the traveling situation recognition unit 121, the dynamic object recognition unit 122, and the lighting state determination unit 123 to the control device 13.

[0020] The driving condition recognition unit 121 uses, for example, the position information acquired by the vehicle behavior acquisition unit 111, the white line information acquired by the vehicle exterior environment acquisition unit 112, and the road information acquired by the road information acquisition unit 113 to recognize the lane in which the vehicle V1 is traveling and to update the position information of the vehicle V1.

[0021] The dynamic object recognition unit 122 recognizes the presence of other vehicles and pedestrians (hereinafter, these may be referred to as dynamic objects) and calculates the traveling direction and speed of the dynamic objects, for example, using the speed information of the host vehicle V1 acquired by the host vehicle behavior acquisition unit 111 and the information of objects existing around the host vehicle V1 acquired by the vehicle exterior environment acquisition unit 112. In addition, by combining with the road information acquired by the road information acquisition unit 113, the dynamic object recognition unit 122 acquires position information on the road on which the dynamic object exists. Thereby, road traffic information is created such as the recognized dynamic object being a parallel running vehicle running in a lane adjacent to the host vehicle lane, a stopped vehicle stopped in a lane adjacent to the host vehicle lane, a crossing vehicle perpendicular to the intersection (running in a lane intersecting the host vehicle lane), a stopped vehicle stopped at the intersection, a pedestrian existing at the intersection, or a pedestrian crossing a crosswalk. In addition, using the time series data of the dynamic object, predicted behavior information is created such as a parallel running vehicle or a crossing vehicle is about to stop before the stop line, or a parallel running vehicle or a crossing vehicle is about to enter the intersection without stopping before the stop line.

[0022] The lighting state determination unit 123 determines the lighting state of a traffic light for the host vehicle V1 (i.e., a traffic light installed on a road on which the host vehicle V1 is traveling) based on, for example, an image captured by an on-board camera used as the vehicle exterior environment acquisition unit 112. The lighting state here indicates any one or a combination of a general red light, yellow light, or green light, as well as the presence or absence and direction of an arrow signal, and a flashing state of a red light.

[0023] Here, a method for determining the lighting state of a traffic light for the vehicle V1 will be described with reference to Fig. 2. Fig. 2 is a flowchart showing a characteristic process flow when the lighting state determination unit 123 determines the lighting state of a traffic light for the vehicle V1. For simplicity, the flowchart shown in Fig. 2 targets traffic lights that are only red, yellow, and green.

[0024] In step S101, the lighting state determination unit 123 determines whether or not there is a self-illuminating object in the captured image acquired by the vehicle exterior environment acquisition unit 112. If it is determined that there is a self-illuminating object, step S102 is executed. If it is determined that there is no self-illuminating object, the process ends as a traffic light not being detected.

[0025] In step S102, the lighting state determination unit 123 calculates position information of the self-light-emitting object. Since the position information obtained from the captured image is a relative position with respect to the vehicle position, the absolute position information of the self-light-emitting object is calculated by combining it with the absolute position information of the vehicle calculated by the driving situation recognition unit 121. When multiple self-light-emitting objects are detected in the captured image, the position information is calculated for all of the self-light-emitting objects.

[0026] In step S103, the lighting state determination unit 123 determines whether the self-luminous object is a traffic light for the vehicle V1 by collating the absolute position information of the self-luminous object with road information obtained from the road information acquisition unit 113. If multiple self-luminous objects are detected, a determination process is executed for all the self-luminous objects. If there is one or more self-luminous objects determined to be a traffic light for the vehicle V1, step S104 is executed. Otherwise, the process is terminated as a traffic light not detected.

[0027] In step S104, the lighting state determination unit 123 extracts an area (self-light-emitting area) of a self-light-emitting object determined to be a traffic light for the vehicle V1 from the captured image acquired by the vehicle exterior environment acquisition unit 112. Then, in step S105, the lighting state determination unit 123 classifies the colors included in the extracted area of ​​the self-light-emitting object for each pixel. Finally, in step S106, the lighting state determination unit 123 calculates the proportion of {red, yellow, blue} included in the area of ​​the self-light-emitting object from the classification result.

[0028] Fig. 3 is an image of the process carried out in step S105 of the flowchart shown in Fig. 2. As shown in Fig. 3, the lighting state determination unit 123 scans all pixels contained in the area of ​​the self-luminous object and classifies them into {red, yellow, blue}. At this time, pixels that do not belong to any color are not included in the classification result as not applicable.

[0029] Figure 4 shows an example of the result of judging the lighting state of a traffic light according to the judgment method shown in Figure 2. The result in Figure 4 shows that a self-luminous object can be detected from the captured image, and the respective proportions of {red, yellow, blue} contained in the area of ​​the self-luminous object are {80%, 10%, 10%}.

[0030] The light-on state determination unit 123 determines that a light-on state in which the ratio is equal to or greater than a reference value (e.g., 75%) is the light-on state of the vehicle V1 based on the classification result as shown in Fig. 4, and outputs the result to the outside. In addition, the unit 123 outputs to the outside the ratio of the color contained in the area of ​​the self-luminous object as shown in Fig. 4 as the reliability of the light-on state of the traffic light. If the ratios of all the light-on states are less than the reference value, that is, if the ratio is ambiguous, the unit 123 terminates the process as a traffic light not being detected.

[0031] In addition to the example shown in Fig. 2, a classifier may be constructed using a machine learning model that uses a neural network, and the on / off state of the traffic light for the vehicle V1 may be determined by inference based on the machine learning method. In this case, the color corresponding to the largest output value among the output values ​​from the output layer of the neural network is determined to be the on / off state of the traffic light, and the output value from the output layer itself is treated as the reliability of the on / off state of the traffic light.

[0032] The control device 13 performs information processing related to driving control of the vehicle using various information output from the input device 11 and the recognition device 12. The control device 13 is mainly composed of a computer including a CPU 13A. The CPU 13A includes, for example, a lighting state evaluation unit 131 and a vehicle control unit 132. The control device 13 is connected to the output device 14, and outputs information created by the lighting state evaluation unit 131 and the vehicle control unit 132 to the output device 14.

[0033] The lighting state evaluation unit 131 uses information on dynamic objects such as other vehicles and pedestrians output from the dynamic object recognition unit 122 to evaluate whether the lighting state of the traffic light output from the lighting state determination unit 123 is reliable (details will be described later). The information on dynamic objects here includes not only simple position information and speed information on dynamic objects, but also road traffic information such as a parallel running vehicle or a crossing vehicle entering an intersection, a parallel running vehicle or a crossing vehicle stopping at a stop line, a pedestrian present at an intersection, and a pedestrian crossing a crosswalk. The information on dynamic objects also includes predicted behavior information such as a parallel running vehicle or a crossing vehicle trying to stop before the stop line and a parallel running vehicle or a crossing vehicle trying to enter an intersection.

[0034] The vehicle control unit 132 calculates a control command value for controlling the running of the host vehicle V1 using various information output from the input device 11, the recognition device 12, and the lighting state evaluation unit 131, and outputs the control command value to the output device 14. The control command value here includes not only control information that changes the physical state of the vehicle, such as running, turning, and stopping, via various actuators 143, but also signal information that provides information to the driver via a display device 141 (meters, etc.) and an audio output device 142 (speakers, etc.).

[0035] 1, the output device 14 includes a display unit 141, an audio output unit 142, and various actuators 143. The output device 14 is connected to the control device 13, and receives a control command value output from the control device 13 to control the display unit 141, the audio output unit 142, and the various actuators 143.

[0036] The display 141 provides various information, for example, operation information of a driving support function, as visual information to the occupants of the vehicle including the driver. The display 141 is, for example, an instrument panel or a display arranged near the driver's seat of the vehicle. The display may be a head-up display. Alternatively, a mobile phone held by the occupant, a portable information terminal including a so-called smartphone, a tablet-type personal computer, or the like may be used as part or all of the display 141.

[0037] The audio output device 142 provides various information, for example, operation information of a driving support function, to the vehicle occupants including the driver as auditory information. The audio output device 142 is, for example, a speaker disposed near the driver's seat of the vehicle. Alternatively, a speaker mounted on a mobile phone held by the occupant, a portable information terminal including a so-called smartphone, or a tablet-type personal computer may be used as part or all of the audio output device 142.

[0038] The various actuators 143 change the steering angle, acceleration / deceleration, and braking pressure of the vehicle based on control command values ​​input from the control device 13.

[0039] Fig. 5 is a diagram showing a state in which the host vehicle V1 equipped with the vehicle control system 1 shown in Fig. 1 approaches an intersection with traffic lights. In this example, the host vehicle V1 observes one vehicle V2 running parallel to the host vehicle V1 and one vehicle V3 crossing the intersection, and the dynamic object recognition unit 122 recognizes their behavior. The host vehicle V1 also detects a traffic light for the host vehicle V1, and the lighting state determination unit 123 determines that the lighting state of the traffic light is a red light and the reliability is {red, yellow, green}={80%, 10%, 10%}.

[0040] The operation of the vehicle control system 1 shown in Fig. 1 when evaluating the lighting state of a traffic light will be described with reference to Fig. 6. Fig. 6 is a flowchart showing a characteristic process flow when the vehicle control system 1 shown in Fig. 1 evaluates the lighting state of a traffic light at an intersection with traffic lights as shown in Fig. 5. The flowchart shown in Fig. 6 is executed periodically (cyclically) in this example, but the process may be started, for example, when approaching an intersection with traffic lights or when detecting a traffic light installed at the intersection.

[0041] In step S201, the lighting state evaluation unit 131 predicts the behavior of a parallel running vehicle depending on the lighting state of a traffic light at the intersection. Specifically, a probability distribution table as shown in FIG. 7A is read. The probability distribution table shown in FIG. 7A is a probability distribution table expressing the behavior prediction of a parallel running vehicle (dynamic object) depending on the lighting state of a traffic light at the intersection in terms of probability, and is a table in which a designer has set in advance the lighting state of a traffic light for the own lane and the probability that a parallel running vehicle will stop before the stop line for each lighting state. For example, when the traffic light for the own lane is red, it shows that the probability that a parallel running vehicle will stop before the stop line is 95%, and the probability that it will not stop is 5%. The probability distribution table shown in FIG. 7A is desirably set by statistically calculating road traffic conditions for each intersection.

[0042] In the probability distribution table shown in Fig. 7A, the behavior of the parallel running vehicle for three patterns of the lighted state of the traffic light, red, yellow, and green, is set, but the behavior of the parallel running vehicle for patterns such as an arrow signal or a flashing red signal may be set. In addition, two patterns of the behavior of the parallel running vehicle, that is, stopping at the stop line and not stopping, are set, but the behavior is not limited to these patterns and any pattern that the parallel running vehicle may take can be set. However, it is preferable that the behavior of the parallel running vehicle set at this time be one that can be properly observed by the vehicle itself. In addition, the probability value set may be a fixed value or may be a function of the speed of the parallel running vehicle and the distance to the stop line.

[0043] In step S202, the lighting state evaluation unit 131 predicts the behavior of a crossing vehicle with respect to the lighting state of a traffic light at the intersection shown in FIG. 5. Specifically, a probability distribution table as shown in FIG. 7B is read. The probability distribution table shown in FIG. 7B is a probability distribution table expressing the behavior prediction of a crossing vehicle (dynamic object) according to the lighting state of a traffic light at the intersection in terms of probability, and is a table in which a designer has set in advance the lighting state of a traffic light for the own lane and the probability that a crossing vehicle will stop before the stop line for each lighting state. For example, when the traffic light for the own lane is red, the traffic light for the crossing vehicle is green, so that it shows that the probability that the crossing vehicle will enter the intersection is 90% and the probability that it will stop at the stop line is 10%. The probability distribution table shown in FIG. 7B is preferably set by statistically calculating road traffic conditions for each intersection.

[0044] In Fig. 7B, similarly to Fig. 7A, the behavior of an intersecting vehicle may be set in response to a pattern such as an arrow signal or a flashing red light as the lighting state of a traffic light. Also, a pattern that an intersecting vehicle may take may be added or changed. However, it is preferable that the behavior of the intersecting vehicle set in this case is one that can be appropriately observed by the vehicle itself. Also, the probability value set may be a fixed value or may be a function of the speed of the intersecting vehicle or the distance to the stop line.

[0045] In step S203, the lighting state evaluation unit 131 acquires the reliability of the lighting state of the traffic light for the vehicle from the lighting state determination unit 123. In the example of the intersection shown in Fig. 4, the lighting state evaluation unit 131 acquires {red, yellow, green} = {80%, 10%, 10%} from the lighting state determination unit 123 as the reliability.

[0046] In step S204, the lighting state evaluation unit 131 acquires information on parallel vehicles and crossing vehicles from the dynamic object recognition unit 122. Specifically, it acquires predicted behavior information such as whether the parallel vehicles set in Fig. 7A are going to stop before the stop line or not. It also acquires road traffic information such as whether the crossing vehicles set in Fig. 7B are entering the intersection or are stopped at the stop line.

[0047] When the dynamic object recognition unit 122 predicts a behavior, if the behavior cannot be clearly predicted, it acquires the probability of each behavior occurring. For example, if a vehicle traveling parallel to the vehicle is decelerating, but the predicted stopping point calculated from the deceleration is beyond the stop line, it acquires a value such as a 70% probability that the vehicle will stop before the stop line, and a 30% probability that the vehicle will not stop.

[0048] In step S205, the lighting state evaluation unit 131 first updates the reliability of the lighting state of the traffic light based on the behavior of the parallel running vehicle. Specifically, the reliability is calculated using the following formula [Mathematical formula 1] by using Bayes' theorem.

number

[0049] P(T i |X j ) is the behavior of the parallel vehicle X j When the traffic light is observed as T i It represents the probability that

[0050] FIG. 8A shows the P(T i |X j) is calculated. For example, when the dynamic object recognition unit 122 observes that a parallel vehicle stops before the stop line, the probability that the traffic light for the vehicle is red is 91%, which means that the probability that the traffic light is red is higher than the reliability of red before the update of 80%. On the other hand, when the dynamic object recognition unit 122 observes that a parallel vehicle does not stop before the stop line in FIG. 8A, the probability that the traffic light for the vehicle is red has decreased to 25% (from red before the update of 80%), and it can be considered that the judgment result of the lighting state judgment unit 123 may be erroneous.

[0051] 8A also shows an example in which the behavior of a parallel running vehicle observed by the dynamic object recognition unit 122 is expressed as a probability. If the probability that the parallel running vehicle stops before the stop line is P(X1) and the probability that the parallel running vehicle does not stop before the stop line is P(X2), the reliability of the lighting state of the traffic light can be calculated by the following formula [Mathematical formula 2]. Here, P X (T i ) is the reliability of the updated traffic light status based on the behavior of parallel vehicles.

number

[0052] In step S205, the lighting state evaluation unit 131 then further updates the reliability of the lighting state of the traffic light based on the behavior of the crossing vehicle. Specifically, the reliability is calculated using the following equation [Mathematical formula 3] by using Bayes' theorem.

number

[0053] P(T i |X j ,Y k ) is the behavior of the parallel vehicle X j , the behavior of the crossing vehicle is Y k When the traffic light is observed as T i It represents the probability that

[0054] FIG. 8B shows the P(T i |X j ,Y k ) is calculated. For example, when the dynamic object recognition unit 122 observes that a parallel vehicle stops before the stop line and that an intersecting vehicle is entering the intersection, the probability that the light status of the traffic light for the vehicle is red is 100%, and compared with the reliability of red before the update = 80%, it is considered that the light is almost certainly red. On the other hand, in FIG. 8B, when it is observed that a parallel vehicle does not stop before the stop line and that an intersecting vehicle has stopped at the stop line, the probability that the light status of the traffic light for the vehicle is red has dropped to 3% (from red before the update = 80%), and it can be considered that the judgment result of the light status judgment unit 123 is erroneous with a high probability.

[0055] 8B also shows an example in which the behavior of a parallel running vehicle and an intersecting vehicle observed by the dynamic object recognition unit 122 is expressed in terms of probability. If the probability that a parallel running vehicle stops before the stop line is P(X1), the probability that a parallel running vehicle does not stop before the stop line is P(X2), the probability that an intersecting vehicle is entering the intersection is P(Y1), and the probability that an intersecting vehicle is stopped at the stop line is P(Y2), then the reliability of the lighting state of the traffic light can be calculated using the following formula [Mathematical formula 4]. Here, P XY (T i ) is the reliability of the updated traffic light status based on the behavior of parallel and intersecting vehicles.

number

[0056] In step S206, the lighting state evaluation unit 131 calculates the degree of deviation between the reliability (before updating) output from the lighting state determination unit 123 and the reliability (after updating) calculated in step S205, thereby evaluating the reliability of the lighting state of the traffic light determined by the lighting state determination unit 123. The degree of deviation is calculated using, for example, the following equation [Mathematical formula 5].

number

[0057] Fig. 9 is an example of calculating the deviation degree for the reliability obtained in Fig. 8B. In the "pattern in which the parallel vehicles do not stop before the stop line and the intersecting vehicles stop at the stop line" where the possibility of the light being red has fallen to 3%, the deviation degree shows a large value compared to other patterns. In other words, when the deviation degree shows a large value, it can be said that there is a high possibility that the light state of the traffic light determined by the light state determination unit 123 is incorrect.

[0058] In step S207, the vehicle control unit 132 makes a decision regarding the driving assistance function at an intersection with a traffic light. The vehicle control unit 132 basically determines the vehicle's driving contents (control contents) based on the lighting state (reliability) of the traffic light determined by the lighting state determination unit 123, but at this time, if the deviation degree of the reliability obtained from the lighting state evaluation unit 131 in step S206 is equal to or greater than a reference value, the lighting state of the traffic light determined by the lighting state determination unit 123 is determined to be incorrect, and the driving assistance function is not activated. If the driving assistance function is already in operation, the driving assistance function is interrupted and the driver is notified of this, or a degenerate operation is performed to avoid danger. In addition, if the deviation degree of the reliability continues to be higher than the reference value, the recognition processing system may be treated as a failure.

[0059] In this way, by setting the behavior (prediction) of other vehicles in response to the lighting state of the traffic lights at the intersection in advance as a probability distribution, the reliability of the lighting state of the traffic lights obtained from the captured image can be updated based on the behavior of other vehicles present around the vehicle. Also, by evaluating the degree of deviation of the reliability before and after the update, and interrupting the driving assistance function or treating it as a failure when the degree of deviation is greater than a reference value, it is possible to prevent the driving assistance function from being executed based on the erroneous lighting state and the vehicle from being controlled even in a scene where the lighting state of the traffic lights is erroneously determined.

[0060] Although the preferred embodiments of the present invention have been described above, the present invention is not limited to the above-described embodiments and can be modified in various ways without departing from the spirit and scope of the present invention.

[0061] For example, even in a case where the vehicle itself judges the lighting state of a pedestrian traffic light, the judgment result can be evaluated by the vehicle control system 1 shown in Fig. 1. In this case, instead of Fig. 7A and Fig. 7B, a probability distribution table can be used in which the probability of each of the pedestrian behaviors {crossing the crosswalk, not crossing the crosswalk} in response to the lighting state of the pedestrian traffic light {red, green, flashing green} is set.

[0062] As described above, the vehicle control system 1 of this embodiment includes a lighting state determination unit 123 that determines the lighting state of traffic lights installed on the road on which the host vehicle is traveling and calculates the reliability of the determined lighting state of the traffic light, a dynamic object recognition unit 122 that recognizes information about dynamic objects present around the host vehicle (the behavior of other vehicles, pedestrians, etc.), and a lighting state evaluation unit 131 that updates the reliability of the lighting state calculated by the lighting state determination unit 123 based on the information recognized by the dynamic object recognition unit 122 (information about dynamic objects present around the host vehicle).

[0063] The vehicle control system 1 (the vehicle control unit 132 thereof) determines the control content of the host vehicle based on the reliability updated by the lighting state evaluation unit 131.

[0064] More specifically, the lighting state evaluation unit 131 calculates the degree of deviation between the reliability of the lighting state calculated by the lighting state determination unit 123 and the reliability updated based on the information recognized by the dynamic object recognition unit 122.

[0065] The vehicle control system 1 (the vehicle control unit 132 thereof) determines the control content of the host vehicle based on the magnitude of the deviation degree calculated by the lighting state evaluation unit 131.

[0066] According to this embodiment, the determination result of the lighting state of a traffic light obtained from a captured image can be evaluated based on the behavior of a dynamic object existing around the vehicle. As a result, even if the captured image contains a lot of noise and the lighting state is determined erroneously, it is possible to provide a vehicle control system 1 that can prevent the vehicle from being controlled according to the erroneous determination result and ensure safety.

[0067] The present invention is not limited to the above-described embodiment, but includes various modified examples. For example, the above-described embodiment has been described in detail to easily explain the present invention, and the present invention is not necessarily limited to the embodiment having all of the described configurations.

[0068] In addition, the above-mentioned configurations, functions, processing units, processing means, etc. may be realized in part or in whole by hardware, for example, by designing them as integrated circuits. In addition, the above-mentioned configurations, functions, etc. may be realized in software by a processor interpreting and executing a program that realizes each function. Information such as the program, table, file, etc. that realizes each function can be stored in a storage device such as a memory, a hard disk, or an SSD (Solid State Drive), or in a recording medium such as an IC card, an SD card, or a DVD.

[0069] In addition, the control lines and information lines shown are those that are considered necessary for the explanation, and not all control lines and information lines in the product are necessarily shown. In reality, it can be considered that almost all components are connected to each other. [Explanation of symbols]

[0070] 1 Vehicle Control System 11 Input Devices 111 Vehicle behavior acquisition unit 112 External environment acquisition department 113 Road information acquisition department 12 Recognition device 121 Driving situation recognition unit 122 Dynamic Object Recognition Unit 123 Lighting status determination unit 13 Control device 131 Lighting status evaluation unit 132 Vehicle control unit 14 Output Devices 141 Display 142 Audio output device 143 Various Actuators

Claims

1. a lighting state determination unit that determines a lighting state of a traffic light installed on a road on which the vehicle is traveling and calculates a reliability of the determined lighting state of the traffic light; a dynamic object recognition unit that recognizes information about dynamic objects present around the host vehicle; a lighting state evaluation unit that updates the reliability of the lighting state calculated by the lighting state determination unit based on information recognized by the dynamic object recognition unit, A vehicle control system characterized in that the lighting state determination unit extracts an area of ​​a self-luminous object from an captured image, and determines the lighting state of the traffic light and calculates its reliability based on the proportion of colors contained in the area of ​​the self-luminous object.

2. a lighting state determination unit that determines a lighting state of a traffic light installed on a road on which the vehicle is traveling and calculates a reliability of the determined lighting state of the traffic light; a dynamic object recognition unit that recognizes information about dynamic objects present around the host vehicle; a lighting state evaluation unit that updates the reliability of the lighting state calculated by the lighting state determination unit based on information recognized by the dynamic object recognition unit, A vehicle control system characterized in that the lighting state evaluation unit updates the reliability of the lighting state using a probability distribution that expresses, in terms of probability, a prediction of the behavior of a dynamic object according to the lighting state of a traffic light.

3. a lighting state determination unit that determines a lighting state of a traffic light installed on a road on which the vehicle is traveling and calculates a reliability of the determined lighting state of the traffic light; a dynamic object recognition unit that recognizes information about dynamic objects present around the host vehicle; a lighting state evaluation unit that updates the reliability of the lighting state calculated by the lighting state determination unit based on information recognized by the dynamic object recognition unit, A vehicle control system characterized in that the lighting state evaluation unit calculates the degree of discrepancy between the reliability of the lighting state calculated by the lighting state determination unit and the reliability updated based on information recognized by the dynamic object recognition unit.

4. 4. The vehicle control system according to claim 1, wherein the dynamic object recognition unit recognizes at least one of the behaviors of a vehicle traveling or stopped in a lane adjacent to the own lane, a vehicle traveling or stopped in a lane intersecting the own lane, and a pedestrian at an intersection.

5. The vehicle control system according to claim 2 or 3, characterized in that the lighting state determination unit extracts an area of ​​a self-luminous object from an image capture image, and determines the lighting state of the traffic light and calculates a reliability based on the proportion of colors contained in the area of ​​the self-luminous object.

6. 4. The vehicle control system according to claim 2, wherein the lighting state determination unit determines the lighting state of the traffic signal and calculates a reliability thereof by inference based on a machine learning technique.

7. The vehicle control system according to claim 3 , characterized in that the lighting state evaluation unit updates the reliability of the lighting state using a probability distribution that expresses a prediction of the behavior of a dynamic object according to the lighting state of a traffic light in terms of probability.

8. 3. The vehicle control system according to claim 1, wherein the vehicle control system determines a control content for the host vehicle based on the reliability updated by the lighting state evaluation unit.

9. The vehicle control system according to claim 3 , wherein the vehicle control system determines a control content for the host vehicle based on the magnitude of the deviation calculated by the lighting state evaluation unit.

10. A lighting state determination unit that determines a lighting state of a traffic light installed on a road on which the vehicle is traveling and calculates a reliability of the determined lighting state of the traffic light; a dynamic object recognition unit that recognizes information about dynamic objects present around the host vehicle; a lighting state evaluation unit that updates the reliability of the lighting state calculated by the lighting state determination unit based on information recognized by the dynamic object recognition unit, A vehicle control system characterized in that the lighting state evaluation unit updates the reliability of the lighting state calculated by the lighting state determination unit based on the direction of travel and speed of the dynamic object recognized by the dynamic object recognition unit.

Images