System and method for traffic light recognition

Abstract

A traffic signal fixture verification system (1) is provided, the system comprising an optical recognition device (15) and a processing device (10), the optical recognition device (15) being configured to obtain one or more images of two or more traffic signal fixtures near an autonomous vehicle (2) and detect the motion state of one or more vehicles near the autonomous vehicle, the processing device (10) being configured to determine a correlation value of two or more traffic signal fixtures identified in the one or more images, monitor the state of each of two or more related traffic signal fixtures having a correlation value greater than a threshold, and actuate an accelerator function or a brake function of the autonomous vehicle based on the correlation value of the two or more related traffic signal fixtures, the corresponding state of each of the two or more related traffic signal fixtures, and optionally, and / or the motion state of one or more vehicles around the vehicle.

Description

Technical Field

[0001] The present disclosure relates to systems and methods for road indicator recognition, and more particularly, to improved recognition of the state of one or more traffic lights. Background Art

[0002] There are various systems for providing assistance to drivers of motor vehicles. For example, one such area of ​​assistance relates to automatic traffic sign recognition.

[0003] Traffic sign recognition can be achieved by using suitable equipment (e.g., optical equipment such as cameras) for sensing or detecting the surroundings of a motor vehicle. Systems are commercially available from, for example, automobile manufacturers as well as manufacturers of portable navigation devices (PNDs). PND systems are based on GPS signals and map data for providing information related to traffic signs.

[0004] Some existing systems implement camera equipment for improving recognition accuracy and robustness. Vehicle manufacturers use front-mounted camera equipment and means for combining the signal with data related to the driving state of the vehicle. A further option is means for combining the signal with navigation equipment.

[0005] US 2016 / 0306361 discloses providing autonomous navigation features using a camera. A traffic light detection system is provided for a vehicle, wherein one or more processing devices associated with the system receive at least one image of an area in which the vehicle is traveling via a data interface, wherein the area includes at least one traffic light fixture having at least one traffic light. The processing device determines whether the vehicle is in a turning lane based on at least one indicator of the vehicle's position. In addition, the processing device processes the received image to determine the state of the traffic light, including whether the traffic light includes an arrow. In addition, the system can cause a system response based on the determination of the state of the traffic light, whether the traffic light includes an arrow, and whether the vehicle is in a turning lane. Summary of the invention

[0006] The inventors have recognized that under certain conditions, it may be difficult or even impossible to verify or even determine the status of a traffic signal applicable to an ego vehicle. Such difficulty may be caused by blocked views (occlusions), inoperable traffic signals, unrecognizable traffic signals, irrelevant traffic signals (e.g., tramway signals), etc.

[0007] Therefore, the present arrangement aims to solve these problems, with the aim of achieving autonomous driving and / or driving assistance with improved safety.

[0008] According to an embodiment of the present disclosure, a traffic signal fixture verification system for an autonomous vehicle is provided. The system includes an optical recognition device and a processing device, wherein the optical recognition device is configured to obtain one or more images of two or more traffic signal fixtures near the autonomous vehicle and detect the motion state of one or more vehicles near the autonomous vehicle. The processing device is configured to determine a correlation value of two or more traffic signal fixtures identified in one or more images, monitor the state of each of two or more related traffic signal fixtures having a correlation value greater than a threshold, and based on the correlation value of the two or more related traffic signal fixtures, the corresponding state of each of the two or more related traffic signal fixtures, and optionally, the motion state of one or more vehicles around the vehicle, actuate an accelerator function or a brake function of the autonomous vehicle.

[0009] By providing such a system, it becomes possible to identify or imply a traffic signal state and verify that state before an action is captured by an autonomous driving system, or before driver assistance is provided (e.g., assisting with braking and / or steering).

[0010] When the state of one of two or more related traffic signal fixtures cannot be determined, the state of another of the two or more related traffic signal fixtures may be used for actuation.

[0011] When the states of two or more correlated traffic signal features are not identical, the state of the correlated traffic signal having a higher correlation value may be used for actuation.

[0012] When the states of two or more related traffic signal features are different, the state of the related traffic signal having the highest safety factor may be used for actuation.

[0013] The state with the highest safety factor may correspond to a red light of a traffic signal fixture.

[0014] The processing device may be configured to receive map data associated with the location of the traffic signal fixture and to use the map data in the correlation determination.

[0015] The distance of each of the two or more relevant traffic signals from the ego vehicle may be considered in the correlation value determination.

[0016] Upon detecting a change in the motion state of one or more vehicles surrounding the vehicle that is greater than a threshold percentage change, the change in motion state may be considered for actuation.

[0017] According to other embodiments of the present disclosure, a method for verifying the state of a traffic signal fixture is provided. The method includes obtaining one or more images of two or more traffic signal fixtures near an autonomous vehicle, detecting the motion state of one or more vehicles near the autonomous vehicle, determining a correlation value of two or more traffic signal fixtures identified in the one or more images, monitoring the state of each of two or more related traffic signal fixtures having a correlation value greater than a threshold, and based on the correlation value of the two or more related traffic signal fixtures, the corresponding state of each of the two or more related traffic signal fixtures, and optionally, the motion state of one or more vehicles around the vehicle, actuating an accelerator function or a brake function of the autonomous vehicle.

[0018] When the state of one of the two or more related traffic signal fixtures cannot be determined, the state of another of the two or more related traffic signal fixtures may be used for actuation.

[0019] When the states of two or more correlated traffic signal features are not identical, the state of the correlated traffic signal having a higher correlation value may be used for actuation.

[0020] When the states of two or more related traffic signal features are different, the state of the related traffic signal having the highest safety factor may be used for actuation.

[0021] The method may include receiving map data associated with the location of the traffic signal fixture and using the map data in the correlation determination.

[0022] Upon detecting a change in the motion state of one or more vehicles surrounding the vehicle that is greater than a threshold percentage change, the change in motion state is considered for actuation.

[0023] Combinations of the elements described above and within the specification are possible unless contradictory.

[0024] It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure, as claimed.

[0025] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments of the present disclosure and, together with the description, serve to explain the principles thereof. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 An exemplary driver assistance system according to an embodiment of the present disclosure is shown;

[0027] Figure 2is a diagram showing an exemplary vehicle including a driver assistance system while at an exemplary intersection on a road;

[0028] Figure 3 is similar to Figure 2 A top view of an exemplary situation;

[0029] Figure 4A Exemplary traffic signals and their associated states are shown;

[0030] Figure 4B is an exemplary actual intersection including a plurality of traffic signals;

[0031] Figure 5A and 5B Two additional exemplary intersections including multiple traffic signals are shown;

[0032] Fig. 6A is a flowchart highlighting an exemplary method according to an embodiment of the present disclosure; and

[0033] Figure 6B An exemplary flow chart for evaluating and verifying one or more security factors according to an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0034] Reference will now be made to exemplary embodiments of the present disclosure, examples of which are illustrated in the accompanying drawings. Wherever possible, the same reference numerals will be used throughout the drawings to refer to the same or like parts.

[0035] Figure 1 An exemplary driver assistance system 1 according to an embodiment of the present disclosure is shown. The driver assistance system 1 may include a processing device (e.g., an electronic control unit (ECU) 10), an image acquisition device 15 (e.g., a camera), one or more sensors 20, and a display / actuator 25, etc. According to some embodiments, a global positioning system (GPS) 27 may be optionally provided as needed, for example, to provide additional data and maps about the location to the ECU 10.

[0036] The display / actuator 25 may be configured to display information provided by the ECU 10 to the driver of the vehicle 2 and / or to issue commands to cause actuation of various vehicle systems in autonomous or assisted driving scenarios (e.g., acceleration, braking, steering, etc.).

[0037] The display 25 may be any suitable device for providing visible and / or audible information to the driver of the vehicle 2. For example, the display 25 may include a heads-up display (e.g., on the windshield in front of the driver), a monitor, a dashboard display, etc. Further, where an actuator 25 is provided, such device may be any suitable device for controlling various vehicle systems.

[0038] The image acquisition device 15 may include, for example, one or more cameras and / or other suitable devices configured to obtain optical data from an area surrounding the vehicle (e.g., in front of the autonomous vehicle). The image acquisition device 15 may be configured to process the data obtained from the surroundings of the vehicle to determine the presence of traffic signals 310, 320, 330, 340, 350 and road intersections (i.e., intersections at which the vehicle may turn or continue straight on the current road). Such image acquisition devices 15 are known in the art, and those skilled in the art will understand that any such image acquisition device 15 may be implemented in the present system without departing from the scope of the present disclosure. Examples of such image acquisition devices are fisheye cameras, telephoto cameras, and typical cameras that are currently on many vehicles in the automotive market.

[0039] Image acquisition devices 15 may be located on the vehicle 2 to provide a reasonable field of view 4 (e.g., a front view and side view spanning approximately 180 degrees) around the vehicle 2. For example, one or more image acquisition devices 15 may be located behind the windshield, on the front bumper, in the side mirrors, in the rearview mirrors, and / or other suitable mounting locations on the vehicle 2 to provide a field of view 4 of traffic signals 310, 320, 330, 340, 350 near the autonomous vehicle and at intersections with adjacent vehicles.

[0040] According to some embodiments, it may be desirable to minimize the visibility of the image acquisition device 15 for aesthetic reasons, and those skilled in the art will appreciate that finding a mounting location suitable to achieve this goal while also providing adequate visibility around the vehicle 2 is a legitimate consideration.

[0041] As used herein, the term "reasonable" when referring to field of view shall mean providing the image acquisition device 15 with a field of view of at least 150 degrees of spherical coverage in front and around the road around the autonomous vehicle, preferably 160 degrees, even more preferably 170 degrees, and even more preferably 180 degrees.

[0042] like Figure 2 and 3As shown, the FOV 4 of the image acquisition device 15 can enable the capture and detection of one or more traffic signals 310, 320, 330, 340, 350 from within the FOV 4. However, some traffic signals (e.g., traffic signals 310 and 350) may be outside the FOV of the image acquisition device 15, or may be blocked from the view of the image acquisition device 15 by (e.g., the vehicle 3). Therefore, it may be difficult to verify the signal state of one or more traffic signals applicable to the autonomous vehicle 2 (e.g., red, yellow, green, red turn signal, yellow turn signal, green turn signal, etc.) in some cases, and as described in more detail below, embodiments of the present disclosure may be implemented to assist in such verification.

[0043] The image acquisition device 15 may be configured to provide the ECU 10 with data about the surroundings of the vehicle 2, including traffic signals 310, 320, 330, 340, 350 and crossroads. Such data may include, for example, the location of the traffic signals relative to the autonomous vehicle 2, the state of each detected traffic signal (e.g., red, yellow, green, turn signal, etc.),

[0044] Image acquisition device 15 may provide such data to ECU 10 via a wired connection, a wireless connection, or other suitable method for transmitting data to ECU 10. For example, image acquisition device 15 may include wireless communication means (e.g., IEEE 802.11 compatible Wi-Fi hardware) for sending data to ECU 10 and / or other devices that may use data from image acquisition device 15. Alternatively or additionally, a wired connection may be provided, such as for safety purposes. For example, such a wired connection may be provided to provide fail-safety in the event that a wireless connection should cease to function.

[0045] When obtaining data related to a traffic signal, the image acquisition device 15 may be configured to assign a time indicator (e.g., a timestamp) and / or a location indicator (e.g., coordinate information) to the data. Alternatively, the ECU 10 may be configured to assign a time indicator to the data upon receipt from the image acquisition device 15. By assigning a time indicator to data obtained by the image acquisition device 15, the age of the data (i.e., the time elapsed since the traffic signal state was identified) may be tracked by the ECU 10 or the like. This may be particularly useful when beginning to "learn" traffic signal timing by the ECU 10.

[0046] The ECU 10 may include any suitable device configured to manipulate data, perform calculations, execute code for decision making, and display information to the operator of the vehicle 2 and / or actuate various systems of the autonomous vehicle 2 (e.g., accelerator, brakes, gear shifting, etc.) in order to perform embodiments of the present disclosure. For example, the ECU 10 may include various analog and / or digital circuits, and may include integrated circuits such as RISC processors, i386 processors, ASIC processors, etc. Typically, on-board computers in modern vehicles include such processors, and those skilled in the art will understand that the present ECU 10 may be included by such on-board computers, or may be provided separately. Those skilled in the art will also understand that the exemplary circuits and processors described herein are not intended to be limiting, and any suitable device may be implemented.

[0047] ECU 10 may be linked to one or more databases and / or other memories (e.g., RAM, ROM, etc.) associated with autonomous vehicle 2 to enable storage of vehicle-related data and values ​​(e.g., thresholds, correlation values) that may be used during processing of vehicle functions, such as the duration of a traffic signal in a particular state (e.g., a typical red light duration is 45 seconds, a yellow light duration is 2 seconds, etc.). Those skilled in the art will recognize that the information discussed herein with respect to any such databases and / or memories is not intended to be limiting.

[0048] The ECU 10 may be configured to receive data from the image acquisition device 15 and one or more sensors 20 that provide functionality associated with the present disclosure. For example, the ECU 10 may receive map data regarding traffic signals from the GPS 27, which may be used to verify a particular traffic signal configuration at an identified or even unidentified intersection. While the GPS 27 may be implemented to provide data such as a failsafe, embodiments of the present disclosure may also function in the absence of such map data.

[0049] Based on the data and signal status information provided to ECU 10 , ECU 10 may be configured to perform various operations to determine relevance, verify status, and set operational flags related to operation of autonomous vehicle 2 based on one or more identified traffic signals.

[0050] Figure 4A The three main state possibilities of a single traffic signal are shown, namely, a green signal on the far left indicating that the vehicle may proceed; a red signal on the far right indicating that the vehicle must stop; and a yellow signal in the middle indicating an urgent change from red to green or from green to red. Similarly, Figure 4B Shown is a situation where vehicles turning left are allowed to proceed (special green arrow signal on the left), but vehicles continuing straight or turning right are required to stop (red signal on the right).

[0051] Figure 5A 2 shows a situation where both traffic signal groups (i.e., Group A and Group B) indicate a green state (i.e., go), while Figure 5B A situation is shown where two overhead traffic signals (Group A) and a side traffic signal (Group B) indicate a red state (i.e., stop). Those skilled in the art will note that in each of the examples provided, occlusion of the image acquisition device 15 and / or FOV 4 may prevent the ECU 10 from providing information about all traffic signals present at the intersection, and therefore, prevent the ECU 10 from making a sufficiently safe determination as to how to notify the driver of the automated vehicle 2 and / or operate the vehicle in an automated driving scenario.

[0052] In a typical scenario, two or more traffic signals at a single intersection may have a signal level above the threshold V th A traffic signal may have a high correlation value, while a second traffic signal may have a lower correlation value; a traffic signal having a high correlation value V th One or more traffic signals may be visible while one or more other traffic signals may be blocked. Therefore, verification of the traffic signal status applicable to autonomous vehicle 2 is desirable.

[0053] FIG. 6 is a flowchart highlighting an exemplary method for performing an embodiment of the present disclosure and can be referenced by Figure 2 For the purpose of explaining this example, assume that the autonomous vehicle 2 is now stopped at an intersection with two or more traffic signals (e.g., Figure 3 However, those skilled in the art will recognize that the systems and methods employed herein may also be used to perform traffic signal verification for a moving vehicle traveling through an intersection or other locations where multiple traffic signals are present and detected.

[0054] When the autonomous vehicle 2 moves through or reaches a traffic scene, the image acquisition device 15 may acquire an image of the surroundings (e.g., lane information) of the autonomous vehicle, and where there are, for example, one or more traffic signals near the autonomous vehicle 2 (step 610). Figure 3 As shown, one or more cameras present on autonomous vehicle 2 may detect traffic signals 320 , 330 , and 340 based on available field of view and occlusions (eg, truck 3 blocking signal 310 ).

[0055] Those skilled in the art will recognize that map information from GPS 27 may be implemented to compare traffic signal configuration and placement at a particular location with information obtained from image acquisition device 15, as well as to determine information regarding the lane that autonomous vehicle 2 is in. Such information from GPS 27 may be useful, for example, in determining relevance values ​​and vehicle direction of travel.

[0056] When the image acquisition device has captured an image that includes one or more traffic signals, a relevance factor may be assigned by the ECU 10 to each of the detected traffic signals in the image (step 615) and the traffic signals grouped by relevance. For example, traffic signals having a relevance value between 8 and 10 may be grouped together, 5 and 8 together, etc. These "groups" of signals may then form the basis for further processing and evaluation within the system.

[0057] When discussing values ​​and threshold correlation values ​​for correlation, it may be assumed that such correlation ranges and thresholds may be arbitrarily set based on the desired level of accuracy and safety factors. For example, correlation values ​​may vary between 0 and 1. Alternatively, correlation values ​​may range from 0 to 10, 1 to 100, etc.

[0058] The relevance value may be determined based on, for example, the location of the traffic signal in the image relative to autonomous vehicle 2, the determined direction of travel of autonomous vehicle 2, the state of the traffic signal, the location relative to other traffic signals, etc. For example, where it is determined that the direction of travel of autonomous vehicle 2 remains unchanged, a traffic signal indicating a "left turn priority" (i.e., a green left turn arrow) state may be determined to have a lower relevance than a threshold for determining an effect on autonomous vehicle 2. Similarly, one or more signals directly ahead or in a straight line in the direction of travel of autonomous vehicle 2 may be given the highest priority (e.g., a value of 10).

[0059] When the ego vehicle 2 is considered to be heading straight ahead, traffic signals that only have turn indicators and / or are in-line with the turn lane may be assigned a relevance value of 5, for example.

[0060] Similarly, tramway signals, e.g., signals marked with a particular color (e.g., white) and / or shape (e.g., white -- and / or |), the system may assume that such signals are not relevant to the autonomous vehicle 2 and may assign, e.g., a relevance value of 0.

[0061] Those skilled in the art will appreciate that tramway signals may generally be disregarded, although they may be helpful for certain verification purposes. For example, where the relevant signal indicates green (i.e., go) but there is a tramway crossing in the driving line and the tramway signal indicates that the tram may go, the ECU 10 may determine that braking or stopping action is required to maintain a safe level.

[0062] According to some embodiments, if a tramway signal is detected (or suspected), ECU 10 may expect relevant traffic signals for vehicles at the intersection. If the information from the tramway signal and the information from the vehicle traffic signal are complementary (i.e., the tramway signal indicates stop, and the traffic signal is green indicating that autonomous vehicle 2 may pass), then the determination regarding the action of autonomous vehicle 2 may be made more reliably. This is true even if other relevant traffic signals are obscured (e.g., by truck 3).

[0063] According to another example, in which a first traffic signal 350 is located in front of the left side of autonomous vehicle 2 and a second traffic signal 310 is located in front of the right side of autonomous vehicle 2, the correlation may be evenly assigned to each traffic signal (e.g., a value of 8) so that in the absence of other traffic signals, for each signal 310 and 350, the correlation with autonomous vehicle 2 remains above a threshold value (e.g., 4), and the signals are combined into a signal group (A).

[0064] After the traffic signals have been identified and grouped, the system determines and monitors the current state of each traffic signal (e.g., red, yellow, green), and performs a comparison between the traffic signals in the signal group (step 620). If the states of all traffic signals in the signal group are consistent (e.g., all signals display green, all signals display red, etc.) (step 620: yes), an operation flag may be set according to the consistent state of the traffic signals in the group (step 635) (e.g., green = maintain / accelerate, yellow / red = slow / stop).

[0065] After setting the operation flag, the ECU 10 may issue commands to various systems of the autonomous vehicle 2 to implement the operation (step 640). For example, in the case of a red / yellow traffic signal state, the braking system may be actuated or maintained. In the case of a green traffic signal state, acceleration or current speed maintenance may be caused. In another example, in the case of a green turn signal, the steering may be actuated to combine the steering with braking or acceleration according to the desired driving comfort / level.

[0066] Those skilled in the art will recognize that agreement on states between the signals of the signal group (step 620: yes) may be interpreted as “semi-definitive.” For example, if all signal states agree, but a pedestrian in the path of the autonomous vehicle 2 is detected by a separate system, the ECU 10 may override steps 635 and 640.

[0067] If the comparison between the identified traffic signals indicates that the traffic signals of the relevant signal group (A; B) are not consistent, and / or if one or more of the multiple traffic signals have been blocked in the field of view 4 so that the comparison can no longer be performed (step 620: No), the ECU 10 can evaluate various safety factors to determine how to set the operating flag (step 650). Such a determination can be based on: the correlation value of two or more related traffic signal fixtures in the group, the corresponding state of each of the two or more related traffic signal fixtures, and in some embodiments, the movement state of one or more vehicles around the vehicle, etc.

[0068] Figure 6B An exemplary flow chart for evaluating and verifying one or more security factors according to an embodiment of the present disclosure is shown. Figure 5A and 5B Describe the device used to evaluate the safety factor.

[0069] The ECU 10 may determine whether the state of at least one signal of the relevant signal group has been determined (step 705). If the state of any relevant signal group has not been obtained (e.g., all signals are blocked) (step 705: No), the ECU 10 may determine that no safety factors may be evaluated and set the operation flag accordingly (step 750). The ECU may then return to Fig. 6A Flowchart of . In the event that the safety factor cannot be verified for whatever reason (step 650: No), a message may be provided to the operator / passenger of the vehicle advising the uncertain state (step 660). Such a message may be displayed on the display device 25 with an indicated level of urgency, intended to attract the attention of the operator / passenger, and may further suggest manual takeover of the vehicle driving function.

[0070] return Figure 6B , if, for example, as one skilled in the art would Figure 5B It is conceivable that the truck 3 is positioned so that one signal of signal group A is blocked, the ECU can still obtain the status of one signal of group A, and the status of group B. In this case (step 705: yes), the ECU 10 can proceed to step 710 to evaluate the determined status.

[0071] When evaluating the state of a visible signal of a signal group, similar to Figure 5B10. In the image shown (i.e., all signals of A and all signals of B indicate red), it can be determined that no signal indicates a green / passive state (step 710: No). In this case, ECU 10 can determine that autonomous vehicle 2 should stop or remain stopped as determined by the current state of autonomous vehicle 2, and the operation flag is set to braking and / or deceleration (step 740).

[0072] If at least one relevant signal state is determined to be green / go (or, for example, green / turn), the relevance value of the traffic signal indicating a green state may be checked to determine whether such state is above a threshold V for determining acceleration or continuation of the action of the autonomous vehicle 2. th (Step 715). As described above, a correlation value may be assigned to each signal in the scene detected by the image acquisition device 15, and a correlation value V may be assigned to each signal in the scene detected by the image acquisition device 15 based on the correlation value V. r Because some signals in the signal group may be blocked, the state is uncertain, and the correlation value V for other signals in the signal group r The evaluation of may become important for safety factor evaluation. For example, in the case where a signal group includes signals with correlations 10, 8, and 7 (i.e., three signals), but only the state of the third signal can be determined (i.e., correlation 7), a safety evaluation may be performed to determine the correlation value V of the signal. r Is it high enough to proceed according to the signal state (step 715). If the detected state of the signal in question has a correlation value V less than a threshold r (Step 715: No), then for the purpose of the current situation, the detected green signal state is ignored and the operation flag is set to braking and / or deceleration (Step 740).

[0073] After it has been determined that the signal having the green signal state also has a value higher than the threshold V th The correlation value V r In the case of a vehicle that is not in a predetermined range, the ECU 10 may request information about the motion of surrounding vehicles from the image acquisition device 15 to determine the confidence level (step 720).

[0074] For example, the image acquisition device may provide information about a number of surrounding vehicles that are moving, slowing down, or even remaining stopped relative to the ego vehicle 2. The percentage of vehicles may then be compared to a second threshold value V2th for each state to determine whether confidence is achieved in determining the green light state (step 720).

[0075] According to some embodiments, in the case that the surrounding vehicles of the second threshold (at least 75%) maintain or start moving (step 720: yes), it can be determined that the confidence level can be obtained in the green signal state for the autonomous vehicle 2, and the operation flag can be set to accelerate / maintain speed (step 730). Those skilled in the art will recognize that the second threshold of 75% is only exemplary, and the second threshold V2 can be set according to the desired safety factor. th For example, the second threshold may be set to 60%, 70%, 80%, 90%, or even 100%, and any value therebetween, in order to satisfy the desired safety level of the autonomous vehicle 2 and surrounding vehicles.

[0076] In the case where the ECU 10 does not detect the second threshold value V2th corresponding to the number of vehicles in motion, the operation flag may be set to braking / decelerating to ensure a safer action at the current intersection (step 740).

[0077] In an alternative to the above scenario, the states of all signals of groups A and B may be detected, but one or more states between the detected signals may not be consistent (i.e., A≠B, A≠A', or B≠B'). In this case, each detected state may be retained by the ECU 10 so that Figure 6B The method shown is further processed and each state passes through the flowchart. Alternatively, the highest correlation value V r The states in such scenarios are processed in order of highest to lowest relevance value in order to determine how the action flags should be set.

[0078] After the operation flag is set following the protocol or safety factor verification between all relevant signals, the process may proceed to step 640 to execute the operation flag as set. In other words, when the signal flag has been set to brake / decelerate, the ECU 10 may cause the vehicle to brake, decelerate and / or maintain in a stopped position.

[0079] Alternatively, in the event that the accelerate / maintain speed flag has been set, the ECU 10 may issue a command to cause such acceleration and / or maintaining speed of the autonomous vehicle 2 .

[0080] Those skilled in the art will recognize that variations of the examples described herein may be considered without departing from the scope of the appended claims. For example, the safety factor validation may take into account currently detected regular traffic patterns as well as historical traffic patterns when evaluating how to proceed. According to some embodiments, the second threshold V2th may be adjusted based on the historical safety level of the intersection. In other words, in the event that the intersection has a history of high accident rates, the second threshold V2th may be adjusted so that the green / passing state is only validated when 100% of other vehicles in operation are detected.

[0081] Additionally, while not discussed in detail herein, yellow / orange warning signals are generally interpreted and equated to red / stop states for an added safety factor. However, this can be adjusted based on the desired system scope, or on an intersection by intersection basis. For example, at a historically "safe" intersection, a yellow light may be interpreted based on the distance of the autonomous vehicle from the intersection and the historical timing of the traffic signal at the intersection. Those skilled in the art will appreciate that how an intersection is designated as historically safe may be determined based on authoritative reports generated by local government or other responsible traffic maintenance entities.

[0082] Furthermore, while the systems and methods described herein are intended to function without map / GPS data provided by an onboard GPS system, such information may be used to further validate / enhance the decisions made by the ECU 10. For example, where the image detection device 15 detects four related traffic signals at an intersection, and based on correlation, is intended to group the four signals into one signal group, the GPS data may be used to confirm the presence of the four signals and their correlation.

[0083] In addition, the ECU 10 may be configured to provide an indication to the operator / passenger via the display 25 of the operation flag that has been set, and a predetermined time may be provided to allow the operator / passenger to override the flagged operation. In doing so, the operator / passenger's attention may be drawn to the display 25 and the existing driving scene, as well as the evaluation made by the operator / passenger regarding the operation determined by the ECU 10.

[0084] Throughout the description, including the claims, the term "comprising" should be understood as being synonymous with "comprising at least one", unless otherwise stated. In addition, any ranges listed in the specification (including the claims) should be understood to include their endpoint values ​​unless otherwise stated. The specific values ​​of the elements described should be understood to be within the accepted manufacturing or industry tolerances known to those skilled in the art, and any use of the terms "substantially" and / or "approximately" and / or "generally" should be understood to be within such accepted tolerance ranges.

[0085] Where any standards of national, international or other standards bodies (e.g., ISO, etc.) are referenced, these references are intended to refer to the standards defined by the national or international standards bodies as of the priority date of this specification. Any subsequent substantive changes to such standards are not intended to modify the scope and / or definition of the present disclosure and / or claims.

[0086] Although the present disclosure has been described herein with reference to specific embodiments, it should be understood that these embodiments are only illustrative of the principles and applications of the present disclosure.

[0087] It is intended that the specification and examples be considered exemplary only, with the true scope of the disclosure being indicated by the following claims.

Claims

1. A traffic signal fixture verification system (1) for an autonomous vehicle (2), comprising: an optical recognition device (15) configured to obtain one or more images of two or more traffic signal fixtures near the autonomous vehicle and detect the motion state of one or more vehicles near the autonomous vehicle; A processing device (10) configured to: determining relevance values ​​for a plurality of traffic signal fixtures identified in the one or more images; combining traffic signal fixtures having the same correlation value among the plurality of traffic signal fixtures into corresponding signal groups; determining a signal group having a correlation value greater than a threshold value among the corresponding signal groups; monitoring the state of each of the traffic signal fixtures in the signal group having a correlation value greater than the threshold value from the corresponding signal group; as well as performing a comparison between all traffic signal fixtures in the monitored signal group to determine whether the states of all traffic signal fixtures are the same; actuating an accelerator function or a brake function of the autonomous vehicle based on the states of all traffic signal fixtures in the monitored signal group when it is determined that the states of all traffic signal fixtures in the monitored signal group are the same; When it is determined that the states of all traffic signal fixtures in the monitored signal group are not the same, an accelerator function or a brake function of the autonomous vehicle is actuated based on (i) the state of the traffic signal fixture with the highest safety factor in the monitored signal group and (ii) at least one of the movement states of the one or more vehicles around the autonomous vehicle.

2. The traffic signal fixture verification system (1) according to claim 1, wherein: When the state of one of the traffic signal fixtures in the monitored signal group cannot be determined, the state of another one of the traffic signal fixtures from another one of the corresponding signal groups is monitored and used for the actuation.

3. The traffic signal fixture verification system (1) according to claim 1, wherein: The state with the highest safety factor corresponds to a stop light or a warning light of a traffic signal fixture.

4. The traffic signal fixture verification system (1) according to claim 1, wherein: The processing device (10) is configured to receive map data associated with the location of the traffic signal fixture and to use the map data in determining the relevance value.

5. The traffic signal fixture verification system (1) according to claim 1, wherein: The distance of the autonomous vehicle (2) from each of the plurality of traffic signal fixtures is taken into account in determining the relevance value.

6. The traffic signal fixture verification system (1) according to claim 1, wherein: When a change in the state of motion of the one or more vehicles surrounding the autonomous vehicle (2) is detected to be greater than a threshold percentage change, the change in the state of motion is taken into account for the braking.

7. A method for verifying the status of a traffic signal fixture from an autonomous vehicle (2), comprising: obtaining one or more images of two or more traffic signal fixtures in a vicinity of the autonomous vehicle; detecting a motion state of one or more vehicles near the autonomous vehicle; determining relevance values ​​for a plurality of traffic signal fixtures identified in the one or more images; combining traffic signal fixtures having the same correlation value among the plurality of traffic signal fixtures into corresponding signal groups; determining a signal group having a correlation value greater than a threshold value among the corresponding signal groups; monitoring the state of each of the traffic signal fixtures in the signal group having a correlation value greater than the threshold value from the corresponding signal group; as well as performing a comparison between all traffic signal fixtures in the monitored signal group to determine whether the states of all traffic signal fixtures are the same; actuating an accelerator function or a brake function of the autonomous vehicle based on the states of all traffic signal fixtures in the monitored signal group when it is determined that the states of all traffic signal fixtures in the monitored signal group are the same; When it is determined that the states of all traffic signal fixtures in the monitored signal group are not the same, an accelerator function or a brake function of the autonomous vehicle is actuated based on (i) the state of the traffic signal fixture with the highest safety factor in the monitored signal group and (ii) at least one of the movement states of the one or more vehicles around the autonomous vehicle.

8. The method for verifying the status of a traffic signal fixture according to claim 7, wherein: When the state of one of the traffic signal fixtures in the monitored signal group cannot be determined, the state of another one of the traffic signal fixtures from another one of the corresponding signal groups is monitored and used for the actuation.

9. The method according to claim 7, comprising: Map data associated with the location of the traffic signal fixture is received and used in determining the relevance value.

10. The method according to claim 7, wherein: Upon detecting a change in the motion state of the one or more vehicles surrounding the autonomous vehicle (2) that is greater than a threshold percentage change, the change in motion state is considered for the actuation.

Images