System and method for risk assessment and warning of traffic signal violation

Abstract

A system and method for vehicle traffic signal violation risk assessment. The system includes a sensor, such as a front-facing camera module (FCM) configured to capture images in a field of view in front of the vehicle; a vehicle communication system configured for at least one of vehicle-to-infrastructure (V2I) communication and telematics communication with the infrastructure unit, where the infrastructure unit is capable of providing intersection information; and a traffic signal violation risk assessment (TSVR) module in communication with the FCM and the vehicle communication system. The TSVR module analyzes the captured images to determine an intersection, a traffic light at the intersection, and a current state (TLCS) of the traffic light; intersection information is collected from the infrastructure unit and a traffic signal violation risk is determined based on the information and actual actions are taken to mitigate the risk.

Description

Technical Field

[0001] The present disclosure relates to assessing the collision risk of a moving vehicle; more particularly, to a system and method for traffic signal violation risk assessment and mitigation. Background Art

[0002] Object recognition systems and collision avoidance systems are common in modern vehicles. Such systems can provide a risk assessment and warning to the vehicle operator regarding objects in the path of the moving host vehicle. Warnings can include: visual indications on the vehicle dashboard, audio warnings such as bells, and / or tactile warnings emitted through feedback devices mounted on control surfaces (such as the steering wheel). The object recognition system can also provide input to active vehicle systems such as an adaptive cruise control system that controls the vehicle speed to maintain an appropriate longitudinal spacing from a leading remote vehicle. The collision avoidance system can provide warnings and automatic braking to avoid an impending collision with an object in the vehicle's path.

[0003] Object recognition systems and collision avoidance systems achieve the purpose of object recognition, collision risk assessment, and mitigation of potential collisions with objects in the vehicle's path. However, there is a need for a system and / or method for traffic signal recognition, risk assessment of violations of the traffic signal by the host vehicle and other vehicles approaching the traffic signal, and mitigation of such risks. Summary of the Invention

[0004] According to several aspects, the present disclosure provides a method for traffic signal violation risk assessment. The method includes approaching an intersection by a host vehicle, determining whether the intersection has a traffic signal, and collecting intersection information. The intersection information includes traffic signal data, which includes the traffic signal current state (TSCS) and the time to the next state (TTNS). The method also includes determining the time of arrival of the host vehicle at the intersection (TAI), determining the risk level of traffic signal violation of the host vehicle based on the TSCS, TTNS, and TAI of the host vehicle, and implementing a first remedial measure in response to a risk level determined to be a low risk, and a second remedial measure in response to a risk level determined to be a high risk. The first remedial measure includes a unicast warning, and the second remedial measure includes a geocast warning. The second remedial measure may also include partial control of the vehicle by a vehicle control module. The partial control includes at least one of emergency braking, steering control, and pre-adjustment (such as pre-tensioning a seat belt) of the vehicle.

[0005] In another aspect of the present disclosure, the TSCS includes one of a GO state and a STOP state. The method further includes: before determining the risk level of a vehicle traffic signal violation, determining that the TSCS is in the STOP state and the TAI is less than the TTNS, or the TSCS is in the GO state and the TAI is greater than the TTNS.

[0006] In another aspect of the present disclosure, the TSCS and TTNS of the traffic signal are wirelessly collected from at least one of a remote server, a roadside unit (RSU), and a mobile edge computer (MEC) that communicates with the traffic signal.

[0007] In another aspect of the present disclosure, the host vehicle includes: a front-view camera configured to capture an image of a dynamic traffic signal in front of the vehicle. The TSCS of the traffic signal is determined based on the captured image.

[0008] In another aspect of the present disclosure, the method further includes determining the current attention state of the operator of the host vehicle and determining the intensity of sunlight based on the horizontal direction of the sunlight and whether the sunlight is directly within the operator's field of view. Wherein, determining the risk level of a host vehicle traffic signal violation is further based on the current attention state of the operator and the intensity of sunlight.

[0009] In another aspect of the present disclosure, the intersection information further includes traffic information about remote vehicles approaching the intersection and determining the number of times the host vehicle has traveled through the intersection. Wherein, determining the risk level of a host vehicle traffic signal violation is further based on the traffic information of the vehicles approaching the intersection and the familiarity with the intersection based on the number of times the host vehicle has traveled through the intersection.

[0010] In another aspect of the present disclosure, determining the risk level of a host vehicle traffic signal violation includes fusing the intersection information, the current attention state of the operator, the intensity of sunlight, and the familiarity with the intersection using one of a look-up table and a fuzzy logic method.

[0011] In another aspect of the present disclosure, the intersection information further includes traffic information about remote vehicles approaching the intersection. The method further includes determining the time of arrival of the remote vehicle at the intersection (TAI), and determining the risk level of a remote vehicle traffic signal violation based on the TSCS, TTNS, and TAI of the remote vehicle.

[0012] In accordance with several aspects, the present disclosure provides a system for assessing the risk of vehicle traffic signal violations. The system includes: a vehicle sensor, a vehicle communication system, and a Traffic Signal Violation Risk (TSVR) assessment module. The vehicle sensor such as a front camera module (FCM) configured to capture image data in the field of view in front of the vehicle; the vehicle communication system is configured for at least one of vehicle-to-infrastructure (V2I) communication and telematics communication with an infrastructure unit, wherein the infrastructure unit can provide intersection information; the Traffic Signal Violation Risk (TSVR) assessment module communicates with the FCM and the vehicle communication system. The TSVR module is configured to: analyze the captured image data to determine the intersection, the traffic lights at the intersection, and the current traffic light state (TLCS); collect intersection information from the infrastructure unit, wherein the intersection information includes the time to the next state (TTN) of the traffic lights at the intersection; determine the traffic signal violation risk based on the TLCS and the TTN; and take practical actions to mitigate the risk of traffic signal violations.

[0013] In another aspect of the present disclosure, determining the traffic signal violation risk includes determining one of a low risk and a high risk. The practical actions include unicasting a warning in response to the determined low risk and geo-broadcasting a warning in response to the determined high risk. Geo-broadcasting includes at least one of changing the current traffic light state and turning on the flashlights at the intersection.

[0014] In another aspect of the present disclosure, the TSVR module is further configured to determine the time of arrival (TAI) of the vehicle at the intersection. The TLCS includes one of a go state and a stop state. Before starting to determine the traffic signal violation risk, the TLCS is in the stop state and the TAI is less than TTNS, or the TSCS is in the go state and the TAI is greater than TTNS.

[0015] In another aspect of the present disclosure, the system further includes an ambient light sensor module configured to determine the intensity of sunlight and a driver monitoring system configured to determine the current attention state of the operator of the host vehicle. The TSVR module is further configured to determine the risk of traffic signal violations based on the TLCS, the TTN, the intensity of sunlight, and the level of the current attention state of the operator.

[0016] In another aspect of the present disclosure, the TSVR module is further configured to fuse intersection traffic information, the intensity of sunlight, the current attention state of the operator, and the familiarity with the intersection using a look-up table or fuzzy logic to determine the risk of at least one host vehicle traffic signal violation.

[0017] According to several aspects, the present disclosure provides a Traffic Signal Violation Risk Assessment (TSVR) module. The TSVR module includes a processor; a computer-readable medium having instructions that cause the processor to perform the following operations: analyze an image of the field of view in front of the vehicle to identify an intersection; determine that the intersection has a traffic signal; communicate with an infrastructure unit to determine the current state of the traffic signal, the time before the next change in the traffic signal state, and the vehicle traffic approaching the intersection; determine a risk level of a vehicle traffic signal violation based on the current state of the traffic signal, the time before the next change in the traffic signal state, and the vehicle traffic approaching the intersection, wherein the risk level includes one of a low risk and a high risk; unicast a warning in response to the determined low risk and geocast a multimodal active warning in response to the determined high risk, while instructing a vehicle controller to at least partially control the vehicle in response to a predetermined high risk.

[0018] Further application areas will become apparent from the description provided herein. It should be understood that these descriptions and specific examples are for illustrative purposes only and are not intended to limit the scope of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The drawings described herein are for illustrative purposes only and are not intended to limit the scope of the present disclosure in any way.

[0020] Figure 1 is an illustration of a plan view of a road intersection with a dynamic traffic signal according to an exemplary embodiment.

[0021] Figure 2 is a high-level functional flowchart of a method for assessing the risk of a vehicle traffic signal violation by a vehicle approaching Figure 1 the intersection;

[0022] Figure 3 is a functional diagram of a connected host vehicle according to an exemplary embodiment;

[0023] Figure 4 is a functional block diagram of a Traffic Signal Violation Risk Assessment (TSVRA) module according to an exemplary embodiment;

[0024] Figure 5 is a flowchart of a method for assessing a traffic signal violation risk assessment according to an exemplary embodiment;

[0025] Figure 6 is a flowchart of a method for determining the initiation of a method for assessing a traffic violation risk assessment according to another exemplary embodiment; and

[0026] Figure 7 is a fusion logic look-up table for assessing the risk level of a traffic signal violation according to an exemplary embodiment. Detailed Implementation Manner

[0027] The following description is merely exemplary in nature and is not intended to limit the present disclosure, its application, or uses. The illustrated embodiments are disclosed with reference to the accompanying drawings, in which like reference numerals indicate corresponding parts in several drawings. The drawings are not necessarily drawn to scale, and some features may be enlarged or minimized to show details of particular features. The specific structural and functional details disclosed are not to be construed as limiting, but rather as a representative basis for teaching one of ordinary skill in the art how to practice the disclosed concepts.

[0028] As used herein, the terms "module", "control module", or "controller" refer to any hardware, software, firmware, electronic control component, processing logic, and / or processor device, either alone or in any combination, including but not limited to: application specific integrated circuits (ASICs), electronic circuits, processors (shared, dedicated, or grouped) executing one or more software or firmware programs and memories, combinational logic circuits, and / or other suitable components that provide the described functionality.

[0029] Embodiments of the present disclosure may be described herein in terms of functional and / or logical block components and various processing steps. It should be understood that such block components may be implemented by any number of hardware, software, and / or firmware components configured to perform the specified functions. For example, embodiments of the present disclosure may employ various integrated circuit components, such as memory elements, digital signal processing elements, logic elements, look-up tables, etc., which may perform various functions under the control of one or more microprocessors or other control devices. Additionally, those of ordinary skill in the art will understand that embodiments of the present disclosure may be practiced in conjunction with any number of systems, and the systems described herein are merely exemplary embodiments of the present disclosure.

[0030] The connecting lines shown in the various figures contained herein are intended to represent example functional relationships and / or physical couplings between the various elements. Conventional techniques may be used for signal processing, data transmission, signaling, control, and other functional aspects of the system (as well as the various operating components of the system) are not described in detail herein. It should be noted that there may be many alternative or additional functional relationships or physical connections in the embodiments of the present disclosure.

[0031] Figure 1FIG. 0 is an illustration of a non-limiting example of a plan view of a road intersection 100 with dynamic traffic signals 102 configured to manage the vehicular traffic flow through the road intersection 100. The road intersection 100 is defined by a first road 104 intersecting a second road 106. Each of the first road 104 and the second road 106 is configured for two-way vehicular travel. The first road 104 includes a road marking 108, shown as a white solid line 108, which is perpendicular to the first road 104 and is located before the intersection 100 in the direction of travel towards the intersection 100. The white solid line 108 is referred to as a stop line 108 and designates the stopping position of a vehicle in response to a stop signal or command issued by the traffic signal 102.

[0032] In the direction of travel towards the intersection, three predetermined decision regions are shown. The predetermined decision regions are designated as Region A, Region B, and Region C. Region A is the farthest from the stop line 108 compared to Region B and Region C. Region B is farther from the stop line 108 than Region C. Region C is adjacent to the stop line 108. Region A is a possible stopping region, Region B is a decision region or dilemma region, and Region C is a possible go-ahead region. The dilemma region (Region B) is the area of the intersection approach where a conditional traffic signal, such as a yellow signal indication, is presented to the vehicle operator. In response to the conditional traffic signal, the vehicle operator needs to decide whether to continue through the intersection 100 or stop before the intersection 100. The dilemma region allows an estimated time between 2.5 seconds and 5.5 seconds to effect a vehicle stop before reaching the stop line 108 in response to a decision to stop before the intersection 100.

[0033] The dynamic traffic signals 102 are disposed at the road intersection 100 and are visible to vehicles approaching the intersection 100 on the first road 104 and the second road 106. The traffic signals 102 are capable of sequencing between visual indicators (also referred to as phases or states) to manage the vehicular traffic flow through the intersection 100. Common visual indicators can include text, symbols, and / or colors. In a non-limiting example, the color indicators include a green state, a yellow state, and a red state to indicate respectively that a vehicle may continue through the intersection 100, prepare to stop before the intersection 100, or stop at the intersection 100.

[0034] Traffic camera 110 is shown as being disposed at intersection 100 near traffic signal 102 and is configured to monitor the vehicular traffic flow approaching and passing through the intersection. Traffic camera 110 can be used to monitor weather and road conditions near intersection 100. Traffic camera 110 can be pointed in the direction of approaching traffic and synchronized with changes in the traffic signal status to detect vehicles at risk of violating the traffic signal. Additional cameras 112 can be installed around intersection 102 to observe vehicular and pedestrian traffic passing through the intersection.

[0035] Pedestrian 114 carrying a personal communication device 116 such as a mobile phone 116 is shown approaching a crosswalk 118 extending across a first road 104. A pedestrian traffic signal 120 is shown at one end of crosswalk 118 and is configured to manage pedestrian traffic across first road 104. Pedestrian traffic signal 120 can include visual and / or audible indicators to indicate to the pedestrian to walk across the road or to prohibit the pedestrian from walking across the road.

[0036] A primary-connected vehicle 200 or a host vehicle 200 is shown approaching an intersection on a first road. A remotely-connected vehicle 124 is shown approaching intersection 100 on a second road 106. An un-remotely-connected vehicle 125 is shown approaching intersection 100 in a direction opposite to that of remotely-connected vehicle 124. Host vehicle 200 communicates with a Traffic Signal Violation Risk Assessment (TSVRA) system 300, which can be located on, outside of, or partially on and outside of host vehicle 200. Traffic Signal Violation Risk Assessment (TSVRA) can be configured for Vehicle-to-Everything (V2X) communication.

[0037] At least one Road Side Unit (RSU) 126 and / or Mobile Edge Computer (MEC) 130 is / are provided near the intersection 100. The RSU 126 and MEC 130 are configured to process intersection data and transmit the intersection data to traffic control devices (such as traffic signals 102, 120) and to a central traffic management center. The RSU 126 and MEC 130 are configured to communicate with a remote server 128 (such as located in the cloud or a back-end server), and / or communicate with a cellular infrastructure 132 to upload or retrieve data related to the safe operation of the intersection. The RSU, MEC 130, remote server 128 and / or cellular infrastructure 132 are capable of wirelessly transmitting the intersection data to connected vehicles operating within a predetermined boundary 131 around the intersection 100. Such intersection data may include, but is not limited to, the location of the intersection, the status of traffic lights, the time to the next state of the traffic lights, information on vehicles approaching on the vehicle, road conditions, weather conditions, sunlight glare, etc. The RSU 126 and MEC 130 are capable of being configured to communicate directly with connected vehicles and personal devices approaching the intersection using infrastructure-to-vehicle (I2V) communication, wireless telematics service 132 and / or the Internet.

[0038] Figure 2 is a high-level functional flowchart of a method for assessing the risk of a vehicle traffic signal violation by a vehicle approaching Figure 1 the intersection. A traffic signal violation is defined as a vehicle passing through the intersection when the traffic signal indicates a stop command for the vehicle. In a non-limiting example, a traffic signal violation may be a vehicle not stopping at a red light. Vehicles may include the host vehicle 200, remotely connected vehicles 124 and non-remotely connected vehicles 125. The TSVRA system 300 collects information from the host vehicle 200, remotely connected vehicles 124 using V2V communication, and infrastructure-based sources using I2V communication. Infrastructure-based sources include, but are not limited to, traffic signals 102, cameras 110, 112, RSU, 126 and MEC 130, remote server 128.

[0039] Information collected from the host vehicle 200 includes, but is not limited to, data collected from a front camera module (FCM), a driver monitoring system (DMS), and an ambient light sensor module. The FCM is configured to detect traffic light status, remote vehicles, and vulnerable road users (VRUs) such as pedestrians and cyclists. The DMS monitors the degree of attention of the vehicle operator in operating the vehicle state. The ambient light sensor module can be used to determine the intensity of sunlight exposure based on the horizontal direction of sunlight and whether the sunlight is directly within the operator's field of view.

[0040] Information collected from infrastructure-based sources includes, but is not limited to: traffic signal status and time to the next state, traffic camera information, historical intersection safety records, and historical host vehicle map data for assessing the risk of red light violations. The historical host vehicle map data may indicate whether the operator of the host vehicle 200 is familiar with the intersection 100.

[0041] Information collected from remotely connected vehicles 124 approaching the intersection includes, but is not limited to: traffic light signals observed by the remotely connected vehicle, time to reach the intersection, and observations of the behavior of other vehicles and / or pedestrians at or near the intersection.

[0042] The collected information is analyzed to determine the risk level of vehicles (including the host vehicle 200, remotely connected vehicles 124, and non-remotely connected vehicles 125) potentially violating the traffic signal 102. Information fusion models (such as look-up tables, fuzzy logic-based methods, and / or weighting schemes, etc.), where the weighting scheme uses weights inversely proportional to the uncertainty of each element, and this weight uses its variance or the Dempster combination rule considering the confidence of each criterion, to determine the risk level from no risk, low risk to high risk.

[0043] In a non-limiting example, if the risk level is determined to be unknown or zero risk, no action is taken; if the risk level is determined to be a low risk level, a unicast warning can be issued to the human-machine interface 250 within the host vehicle 200; or, if the risk level is determined to be a high risk level, a geocast warning can be broadcast to the personal communication device 116 of the VRU and the remotely connected vehicles 124. The geocast warning can also be broadcast to the traffic signal 102 and / or the pedestrian traffic signal 120 to change the current traffic signal state and / or turn on the flashers, thereby mitigating the high risk level. Additionally, if the host vehicle 200 is equipped with autonomous driving features such as braking or steering, the TSVRA system 300 can activate the ADS and / or ADS to reduce the likelihood of a collision event.

[0044] Figure 3 It is a functional diagram of a connected host vehicle 200 communicating with a traffic signal violation risk assessment (TSVRA) system 300. Although the TSVRA system 300 is shown outside the host vehicle 200, it is contemplated that some or all of the components defining the TSVRA system 300 may be part of the host vehicle 200. In one non-limiting example, the TSVRA system 300 can be a cloud-based system wirelessly communicating with the host vehicle 200. In another non-limiting example, the TSVRA system 300 can be a vehicle system communicating with various other vehicle systems on the vehicle.

[0045] Vehicle 200 generally includes a body 206, front wheels 208, and rear wheels 210. The body 206 substantially encloses the systems and components of the vehicle 200. The front wheels 208 and the rear wheels 210 are each rotatably coupled to the body 206 near respective corners of the body 206. Although the primary vehicle 200 is shown as a sedan, it is contemplated that the primary vehicle 200 can be another type of vehicle, such as a pickup truck, a coupe, a sport utility vehicle (SUV), a recreational vehicle (RV), a motorcycle, and a road non-motor vehicle (such as a bicycle, etc.), capable of wirelessly connecting to an infrastructure and / or the Internet using a personal communication device 116 or a similar device.

[0046] Regardless of the vehicle type, the primary vehicle 200 is capable of communicating with other connected vehicles, networked roadside units (RSUs), or cloud computing services using wireless cellular communication, dedicated short-range communication (DSRC), and cellular vehicle-to-everything communication (C-V2X) technology. The primary vehicle 200 can be an intelligent vehicle equipped with an advanced driver assistance system (ADAS) and / or an autonomous driving system (ADS). The advanced driver assistance system is configured to enhance the safe operation of the vehicle, and the autonomous driving system is capable of operating from level 0 (no driving automation) to level 5 (full driving automation) according to the SAE J3016 driving automation levels.

[0047] As shown, the vehicle 200 generally includes a propulsion system 220, a driveline 222, a steering system 224, a braking system 226, a vehicle communication system 230, a driver monitoring system (DMS) 232, an ADAS 238, and / or an ADS 239, a plurality of sensors 240, and actuators 242. The vehicle 200 also includes one or more vehicle controllers 243 (also referred to as controller 243) that communicate with one or more vehicle systems, sensors 240, 234, 236, and actuators 242. The controller 243 also communicates with the TSVRA system 300 to evaluate the risk of traffic signal violations and communicates with the ADAS 238 and / or the ADS 239 to mitigate such risks.

[0048] The plurality of sensors 240 collect information and generate sensor data indicative of the collected information. As a non-limiting example, the sensors 240 can include a global navigation satellite system (GNSS) transceiver or receiver, a yaw rate sensor, a speed sensor, lidar, radar, ultrasonic sensors, and external and in-cabin cameras, etc. The GNSS transceiver or receiver is configured to detect the position of the primary vehicle 200. The speed sensor is configured to detect the speed of the primary vehicle 200. The yaw rate sensor is configured to determine the forward direction of the primary vehicle 200.

[0049] The sensor 240 also includes a front camera module (FCM) 234 and an ambient light sensor module 236. The FCM 234 has a field of view large enough to capture an image in front of the vehicle 200. The FCM 234 is configured to detect dynamic traffic signals (such as traffic lights), and identify changes in the states of the dynamic traffic signals. It should be understood that the vehicle sensor 240 may include a light detection and ranging (LIDAR) unit, a thermal scanner, and other external data sensors, which can be configured to detect dynamic traffic signals and identify changes in the states of the dynamic traffic signals. The ambient light sensor module 236 is configured to detect the intensity of sunlight based on the horizontal direction of the sunlight and whether the sunlight is directly within the operator's field of view.

[0050] The vehicle controller 243 includes at least one vehicle processor 244 and a vehicle non-transitory computer-readable storage device or medium 246. The vehicle processor 244 can be a customized or commercially available processor, a central processing unit (CPU), a graphics processing unit (GPU), an auxiliary processor among multiple processors associated with the vehicle controller 243, a semiconductor-based microprocessor (in the form of a microchip or a chipset), a macroprocessor, a combination thereof, or generally a device for executing instructions. As a non-limiting example, the vehicle computer-readable storage device or medium 246 may include volatile and non-volatile storage devices such as read-only memory (ROM), random access memory (RAM), and keep-alive memory (KAM). The KAM is a persistent or non-volatile memory, which can be used to store various operating variables when the vehicle processor 244 is powered off. The vehicle computer-readable storage device or medium 246 of the vehicle controller 243 can be implemented using multiple storage devices, such as PROM (programmable read-only memory), EPROM (electrical PROM), EEPROM (electrically erasable PROM), flash memory, or other electrical, magnetic, optical, or combined storage devices capable of storing data, some of which represent executable instructions used by the vehicle controller 243 when controlling the main vehicle 200. The vehicle non-transitory computer-readable storage device or medium 246 can store map data and / or sensor data received from one of the multiple sensors 240. The sensor data may include positioning data received from a GNSS transceiver. The map data includes a navigation map.

[0051] The vehicle communication system 230 may include one or more communication transceivers 247. Each communication transceiver 247 is configured to wirelessly transmit information to and receive information from other remote entities, such as a remotely connected vehicle 124, an RSU 126, an MEC 130, infrastructure (via "V2I" communication), a remote server 128 at the cloud or back office or remote call center, and / or a personal electronic device 116 such as a smart phone. The communication transceiver 247 can be configured to communicate using the IEEE 802.11 standard or via a wireless local area network (WLAN) using cellular data communication. However, additional or alternative communication methods such as cellular V2X (C-V2X), dedicated short range communication (DSRC), etc. are also considered within the scope of the present disclosure. The DSRC channel refers to a one-way or two-way short-to-medium range wireless communication channel designed specifically for automotive use, as well as a set of corresponding protocols and standards. Accordingly, the communication transceiver 247 may include one or more antennas for receiving and / or transmitting signals (such as cooperative sensing messages (CSMs)). The communication transceiver 247 can be considered a data source.

[0052] Reference Figure 4 ,The TSVRA system 300 includes one or more TSVRA controllers 334. The TSVRA controller 334 is configured to consider potential causes of traffic signal violations, such as sunlight glare (e.g., being blinded by sunlight, preventing the operator from seeing a red light), distraction (e.g., driver distraction or inattentiveness), driving score / record (e.g., an aggressive driver), uncertainty (e.g., a driver being unfamiliar with an intersection, and a novice driver).

[0053] The TSVRA controller 334 includes at least one system processor 344 and a system non-transitory computer-readable storage device or medium 346. The system processor 344 can be a custom or commercially available processor, a central processing unit (CPU), a graphics processing unit (GPU), an auxiliary processor among several processors associated with the TSVRA controller 334, a semiconductor-based microprocessor (in the form of a microchip or chipset), a macroprocessor, a combination thereof, or generally a device for executing instructions. The system computer-readable storage device or medium 346 can include volatile and non-volatile storage devices such as, for example, read-only memory (ROM), random access memory (RAM), and keep-alive memory (KAM). The system computer-readable storage device or medium of the TSVRA controller 334 can be implemented using a variety of storage devices, such as PROM (programmable read-only memory), EPROM (electrical PROM), EEPROM (electrically erasable PROM), flash memory, or other electrical, magnetic, optical, or combined storage devices capable of storing data, some of which represent executable instructions.

[0054] The system non - transitory computer - readable storage device or medium 346 and / or the vehicle non - transitory computer - readable storage device or medium 346 includes machine - readable instructions that, when executed by one or more system processors 344 and / or vehicle processors 244, cause the system processor 344 and / or the vehicle processor 244 to perform the following method. The TSVRA controller 334 communicates with the vehicle controller 243, vehicle systems, and sensors (including but not limited to the communication system 230, DMS 232, FCM 234, and ambient light sensor module 236). The TSVRA controller 334 also communicates with an infrastructure unit that includes but is not limited to the RSU 126, remote server 128, MEC 130, and personal communication device 116.

[0055] When the host vehicle 200 is traveling on a road, method 400 begins at block 402. At block 404, the TSVRA controller 334 monitors the position of the host vehicle 200 and determines whether the host vehicle 200 is approaching an intersection with a dynamic traffic signal. Proceeding to block 406, if the host vehicle 200 is not approaching an intersection with a dynamic traffic signal, method 400 returns to block 402. Returning to block 406, if the host vehicle is approaching an intersection with a dynamic traffic signal, the method proceeds to block 408.

[0056] At block 408, the TSVRA controller 334 determines whether the host vehicle 200 is within a predetermined decision region relative to a stop line at the intersection. If the host vehicle 200 is not within the predetermined decision region, method 400 returns to block 402. Returning to block 408, if the vehicle is within the predetermined decision region, method 400 proceeds to block 410.

[0057] At block 410, the TSVRA controller 334 collects and analyzes information from the host vehicle 200; traffic signal current status and time information to the next state obtained from the RSU 126, MEC 130, and / or remote server 128 in the cloud or back office; and traffic information from remotely connected vehicles 124 approaching the intersection. The TSVRA controller 334 can also analyze information obtained from traffic cameras 110, 112 to determine whether other remote vehicles 125 (whether connected or not) are approaching the intersection. Host vehicle 200 information includes but is not limited to the current traffic signal status from the FCM 234, the operator distraction status from the DMS 232, the amount of sunlight glare from the ambient light sensor module 236, familiar intersections from historical map data, and the number of times the host vehicle 200 has traveled through the intersection, as well as other information such as the driver's historical driving score and intersection scores based on accident records related to the potential traffic signal violation risk at a specific intersection.

[0058] Proceeding to block 410, an information fusion method can be used to fuse relevant information (i.e., current signal status detection, signal phase and timing of traffic signals, warnings from infrastructure or connected vehicles, driver distraction status, impact of sunlight glare, and unfamiliar intersections) to evaluate the risk level of traffic signal violations. The information fusion module can be a simple look-up table, a fuzzy logic-based method, or a weight scheme that combines three criteria using weights inversely proportional to the uncertainty of each element, where the weights use their variances or the Dempster combination rule that takes into account the degree of belief of each criterion.

[0059] In a non-limiting example, a fuzzy logic-based TSLVR (also known as Red Light Violation (RLV) Risk) can be used to estimate whether the risk of traffic signal violations is unknown, zero, low risk, or high risk. Metrics such as FCM-based red light detection, I2V-based signal phase and timing, V2V-based warnings, DMS-based distracted driver detection, sun glare detection, and unfamiliar intersections are all fuzzified. The fuzzification of the 6 metrics is input into an inference mechanism operating on a predefined rule base to estimate the risk level of RLV.

[0060] Input set: X = {I FCM , I I2V , I V2V , I DMS , I sunglare , I unfamiliarity}

[0061] Output set: M = {Red Light Violation (RLV) Risk} = {m}

[0062] The total number of possible rules for controlling the fuzzy logic-based strategy is given as l = k 1 k 2 …k n , where:

[0063] n is the number of non-interactive inputs; and

[0064] k i is the number of partitions of the input universe of discourse i, where i ∈ {1, 2, …, n}

[0065] For six metrics with four membership functions such as "none", "unknown", "low", "high", the number of possible rules is 4 6 = 4096. In fact, due to the interpolation inference ability of the fuzzy model and due to the overlap of the fuzzy functions of the partitions, the actual number of rules required for the fuzzy inference rule base is much less than l. The strategy can be adjusted by adding or changing rules and by adjusting the set boundaries. These rules can be created manually based on conducting some experiments to understand the relationship between the context information and the signal conditioning mode. Learning algorithms such as neural networks or decision trees can also be used to automatically learn rules from the example data. A fused neuro-fuzzy system such as the Dynamic Evolving Neural Fuzzy Inference System (DENFIS) can also be used

[0066] The estimated RLV risk can be regarded as the explicitly related context expressed as a linguistic variable, and its term set T(b) can be: T(b) = {none, unknown, low, high}. Each term in T(b) is characterized by a fuzzy set in the universe of discourse U = [-3, 3]

[0067] In a non-limiting example: If I FCM = 70% high risk, and I I2V = 85% high risk, and I V2V = 90% unknown, and I DMS = 83% yes, and I sunglare = 30% yes, and I unfamiliarity = 15% yes, then the RLV risk = 90% high risk

[0068] In another non-limiting example: If I FCM = 82% no, and I I2V = 75% no, and I V2V = 84% no, and I DMS = 87% no, and I sunglare = 75% no, and I unfamiliarity= 92% yes, then the RLV risk = 85% risk-free.

[0069] Other methods can be used to fuse the six metrics, such as using a lookup table or a weighting scheme as Figure 7 shown, which uses weights inversely proportional to the uncertainty of each element to combine three criteria, where the weights use their variances or the Dempster combination rule that takes into account the confidence of each criterion.

[0070] Proceeding to block 412, mitigation actions can be activated based on the determined risk level. If the risk level is determined to be zero risk, no action is taken. Active warnings can be geocast or unicast to surrounding vehicles via V2V, geocast or unicast to infrastructure via V2I, or geocast or unicast to vulnerable road users (VRUs) such as pedestrians / cyclists via cellular-based V2VRU communication through a safety app, and this communication can be achieved by a safety application (such as the OnStar Guardian App) that the VRU may subscribe to.

[0071] In a non-limiting example, if the risk level is determined to be low risk, a single-mode active warning can be generated to alert the operator of the host vehicle 200. The active warning can be a series of messages that can be pre-programmed to be transmitted via V2V communication to connected vehicle communications, communicated with VRUs via a personal communication device 116, and / or an audible message broadcast by infrastructure within the boundary 131 of the intersection 100. The single-mode warning can be an audio warning, a visual warning on an HMI device, or a tactile warning. If the risk level is determined to be high risk, a multi-mode active warning can be generated to alert connected vehicles 124 and VRUs within a predefined perimeter 131 that defines the intersection 102. The active warning can be directly unicast to specific connected vehicles or vulnerable users (VRUs) within the perimeter via the host vehicle V2V communication system, RSU / MEC, and / or the cloud using V2V, V2I, and / or telematics communication, or geocast to multiple connected vehicles and / or VRUs. If the host vehicle 200 is equipped with ADAS and / or ADS, partial or full control of the host vehicle (such as emergency braking or steering) and pre-conditioning of the vehicle (such as pre-tensioning the seatbelt) can be achieved by the vehicle control module to mitigate the high-risk determination. The method ends at block 414.

[0072] Figure 6A flowchart is shown that takes into account the time of arrival (TAI) of the host vehicle at the stop line 108 at intersection 100, the current state of the traffic light (TLCS) or the current state of the traffic signal (TSCS), and the time to the next state (TTNS) when determining the traffic signal violation risk assessment. Method 500 begins at block 502, where the host vehicle 200 is traveling on the road. Proceeding to block 504, the host vehicle 200 retrieves map information from a remote server 128 located on a cloud server and / or in a back office. The map information includes the location of the upcoming intersection and whether the intersection has a traffic signal such as a traffic light.

[0073] Proceeding to block 506, if the approaching intersection does not have a traffic light, method 500 returns to block 502. Returning to reference block 506, if the intersection does have a traffic light and the host vehicle 200 is within a predetermined distance of the intersection, method 500 proceeds to block 508.

[0074] At block 508, the host vehicle 200 collects information from the FCM 234 to determine whether the current state of the traffic signal is a red state, a yellow state, or a green state. The host vehicle 200 attempts to collect information from a remote server or RSU to confirm the current state of the traffic light (TLCS) and the time to the next state (TTNS). The TSVRA controller 334 calculates the TAI based on the current speed of the host vehicle 200.

[0075] Proceeding to block 510, if the TAI is greater than a predetermined TAI threshold, method 500 returns to block 502. A non-limiting example of the TAI threshold could be approximately 5.5 seconds. Returning to reference block 510, if the TAI is less than the predetermined TAI threshold, method 500 proceeds to block 512.

[0076] At block 512, if the TTNS is known, method 500 proceeds to block 514. At block 514, the information from the FCM 234 is analyzed to determine whether the TLCS is in a no-go command (red) or a prepare-to-stop command (yellow). If the TLCS is: (i) red and the TAI to the stop line is less than the TTNS, or (ii) the TLCS is green and the TAI is greater than the TTNS, method 500 proceeds to block 518 to perform a traffic signal violation risk assessment according to method 400. Otherwise, method 500 returns to block 502.

[0077] Return to reference box 512. If TTNS is unknown, method 500 proceeds to box 516. At box 516, if TLCS is neither red nor yellow, method 500 returns to box 502. Return to reference box 516. If the current status is red or yellow, method 500 proceeds to box 518. At box 518, perform a traffic signal violation risk assessment according to method 400. If the traffic signal violation risk assessment is determined to be a low risk, the mitigation effort may include unicasting to warn the host vehicle.

[0078] Proceed to box 520 to determine whether TAI is less than the time required to stop at the stop line (braking threshold). If TAI is not less than the braking threshold, the method returns to box 502. Return to reference box 520. If TAS is less than the braking threshold, the method proceeds to box 522. At box 522, if it is determined that the traffic signal violation risk assessment is at a high risk level, perform a geocast warning and, if the host vehicle 200 is equipped with ADAS 238 and / or ADS 239, implement a collision mitigation action such as automatic braking.

[0079] The present disclosure provides a wide range of signals and information for red light violation risk assessment and subsequent actions sent to vehicles and pedestrians through multiple channels such as unicasting, geocasting, vision, etc. The description of the present disclosure is merely exemplary, and variations that do not depart from the general meaning of the present disclosure are within the scope of the present disclosure. These variations should not be regarded as departing from the spirit and scope of the present disclosure.

Claims

1. A method for assessing traffic signal violation risk, comprising: Approaching the intersection by the host vehicle; Determining, by a module, that the intersection has a traffic signal; Collecting intersection information, wherein the intersection information includes traffic signal data, and the traffic signal data includes a current state TSCS of the traffic signal and a time to the next state TTNS; Determine a time TAI at which the host vehicle arrives at the intersection; determining a risk level of traffic signal violation of the host vehicle based on the TSCS, TTNS, and TAI of the host vehicle, wherein the risk level of traffic signal violation comprises one of: no risk level, low risk level, and high risk level; and implementing a first remedial measure in response to the risk level being determined to be a low risk, and implementing a second remedial measure in response to the risk level being determined to be a high risk; and Wherein, the first remedial measure and the second remedial measure include one of a unicast warning and a geographic broadcast warning.

2. The method according to claim 1, wherein: The second remedial action further includes partially controlling the host vehicle by a vehicle control module, wherein the partial control includes at least one of emergency braking, steering control, and pre-regulation of the vehicle.

3. The method according to claim 1, wherein: The TSCS includes one of a moving state and a stopping state; and further includes: before determining the risk level of the host vehicle's traffic signal violation, determining that the TSCS is the stopping state and the TAI is less than the TTNS, or that the TSCS is the moving state and the TAI is greater than the TTNS.

4. The method according to claim 1, wherein: The host vehicle includes a forward-looking camera configured to capture an image of a dynamic traffic signal in front of the host vehicle; and wherein the TSCS of the traffic signal is determined based on the captured image.

5. The method according to claim 1, wherein: The TSCS and TTNS of the traffic signal are wirelessly collected from at least one of a remote server, a roadside unit RSU, and a mobile edge computer MEC in communication with the traffic signal.

6. The method according to claim 1, wherein: Also includes: determining, by a driver monitoring system (DMS), a current state of attention of an operator of the host vehicle; as well as The ambient light sensor module determines the intensity of sunlight based on the horizontal direction of the sunlight and whether the sunlight is directly within the operator's field of view; as well as Wherein, determining the risk level of the host vehicle's traffic signal violation is also based on the operator's current attention state and the intensity of the sunlight.

7. The method according to claim 6, wherein: The intersection information also includes traffic information about remote vehicles approaching the intersection; and wherein the risk level of traffic signal violation by the host vehicle is also determined based on the traffic information about remote vehicles approaching the intersection.

8. The method according to claim 7, wherein: Also includes: Obtaining the location of the intersection; determining at least one of the intersection score and a number of times the host vehicle has traveled through the intersection; as well as Familiarity with the intersection is determined based on a number of times the host vehicle has traveled through the intersection.

9. The method according to claim 8, wherein: Determining the risk level of the host vehicle's traffic signal violation includes fusing the intersection information, the operator's current state of attention, sunlight intensity, and familiarity with the intersection using one of a lookup table and a fuzzy logic method.

10. The method according to claim 1, wherein: The intersection information also includes traffic information about a remote vehicle approaching the intersection; and the method further includes: Determining a time TAI at which the remote vehicle arrives at the intersection; and A risk level of traffic signal violation of the remote vehicle is determined based on the TSCS, TTNS, and TAI of the remote vehicle.

Images