Early warning and collision avoidance

Abstract

To provide early warning of a dangerous situation for assisting collision avoidance.SOLUTION: An apparatus is located at an intersection and receives data from a sensor oriented so as to monitor ground transportation entities at or near the intersection. A wireless communication device transmits an alert regarding a dangerous situation at or near the intersection to one device of the ground transportation entities. A machine learning model is stored that may predict behavior of the ground transportation entity at or near the intersection at a current time. Current operation data received from the sensors is applied to the machine learning model in order to predict imminent behaviors of the ground transportation entities. The wireless communication device transmits the alert regarding the dangerous situation to one device of the ground transportation entities.SELECTED DRAWING: Figure 6

Description

[Technical Field]

[0001] This application is entitled to the benefit of the filing dates of U.S. patent application Ser. Nos. 15 / 994,568, filed May 31, 2018, 15 / 994,826, filed May 31, 2018, 15 / 994,702, filed May 31, 2018, 15 / 994,915, filed May 31, 2018, 16 / 222,536, filed December 17, 2018, and 15 / 994,850, filed May 31, 2018, all of which claim priority to and the benefit of U.S. provisional patent application Ser. No. 62 / 644,725, filed March 19, 2018, the entire contents of which are incorporated herein by reference. [Background technology]

[0002] This description relates to early warning and collision avoidance.

[0003] Collision avoidance systems are on the rise. King et al. (US Patent Application Publication No. 2007 / 0276600(A1), 2007) described, for example, placing sensors in front of an intersection and applying physics-based decision rules to predict whether two vehicles are about to collide at the intersection based on their direction of travel and speed.

[0004] In Aoude et al. (U.S. Patent No. 9,129,519(B2), 2015, the entire contents of which are incorporated herein by reference), driver behavior is monitored and modeled to enable prediction and prevention of violations in traffic situations at intersections. Summary of the Invention [Problem to be solved by the invention]

[0005] Collision avoidance is the primary defense against injury and loss of life and property in ground transportation. Providing early warning of dangerous situations assists in collision avoidance. [Means for solving the problem]

[0006] In general, in one aspect, a device is located at an intersection of a transportation network. The device includes an input that receives data from a sensor oriented to monitor ground transportation entities at or near the intersection. The wireless communication device transmits a warning regarding a dangerous situation at or near the intersection to a device of one of the ground transportation entities, and there is a processor and storage for instructions executable by the processor to perform operations including: a machine learning model capable of predicting a behavior of the ground transportation entity at or near the intersection at a current time is stored; the machine learning model is based on training data regarding previous operations and associated behaviors of the ground transportation entity at or near the intersection; current operation data received from the sensor regarding the ground transportation entity at or near the intersection is applied to the machine learning model to predict an impending behavior of the ground transportation entity; an impending dangerous situation for one or more of the ground transportation entities at or near the intersection is inferred from the predicted impending behavior; and the wireless communication device transmits a warning regarding the dangerous situation to a device of the one of the ground transportation entities.

[0007] Implementations may include one or a combination of two or more of the following features: The wireless communication device transmits a warning about the hazardous situation to a sign or other infrastructure presentation device. The warning includes an instruction or command capable of controlling a particular ground transportation entity. The equipment includes a roadside device. There is a housing for the equipment, with a sensor attached to the housing. The warning is transmitted by broadcasting the warning for reception by any of the ground transportation entities at or near the intersection. The machine learning model includes an artificial intelligence model. The training data and behavior data include at least one of speed, position, or heading. The training data and behavior data may further include intent, posture, looking direction, or interaction with other vulnerable road users, for example, in a group. The processor is configured to enable generation of the machine learning model in the device. The training data is stored in the device. The intersection includes an intersection without a traffic light. The intersection includes an intersection with a traffic light. The transportation network includes a road network. The ground transportation entity includes a vulnerable road user. The ground transportation entity includes a vehicle. The imminent danger situation includes a collision or a near miss. The ground transportation entity includes a pedestrian crossing a road at a crosswalk. There is another communication device in communication with the central server. One device of the ground transportation entity includes a mobile communication device.

[0008] In general, in one aspect, an apparatus is located at an intersection of a transportation network. The apparatus includes an input that receives data from a sensor oriented to monitor ground transportation entities at or near the intersection. The wireless communication device transmits a warning regarding a dangerous situation at or near the intersection to a device of one of the ground transportation entities. There is a processor and storage for instructions executable by the processor for storing a machine learning model capable of predicting the behavior of the ground transportation entity at or near the intersection at a current time. The machine learning model is based on training data regarding previous operations and associated behaviors of the ground transportation entities at or near the intersection. Current operational data received from the sensor regarding the ground transportation entities at or near the intersection is applied to the machine learning model to predict impending behavior of the ground transportation entities, including ground transportation entities for which associated devices (devices of the ground transportation entities) are unable to receive warnings from the wireless communication device. An impending dangerous situation is inferred for a ground transportation entity for which associated devices (devices of the ground transportation entities) may receive warnings from the wireless communication device. The imminent hazardous situation is the result of a predicted imminent behavior of a ground transportation entity that is unable to receive the warning. The warning about the hazardous situation is transmitted to a device of the ground transportation entity that can receive the warning from the wireless communication device.

[0009] Implementations may include one or a combination of two or more of the following features: The equipment includes a roadside equipment; There is a housing for the equipment, with a sensor attached to the housing; The alert is sent by broadcasting the alert for reception by any of the ground traffic entities at or near the intersection that may receive the alert; The machine learning model includes an artificial intelligence model; The intersection includes an intersection without a traffic light; The intersection includes an intersection with a traffic light; The traffic network includes a road network; The ground traffic entity includes a road user that may be harmed; The ground traffic entity includes a vehicle; The imminent hazard situation includes a collision; The ground traffic entity whose associated device cannot receive the alert from the wireless communication device includes a vehicle; The ground traffic entity whose associated device may receive the alert from the wireless communication device includes a pedestrian crossing a road at a crosswalk; There is another communication device in communication with a central server; One of the ground traffic entities includes a mobile communication device.

[0010] Generally, in one aspect, a road vehicle traveling within a ground transportation network receives messages and data including messages from external sources regarding the location, operation, and status of other ground transportation entities, data from on-board sensors regarding road and driving conditions and regarding the location of static objects and moving ground transportation entities in the vehicle's vicinity, data regarding the quality of driving by the road vehicle driver, and basic safety messages from other ground transportation entities and personal safety messages from potential road users. The received data and messages are fused and applied to an artificial intelligence model to predict the behavior of the road vehicle driver or potential road users, and / or collision risk to the road vehicle.

[0011] Implementations may include one or a combination of two or more of the following features: The road vehicle generates a map of static objects and moving ground traffic entities in the vicinity of the road vehicle. The driver of the road vehicle is alerted to a collision risk. The collision risk is identified based on the probability of predicted trajectories of other nearby moving ground traffic entities. The basic safety messages and personal safety messages are filtered to reduce the number of alerts provided to the driver of the road vehicle.

[0012] In general, in one aspect, electronic sensors located near a crosswalk across a roadway are used to monitor an area within or near the crosswalk. The electronic sensors generate motion data regarding potential roadway users within or near the crosswalk. The generated motion data is applied to a machine learning model operating on a device located near the crosswalk to predict when one of the potential roadway users is about to enter the crosswalk. Before the potential roadway user enters the crosswalk, a warning is wirelessly transmitted to at least one of a device associated with the potential roadway user or a device associated with another ground transportation entity approaching the crosswalk on the roadway.

[0013] Implementations may include one or a combination of two or more of the following features: The equipment includes roadside equipment; The potentially vulnerable roadway users include pedestrians, animals, or cyclists; The device associated with the potentially vulnerable roadway users includes a smart watch or other wearable device, a smartphone, or another mobile device; The other ground transportation entities include motorized vehicles; The device associated with the other ground transportation entities includes a smartphone or another mobile device; The machine learning model is provided to the equipment located near the crosswalk by a remote server over the internet; The machine learning model is generated in the equipment located near the crosswalk; The machine learning model is trained using motion data generated by sensors located near the crosswalk; The motion data generated by the sensors located near the crosswalk is transmitted to the server for use in training the machine learning model; The motion data generated by the sensors located near the crosswalk is segmented based on corresponding zones in the vicinity of the crosswalk; Electronic sensors are used to generate motion-related data representing physical characteristics of the potentially vulnerable road users; Trajectory information regarding the potentially vulnerable road users is derived from the motion data generated by the sensors.

[0014] In general, in one aspect, electronic sensors located near intersections are used to monitor intersections and approaches to the intersections of a ground transportation network. The electronic sensors generate operational data regarding ground transportation entities moving on the approaches or at the intersection. One or more of the ground transportation entities are unable to transmit basic safety messages to other ground transportation entities in the vicinity of the intersection. Based on the operational data generated by the electronic sensors, virtual basic safety messages are transmitted to one or more of the ground transportation entities capable of receiving the messages. The virtual basic safety messages incorporate information regarding one or more of the ground transportation entities unable to transmit the basic safety messages. The incorporated information in each virtual basic safety message includes at least one of a position, heading, speed, and predicted future trajectory of one of the ground transportation entities unable to transmit the basic safety message.

[0015] Embodiments may include one or a combination of two or more of the following features: The equipment includes roadside equipment; The incorporated information includes a subset of information that would be incorporated into a basic safety message generated by a ground transportation entity assuming the ground transportation entity is capable of transmitting the basic safety message; The generated operational data is applied to a machine learning model operating on the equipment located near the intersection to predict a trajectory of a ground transportation entity that is unable to transmit the basic safety message; One of the ground transportation entities includes a motorized vehicle; The machine learning model is provided to the equipment located near the intersection by a remote server over the internet; The machine learning model is generated on the equipment located near the intersection; The machine learning model is trained using operational data generated by a sensor located near the intersection; The operational data generated by the sensor located near the intersection is transmitted to the server for use in training the machine learning model.

[0016] In general, in one aspect, electronic sensors located near intersections are used to monitor intersections and approaches to the intersections of a ground transportation network. The electronic sensors generate operational data regarding ground transportation entities moving on the approaches or moving at the intersections. Separate virtual zones are defined at the intersections and the approaches to the intersections. The operational data is segmented according to the corresponding virtual zones to which the generated operational data relates. The generated operational data is applied to a machine learning model running on equipment located near the intersection to predict, for each respective segment, an imminent hazardous situation at the intersection or one of the approaches involving one or more of the ground transportation entities. Before the imminent hazardous situation becomes an actual hazardous situation, a warning is wirelessly transmitted to a device associated with at least one of the involved ground transportation entities.

[0017] Implementations may include one or a combination of two or more of the following features: The equipment includes roadside equipment; The device associated with each of the ground transportation entities includes a wearable device, a smartphone, or another mobile device; One of the ground transportation entities includes a motorized vehicle; The machine learning model is provided to the equipment located near the intersection by a remote server over the internet; The machine learning model is generated on the equipment located near the intersection; The machine learning model is trained using motion data generated by sensors located near the intersection; The motion data generated by the sensors located near the intersection is transmitted to the server for use in training the machine learning model; Electronic sensors are used to monitor an area within or near a crosswalk crossing one of the approaches to the intersection; The electronic sensors are used to generate motion-related data representing physical characteristics of vulnerable road users near the crosswalk; Trajectory information regarding the vulnerable road users is derived from the motion data generated by the sensors; A machine learning model exists for each approach to the intersection; A determination of whether to send an alert is further based on motion data generated by sensors associated with another nearby intersection. A determination is made whether to send a warning further based on information received from a ground traffic entity moving on the approach road or a ground traffic entity moving at the intersection. The intersection is signalized and information regarding the status of the signal is received. The intersection is not signalized and is controlled by one or more signs. The defined virtual zone includes one or more approach roads controlled by signs. The signs include a stop sign or a yield sign. One of the ground traffic entities includes a rail vehicle.

[0018] In general, in one aspect, equipment is located in or on a ground transportation entity. The equipment includes an input that receives data from sensors in or on the ground transportation entity oriented to monitor nearby features of the ground transportation network and other information related to a context in which the ground transportation entity is traversing the ground transportation network. A wireless communication device receives the information related to the context. A signal processor applies signal processing to the data from the sensors and the other information related to the context. There is a processor and storage for instructions executable by the processor to perform operations including: storing a machine learning model capable of predicting the behavior of an operator of the ground transportation entity and the intentions and movements of other ground transportation entities in the vicinity; and applying currently received data from the sensors and other information related to the context to predict the behavior of the operator and the intentions and movements of other ground transportation entities in the vicinity.

[0019] Implementations may include one or a combination of two or more of the following features: The equipment includes roadside equipment. The instructions are executable by the processor to monitor users or occupants of the ground transportation entity. The other contextual information includes emergency broadcasts, roadside equipment traffic and safety messages, and messages from other ground transportation entities regarding safety, location, and other operational information. The sensors include cameras, distance sensors, vibration sensors, microphones, seat sensors, hydrocarbon sensors, volatile organic compound and other toxic substance sensors, and kinematic sensors, or combinations thereof. The instructions are executable by the processor to filter received alerts received by the vehicle by applying the alerts to machine learning models to predict which alerts are significant in relation to the current location, environmental conditions, driver behavior, vehicle health and status, and kinematics.

[0020] In general, in one aspect, operational data is obtained for unconnected ground transportation entities moving in a transportation network, and a virtual safety message incorporating information about the operational data for the unconnected ground transportation entities is transmitted to connected ground transportation entities in proximity to the unconnected ground transportation entities.

[0021] Implementations may include one or a combination of two or more of the following features: The virtual safety message is a substitute for a safety message that would be sent by the unconnected ground transportation entity if the unconnected ground transportation entity were connected; The unconnected ground transportation entity comprises a vehicle, and the virtual safety message is a substitute for a basic safety message; The unconnected ground transportation entity comprises a vulnerable road user, and the virtual safety message is a substitute for a personal safety message; The operational data is detected by infrastructure sensors.

[0022] In general, in one aspect, equipment located at an intersection of a transportation network includes an input for receiving data from sensors oriented to monitor ground transportation entities at or near the intersection. The data from each of the sensors represents at least one position or operating parameter of at least one of the ground transportation entities. The data from each of the sensors is represented in a native format. The data received from at least two of the sensors does not match with respect to the position or operating parameter, the native format, or both. Storage exists for processor-executable instructions for converting the data from each of the sensors into data having a common format independent of the native format of the sensor data. The data having the common format is incorporated into a global integrated representation of ground transportation entities monitored at or near the intersection. The global integrated representation includes the position, speed, and heading of each of the ground transportation entities. Relationships between the positions and operations of two of the ground transportation entities are identified using the global integrated representation. A dangerous situation involving the two ground transportation entities is predicted, and a message is transmitted to the at least one ground transportation entity of the two ground transportation entities alerting the at least one ground transportation entity of the dangerous situation.

[0023] Implementations may include one or a combination of two or more of the following features: The sensors include at least two of a radar, a lidar, and a camera. The data received from one of the sensors includes image data of a field of view at successive instants. The data received from one of the sensors includes reflection points in 3D space. The data received from one of the sensors includes a distance from the sensor and a speed. The global integrated representation represents the position of the ground traffic entity in a common reference frame. The two sensors from which the data is received are mounted at fixed positions at or near the intersection and have at least partially non-overlapping fields of view. One of the sensors includes a radar, and the transformation of the data includes determining the position of the ground traffic entity from a known position of the radar and the distance from the radar to the ground traffic entity. One of the sensors includes a camera, and the transformation of the data includes determining the position, direction of view, and tilt of the camera of the ground traffic entity from a known position, and the position of the ground traffic entity within an image frame of the camera.

[0024] In general, in one aspect, an apparatus is located at an at-grade intersection of a transportation network including a road intersection, a crosswalk, and railroad tracks. The apparatus includes inputs for receiving data from sensors oriented to monitor road vehicles and pedestrians at or near the at-grade intersection and for receiving phase and timing data for signals on the roads and on the railroad tracks. A wireless communication device is included for transmitting a warning to one of a ground transportation entity, a pedestrian, or a rail vehicle on the railroad tracks regarding a dangerous condition at or near the at-grade intersection. Storage is present for processor-executable instructions for storing a machine learning model capable of predicting the behavior of a ground transportation entity at or near the at-grade intersection at a current time. The machine learning model is based on training data regarding previous actions and associated behaviors of road vehicles and pedestrians at or near the intersection. Current behavior data received from the sensors regarding road vehicles and pedestrians at or near the at-grade intersection is applied to the machine learning model to predict the impending behavior of the road vehicles and pedestrians. An imminent hazardous situation for a rail vehicle on the rail track at or near the intersection is inferred from the predicted imminent behavior, and the wireless communication device is caused to transmit a warning about the hazardous situation to at least one of a road vehicle, a pedestrian, and a rail vehicle device.

[0025] Implementations may include one or a combination of two or more of the following features: The warning is transmitted to onboard equipment on the rail vehicle; The rail tracks are on a separated rail right-of-way; The rail tracks are not on a separated rail right-of-way; The equipment comprises roadside equipment; The warning is transmitted by broadcasting a warning for reception by any of ground traffic entities, pedestrians, or rail vehicles at or near the grade crossing; The imminent hazardous situation comprises a collision or near-miss.

[0026] In general, in one aspect, data representing the location and behavior of road vehicles operating in a ground transportation network or walking pedestrians is received from infrastructure sensors. The data is received in virtual basic safety messages and virtual personalized safety messages regarding the status of the road vehicles and pedestrians. The received data is applied to a machine learning model that is trained to identify risky driving or walking behavior of one of the road vehicles or pedestrians. The risky driving or walking behavior is reported to authorities.

[0027] Implementations may include one or a combination of two or more of the following features: The road vehicle is identified based on plate number recognition; The pedestrian is identified based on biometric recognition; The road vehicle or pedestrian is identified based on social networking.

[0028] These and other aspects, features, and implementations may be expressed as methods, apparatus, systems, components, program products, ways of doing business, means or steps for performing a function, and other techniques.

[0029] These and other aspects, features, and embodiments will become apparent from the following description, including the claims. [Brief explanation of the drawings]

[0030] [Figure 1] FIG. [Figure 2] FIG. [Figure 3] FIG. [Figure 4] 1 is a schematic diagram of a road network from above. [Figure 5] 1 is a schematic diagram of a road network from above. [Figure 6] FIG. 1 is an annotated perspective view of an intersection. [Figure 7] FIG. 1 is an annotated perspective view of an intersection. [Figure 8] 1 is a schematic diagram of a road network from above. [Figure 9] 1 is a schematic diagram of a road network from above. [Figure 10] 1 is a schematic diagram of a road network from above. [Figure 11] 1 is a schematic diagram of a road network from above. [Figure 12] FIG. 1 is a schematic side view of a road network. [Figure 13] 1 is a schematic diagram of a road network from above. [Figure 14] 1 is a schematic diagram of a road network from above. [Figure 15] FIG. [Figure 16] FIG. 1 is a schematic perspective view of a road network. [Figure 17] 1 is a schematic diagram of a road network from above. [Figure 18] 1 is a schematic diagram of a road network from above. DETAILED DESCRIPTION OF THE INVENTION

[0031] With advances in sensor technology and computers, it has become feasible to predict dangerous situations (and provide early warning of dangerous situations) and thus prevent collisions and near misses between ground traffic entities in ground traffic operations (i.e., enable collision avoidance).

[0032] The term "ground transportation" is used broadly herein to encompass, for example, any mode or medium of moving from place to place on the Earth's surface involving contact with land or water, such as walking or running (or performing other pedestrian behavior), non-motorized vehicles, motorized vehicles (autonomous, semi-autonomous, and non-autonomous), and rail vehicles.

[0033] The term "ground transportation entity" (or sometimes simply "entity") is used broadly herein to encompass, for example, a person or a discrete motorized or non-motorized vehicle engaged in a mode of ground transportation, such as, among other things, a pedestrian, bicycle rider, boat, car, truck, tram, streetcar, or train. In some cases, the term "vehicle" or "road user" is used herein as a shorthand reference to a ground transportation entity.

[0034] The term "hazardous situation" is used broadly herein to encompass any event, occurrence, sequence, context, or other circumstance that may result in, for example, imminent property damage or personal injury or death, and that may be mitigable or avoidable. The term "hazard" is sometimes used interchangeably herein with "hazardous situation." The phrases "violation" or "violating" are sometimes used herein with respect to behavior of an entity that results in, may lead to, or will lead to a dangerous situation.

[0035] In some implementations of the technology discussed herein, the ground transportation network is used by a mix of ground transportation entities that do not have or do not use transportation connectivity, and ground transportation entities that do have and use transportation connectivity.

[0036] The term "connectivity" is used broadly herein to encompass any capability of a ground transportation entity to, for example, (a) perceive and act on knowledge of its surroundings, other ground transportation entities in its vicinity, and traffic conditions associated with the ground transportation entity; (b) broadcast or otherwise transmit data regarding the state of the ground transportation entity; or (c) do both (a) and (b). The transmitted data may include its position, heading, speed, or the internal state of its components related to the traffic situation. In some examples, the ground transportation entity's awareness is based on wirelessly received data regarding other ground transportation entities or traffic conditions related to the operation of the ground transportation entity. The received data may originate from other ground transportation entities, from infrastructure devices, or both. Typically, connectivity involves transmitting or receiving data in real time, or substantially in real time, or in time for one or more of the ground transportation entities to act on the data in the traffic situation.

[0037] The term "traffic situation" is used broadly herein to encompass any situation in which two or more ground transportation entities operate in proximity to one another and in which the operation or status of each of the entities may affect or be related to the operation or status of the others.

[0038] At times herein, a ground transportation entity that does not have or does not use connectivity or connectivity aspects is referred to as a "disconnected ground transportation entity" or simply an "disconnected entity." At times herein, a ground transportation entity that has and uses connectivity or connectivity aspects is referred to as a "connected ground transportation entity" or simply a "connected entity."

[0039] The term "associated entity" is sometimes used herein to refer to a ground transportation entity that broadcasts data to its environment, including, for example, location, heading, speed, or the status of onboard safety systems (e.g., brakes, lights, and wipers).

[0040] In some cases, the term "uncoordinated entity" is used herein to refer to a ground transportation entity that does not broadcast one or more types of data, such as the ground transportation entity's position, speed, heading, or status, to the ground transportation entity's environment.

[0041] In some cases, the term "neighborhood" of a ground transportation entity is used broadly herein to encompass, for example, the area where a broadcast by the entity can be received by other ground transportation entities or infrastructure devices. In some instances, neighborhood varies with the entity's location and the number and characteristics of obstacles around the entity. An entity traveling on open roads in the desert has a very large neighborhood because there are no obstacles that prevent a broadcast signal from the entity from reaching long distances. Conversely, neighborhood in an urban canyon is smaller due to buildings around the entity. Furthermore, there may be electromagnetic noise sources that degrade the quality of the broadcase signal, thus shortening the reception distance (neighborhood).

[0042] 14, the neighborhood of entity 7001 traveling along road 7005 may be represented by concentric circles, with outermost circle 7002 representing the outermost extent of the neighborhood. Any other entity present within circle 7002 is in the neighborhood of entity 7001. Any other entity located outside circle 7002 is outside the neighborhood of entity 7001 and cannot receive broadcasts by entity 7001. Entity 7001 is invisible to all entities and infrastructure devices outside its neighborhood.

[0043] Typically, coordinated entities continuously broadcast their status data. Connected entities in the vicinity of the broadcasting entities can receive these broadcasts, process the received data, and act based on the received data. For example, if a vulnerable road user carries a wearable device that can receive a broadcast from an entity, such as an approaching truck, the wearable device can process the received data and inform the vulnerable user when it is safe to cross the road. As long as the user's device can receive the broadcast, i.e., is within the vicinity of the coordinated entities, this action occurs without regard to the location of the coordinated entities or the vulnerable user relative to the "smart" intersection.

[0044] The terms "vulnerable road user" or "vulnerable road user" are used broadly herein to encompass any user of a roadway or other feature of a road network who is not using, for example, a motorized vehicle. If the vulnerable road user is struck by a motorized vehicle, the vulnerable road user is generally not protected against injury or death or property damage. In some examples, the vulnerable road user may be a person walking, running, bicycling, or performing any type of activity that exposes the vulnerable road user to the risk of direct physical contact by a vehicle or other ground transportation entity in the event of a collision.

[0045] In some embodiments, the collision avoidance techniques and systems described herein (sometimes referred to herein simply as "systems") use sensors mounted on infrastructure facilities to monitor, track, detect, and predict the actions (e.g., speed, direction, and location), behavior (e.g., high speeds), and intentions (e.g., intending to run a stop sign) of ground transportation entities and their drivers and operators. The information provided by the sensors ("sensor data") enables the system to predict dangerous situations and provide early warnings to the entities to increase the chances of collision avoidance.

[0046] The term "collision avoidance" is used broadly herein to encompass any situation in which a collision or near miss between two or more ground transportation entities, or between a ground transportation entity and another object in the environment, that may result from, for example, a dangerous situation, is prevented, or the opportunity for such an interaction is reduced.

[0047] The term "early warning" is used broadly herein to encompass any notification, alarm, instruction, command, broadcast, transmission, or other transmission or receipt of information useful, for example, for identifying, suggesting, or in some manner indicating a dangerous situation and for collision avoidance.

[0048] Road intersections are prime locations where dangerous situations can occur. The technology described herein may include intersections that include infrastructure devices, including sensors, computing hardware, and intelligence, that enable simultaneous monitoring, detection, and prediction of dangerous situations. Data from these sensors is normalized to a single reference coordinate system and then processed. Artificial intelligence models of traffic flow along different approaches to the intersection are constructed. These models are useful, for example, to entities that are likely to violate traffic rules. The models are configured to detect dangerous situations before an actual violation and may therefore be considered predictive. Based on the prediction of a dangerous situation, an alert is sent from the infrastructure device at the intersection to all connected entities in the vicinity of the intersection. Each entity that receives the alert processes the data in the alert and performs alert filtering. Alert filtering is a process of discarding or ignoring alerts that are not useful to the entity. If an alert, such as an alert of an impending collision, is deemed useful (i.e., not ignored as a result of filtering), the entity automatically reacts to the alert (e.g., by applying the brakes), or a notification is presented to the driver, or both.

[0049] The system may be used on, but is not limited to, roadways, waterways, and railroads. These and other similar transportation contexts are sometimes referred to herein as "surface transportation networks."

[0050] Although the present specification often discusses the system in the context of intersections, the system may also be applied in other contexts.

[0051] The term "intersection" is used broadly herein to encompass, for example, any real arrangement of roads, rails, bodies of water, or other paths of travel where two or more ground transportation entities traveling along a route in a ground transportation network may occupy the same location at some time and location and thus collide.

[0052] Ground transportation entities using the ground transportation network move at various speeds and may arrive at a given intersection at different speeds and times. If the entity's speed and distance from the intersection are known, dividing the distance by the speed (both expressed in the same unit system) gives the arrival time at the intersection. However, the expected arrival time at the intersection changes continuously as the intended speed changes due to, for example, traffic conditions, speed limits on the route, traffic signals, and other factors. This dynamic change in expected arrival time makes it impossible to predict the actual arrival time with 100% confidence.

[0053] Considering factors that influence an entity's motion requires applying multiple relationships between the entity's velocity and various influencing factors. The absolute state of an entity's motion can be observed by sensors tracking the entity from the entity or from an external location. Data captured by these sensors can be used to model patterns of the entity's motion, behavior, and intentions. Machine learning can be used to generate complex models from vast amounts of data. Patterns that cannot be directly modeled using the entity's kinematics can be captured using machine learning. A trained model can predict whether an entity is about to move or stop at a particular point by using the entity's tracking data from the sensors tracking them.

[0054] In other words, in addition to detecting information about ground transportation entities directly from sensor data, the system uses artificial intelligence and machine learning to process vast amounts of sensor data to learn patterns of ground transportation entity behavior, behavior, and intentions, for example, at intersections in the ground transportation network, on approaches to such intersections, and at crosswalks in the ground transportation network. Based on the direct use of current sensor data and the results of applying artificial intelligence and machine learning to the current sensor data, the system generates early warnings, such as alerts of dangerous situations, which in turn assist in collision avoidance. In connection with early warnings in the form of instructions or commands, the commands or instructions may be targeted to specific autonomous or human-driven entities to directly control the vehicle. For example, the instructions or commands may slow down or stop an entity driven by a malicious person that is determined to be running a red light with the intent of injuring someone.

[0055] The system can be tuned to make predictions for that particular intersection and send alerts to entities in the vicinity of the device broadcasting the alert. To this end, the system derives data about dangerous entities using sensors and passes current readings from the sensors through a trained model. The model's output can then predict a dangerous situation and broadcast a corresponding alert. The alert received by connected entities in the vicinity contains information about the dangerous entity, and the receiving entities can then analyze the information to assess the threat posed to the receiving entity by the dangerous entity. If a threat exists, the receiving entity can either take action itself (e.g., slow down) or notify its driver using a human-machine interface based on visual, audio, tactile, or any type of sensory stimulus. The autonomous entity can then take action itself to avoid the dangerous situation.

[0056] The alert may also be transmitted directly over a cellular or other network to a mobile phone or other device equipped to receive the alert and carried by the pedestrian. The system identifies potentially dangerous entities at an intersection and broadcasts (or transmits directly) an alert to the pedestrian's personal device that includes a communication unit. The alert may, for example, prevent the pedestrian from entering the crosswalk, thus avoiding a potential accident.

[0057] The system may further track pedestrians and broadcast information related to the pedestrian's status (position, speed, and other parameters) to other entities so that they may take action to avoid dangerous situations.

[0058] As shown in FIG. 1, the system includes at least the following types of components:

[0059] 1. Roadside Equipment (RSE) 10 includes or uses sensors 12 to monitor, track, detect, and predict the behavior (e.g., speed, direction, and location), behavior (e.g., high speed), and intention (e.g., violating stop signs) of ground transportation entities 14. The RSE may also include or use data processing units 11 and data storage 18. Ground transportation entities exhibit a wide range of behaviors that depend on the infrastructure of the ground transportation network, as well as the state of the entity itself, the driver, and other ground transportation entities. To capture the entity's behavior, the RSE collects information from sensors, other RSEs, OBEs, OPEs, local or central servers, and other data processing units. The RSE may also store data received by the RSE and store data processed during some or all of the steps in the pipeline.

[0060] The RSE may store data on a local storage device or remote storage. The collected data is processed in real time using predefined logic or dynamically based on the collected data, meaning the RSE can automatically update its logic. The data may be processed in one processing unit or in a cluster of processing units to obtain faster results. The data may be processed in a local or remote processing unit or a local or remote cluster of processing units. The RSE may use simple logic or advanced models trained on the collected data. The models may be trained locally or remotely.

[0061] The RSE may preprocess data before using the trained model to filter outliers. Outliers may appear due to noise or reflections in the sensor or due to some other artifact. The resulting outliers may result in false alarms that can affect the performance of the RSE overall. The filtration method may be based on data collected by the RSE, OBE, OPE, or online resources. The RSE may interface with other controllers, such as traffic light controllers at intersections or other locations, to extract information for use in the data processing pipeline.

[0062] The RSE may also include or use communications equipment 20 for communicating, wired or wirelessly, with other RSEs, and with OBEs, OPEs, local or central servers, and other data processing units. The RSE may use any available standard for communicating with other equipment. The RSE may use a wired or wireless Internet connection to download and upload data to other equipment, a cellular network to send messages to and receive messages from other cellular devices, and dedicated radio devices to communicate with infrastructure devices and other RSEs at intersections or other locations.

[0063] RSEs can be installed in close proximity to different types of intersections. For example, at signalized intersections (e.g., intersections where traffic is controlled by signal lights), the RSE 10 is installed near the traffic light controller 26, either in the same enclosure or in a nearby enclosure. Data (e.g., traffic light phase and timing) is intended to flow 28 between the traffic light controller and the RSE. At non-signalized intersections, the RSE 10 is typically located to simplify connecting the RSE 10 to sensors 12 used to monitor other features of the roadway or ground transportation network in the vicinity of the intersection. The proximity of the RSE to the intersection helps maintain a low latency system, which is important to provide the receiving ground unit with maximum time to respond to an alert.

[0064] 2. Onboard Equipment (OBE) 36 mounted on, supported by, or within the ground transportation entity 14, including sensors 38 that determine the entity's position and kinematics (operational data) as well as safety-related data about the entity. The OBE further includes a data processing unit 40, data storage 42, and communications equipment 44 that can wirelessly communicate with other OBEs, OPEs, RSEs, and possibly servers and computing units.

[0065] 3. On Person Equipment (OPE) 46, which may be, but is not limited to, a mobile phone, a wearable device, or any other device capable of being worn by, held by, attached to, or otherwise associated with a person or animal. The OPE may include, or be coupled to, a data processing unit 48, data storage 50, and communication equipment 52, as needed. In some embodiments, the OPE serves as a dedicated communication unit for vulnerable road users who are not vehicles. In some instances, the OPE may be used for other purposes as well. The OPE may include components for providing visual, audio, or tactile alerts to vulnerable road users.

[0066] Potentially affected road users may include pedestrians, cyclists, road workers, people in wheelchairs, scooters, self-balancing devices, or battery-powered personal mobility devices, animal-powered carriages, guide or police animals, livestock, herds, and pets.

[0067] Typically, an OPE is carried by a vulnerable road user and is capable of sending and receiving messages. An OPE may be attached to or integrated into a mobile phone, tablet, personal mobility device, bicycle, wearable device (e.g., watch, bracelet, anklet), or attached to a pet collar.

[0068] Messages transmitted by the OPE may include kinematic information related to the potential road user, including but not limited to time, 3D location, heading, speed, and acceleration. The transmitted message may further convey data representing the alert level, current behavior, and future intentions of the potential road user, such as a potential road user currently crossing a crosswalk, listening to music, or about to cross a crosswalk. Among other possibilities, the message may convey the potential road user's blob size or data size, whether there is an external device (e.g., a stroller, cart, or other device) with the potential road user, whether the potential road user has a physical disability, or whether the potential road user is using any personal assistance. If the potential road user is a worker, the message may convey the worker's category and further describe the type of activity being performed by the worker. If a cluster of similar potential road users (e.g., a group of pedestrians) has similar characteristics, a single message may be transmitted to avoid multiple message broadcasts.

[0069] Typically, messages received by the OPE are warning messages from roadside equipment or entities. The OPE can act on the received messages by issuing warnings to vulnerable road users. Warning messages convey data useful in providing customized warnings for vulnerable road users. For example, a warning to a vulnerable road user can indicate the type of dangerous situation and suggest possible actions to take. The OPE can apply warning filtering to all received messages and present only relevant messages to the vulnerable road users.

[0070] The alert filtering is based on the results of applying a learning algorithm to historical data associated with the OPE, allowing the alert filtering to be custom tailored to each vulnerable road user. The OPE learning algorithm tracks the vulnerable road user's responses to received alerts and tailors future alerts to elicit the best response time and attention from the vulnerable road users. The learning algorithm may also be applied to data conveyed in transmitted messages.

[0071] 4. A data storage server 54, which may be, but is not limited to, cloud storage, local storage, or any other storage facility that allows for the storage and retrieval of data. The data storage server is accessible by the RSE, by the computing unit, and possibly by the OBE, OPE, and data server, for example, to store data related to early warning and collision avoidance. The data storage server is accessible from the RSE, and possibly from the OBE, OPE, and data server, for fetching the stored data. The data may be raw sensor data, data processed by the processing unit, or any other information generated by the RSE, OBE, and OPE.

[0072] Sensors at intersections that continuously monitor ground traffic entities can generate large amounts of data daily. The volume of this data depends on the number and type of sensors. The data is both processed in real time and stored for future analysis, which requires, for example, a data storage unit (e.g., hard disk drive, solid state drive, or other mass storage device) locally at the intersection. The local storage device fills up after a period of time depending on the storage capacity of the local storage device, the volume of data generated, and the rate at which the data is generated. To preserve the data for future use, the data is uploaded to a remote server with larger capacity. The remote server can upgrade its storage capacity on demand as needed. The remote server may use a data storage device similar to the local storage (e.g., hard disk drive, solid state drive, or other mass storage device) accessible through a network connection.

[0073] Data stored locally and on a server for future analysis may include data broadcast by ground transportation entities and received by the RSE that is saved for future analysis. Stored data may be downloaded from a server or other remote source for processing at the RSE. For example, a machine learning model of an intersection where the RSE is located may be stored on a server or other remote storage and downloaded by the RSE for use in analyzing current data received at the RSE from a local source.

[0074] 5. Computing unit 56, a powerful computing machine located in the cloud, locally (e.g., as part of the RSE), or a combination thereof. Among other functions, the computing unit processes available data to generate predictions and machine learning-based models of the behavior, behavior, and intentions of vehicles, pedestrians, or other ground transportation entities using the transportation network. Each computing unit may include specialized hardware for processing the corresponding type of data (e.g., a graphics processing unit for processing images). Under heavy processing loads, the computing units in the RSE may become overloaded. This may occur, for example, if additional data-generating units (e.g., sensors) are added to the system, causing a computational overload. Overload may also occur if the logic running in the computing unit is replaced with more computationally intensive logic. An overload may occur due to an increase in the number of ground transportation entities being tracked. In the event of a local computational overload, the RSE may offload some of its tasks to another computing unit. Such another computing unit may be local to the RSE or remote, such as a server. Computational tasks may be prioritized, and non-time-critical tasks may be performed on such other computing units, with results available to the local computing unit. For example, a computing unit in an RSE may request another computing unit to run a job to analyze stored data and to train a model using the data. The trained model is then downloaded by the computing unit in the RSE for storage and use therein.

[0075] Compute units in the RSE may use other smaller compute units to perform computationally intensive jobs more efficiently, reducing the time required. Available compute units are used wisely to perform most tasks in the shortest amount of time, for example, by dividing tasks between the RSE compute units and other available compute units. Compute units may also be attached to the RSE as external devices to add more computing power to the compute units in the RSE. Externally attached compute units may include the same or different architecture as the compute units in the RSE. Externally attached compute units may communicate with existing compute units using any available communication ports. RSE compute units may request more computing power from external compute units as needed.

[0076] The remainder of this specification will, among other things, explain in detail the roles and functions of the above-mentioned components in the system.

[0077] Roadside Equipment (RSE)

[0078] As shown in FIG. 2, the RSE may include, but is not limited to, the following components:

[0079] 1. One or more communication units 103, 104 that enable reception and / or transmission of operational and other data related to ground transportation entities and road safety data from and to nearby vehicles or other ground transportation entities, infrastructure, and remote servers and data storage systems 130. In some examples, this type of communication is known as infrastructure-to-everything (I2X), where I2X includes, but is not limited to, infrastructure-to-vehicles (I2V), infrastructure-to-pedestrians (I2P), infrastructure-to-infrastructure (I2I), and infrastructure-to-devices (I2D), and combinations thereof. Communications may be wireless or wired and may conform to a wide variety of communication protocols.

[0080] 2. Communication unit 103 is used for communication with ground transportation entities, and unit 104 is used for communication with a remote server and data storage system 130 over the Internet.

[0081] 3. Local storage 106 for storing programs, intersection models, and behavior and traffic models. The local storage 106 may also be used for temporary storage of data collected from the sensors 101.

[0082] 4. Sensors 101 and sensor controllers 107 that enable monitoring (e.g., generating data regarding) moving objects, such as ground traffic entities typically near the RSE. The sensors may include, but are not limited to, cameras, radar, lidar, ultrasonic detectors, or any other hardware that can detect or infer, from detected data, the distance to a ground traffic entity, or the speed, heading, or position of the ground traffic entity, or a combination thereof, among other things. Sensor fusion is performed using the collection or combination of data from two or more sensors 101.

[0083] 5. A position receiver (102) (e.g., a GPS receiver) that provides positioning data (e.g., coordinates of the RSE's location) and helps correct positioning errors in the location of ground transportation entities.

[0084] 6. A processing unit 105 that acquires and uses data generated from the sensors and incoming data from the communication units 103, 104. The processing unit processes and stores the data locally and, in some embodiments, transmits the data for remote storage and further processing. The processing unit also generates messages and alerts that are broadcast or otherwise transmitted over wireless communication facilities to nearby pedestrians, motor vehicles, or other ground transportation entities, and in some examples to signs or other infrastructure indication devices. The processing unit also periodically reports the health and status of all RSE systems to a remote server for monitoring.

[0085] 7. Expansion connector 108 to allow control and communication between the RSE and other hardware or other components, such as temperature and humidity sensors, traffic light controllers, other computing units as mentioned above, and other electronic devices that may become available in the future.

[0086] Onboard equipment (OBE)

[0087] The on-board equipment may typically be original equipment for the ground transportation entity or may be added to the entity by a third-party supplier. As shown in Figure 3, the OBE may include, but is not limited to, the following components:

[0088] 1. A communications unit 203 that enables transmission and / or reception of data to and from nearby vehicles, pedestrians, cyclists, or other ground transportation entities and infrastructure, and combinations thereof. The communications unit further enables transmission and / or reception of data between vehicles or other ground transportation entities and a local or remote server 212 for purposes of machine learning and for remote monitoring of the ground transportation entities by the server. In some examples, this type of communication is known as vehicle-to-everything (V2X), where V2X includes, but is not limited to, vehicles-to-vehicles (V2V), vehicles-to-pedestrians (V2P), vehicle-to-infrastructure (V2I), vehicle-to-devices (V2D), and combinations thereof. Communication may be wireless or wired and may conform to a wide variety of communication protocols.

[0089] The communication unit 204 allows the OBE to communicate with remote servers over the Internet for program updates, data storage, and data processing.

[0090] 2. Local storage 206 for storing programs, intersection models, and traffic models. The local storage 206 may be used for temporary storage of data collected from the sensors 201.

[0091] 3. Sensors 201 and sensor control unit 207, which may include, but are not limited to, external cameras, lidar, radar, ultrasonic sensors, or any device that can be used to detect nearby objects or people or other ground traffic entities. Sensors 201 may include additional kinematic sensors, global positioning receivers, and internal and local microphones and cameras.

[0092] 4. A location receiver 202 (eg, a GPS receiver) that provides location data (eg, coordinates of the location of a ground transportation entity).

[0093] 5. A processing unit 205 for acquiring, using, generating, and transmitting data, including consuming data from the communication unit and transmitting data to the communication unit and consuming data from sensors in or on the ground transportation entity.

[0094] 6. Expansion connector 208 that allows control of the OBE and other hardware and communication between the OBE and other hardware.

[0095] 7. An interface unit that may be retrofitted or integrated into the head unit, steering wheel, or driver mobile device in one or more ways, for example, using visual, audible, or tactile feedback.

[0096] Smart OBE (SOBE: Smart OBE)

[0097] In a world where all vehicles and other ground transportation entities are connected entities, each vehicle or other ground transportation entity may be a coordinated entity and may report its current location, safety status, intentions, and other information to others. Currently, nearly all vehicles are not connected entities, are unable to report such information to other ground transportation entities, and are operated by humans with different levels of skill, happiness, stress, and behavior. Without such connectivity and communication, it becomes difficult to predict the next move of a vehicle or a ground transportation entity, resulting in a reduced ability to implement collision avoidance and provide early warnings.

[0098] A smart OBE monitors the environment and the user or occupants of the ground transportation entity. The smart OBE also monitors the health and status of the entity's different systems and subsystems. The SOBE monitors the outside world, for example, by listening to radio transmissions from emergency broadcasts, traffic and safety messages from nearby RSEs, and messages regarding safety, location, and other operational information from other connected vehicles or other ground transportation entities. The SOBE also interfaces with onboard sensors that can view road and driving conditions, such as cameras, distance sensors, vibration sensors, microphones, or any other sensors that enable such monitoring. The SOBE also monitors its immediate environment and generates a map of all stationary and moving objects.

[0099] A SOBE may also monitor the behavior of users or occupants of a vehicle or other ground transportation entity. A SOBE may use microphones to monitor conversation quality. A SOBE may also use other sensors, such as seat sensors, cameras, hydrocarbon sensors, and sensors for volatile organic compounds and other toxic substances. A SOBE may also use kinematic sensors to measure driver reactions and behavior and infer driving quality therefrom.

[0100] SOBE also receives vehicle-to-vehicle messages from other ground transportation entities (e.g., basic safety messages (BSM)) and vehicle-to-pedestrian messages from vulnerable road users (e.g., personal safety messages (PSM)).

[0101] The SOBE then fuses data from this array of sensors, sources, and messages. The SOBE then applies the fused data to an artificial intelligence model that can predict not only the next action or reaction of the driver or user of a vehicle or other ground transportation entity, or of a potential road user, but also the intentions and future trajectories and the associated near-miss or collision risk due to other vehicles, ground transportation entities, and nearby potential road users. For example, the SOBE may use the BSM received from a nearby vehicle to predict that the nearby vehicle is about to enter a lane-changing maneuver that poses a risk to its own host vehicle and may alert the driver of the impending risk. The risk is calculated by the SOBE based on the nearby vehicle's various future predicted trajectories (e.g., going straight, changing lanes to the right, changing lanes to the left) and the probability of the associated collision risk with the host vehicle for each of those trajectories. If the collision risk is higher than a certain threshold, a warning is displayed to the driver of the host vehicle.

[0102] Due to the complexity of modeling human driver behavior, which is also affected by external factors (e.g., changing environmental and weather conditions), machine learning is typically required to predict intent and future trajectories.

[0103] SOBE is characterized by powerful computing power that can process multiple data feeds, some of which provide several megabytes of data per second. The amount of available data is, in turn, proportional to the level of detail required from each sensor.

[0104] SOBE also includes powerful signal processing equipment that can extract useful information from environments known to have high (signal) noise levels and low signal-to-noise ratios. SOBE also protects the driver from the overwhelming number of alerts the vehicle receives by providing smart alert filtering. Alert filtering is the result of machine learning models that can distinguish which alerts are significant in the context of current location, environmental conditions, driver behavior, vehicle health and status, and kinematics.

[0105] Smart SOBEs are important for collision avoidance and early warning, and for achieving a safer transportation network for all users, not just the occupants or users of vehicles that contain SOBEs. SOBEs can detect and predict the movements of different entities on the road, and thus assist in collision avoidance.

[0106] Person equipment (OPE)

[0107] As described above, person-on-board equipment (OPE) includes any device that can be held, worn, or otherwise directly associated with a pedestrian, jogger, or other person who is a ground transportation entity or otherwise present on or using the ground transportation network. Such a person may be, for example, a road user who may be vulnerable to being struck by a vehicle. OPE may include, but is not limited to, mobile devices (e.g., smartphones, tablets, digital assistants), wearable devices (e.g., eyewear, watches, bracelets, anklets), and implants. Existing components and features of OPE may be used to track and report location, speed, and heading. OPE may also be used to receive and process data and display alerts to the user through various modes (e.g., visual, audible, tactile).

[0108] Honda has developed a communication system and method for V2P applications that focuses on direct communication between vehicles and pedestrians using an OPE. In one example, a vehicle is equipped with an OBE for broadcasting messages to the OPEs of nearby pedestrians. The messages convey the vehicle's current status, including, for example, vehicle parameters, speed, and direction of travel. For example, the messages can be basic safety messages (BSMs). When necessary, the OPEs present pedestrians with warnings about predicted dangerous situations tailored to the pedestrian's level of distraction to avoid a collision. In another example, the pedestrian's OPE broadcasts a message (e.g., a personal safety message (PSM)) to the OBEs of nearby vehicles that a pedestrian may cross the vehicle's intended path. When necessary, the vehicle's OBE displays a warning to the vehicle user about the predicted danger to avoid a collision. See Strickland, Richard Dean, et al., "Vehicle to pedestrian communication system and method." U.S. Patent No. 9,421,909.

[0109] The system described herein uses an I2P or I2V approach that uses sensors external to vehicles and pedestrians (primarily in infrastructure) to track and collect data about pedestrians and other vulnerable road users. For example, sensors may track pedestrians crossing streets and vehicles operating at or near crossing locations. The collected data is then used to build predictive models of pedestrian and vehicle driver intent and behavior on roads using rule-based and machine learning methods. These models help analyze the collected data and make predictions of pedestrian and vehicle paths and intents. When a hazard is predicted, a message is broadcast from the RSE to the OBE and / or OPE, alerting each entity in the other's intended path, allowing each to take preemptive action with enough time to avoid a collision.

[0110] Remote Computing (Cloud Computing and Storage)

[0111] Data collected from sensors connected to or embedded in RSEs, OBEs, and OPEs needs to be processed so that effective mathematical machine learning models can be generated. This processing requires a lot of data processing power to shorten the time required to generate each model. The required processing power is much more than what is typically available locally in the RSE. To solve this, data can be sent to a remote computing facility that provides the necessary capacity and can be scaled on demand. Herein, the remote computing facility is referred to as a "remote server," in keeping with the terminology used in the computing literature. In some instances, it may be possible to perform some or all of the processing in the RCE by equipping the RCE with high-powered computing power.

[0112] Rule-Based Processing

[0113] Unlike artificial intelligence and machine learning technologies, rule-based processing can be applied at any time without the need for data collection, training, and model building. Rule-based processing can be deployed from the beginning of system operation, typically until sufficient training data is acquired to generate a machine learning model. After a new installation, rules are set up to process incoming sensor data. This is not only useful for improving road safety but also serves as a suitable test case to ensure all components of the system are operating as expected. Rule-based processing can be added and used later as an additional layer to capture rare cases where machine learning cannot make accurate predictions. The rule-based approach is based on simple associations between collected data parameters (e.g., speed, range, etc.). The rule-based approach can also provide a baseline for evaluating the performance of machine learning algorithms.

[0114] In rule-based processing, sensors monitor vehicles or other ground traffic entities traversing a portion of a ground transportation network. If their current speed and acceleration exceed thresholds that prevent them from stopping before a stop bar (line) on the road, for example, an alert is generated. A variable area is assigned to each vehicle or other ground traffic entity. The area is labeled as a dilemma zone, and within the dilemma zone, the vehicle has not yet been labeled as a violating vehicle. If the vehicle crosses the dilemma zone and enters a danger zone because its speed, acceleration, or both are higher than a predetermined threshold, the vehicle is labeled as a violating entity and an alert is generated. The thresholds for speed and acceleration are based on physics and kinematics and vary depending on each ground traffic entity approaching, for example, an intersection.

[0115] Two traditional rule-based approaches are 1) static TTI (Time-To-Intersection) and 2) static RDP (Required Deceleration Parameter). See Aoude, Georges S. et al., "Driver behavior classification at intersections and validation on large naturalistic data set." IEEE Transactions on Intelligent Transportation Systems 13.2(2012):724-736.

[0116] Static TTI (Time to Intersection) uses the estimated time to reach an intersection as a classification criterion. In its simplest form, TTI is:

number

[0117] The static RDP (requested deceleration parameter) calculates the deceleration required for a vehicle to come to a safe stop, given the vehicle's current speed and position on the road.

number

number

[0118] Similar to the static TTI algorithm, RDP alert The parameter reflects the conservatism level of the rule-based algorithm.

[0119] Rule-based approaches are used herein as a baseline for evaluating the performance of the machine learning algorithms of the present disclosure, and in some cases they are run in parallel with the machine learning algorithms to capture rare cases that machine learning may not be able to predict.

[0120] Machine Learning

[0121] Modeling driver behavior has been shown to be a complex task given the complexity of human behavior. See H.M. Mandalia and D.D. Dalvucci, "Using Support Vector Machines for Lane-Change Detection," Human Factors and Ergonomics Society Annual Meeting Proceedings, vol. 49, pp. 1965-1969, 2005. Machine learning techniques are well suited to modeling human behavior, but they require "learning" using training data to operate properly. To provide superior detection and prediction results, machine learning is used herein to model detected traffic at intersections or other features of the ground transportation network during a training period before alert processing is applied to current traffic during the deployment phase. Machine learning can further be used to model driver responses using in-vehicle data from on-board equipment (OBE) and can be further based on in-vehicle sensors and driving record history and preferences. Machine learning models are also used herein to detect and predict the trajectories, behaviors, and intentions of vulnerable road users (e.g., pedestrians). Machine learning can also be used to model the responses of vulnerable road users from operator-on-board equipment (OPE). These models can include interactions between entities, between vulnerable road users, and between one or more entities and one or more vulnerable road users.

[0122] Machine learning techniques may also be used to model the behavior of non-autonomous ground traffic entities. By observing and / or communicating with the non-autonomous ground traffic entities, machine learning may be used to predict the intentions of the non-autonomous ground traffic entities, and to communicate with the non-autonomous ground traffic entities and with other involved entities when near misses, accidents, or other dangerous situations are predicted.

[0123] The machine learning mechanism works well in two phases: 1) training, and 2) deployment.

[0124] Training Phase

[0125] After installation, the RSE begins collecting data from sensors it can access. Because AI model training requires significant computing power, it is typically performed on powerful servers containing multiple parallel processing modules to speed up the training phase. For this reason, data acquired at the RSE's location in the ground transportation network can be packaged and sent to a remote, powerful server immediately after acquisition. This is done using an internet connection. The data is then prepared automatically or with the assistance of a data scientist. An AI model is then built to capture important characteristics of the traffic flow of vehicles and other ground transportation entities relative to that intersection or other aspect of the ground transportation network. The captured data features may include the position, direction, and movement of vehicles or other ground transportation entities, which can then be converted into intent and behavior. Knowing the intent allows the AI ​​model to predict, with high accuracy, the actions and future behavior of vehicles or other ground transportation entities approaching the traffic location. The trained AI model is then tested on a subset of data that was not included in the training phase. If the AI ​​model's performance meets expectations, training is considered complete. This phase is repeated repeatedly using different model parameters until a satisfactory performance of the model is achieved.

[0126] Deployment Phase

[0127] In some embodiments, the completed and tested AI model is then transmitted via the Internet to RSEs at traffic locations in the ground transportation network. The RSEs are then ready to process new sensor data and predict and detect dangerous situations, such as traffic light violations. If a dangerous situation is predicted, the RSE generates an appropriate warning message. Before the predicted dangerous situation occurs, the dangerous situation may be predicted, a warning message may be generated, and the warning message may be broadcast to and received by vehicles and other ground transportation entities in the vicinity of the RSE. This provides sufficient time for operators of vehicles or other ground transportation entities to react and to implement collision avoidance. The output of the AI ​​model from various intersections where the corresponding RSEs are located may be recorded and made available online in a dashboard incorporating all the generated and displayed data in an intuitive and user-friendly manner. Such a dashboard may be used as an interface with customers of the system (e.g., city traffic engineers or planners). An example dashboard is a map with markers showing the locations of monitored intersections, violation events that have occurred, and statistical and analytical results based on AI predictions and actual outcomes.

[0128] Smart RSE (SRSE) and connected / disconnected entity bridging

[0129] As already alluded to, there are gaps between the capabilities and behaviors of connected and unconnected entities. For example, connected entities are typically coordinated entities that continuously advertise their location and safety system status, e.g., speed, heading, brake status, and headlight status, to the world. Unconnected entities are unable to coordinate and communicate in these ways. Thus, connected entities are unaware of unconnected entities that are not in the connected entity's vicinity or that are outside their detection range due to interference, distance, or lack of vantage point.

[0130] With the appropriate equipment and configuration, RSEs can be enabled to detect all entities using the ground transportation network in their vicinity, including unconnected entities. Specialized sensors can be used to detect different types of entities. For example, radar is suitable for detecting moving metal objects, such as cars, buses, and trucks. Such road entities are most likely moving in one direction toward an intersection. Cameras are suitable for detecting vulnerable road users who may be pondering around an intersection, waiting for a safe time to cross.

[0131] Placing sensors on components of the ground transportation network has at least the following advantages:

[0132] - Vantage Points: Infrastructure poles, beams, and support cables typically include high vantage points. High vantage points allow for a more comprehensive view of the intersection. This is similar to a control tower at an airport, where air traffic controllers have a panoramic view of most of the critical and vulnerable users on the ground. For ground traffic entities, in contrast, the view from a sensor's (camera, lidar, radar, etc., or other) vantage point can be obstructed or obstructed by trucks in nearby lanes, direct sunlight, or other interference. Sensors at intersections can be selected to be resistant to and less susceptible to such interference. For example, radar is not affected by sunlight and remains effective during the evening commute. Thermal cameras are more likely to detect pedestrians in bright light conditions where an optical camera's view is obstructed.

[0133] Fixed position: A sensor located at an intersection can be adjusted and fixed to detect in a specific direction that may be optimal for detecting important targets. This helps the processing software to detect objects better. As an example, if a camera has a fixed view, the background information (of stationary objects and structures) in the fixed view can be easily detected and used to improve the identification and classification of relatively important moving entities.

[0134] Fixed sensor locations also allow for easier placement of each entity in an integrated global view of the intersection. Because the sensor view is fixed, measurements from the sensors can be easily mapped to an integrated global position map of the intersection. Such an integrated map is useful when conducting a global analysis of traffic movement from all directions to investigate the interactions and dependencies of one traffic flow on another. One example is detecting near misses (hazardous situations) before they occur. When two entities are traveling along intersecting paths, the global and integrated view of the intersection allows for the calculation of each entity's arrival time at the intersection on their respective paths. If the time is within a certain limit or tolerance, the near miss can be flagged (e.g., the subject of a warning message) before the near miss occurs.

[0135] With the help of sensors installed on infrastructure components, smart RSE (SRSE) can bridge this gap and enable connected entities to recognize "dark" or unconnected entities.

[0136] FIG. 8 shows a scenario illustrating how strategically placed sensors can help a connected entity identify the velocity and location of an unconnected entity.

[0137] Connected entity 1001 travels along path 1007. Entity 1001 has green light 1010 on. Unconnected entity 1002 travels along path 1006. Unconnected entity 1002 is subject to red light 1009 but intends to turn right along path 1006 on the red light. This places unconnected entity 1002 directly in entity 1001's path. A dangerous situation is imminent because entity 1001 does not see entity 1002. Because entity 1002 is an unconnected entity, it cannot broadcast (e.g., advertise) its location and heading to other entities sharing the intersection. Furthermore, even if entity 1001 were connected, entity 1001 would not be able to "see" entity 1002, who is obscured by building 1008. There is a risk of entity 1001 traveling straight through the intersection and colliding with entity 1002.

[0138] If the intersection is configured as a smart intersection, radar 1004 mounted on a beam 1005 above the road at the intersection detects the entities 1002 and the speed and distance of the entities 1002. This information can be relayed to the connected entities 1001 through the SRSE 1011, which acts as a bridge between the unconnected entities 1002 and the connected entities 1001.

[0139] Artificial Intelligence and Machine Learning

[0140] Smart RSEs also rely on learning traffic patterns and entity behavior to better predict and prevent dangerous situations and to avoid collisions. As shown in FIG. 8 , radar 1004 is constantly detecting and providing data about each entity moving along the approach road 1012. This data is collected and transmitted to the cloud, either directly or through the RSE, for example, for analysis and to build and train a model that adequately represents traffic along the approach road 1012. When the model is complete, it is downloaded to the SRSE 1011. This model can then be applied to each entity moving along the approach road 1012. If an entity is classified by the model as violating (or attempting to violate) a traffic rule, a warning (alert) can be broadcast by the SRSE to all connected entities in the vicinity. This warning, known as an intersection collision avoidance warning, can be received by the connected entities and influence them to consider the dangerous situation and avoid collisions. By using an appropriate traffic model, violating entities can be detected in advance, giving connected entities using the intersection sufficient time to react and avoid a dangerous situation.

[0141] With the help of multiple sensors (some mounted high on components of the ground transportation network's infrastructure), artificial intelligence models, and accurate traffic models, the SRSE can obtain a virtual overview of the ground transportation network and recognize each entity in its field of view, including unconnected entities in its field of view that are not "visible" to connected entities in its field of view. The SRSE can use this data to feed the AI ​​model and provide alerts to connected entities on behalf of unconnected entities. Otherwise, connected entities would not know that there are unconnected entities sharing the road.

[0142] The SRSE can utilize large-capacity computing power at its location, either within the same housing, through connection to a nearby unit, or through connection to a server over the internet. The SRSE can process data received directly from sensors or via broadcast from nearby SRSEs, emergency and weather information, and other data. The SRSE also includes large-capacity storage to help store and process data. High-bandwidth connectivity is also required to help transmit raw data and AI models between the SRSE and a more powerful remote server. The SRSE enhances other AI-based traffic hazard detection technologies to achieve high accuracy and provide additional time to react and avoid collisions.

[0143] The SRSE can still be compatible with current and new standardized communication protocols, so the SRSE can seamlessly interface with equipment already deployed in the field.

[0144] The SRSE may also reduce network congestion by sending messages only when necessary.

[0145] Global and integrated intersection topology

[0146] Effective traffic monitoring and control at an intersection benefits from a bird's-eye view of the intersection unobstructed by obstacles, lighting, or any other interference.

[0147] As described above, different types of sensors can be used to detect different types of entities. The information from these sensors can differ, e.g., inconsistent with respect to the location or motion parameters the data represents, the native format of the data, or both. For example, radar data typically includes speed, distance, and sometimes additional information, such as the number of moving and stationary entities within the radar's field of view. Camera data, in contrast, can represent an image of the field of view at any point in time. Lidar data can provide the location of a point in 3D space corresponding to the reflection point of a laser beam emitted by the lidar at a specific time and direction of travel. Generally, each sensor provides data in a native format that adequately represents the physical quantity the sensor measures.

[0148] To obtain a unified view (representation) of an intersection, it is useful to fuse data from different types of sensors. For the purpose of fusion, data from the various sensors is converted into a common (unified) format that is independent of the sensors used. The data contained in the unified format from all of the sensors includes the global position, speed, and heading of each entity using the intersection, regardless of how it was detected.

[0149] By using this integrated global data, the smart RSE can not only detect and predict the movements of entities, but also identify the relative positions and headings of different entities with respect to each other. Thus, the SRSE can achieve improved detection and prediction of dangerous situations.

[0150] For example, in the scenario shown in Figure 9, motorized entity 2001 and vulnerable road user 2002 share the same crosswalk. Entity 2001 is moving along road 2007 and is detected by radar 2003. Vulnerable road user 2002, walking along sidewalk 2006, is detected by camera 2004. Vulnerable road user 2002 may decide to cross road 2007 using crosswalk 2005. Doing so results in road user 2002 in entity 2001's path creating a potentially dangerous situation. Because each of the sensors may only detect entities in its respective field of view, if the data from each of sensors 2003 and 2004 is considered independently and no other information is taken into account, the dangerous situation will not be identified. Furthermore, each of the sensors may be unable to detect objects that the sensor is not designed to detect. However, when a synthetic view is considered by the SRSE, the position and dynamics of the entity 2001 and of the potentially affected road user 2002 may be located in the same reference frame, i.e., a geographical coordinate system, e.g., a map projection, or other coordinate system. When considered in a common reference frame, the fused data from the sensors may be used to detect and predict potentially dangerous situations between the two entities 2001 and 2002. In the following paragraphs, the transformation between sensor space and synthetic space is discussed.

[0151] Conversion of radar data to a synthetic standard

[0152] As shown in FIG. 10 , radar 3001 is used to monitor road entities traveling along a road including two lanes 3005 and 3008, each including centerlines 3006 and 3007. Stop bars 3003 indicate the ends of lanes 3005 and 3008. T 3006 can be defined by a set of markers 3003 and 3004. While FIG. 10 shows only two markers, the centerline is generally a piecewise linear function. The global positions of markers 3003 and 3004 (and other markers not shown) are predetermined by the roadway design and are known to the system. The precise global position of radar 3001 can be further determined. Thus, distances 3009 and 3010 of markers 3003 and 3004 from radar 3001 can be calculated. Distance 3011 of entity 3002 from radar 3001 can be measured by radar 3001. Using simple geometry, the system can determine the location of entity 3002 using the measured distance 3011. The result is a global position because it is derived from the global positions of markers 3003, 3004 and radar 3001. Since each roadway can be approximated by a generalized piecewise linear function, the above method can be applied to any roadway that can be monitored by radar.

[0153] 11 shows a similar scenario on a curved road. Radar 4001 monitors an entity moving along road 4008. Markers 4003 and 4004 represent linear segments 4009 (of a piecewise linear function) of centerline 4007. Distances 4005 and 4006 typically represent the normal distances between the plane 4010 of radar 4001 and markers 4003 and 4004, respectively. Distance 4007 is the measured distance of entity 4002 from radar plane 4010. Following this, given the global positions of radar 4001 and markers 4003 and 4004, the global position of entity 4002 can be calculated using simple ratio arithmetic.

[0154] Transforming camera data into a synthetic reference

[0155] By knowing the camera's height, global position, orientation, tilt, and field of view, calculating the global position of each pixel in the camera image becomes simple using existing 3D geometric laws and transformations. As a result, when an object is identified in an image, its global position can be easily estimated by knowing the pixel it occupies. It is useful to note that if the camera's specifications, such as sensor size, focal length, or field of view, or a combination thereof, are known, the type of camera is irrelevant.

[0156] 12 shows a side view of camera 5001 looking at entity 5002. The height 5008 and tilt angle 5006 of camera 5001 can be determined at installation. The field of view 5007 can be known from the specifications of camera 5001. The global position of camera 5001 can also be specified at installation. From the known information, the system can determine the global positions of points 5003 and 5004. The distance between points 5003 and 5004 is further divided into pixels in the image generated by camera 5001. The number of pixels is known from the specifications of camera 5001. The pixel occupied by entity 5002 can be identified. Thus, distance 5005 can be calculated. The global position of entity 5002 can also be calculated.

[0157] By fusing information from various sensors, a global, integrated view of any intersection can be pieced together. FIG. 13 shows a top view of a four-way intersection. Each leg of the intersection is divided by a median 6003. The intersection in the figure is monitored by two different types of sensors: radar and camera; the principles described herein can be generalized to other types of sensors. In this example, the radar monitoring area 6001 overlaps the camera monitoring area 6002. By using a unified global view, each entity moving between areas continues to be tracked within the unified global view. This, for example, allows the SRSE to easily identify relationships between the actions of different entities. Such information allows for a truly universal, bird's-eye view of the intersection and roadways. The integrated data from the sensors can then be fed to an artificial intelligence program, as described in the following paragraphs.

[0158] FIG. 2 above illustrates the components of an RSE. Additionally, in an SRSE, the processing unit may further include one or more specialized processing units that can process data in parallel. An example of such a unit is a graphics processing unit, or GPU. With the assistance of a GPU or similar hardware, machine learning algorithms can operate much more efficiently in an SRSE and provide results in real time. Such a processing architecture enables real-time prediction of dangerous situations and thus allows for early warnings to be sent to provide entities with sufficient time to react and avoid collisions. Additionally, because the SRSE may perform processing that may use data from different sensors and different types of sensors, the SRSE may build an integrated view of an intersection that is useful for analyzing traffic flow and detecting and predicting dangerous situations.

[0159] Usage example

[0160] A wide variety of scenarios may benefit from the system and the early warning it may provide for collision avoidance. Examples are provided herein.

[0161] Example 1: Vulnerable Ground Transportation Entities

[0162] As shown in FIG. 4 , a roadway crossing a typical intersection 409 may include crosswalks with specific crossing areas 401, 402, 403, and 404 that pedestrians and other vulnerable road users (vulnerable road users) can use to walk across the roadway. Sensors suitable for detecting such crossings or other vulnerable users are located at one or more vantage points that enable monitoring of the crosswalks and their surroundings. During the training phase, collected data may be used to train an artificial intelligence model to learn about the behavior of vulnerable road users at the intersection. Then, during the deployment phase, the AI ​​model may use current data about vulnerable road users to, for example, predict that a vulnerable road user is about to cross the roadway and make that prediction before the vulnerable road user begins to cross. If the behavior and intentions of pedestrians and other vulnerable road users, drivers, vehicles, and other people and ground traffic entities can be predicted in advance, early warnings (e.g., alerts) may be sent to any or all of them. Early warnings may allow vehicles to stop, slow down, reroute, or a combination thereof, and may allow vulnerable road users to refrain from crossing the road if a dangerous situation is predicted to be imminent.

[0163] Generally, sensors are used to monitor all areas near intersections for possible movement of vulnerable road users and vehicles. The type of sensor used depends on the type of object being monitored and tracked. Some sensors are better at tracking people and bicycles or other non-motorized vehicles. Some sensors are better at monitoring and tracking motorized vehicles. The solution described herein is sensor and hardware agnostic, as the type of sensor is irrelevant if it provides appropriate data at a sufficient data rate, which may depend on the type of object being monitored and tracked. For example, Doppler radar is an appropriate sensor for monitoring and tracking vehicle speed and distance. The data rate, or sampling rate, is the rate at which the radar can provide successive new data values. The data rate must be fast enough to capture the dynamics of the motion of the monitored and tracked object. The higher the sampling rate, the more detail is captured, making the data's representation of motion more robust and accurate. If the sampling rate is too low, and if the vehicle travels a significant distance between two sample instances, it becomes difficult to model the behavior due to details that are missed during the intervals where no data is generated.

[0164] For crosswalks, sensors monitor the intersection and areas near the intersection for crossing pedestrians and other vulnerable road users (e.g., cyclists). Data from these sensors can be segmented to represent conditions using different virtual zones to aid in detection and localization. Zones can be selected to correspond to critical areas where dangerous situations may be expected, such as sidewalks, sidewalk entrances, and road approaches 405, 406, 407, 408 to the intersection. Behavior and other conditions in each zone are recorded. Recordings can include, but are not limited to, kinematics (e.g., position, heading, speed, and acceleration) and facial and body features (e.g., eyes, posture).

[0165] The number of sensors, the number of zones, and the shape of the zones are specific to each intersection and to each approach to the intersection.

[0166] FIG. 5 shows a floor plan of a typical example setup showing different zones used to monitor and track the movements and behavior of pedestrians or other vulnerable road users, motorized and non-motorized vehicles, and other ground transportation entities.

[0167] Sensors are configured to monitor crosswalks across a roadway. Virtual zones (301, 302) may be located on the sidewalk and along the crosswalk. Other sensors are positioned to monitor vehicles and other ground traffic entities traveling on the path leading to the crosswalk, with virtual zones (303, 304) strategically placed to help detect, for example, incoming vehicles and other ground traffic entities, their distance from the crosswalk, and their speed.

[0168] The system (e.g., the RSE or SRSE associated with the sensors) collects streams of data from all sensors. When the system is first operational, an initial rule-based model may be deployed to aid in equipment calibration and function. Meanwhile, sensor data (e.g., speed and distance from the radar unit, images and video from the cameras) is collected and stored locally on the RSE in preparation for transmission to a remote computer powerful enough to use this collected data to build AI models of the behavior of different entities at the intersection. In some examples, the RSE itself is an SRSE capable of generating AI models.

[0169] Thus, data is prepared and trajectories are constructed for each ground traffic entity passing through the intersection. For example, trajectories can be derived from radar data by connecting together points at different distances that belong to the same entity. Pedestrian trajectories and behaviors can be derived, for example, from camera and video recordings. By implementing video and image processing techniques, pedestrian movements can be detected in images and videos and their respective trajectories can be estimated.

[0170] For human behavior, intelligent machine learning-based models typically outperform simple rules based on simple physics because human intent is difficult to capture and large data sets are required to be able to detect patterns.

[0171] When the machine learning (AI) model is completed on the server, the AI ​​model is downloaded to the RSE, e.g., over the internet. The RSE then applies current data captured from the sensors to the AI ​​model to result in it predicting intent and behavior, identify when a dangerous situation is imminent, and trigger a corresponding alert that is disseminated (e.g., broadcast) to vehicles and other ground traffic entities as an early warning and to potential road users and drivers in time to enable them to take collision avoidance steps.

[0172] This exemplary setup can be combined with any other use case, for example traffic at a traffic lighted intersection or at grade intersection.

[0173] Example 2: Intersection with traffic lights

[0174] For a signalized intersection (e.g., controlled by traffic lights), the overall configuration of the system is as in Example 1. One difference may be the type of sensors used to monitor or track vehicle speed, direction, distance, and location. The crosswalk configuration in Example 1 may also be combined with a signalized intersection configuration for a more general solution.

[0175] The concept of operation for the traffic lighted intersection use case is to track road users around the intersection using external sensors that collect data about the users or data communicated by the users themselves, predict the users' behavior, and broadcast warnings through different communication means about approaching dangerous situations, typically due to violation of traffic rules at the intersection, e.g., violating a red light.

[0176] Data about road users can be collected using (a) entity data broadcast by each entity itself regarding its current state, for example, through a BSM or PSM, and (b) sensors installed externally on the infrastructure or on vehicles, such as Doppler radar, ultrasonic sensors, visual or thermal cameras, and lidar. As mentioned above, the type of sensor selected and its location and orientation at the intersection must provide the widest coverage of the intersection or the portion under investigation, and the collected data about entities approaching the intersection is the most accurate. The collected data therefore enables the reconstruction of the current state of road users and the generation of accurate, timely, and useful virtual basic safety messages (VBSMs) or virtual personal safety messages (VPSMs). The frequency with which data must be collected depends on the potential danger of each type of road user and the criticality of potential violations. For example, a motorized vehicle moving at high speed at an intersection typically requires 10 data updates per second to achieve real-time collision avoidance, while a pedestrian crossing the intersection at a much slower speed may require as few as one data update per second.

[0177] As previously described, FIG. 4 shows an example plan view of a signalized intersection including detection virtual zones. These zones may segment each approach to the intersection into separate lanes 410, 411, 412, 413, 405, 406, 407, and 408, and may further divide each lane into areas corresponding to typical ranges of distance from the stop bar. The selection of these zones may generally be performed empirically to suit the characteristics of the particular approach and intersection. Segmenting the intersection allows for more accurate identification of the relative heading, speed, acceleration, and position of each road user, which in turn allows for better assessment of the potential hazards that road users present to other ground traffic entities.

[0178] To determine whether an observed traffic situation is unsafe, the system also needs to compare the results of the predicted situation with the status of traffic lights and consider local traffic rules (e.g., left-turn lanes, right turns on red, etc.). Therefore, collecting and using signal phase and timing (SPaT) information for an intersection is required. SPaT data can be collected by directly interfacing with traffic light controllers at the intersection, typically by reading data through a wired connection, or by interfacing with a traffic management system to receive requested data, for example, through an API. To ensure that road user status is always synchronized with traffic signal status, it is important to collect SPaT data at a rate as close as possible to the rate at which road user data is collected. An additional complication to the need for SPaT information is that modern traffic control techniques used to regulate traffic flow near intersections are not based on fixed timing but use algorithms that can dynamically adapt to real-time traffic conditions. Therefore, incorporating SPaT data prediction algorithms is important to ensure the highest accuracy in violation prediction. These SPaT data prediction algorithms can be built using rule-based or machine learning methods.

[0179] For each approach to an intersection, data is collected by the RSE (or SRSE), and a machine learning (AI) model is constructed to explain vehicle behavior corresponding to the collected data. Current data collected at the intersection is then applied to the AI ​​model to generate an early prediction of whether a vehicle or other ground traffic entity traveling on one of the approaches to the intersection is about to violate, for example, a traffic light. If a violation is imminent, a message is relayed (e.g., broadcast) from the RSE to nearby ground traffic entities. Vehicles (including the violating vehicle) and pedestrians or other potentially affected road users receive the message and have time to take appropriate preemptive measures to avoid a collision. The message may be delivered to the ground traffic entity by one or more of the following techniques: flashing lights, signs, or radio signals, among other possibilities.

[0180] If a vehicle or other entity approaching the intersection is equipped with an OBE or OPE, it can receive a message broadcast from the RSE that a potential hazard has been predicted at the intersection. This allows the user to be alerted and take appropriate preemptive measures to avoid a collision. If a violating road user at the intersection is also equipped with an OBE or OPE, the user also receives the broadcast alert. An algorithm in the OBE or OPE can then match the user's violating behavior with the message and alert the user appropriately.

[0181] The decision to send an alert depends not only on the vehicle's behavior as indicated by data collected by sensors at the intersection. While sensors play a large role in the decision, other inputs are also considered. These inputs may include, but are not limited to, information from nearby intersections (if a vehicle runs a red light at a nearby intersection, there is a higher probability that the vehicle will do the same at this intersection), information from other associated vehicles, or even information from the vehicle itself, for example, if the vehicle reports that it is experiencing an anomaly.

[0182] Example 3: Intersection without traffic lights

[0183] Controlled intersections without traffic lights, such as those controlled by stop signs or yield signs, can also be monitored. Sensors can be used to monitor approach paths controlled by traffic lights, and predictions can be made regarding incoming vehicles, similar to predictions regarding incoming vehicles at approaches to signalized intersections. The rules of the road at controlled intersections without traffic lights are typically clearly defined. Ground traffic entities at approach paths controlled by stop signs must come to a complete stop. At multi-way stop intersections, the order in which ground traffic entities reach the intersection determines right-of-way. Special cases involving one-way stops can be considered. A set of sensors can also monitor approach paths without stop signs. Such a configuration can assist in stop sign gap negotiation. For intersections controlled by yield signs, ground traffic entities at approach paths controlled by yield signs must reduce their speed to grant right-of-way to other ground traffic entities at the intersection.

[0184] A major challenge is that due to internal factors (e.g., driver distraction) or external factors (e.g., lack of visibility), ground transportation entities violate the rules of the road and put other ground transportation entities at risk.

[0185] In the general example of a stop sign controlled intersection (i.e., each approach is controlled by a stop sign), the overall configuration of the system is as in Example 1. One difference may be the type of sensors used to monitor or track vehicle speed, direction, distance, and location. Another difference is that the rules of the road are indicated by road signs, without including a traffic light controller. The configuration for pedestrian crossings in Example 1 may also be combined with the configuration for a controlled intersection without traffic lights for a more general solution.

[0186] 4 can also be understood to illustrate an example of a plan view of a four-way stop intersection including detection virtual zones. These zones may segment each approach to the intersection into separate lanes 410, 411, 412, 413, 405, 406, 407, 408, and further divide each lane into areas corresponding to common ranges of distances from the stop bar. The selection of these zones may generally be done empirically to suit the characteristics of the particular approach and intersection.

[0187] In a similar manner to that described above for Figure 4, current data collected at the intersection is applied to an AI model to generate an early prediction of whether a vehicle or other ground traffic entity traveling on one of the approaches to the intersection is about to violate a stop sign. If a violation is imminent, the message may be treated similarly to the previous example involving a traffic light violation.

[0188] Further, as discussed above, the decision to send an alert may be based on the factors discussed above, as well as other information, such as whether the vehicle ran a stop sign at a nearby intersection (suggesting that the vehicle is more likely to do the same at this intersection).

[0189] Figure 18 shows a use case for an intersection without controlled traffic lights. It illustrates how an SRSE containing strategically placed sensors can alert connected entities to impending dangerous situations arising from unconnected entities.

[0190] Connected entity 9106 is traveling along path 9109. Entity 9106 has the right-of-way. Unconnected entity 9107 is traveling along path 9110. Entity 9107 is subject to yield sign 9104, which is attempting to merge onto path 9109 without granting entity 9106 the right-of-way, and is attempting to place entity 9107 directly into entity 9106's path. A dangerous situation is imminent because entity 9106 is unaware of entity 9107. Because entity 9107 is an unconnected entity, it cannot advertise (broadcast) its position and heading to other entities sharing the intersection. Furthermore, entity 9106 may be unable to "see" entity 9107 that is not within its direct line of sight. If entity 9106 continues along its path, it may eventually collide with entity 9107.

[0191] Because the intersection is a smart intersection, radar 9111 mounted on a beam 9102 above the road detects entity 9107. Radar 9111 also detects the speed and distance of entity 9107. This information can be relayed as an alert through SRSE 9101 to connected entities 9106. SRSE 9101 contains a machine learning model for entities moving along approach road 9110. If entity 9107 is classified by the model as a potential violator of traffic rules, a warning (alert) is broadcast to connected entities 9106. This warning is sent in advance, giving entity 9106 enough time to react to and prevent a dangerous situation.

[0192] Example 4: At-grade intersection

[0193] At-grade intersections are dangerous because they can carry motorized vehicles, pedestrians, and rail vehicles. Often, the road leading to the at-grade intersection is in the blind spot of train or other rail vehicle operators (e.g., conductors). Because rail vehicle operators operate primarily based on line-of-sight information, this increases the likelihood of accidents when road users violate the rail vehicle's right-of-way and when road users cross the at-grade intersection when they are not authorized to cross.

[0194] The behavior of the at-grade intersection use case is similar to that of a signalized intersection in the sense that at-grade intersections are often a collision point between road and rail traffic, regulated by traffic rules and signals. Therefore, this use case also requires collision avoidance warnings to increase safety near at-grade intersections. Rail traffic may have planned separated rail rights-of-way (e.g., high-speed rail) or may not have separated rail rights-of-way (e.g., light urban rail or tram). With light rail and tram services, this use case becomes even more important because these rail vehicles operate on more active roads and must follow the same traffic rules as road users.

[0195] Figure 6 shows a common use case for an at-grade intersection where a road and pedestrian crossing cross a railroad. As with the pedestrian crossing use case, sensors are deployed to collect data on pedestrian movements and intentions. Other sensors are used to monitor and predict the movements of road vehicles approaching the intersection. Data on road users can also be collected from road user broadcasts (e.g., BSM or PSM). Data from nearby intersections, vehicles, and remote command and control centers can be used to determine whether to trigger an alert.

[0196] Data on SPaT for road and rail approaches also needs to be collected to properly assess potential violations.

[0197] Similar to the signalized intersection use case, the collected data allows for the generation of predictive models using rule-based and machine learning algorithms.

[0198] In this use case, rail vehicles are equipped with an OBE or OPE to receive collision avoidance warnings. If a violation of the rail vehicle's right-of-way is predicted, the RSE broadcasts a warning message, alerting the rail vehicle driver that a road user is in its intended path and allowing the rail vehicle driver to take preemptive action with enough time to avoid a collision.

[0199] If the violation road user is also equipped with an OBE or an OPE, the message broadcast by the RSE will also be received by the violation road user. An algorithm in the OBE or OPE can then match the user's violation behavior with the received message and warn the user appropriately.

[0200] (Bridging the Gap) Virtual Connected Ground Transportation Environment

[0201] As described thus far, a useful application of the system is to generate a virtual connected environment in place of disconnected ground transportation entities. An obstacle to the adoption of connected technologies is not only the lack of infrastructure installations, but also the near-existence of connected vehicles, connected vulnerable road users, and other connected ground transportation entities.

[0202] In relation to connected vehicles, in some regulatory regimes, such vehicles constantly transmit what are called basic safety messages (BSMs). BSMs include, among other information, the vehicle's location, direction of travel, speed, and future route. Other connected vehicles can pay attention to these messages and use them to generate a map of vehicles in their environment. By knowing where surrounding vehicles are, vehicles, whether autonomous or not, have useful information to maintain a high level of safety. For example, an autonomous vehicle can maneuver to avoid a connected vehicle in its path. Similarly, a driver can receive an alert if any other vehicle is on the path the driver plans to follow, such as a sudden lane change.

[0203] Until all ground transportation entities are equipped to send and receive traffic safety messages and information, some road entities will be "dark" or invisible to the rest of the road entities. Dark road entities pose a risk of hazardous conditions.

[0204] Because dark road entities do not advertise (e.g., broadcast) their locations, they are invisible to connected entities, which may expect all road entities to broadcast their information (i.e., be connected entities). While on-board sensors can detect obstacles and other road entities, the range of these sensors tends to be too short to be effective in preventing dangerous situations and collisions. Thus, there is a gap between the connectivity of connected vehicles and the lack of connectivity of unconnected vehicles. The technology described below aims to bridge this gap by using infrastructure intelligence that can detect all vehicles or other components of the ground transportation network at intersections and send messages on behalf of unconnected vehicles.

[0205] The system may establish a virtual connected ground traffic environment that may bridge the gap between a future in which most vehicles (and other ground traffic entities) are envisioned to be connected entities and the present in which most vehicles and other ground traffic entities lack connectivity, for example at intersections. In the virtual connected ground traffic environment, smart traffic lights and other infrastructure installations may use sensors to track all vehicles and other ground traffic entities (connected, disconnected, semi-autonomous, autonomous, non-autonomous) and (in the case of vehicles) generate virtual BSM messages (VBSMs) on their behalf.

[0206] A VBSM message can be considered a subset of a BSM. It may not contain all fields required to generate a BSM, but it may contain all localization information, including position, heading, speed, and trajectory. Because V2X communications are standardized and anonymous, VBSM and BSM cannot be easily distinguished and follow the same message structure. The main difference between the two messages is the availability of the source of the information contained in these messages. A VBSM may not include data and information that is not easily generated by external sensors, such as steering wheel angle, brake status, tire pressure, or wiper activation.

[0207] With the appropriate sensors installed, an intersection containing a smart RSE can detect all road entities moving through the intersection. The RSE can then convert all data from multiple sensors into a global integrated coordinate system. This global integrated system is represented by the geographic location, speed, and direction of travel of each road entity. Each road entity, whether connected or not, is detected by intersection equipment, and a global integrated position is generated on behalf of each road entity. Therefore, standard safety messages can be broadcast on behalf of road entities. However, if the RSE broadcasts safety messages for all entities it detects, the RSE may send messages on behalf of connected road entities. To resolve conflicts, the RSE can filter connected road entities from its list of dark entities. This can be achieved because the RSE continuously receives safety messages from connected vehicles and because the RSE sensors continuously detect road entities passing through the intersection. If the detected location of a road entity matches a location from which a safety message was received by the RSE receiver, the road entity is inferred to be connected and no safety messages are broadcast by the RSE on behalf of that road entity. This is shown in Figure 15.

[0208] By creating a bridge between connected and unconnected vehicles, connected entities (including autonomous vehicles) may safely maneuver through intersections with full awareness of all nearby road entities.

[0209] This aspect of the technology is illustrated in FIG. 17. An intersection 9001 contains multiple road entities at a given time. Some of these entities are unconnected 9004, 9006, and others are connected 9005, 9007. Vulnerable road users 9004, 9007 are detected by a camera 9002. Motorized road entities 9005, 9006 are detected by a radar 9003. The location of each road entity is calculated. Broadcasts from connected road entities are also received by an RSE 9008. The location of the entity from which the message was received is compared to the location at which the entity was detected. If the two entities match within a predetermined tolerance, the entity at that location is considered connected, and no safety messages are sent on behalf of that entity. The remaining road entities without matching receiving locations are considered dark. Safety messages are broadcast in their place.

[0210] For collision and intersection violation warnings, which are an integrated part of the V2X protocol, each entity must be connected for the system to be effective. That requirement poses a hurdle in the deployment of V2X devices and systems. Intersections with smart RSEs address that concern by providing a virtual bridge between connected and unconnected vehicles.

[0211] The U.S. Department of Transportation (DOT) and National Highway Traffic Safety Administration (NHTSA) have identified many connected vehicle applications that use BSM to help substantially reduce non-impaired crashes and fatalities. These applications include, but are not limited to, Forward Collision Warning (FCW), Intersection Movement Assist (IMA), Left Turn Assist (LTA), Do Not Pass Warning (DNPW), and Blind Spot / Lane Change Warning (BS / LCW). The U.S. DOT and NHTSA have defined these applications as follows:

[0212] FCW resolves rear-end collisions and warns drivers of stopped, slowing, or slower vehicles ahead. IMA is designed to avoid intersection crossing collisions, warning drivers of vehicles approaching an intersection from the side and covering two main scenarios: same- or opposite-direction turn paths and straight-through crossing paths. LTA resolves collisions when one vehicle involved turns left at an intersection and another vehicle is moving straight-through from the opposite direction and warns drivers of approaching opposite-direction traffic when making a left turn. DNPW assists drivers in avoiding opposite-direction collisions caused by overtaking maneuvers and warns drivers of approaching opposite-direction vehicles when attempting to overtake a slower vehicle on an undivided two-lane roadway. BS / LCW resolves collisions when vehicles perform lane change / merging maneuvers before a collision and alerts drivers of approaching vehicles or the presence of vehicles in their blind spots in adjacent lanes.

[0213] The V2X protocol specifies that these applications must be accomplished using vehicle-to-vehicle (V2V) communications, where one connected remote vehicle broadcasts basic safety messages to a connected host vehicle. The host vehicle's OBE then attempts to match its own vehicle parameters, such as speed, heading, and trajectory, with these BSMs to determine whether there is a potential hazard or threat posed by the remote vehicle, as described herein thus far. Furthermore, autonomous vehicles particularly benefit from such applications because they allow surrounding vehicles to communicate their intent, a critical piece of information not included in data collected from their onboard sensors.

[0214] However, current vehicles are not connected, and as noted above, it will be a very long time before the percentage of connected vehicles is high enough for the BSM to function properly as described above. Thus, in an environment with a small percentage of connected vehicles, there is no need for connected vehicles to receive and analyze the large number of BSMs that connected vehicles would receive if the percentage of connected vehicles was high enough to enable the above-described applications and to fully benefit from V2X communications.

[0215] VBSMs can help bridge the gap between a current environment with mostly unconnected entities and a future environment with mostly connected entities, enabling the above-mentioned applications in the interim. In the technology described herein, a connected vehicle receiving a VBSM processes the VBSM as a regular BSM in that application. Because VBSMs and BSMs follow the same message structure, and because VBSMs contain substantially the same basic information as BSMs—e.g., speed, acceleration, heading, past trajectory, and predicted trajectory—the results of applying the message to a given application are substantially the same.

[0216] For example, consider an intersection with an unprotected left turn, where a connected host vehicle is attempting to turn left at the same moment that an unconnected remote vehicle is traveling straight from the opposite direction with the right-of-way. This is a situation where the execution of the maneuver depends on the host vehicle's driver's judgment of the situation; an improper assessment of the situation could result in a conflict and a potential near-miss or collision. External sensors installed in the surrounding infrastructure could detect and track the remote vehicle, or even both vehicles, collect basic information (e.g., speed, acceleration, heading, and past trajectory), and transmit it to the RSE, which could then use rule-based and / or machine learning algorithms to construct a predicted trajectory for the remote vehicle, fill in the fields required for the VBSM, and broadcast it on behalf of the unconnected remote vehicle. The host vehicle's OBE receives the VBSM containing information about the remote vehicle and processes the VBSM in its LTA application to determine whether the driver's maneuver poses a potential hazard and whether the OBE should display a warning to the host vehicle's driver to take proactive or corrective action to avoid the collision. A similar result can be achieved when a distant vehicle is connected and receives data from the RSE and sensors that an oncoming vehicle is about to make a left turn that would result in a collision.

[0217] VBSM can also be used in lane-change maneuvers. Such maneuvers can be dangerous if a vehicle changing lanes fails to take the necessary steps to ensure the maneuver is safe, for example, by checking its rearview and side mirrors and blind spots. New advanced driver assistance systems, such as blind spot warnings using onboard ultrasonic sensors, have been developed to help prevent vehicles from making dangerous lane changes. However, these systems can have drawbacks when the sensors are dirty or their field of view is blocked. Furthermore, existing systems do not attempt to warn compromised vehicles about other vehicles attempting to change lanes. While V2X communication can help solve this problem through applications such as BS / LCW using BSM, the vehicle attempting to change lanes may be an unconnected vehicle and therefore unable to communicate its intentions. VBSM can help achieve this goal. Similar to the LTA use case, external sensors installed in the surrounding infrastructure could detect and track unconnected vehicles attempting to perform lane-change maneuvers, collecting basic information such as speed, acceleration, heading, and past trajectory and transmitting it to the RSE. The RSE then uses rule-based and machine learning algorithms to construct a predicted trajectory for the lane-changing vehicle, fills in the required fields for the VBSM, and broadcasts it instead of the unconnected, distant vehicle. The OBE of the at-risk vehicle then receives the VBSM containing information about the vehicle attempting to merge into the same lane, processes the VBSM, and determines whether the maneuver poses a potential hazard and whether it should display a lane-change warning to the vehicle's driver. If the lane-changing vehicle is a connected vehicle, its OBE may similarly receive VBSMs from the RSE for vehicles in its blind spot and determine whether the lane-changing maneuver poses a potential hazard to surrounding traffic and whether it should display a blind-spot warning to the vehicle's driver.If both vehicles are connected, they can broadcast their BSM to each other, enabling BS / LCW applications, but these applications still benefit from applying the same rule-based and / or machine learning algorithms to the BSM data as described above to make early predictions of the lane-changing vehicle's intent, along with whether the OBE should display a warning.

[0218] Autonomous Vehicles

[0219] The lack of connectivity for unconnected road entities impacts autonomous vehicles. Sensors in autonomous vehicles have short ranges or narrow fields of view. They cannot detect vehicles approaching buildings, for example, on street corners. They also cannot detect vehicles that may be hidden behind delivery trucks. If these hidden vehicles are unconnected entities, they are invisible to autonomous vehicles. These circumstances affect the ability of autonomous vehicle technology to achieve the level of safety necessary for mass adoption of the technology. Smart intersections can help mitigate this gap and aid in public acceptance of autonomous vehicles. Autonomous vehicles are only as good as their sensors. Intersections equipped with smart RSEs can extend the range of onboard sensors around blind corners or beyond large trucks. Such extensions allow autonomous entities and other connected entities to coexist with traditional, unconnected vehicles. Such coexistence can accelerate the adoption of autonomous vehicles and the benefits they bring.

[0220] The virtual connected ground transportation environment includes VBSM messages that enable the implementation of vehicle-to-vehicle (V2V), vehicle-to-pedestrian (V2P), and vehicle-to-device (V2D) applications that would otherwise be difficult to implement.

[0221] The system may use machine learning to quickly and accurately generate the fields of data required for various safety messages, pack them into a VBSM message structure, and transmit the messages to nearby ground transportation entities using various mediums, such as, but not limited to, DSRC, Wi-Fi, cellular, or traditional road signs.

[0222] Virtual personal safety messages (VPMS)

[0223] The ground traffic environment may include not only disconnected vehicles, but also disconnected people and other vulnerable road users.

[0224] In some regulatory regimes, connected vulnerable ground transportation entities continuously transmit personal safety messages (PSMs). The PSMs include, among other information, the location, heading, speed, and future route of the vulnerable ground transportation entity. Connected vehicles and infrastructure can receive these messages and use them to generate maps that include vulnerable entities and increase the level of safety in the ground transportation network.

[0225] Thus, a virtual connected ground transportation environment may bridge the gap between a future in which most vulnerable ground transportation entities are assumed to be connected and a present in which most vulnerable ground transportation entities lack connectivity. In a virtual connected ground transportation environment, smart traffic lights and other infrastructure installations may use sensors to track all vulnerable ground transportation entities (connected and unconnected) and generate VPSMs on their behalf.

[0226] A VPSM message may be considered a subset of a PSM. A VPSM need not include all fields required to generate a PSM, but may include data necessary for safety assessment and prevention of unsafe situations, including location-specific information, including position, heading, speed, and trajectory. In some instances, non-standard PSM fields, such as driver intent, posture, or looking direction, may also be included in a VPSM.

[0227] The system may use machine learning to quickly and accurately generate these fields, pack them into a VPSM message structure, and transmit it to nearby ground transportation entities using a variety of mediums, such as, but not limited to, DSRC, Wi-Fi, cellular, or traditional road signs.

[0228] VPSM messages enable the implementation of pedestrian-to-vehicle (P2V), pedestrian-to-infrastructure (P2I), pedestrian-to-device (P2D), vehicle-to-pedestrian (V2P), infrastructure-to-pedestrian (I2P), and device-to-pedestrian (D2P) applications that would be difficult to implement using other techniques.

[0229] FIG. 16 shows a pedestrian 8102 crossing a crosswalk 8103. The crosswalk 8103 may be at an intersection or a mid-block crosswalk spanning a stretch of road between intersections. A camera 8101 is used to monitor the sidewalk 8104. The global position of the boundary of the camera's 8101 field of view 8105 may be determined at installation. The field of view 8105 is covered by a predetermined number of pixels reflected by the camera's 8101 specifications. A road entity 8102 may be detected within the camera's field of view, and its global position may be calculated. The speed and direction of the road entity 8102 may further be determined from its displacement at time s. The path of the road entity 8102 may be represented by a breadcrumb 8106, which is a sequence of locations crossed by the entity 8102. This data may be used to construct a virtual PSM message. The PSM message may then be broadcast to all entities near the intersection.

[0230] Traffic and behavior enforcement at unsignalized intersections

[0231] Another useful application of the system is traffic enforcement (eg, stop signs, yield signs) at intersections without traffic lights, and enforcement of good driving behavior anywhere on the surface transportation network.

[0232] As a by-product of generating VBSM and VPSM, the system may track and detect road users who do not comply with traffic laws and increase the probability of unsafe situations and collisions. Prediction of unsafe situations may be extended to include enforcement. Unsafe situations do not necessarily result in collisions. Near misses are common and can increase driver stress levels and lead to subsequent accidents. The frequency of near misses is positively correlated with lack of enforcement.

[0233] Additionally, using VBSM, the system may detect inappropriate driving behavior, such as abrupt lane changes and other forms of reckless driving. Data collected by the sensors may be used to train and enable machine learning models to flag ground traffic entities that are engaging in risky driving behavior.

[0234] Law enforcement authorities typically enforce the rules of the road against ground traffic entities that contain vulnerable road users, but the authorities need to be present near intersections to monitor, detect, and report violations. By using VBSM and VPSM to track unconnected ground traffic entities that contain vulnerable road users, a smart RSE can act as a law enforcement authority and enforce the rules of the road at intersections. For example, an unconnected vehicle tracked by a smart RSE can be detected, identified, and reported to authorities for violating a stop sign or a yield sign. Similarly, a vulnerable road user near an intersection tracked by a smart RSE can be detected, identified, and reported to authorities for illegally crossing the intersection.

[0235] For enforcement and other purposes, ground transportation entities may be identified using unique identification information, including but not limited to plate number recognition. Potentially victimized road users may be identified using biometric recognition, including but not limited to facial recognition, retinal recognition, and voice waveform recognition. In special cases, including civil or criminal investigations, social media networks (e.g., Facebook, Instagram, Twitter) may further be used to support the identification of offending ground transportation entities or potentially victimized road users. An example of a social network being utilized is uploading a captured photo of a violator to the social network and requesting social network users who recognize the violator to provide information to law enforcement authorities that will help identify the violator.

[0236] Other embodiments are within the scope of the following claims. (Additional note 1) An apparatus comprising a device located at an intersection of a transportation network, The device comprises: an input for receiving data from a sensor oriented to monitor ground traffic entities at or near the intersection; a wireless communication device that transmits a warning to a device of one of the ground transportation entities regarding a dangerous condition at or near the intersection; a processor; Storage and Equipped with The storage storing a machine learning model capable of predicting a behavior of a ground traffic entity at or near the intersection at a current time, the machine learning model being based on training data regarding previous actions and associated behaviors of ground traffic entities at or near the intersection; applying current motion data received from the sensors regarding ground traffic entities at or near the intersection to the machine learning model to predict impending behavior of the ground traffic entities; inferring an imminent hazardous situation for one or more of the ground traffic entities at or near the intersection from the predicted imminent behavior; and causing the wireless communication device to transmit the warning regarding the hazardous situation to the device of one of the ground transportation entities; the storage for instructions executable by the processor to Device. (Additional note 2) the equipment includes roadside equipment; Item 1. The device of item 1. (Additional note 3) the apparatus comprising a housing for the device; The sensor is mounted in the housing. Item 1. The device of item 1. (Additional note 4) transmitting the alert by broadcasting the alert for receipt by any of the ground transportation entities at or near the intersection; Item 1. The device of item 1. (Additional note 5) the machine learning model comprises an artificial intelligence model; Item 1. The device of item 1. (Additional note 6) the training data and the motion data include at least one of speed, position, heading, intent, attitude, or looking direction; Item 1. The device of item 1. (Additional note 7) the processor is configured to enable generation of the machine learning model at the device; Item 1. The device of item 1. (Additional note 8) The machine learning model is generated on the device. Item 1. The device of item 1. (Additional note 9) the instructions are executable by the processor to store the training data at the device. Item 1. The device of item 1. (Additional note 10) The intersection includes an intersection without a traffic light. Item 1. The device of item 1. (Additional note 11) The transportation network includes a road network. Item 1. The device of item 1. (Additional note 12) the ground transportation entities include road users that may be affected; Item 1. The device of item 1. (Additional note 13) the ground transportation entity includes a vehicle; Item 1. The device of item 1. (Additional note 14) The imminent hazardous situation includes a collision or near miss. Item 1. The device of item 1. (Additional note 15) the ground transportation entity includes a pedestrian crossing a roadway at a crosswalk; Item 1. The device of item 1. (Additional note 16) a separate communication device for communicating with the central server; Item 1. The device of item 1. (Additional note 17) the device of one of the ground transportation entities includes a mobile communications device; Item 1. The device of item 1. (Additional note 18) 1. An apparatus comprising equipment located in or on a ground transportation entity, The device comprises: an input that receives data from a sensor in or on the ground transportation entity that is oriented to monitor nearby features of a ground transportation network and other information related to the context in which the ground transportation entity traverses the ground transportation network; a wireless communication device that receives information about the context; a signal processor that applies signal processing to data from the sensors and other information related to the context; a processor; Storage and Equipped with The storage storing a machine learning model capable of predicting the behavior of an operator of the ground transportation entity and the intentions and movements of other ground transportation entities in the vicinity; applying the currently received data from the sensors and other information about the context to predict the behavior of the operator and the intentions and movements of other ground transportation entities in the vicinity; storage for instructions executable by the processor for Device. (Additional note 19) the instructions are executable by the processor to monitor users or passengers of the ground transportation entity; Item 19. The device according to item 18. (Additional note 20) the other information relating to the context includes emergency broadcasts, traffic and safety messages from roadside equipment, and messages relating to safety, location, and other operational information from other ground transportation entities; Item 19. The device according to item 18. (Additional note 21) The sensors include cameras, distance sensors, vibration sensors, microphones, seat sensors, hydrocarbon sensors, volatile organic compounds and other toxic substance sensors, and kinematic sensors, or combinations thereof; Item 19. The device according to item 18. (Additional note 22) instructions executable by the processor to filter the alerts received at the vehicle by applying the alerts to the machine learning model to predict which alerts are significant in relation to current location, environmental conditions, driver behavior, vehicle health and status, and kinematics; Item 19. The device according to item 18. (Additional note 23) the wireless communication device transmitting a warning about the dangerous situation to a sign or other infrastructure presentation device; Item 19. The device according to item 18. (Additional note 24) the warning includes an instruction or command capable of controlling a particular ground transportation entity; Item 19. The device according to item 18. (Additional note 25) storing a machine learning model capable of predicting a current behavior of a ground transportation entity at an intersection of a transportation network or near the intersection of the transportation network, the machine learning model being based on training data regarding previous actions and associated behaviors of ground transportation entities at the intersection or near the intersection; applying current motion data received from sensors relating to ground traffic entities at or near the intersection to the machine learning model to predict impending behavior of the ground traffic entities; inferring an imminent hazardous situation for one or more of the ground traffic entities at or near the intersection from the predicted imminent behavior; and causing a wireless communication device to transmit a warning regarding the hazardous situation to a device of one of the ground transportation entities; A method comprising: (Additional note 26) transmitting the alert by broadcasting the alert for receipt by any of the ground transportation entities at or near the intersection; The method described in appended item 25. (Additional note 27) the machine learning model comprises an artificial intelligence model; The method described in appended item 25. (Additional note 28) the training data and the motion data include at least one of speed, position, heading, intent, attitude, or looking direction; The method described in appended item 25. (Additional note 29) generating the machine learning model on a device. The method described in appended item 25. (Additional note 30) storing the training data in a device. The method described in appended item 25. (Additional note 31) receiving data from sensors in or on a ground transportation entity oriented to monitor nearby features of a ground transportation network and other information related to the context in which the ground transportation entity traverses the ground transportation network; receiving information about the context; storing a machine learning model capable of predicting the behavior of an operator of the ground transportation entity and the intentions and movements of other ground transportation entities in the vicinity; applying the currently received data from the sensors and other information about the context to predict the operator's behavior and the intentions and movements of other ground transportation entities in the vicinity; A method comprising: (Additional note 32) monitoring users or passengers of said ground transportation entities; The method described in appended item 31. (Additional note 33) the other information relating to the context includes emergency broadcasts, traffic and safety messages from roadside equipment, and messages relating to safety, location, and other operational information from other ground transportation entities; The method described in appended item 31. (Additional note 34) The sensors include cameras, distance sensors, vibration sensors, microphones, seat sensors, hydrocarbon sensors, volatile organic compounds and other toxic substances sensors, and kinematic sensors, or combinations thereof; The method described in appended item 31. (Additional note 35) filtering the alerts received at the vehicle by applying the alerts to a machine learning model to predict which alerts are significant in relation to current location, environmental conditions, driver behavior, vehicle health and status, and kinematics; The method described in appended item 31. (Additional note 36) including transmitting warnings about dangerous situations to signs or other infrastructure presentation devices; The method described in appended item 31. (Additional note 37) the warning includes an instruction or command capable of controlling a particular ground transportation entity; Item 32. The device according to item 31. (Additional note 38) In road vehicles moving within a ground transportation network, receiving messages from external sources regarding the location, operation, and status of other ground transportation entities; receiving data from on-board sensors regarding road and driving conditions and regarding the positions of static objects and moving ground traffic entities in the vicinity of said vehicle; receiving data relating to the quality of driving by a driver of said road vehicle; receiving basic safety messages from other ground transportation entities and personal safety messages from vulnerable road users; aggregating the received data and the message; applying the fused data and messages to an artificial intelligence model to predict the behaviour of a driver of the road vehicle or the behaviour of a vulnerable road user, or a collision risk to the road vehicle, or both the behaviour of a driver of the road vehicle or the behaviour of a vulnerable road user and a collision risk to the road vehicle; A method comprising: (Additional note 39) generating, at the road vehicle, a map of the static objects and the moving ground traffic entities in a vicinity of the road vehicle; The method described in appended item 38. (Additional note 40) alerting the driver of the road vehicle of a collision risk; The method described in appended item 38. (Additional note 41) determining the collision risk based on probabilities of predicted trajectories of other nearby moving ground transportation entities; The method described in appended item 38. (Additional note 42) filtering received basic safety messages and personal safety messages to reduce the number of alerts provided to the driver of the road vehicle. The method described in appended item 38. (Additional note 43) acquiring motion data for unconnected ground transportation entities moving in the transportation network; transmitting a virtual safety message to a connected ground transportation entity in the vicinity of the unconnected ground transportation entity, the virtual safety message incorporating information regarding the operational data for the unconnected ground transportation entity; A method comprising: (Additional note 44) the virtual safety message is in lieu of a safety message that would be transmitted by the unconnected ground transportation entity if the unconnected ground transportation entity were connected; The method described in appendix 43. (Additional note 45) the unconnected ground transportation entities include vehicles; the virtual safety message is a substitute for a primary safety message; The method described in appendix 43. (Additional note 46) the unconnected ground transportation entities include vulnerable road users; the virtual safety message is a substitute for a personal safety message; The method described in appendix 43. (Additional note 47) the operational data is acquired by infrastructure sensors; The method described in appendix 43. (Additional note 48) An apparatus comprising a device located at an intersection of a transportation network, The device comprises: an input for receiving data from a sensor oriented to monitor ground traffic entities at or near the intersection; a wireless communication device that transmits a warning to a device of one of the ground transportation entities regarding a dangerous condition at or near the intersection; a processor; Storage and Equipped with The storage storing a machine learning model capable of predicting a behavior of a ground traffic entity at or near the intersection at a current time, the machine learning model being based on training data regarding previous actions and associated behaviors of ground traffic entities at or near the intersection; applying current operational data received from the sensors regarding ground transportation entities at or near the intersection to the machine learning model to predict impending behavior of the ground transportation entities, including ground transportation entities whose participating devices are unable to receive warnings from the wireless communication devices; inferring an imminent hazardous situation for a ground transportation entity whose participating devices are able to receive a warning from the wireless communication device, wherein the imminent hazardous situation is a result of a predicted imminent behavior of the ground transportation entity that is unable to receive the warning; and transmitting the warning regarding the hazardous situation to the device of the ground transportation entity that can receive the warning from the wireless communication device; storage for instructions executable by the processor for Device. (Additional note 49) the equipment includes roadside equipment; Item 49. The device described in paragraph 48. (Additional note 50) the apparatus comprising a housing for the device; The sensor is mounted in the housing. Item 49. The device described in paragraph 48. (Additional note 51) transmitting the warning by broadcasting the warning for receipt by any of the ground transportation entities at or near the intersection that can receive the warning; Item 49. The device described in paragraph 48. (Additional note 52) the machine learning model comprises an artificial intelligence model; Item 49. The device described in paragraph 48. (Additional note 53) The intersection includes an intersection without a traffic light. Item 49. The device described in paragraph 48. (Additional note 54) The intersection includes a traffic light intersection. Item 49. The device described in paragraph 48. (Additional note 55) The transportation network includes a road network. Item 49. The device described in paragraph 48. (Additional note 56) the ground transportation entities include road users that may be affected; Item 49. The device described in paragraph 48. (Additional note 57) the ground transportation entity includes a vehicle; Item 49. The device described in paragraph 48. (Additional note 58) the imminent danger situation includes a collision; Item 49. The device described in paragraph 48. (Additional note 59) the ground transportation entity, the device of which is unable to receive the alert from the wireless communication device, comprises a vehicle; the ground transportation entities whose participating devices may receive the alert from the wireless communication device include a pedestrian crossing a road at a crosswalk; Item 49. The device described in paragraph 48. (Additional note 60) a separate communication device for communicating with the central server; Item 49. The device described in paragraph 48. (Additional note 61) the device of one of the ground transportation entities comprises a mobile communications device; Item 49. The device described in paragraph 48. (Additional note 62) Using electronic sensors located near intersections of a ground transportation network to monitor the intersections and approaches to the intersections, the electronic sensors generating operational data regarding ground transportation entities moving on the approaches or at the intersections, one or more of the ground transportation entities being unable to transmit safety messages to other ground transportation entities in the vicinity of the intersections; transmitting a virtual safety message to one or more of the ground transportation entities capable of receiving the message based on the operational data generated by the electronic sensors; Incorporating information about one or more of the ground transportation entities that are unable to transmit safety messages into the virtual safety messages, wherein the incorporated information in each of the virtual safety messages includes at least one of a location, a heading, a speed, and a predicted future trajectory of one of the ground transportation entities that are unable to transmit safety messages; A method comprising: (Additional note 63) the embedded information includes a subset of information that would be embedded in a basic safety message or a personal safety message generated by the ground transportation entity if the ground transportation entity were capable of transmitting the basic safety message or the personal safety message; The method described in appended paragraph 62. (Additional note 64) applying the generated operational data to a machine learning model operating on equipment located near the intersection to predict a trajectory of the ground transportation entity that is unable to transmit a safety message; The method described in appended paragraph 62. (Additional note 65) at least one of the ground transportation entities includes a motorized vehicle; The method described in appended paragraph 62. (Additional note 66) the machine learning model is provided by a remote server over the internet to a device located near the intersection; The method described in appended paragraph 62. (Additional note 67) The machine learning model is generated on a device located near the intersection. The method described in appended paragraph 62. (Additional note 68) training a machine learning model using operational data generated by the sensors located near the intersection; The method described in appended paragraph 62. (Additional note 69) transmitting operational data generated by the sensors located near the intersection to a server for use in training a machine learning model. The method described in appended paragraph 62. (Additional note 70) using electronic sensors located near a crosswalk across a roadway to monitor an area within or near the crosswalk, the electronic sensors generating and using motion data related to vulnerable roadway users within or near the crosswalk; applying the generated motion data to a machine learning model running on a device located near the crosswalk to predict when one of the vulnerable roadway users is about to enter the crosswalk; wirelessly transmitting a warning to at least one of a device associated with the vulnerable roadway user or a device associated with another ground transportation entity approaching the crosswalk on the roadway before the vulnerable roadway user enters the crosswalk; A method comprising: (Additional note 71) The potentially affected roadway users include pedestrians, animals, or cyclists; The method described in appended paragraph 70. (Additional note 72) the device associated with the potentially vulnerable roadway user includes a smartwatch or other wearable device, a smartphone, or another mobile device; The method described in appended paragraph 70. (Additional note 73) the other ground transportation entities include motorized vehicles; The method described in appended paragraph 70. (Additional note 74) the device associated with the other ground transportation entity includes a smartphone or another mobile device; The method described in appended paragraph 70. (Additional note 75) the machine learning model is provided to the device located near the crosswalk by a remote server over the internet; The method described in appended paragraph 70. (Additional note 76) the machine learning model is generated on the device located near the crosswalk; The method described in appended paragraph 70. (Additional note 77) training the machine learning model using motion data generated by the sensor located near the crosswalk. The method described in appended paragraph 70. (Additional note 78) transmitting motion data generated by the sensors located near the crosswalk to a server for use in training the machine learning model. The method described in appended paragraph 70. (Additional note 79) segmenting the motion data generated by the sensors located near the crosswalk based on corresponding zones in the vicinity of the crosswalk. The method described in appended paragraph 70. (Additional note 80) using said electronic sensors to generate motion-related data representative of a physical characteristic of said vulnerable road users; The method described in appended paragraph 70. (Additional note 81) deriving trajectory information about the vulnerable road users from the motion data generated by the sensors. The method described in appended paragraph 70. (Additional note 82) 1. An apparatus comprising equipment located at an at-grade intersection of a transportation network, the at-grade intersection includes a road intersection, a pedestrian crossing, and a railroad track; The device comprises: an input for receiving data from sensors oriented to monitor road vehicles and pedestrians at or near the grade crossing, and for receiving phase and timing data for signals on the road and signals on the railroad tracks; a wireless communication device that transmits a warning about a dangerous condition at or near the grade crossing to one of a ground transportation entity, a pedestrian, or a rail vehicle on the railroad track; a processor; Storage and Equipped with The storage storing a machine learning model capable of predicting a behavior of ground traffic entities at or near the grade intersection at a current time, the machine learning model being based on training data relating to previous actions and associated behaviors of road vehicles and pedestrians at or near the intersection; applying current motion data received from the sensors relating to road vehicles and pedestrians at or near the grade intersection to the machine learning model to predict impending behavior of the road vehicles and pedestrians; inferring an imminent danger situation for a rail vehicle on the railroad track at the intersection or a rail vehicle on the railroad track near the intersection from the predicted imminent behavior; causing the wireless communication device to transmit the warning regarding the dangerous situation to at least one of the road vehicle, the pedestrian, and the rail vehicle; the storage for instructions executable by the processor to Device. (Additional note 83) the warning is transmitted to an on-board device of the rail vehicle; Item 83. The device described in paragraph 82. (Additional note 84) the railroad tracks are on a separated railroad right-of-way; Item 83. The device described in paragraph 82. (Additional note 85) The railroad tracks are not located on a separate railroad right-of-way; Item 83. The device described in paragraph 82. (Additional note 86) the equipment includes roadside equipment; Item 83. The device described in paragraph 82. (Additional note 87) transmitting the warning by broadcasting the warning for reception by any of the ground transportation entities, the pedestrians, or the rail vehicles at or near the grade crossing; Item 83. The device described in appended item 82. (Additional note 88) The imminent hazardous situation includes a collision or near miss. Item 83. The device described in appended item 82. (Additional note 89) receiving data from infrastructure sensors representing the location and movement of an operating road vehicle or a walking pedestrian in a ground transportation network; receiving data in virtual basic safety messages and virtual personal safety messages relating to the status of the road vehicle and the pedestrian; applying the received data to a trained machine learning model to identify risky driving or walking behavior of one of the road vehicles or the pedestrians; reporting said unsafe driving or walking behavior to authorities; A method comprising: (Additional note 90) identifying said road vehicle based on plate number recognition; The method described in appended item 89. (Additional note 91) identifying the pedestrian based on biometric recognition; The method described in appended item 89. (Additional note 92) identifying the road vehicle or the pedestrian based on social networking; The method described in appended item 89. (Additional note 93) using electronic sensors located near intersections of a ground transportation system to monitor the intersections and approaches to the intersections, the electronic sensors generating operational data regarding ground transportation entities moving on the approaches or at the intersections; defining separate virtual zones at the intersection and the approaches to the intersection; segmenting the generated motion data according to the corresponding virtual zone to which the motion data relates; applying the generated operational data for each respective segment to a machine learning model operating on equipment located near the intersection to predict an imminent hazardous situation at the intersection or on one of the approach roads involving one or more of the ground transportation entities; wirelessly transmitting a warning to a device associated with at least one of the involved ground transportation entities before the imminent hazardous situation becomes an actual hazardous situation; A method comprising: (Additional note 94) the device associated with each of the ground transportation entities includes a wearable device, a smartphone, or another mobile device; The method described in appendix 93. (Additional note 95) at least one of the ground transportation entities includes a motorized vehicle; The method described in appendix 93. (Additional note 96) the machine learning model is provided to the device located near the intersection by a remote server over the internet; The method described in appendix 93. (Additional note 97) the machine learning model is generated on the device located near the intersection; The method described in appendix 93. (Additional note 98) training the machine learning model using operational data generated by the sensors located near the intersection. The method described in appendix 93. (Additional note 99) transmitting the motion data generated by the sensors located near the intersection to a server for use in training the machine learning model. The method described in appendix 93. (Additional note 100) using the electronic sensor to monitor an area in or near a crosswalk across one of the approaches to the intersection. The method described in appendix 93. (Additional note 101) using said electronic sensors to generate motion-related data representative of physical characteristics of vulnerable road users in the vicinity of a crosswalk; The method described in appendix 93. (Additional note 102) deriving trajectory information about vulnerable road users from the motion data generated by the sensors; The method described in appendix 93. (Additional note 103) a machine learning model exists for each of the approaches to the intersection; The method described in appendix 93. (Additional note 104) determining whether to send the alert further based on operational data generated by a sensor associated with another nearby intersection. The method described in appendix 93. (Additional note 105) determining whether to send the warning further based on information received from a ground traffic entity operating on the approach road or a ground traffic entity operating at the intersection. The method described in appendix 93. (Additional note 106) The intersection is equipped with a traffic light, Information regarding the state of the signal is received; The method described in appendix 93. (Additional note 107) the intersection is not traffic lighted but is controlled by one or more signs; The method described in appendix 93. (Additional note 108) The defined virtual zone includes one or more of the access routes controlled by the signs; The method described in appended item 107. (Additional note 109) The sign includes a stop sign or a yield sign; The method described in appended item 107. (Supplementary Note 110) one of the ground transportation entities includes a rail vehicle; The method described in appended item 107. (Additional note 111) An apparatus comprising a device located at an intersection of a transportation network, The device comprises: an input receiving data from sensors oriented to monitor ground transportation entities at or near the intersection, the data from each of the sensors representing at least one position or operating parameter of at least one of the ground transportation entities, the data from each of the sensors being represented in a native format, and the data received from at least two of the sensors not matching with respect to the position or operating parameter, or with respect to the native format, or with respect to both the position or operating parameter and the native format; a processor; Storage and Equipped with The storage converting the data from each of the sensors into data having a common format that is independent of the native format of the data of the sensors; incorporating the data having the common format into a global integrated representation of the ground traffic entities monitored at or near the intersection, the global integrated representation including the position, speed, and heading of each of the ground traffic entities; identifying a relationship between positions and movements of two of the ground transportation entities using the global integrated representation; and predicting a dangerous situation involving the two ground transportation entities; sending a message to at least one of the two ground transportation entities alerting the at least one of the two ground transportation entities about the hazardous situation; the storage for instructions executable by the processor to Device. (Additional Note 112) the sensors include at least two of a radar, a lidar, and a camera; Item 112. The device described in item 111. (Additional note 113) the data received from one of the sensors includes image data of a field of view at successive instants; Item 112. The device described in item 111. (Additional note 114) the data received from one of the sensors includes reflection points in 3D space; Item 112. The device described in item 111. (Additional note 115) the data received from one of the sensors includes distance and velocity from the sensor; Item 112. The device described in item 111. (Additional note 116) the global integrated representation represents the positions of the ground transportation entities in a common reference frame; Item 112. The device described in item 111. (Additional note 117) the device comprises at least two of the sensors; the data is received from the at least two sensors; two said sensors mounted at fixed positions at or near said intersection and having at least partially non-overlapping fields of view; Item 112. The device described in item 111. (Additional note 118) one of the sensors includes a radar; converting the data includes determining a location of a ground traffic entity from a known location of the radar and a distance of the ground traffic entity from the radar; Item 118. The device described in item 117. (Additional note 119) one of the sensors includes a camera; converting the data includes determining a position of a ground traffic entity from a known position, a direction of view, and a tilt of the camera, and a position of the ground traffic entity within an image frame of the camera; Item 118. The device described in item 117. (Supplementary Note 120) receiving data from sensors oriented to monitor ground transportation entities at or near an intersection of a ground transportation network, the data from each of the sensors representing at least one position or operational parameter of at least one of the ground transportation entities, the data from each of the sensors being represented in a native format, and the data received from at least two of the sensors not matching with respect to the position or operational parameter, with respect to the native format, or with respect to both the position or operational parameter and the native format; converting the data from each of the sensors into data having a common format that is independent of the native format of the data of the sensors; incorporating the data having the common format into a global integrated representation of the ground traffic entities monitored at or near the intersection, the global integrated representation including the position, speed, and heading of each of the ground traffic entities; identifying a relationship between positions and movements of two of the ground transportation entities using the global integrated representation; and predicting a dangerous situation involving the two ground transportation entities; sending a message to at least one of the two ground transportation entities alerting the at least one of the two ground transportation entities about the hazardous situation; A method comprising: (Additional note 121) the sensors include at least two of a radar, a lidar, and a camera; The method described in appendix 120. (Additional note 122) the data received from one of the sensors includes image data of a field of view at successive instants; The method described in appended item 120. (Additional note 123) the data received from one of the sensors includes reflection points in 3D space; The method described in appendix 120. (Additional note 124) the data received from one of the sensors includes distance and velocity from the sensor; The method described in appendix 120. (Additional note 125) the global integrated representation represents the positions of the ground transportation entities in a common reference frame; The method described in appendix 120. (Additional note 126) the data is received from at least two of the sensors mounted at fixed locations at or near the intersection and having at least partially non-overlapping fields of view; The method described in appendix 120. (Additional note 127) one of the sensors includes a radar; the method including transforming the data, including determining a location of a ground traffic entity from a known location of the radar and a distance from the radar to the ground traffic entity; The method described in appended item 126. (Additional note 128) one of the sensors comprises a camera; the method including transforming the data, including determining the position of a ground traffic entity from a known location, the direction of view, and the tilt of the camera, and the position of the ground traffic entity within an image frame of the camera; The method described in appended item 126. (Aspect 1) using electronic sensors located near a crosswalk across a roadway to monitor an area within and near the crosswalk, the electronic sensors generating motion data regarding vulnerable roadway users within or near the crosswalk, the motion data including the location and direction of the vulnerable roadway users; applying the generated motion data to a machine learning model operating on a device located near the crosswalk to make a prediction of one of the vulnerable roadway users' intention to enter the road within or near the crosswalk, the prediction being made before the vulnerable roadway user enters the road within or near the crosswalk, the machine learning model being trained using motion data generated in the vicinity of the crosswalk, the motion data indicative of intentions or behaviors of vulnerable roadway users who were previously in or near the crosswalk, the motion data including position, speed, acceleration, and orientation; Before the vulnerable roadway user enters the roadway within or near the crosswalk, transmitting a warning to at least one of a device associated with the vulnerable roadway user, a device associated with another ground transportation entity approaching the crosswalk on the roadway, and a road sign configured to alert the vulnerable roadway user or driver; A method comprising: (Aspect 2) The potentially affected roadway users include pedestrians, animals, or cyclists; 2. The method of embodiment 1. (Aspect 3) the device associated with the potentially vulnerable roadway user includes a smartwatch or other wearable device, a smartphone, or another mobile device; 2. The method of embodiment 1. (Aspect 4) the other ground transportation entities include motorized vehicles; 2. The method of embodiment 1. (Aspect 5) the device associated with the other ground transportation entity includes a smartphone or another mobile device; 2. The method of embodiment 1. (Aspect 6) the machine learning model is provided to the device located near the crosswalk by a remote server over the internet; 2. The method of embodiment 1. (Aspect 7) the machine learning model is generated on the device located near the crosswalk; 2. The method of embodiment 1. (Aspect 8) training the machine learning model using motion data generated by the sensor located near the crosswalk. 2. The method of embodiment 1. (Aspect 9) transmitting motion data generated by the sensors located near the crosswalk to a server for use in training the machine learning model. 2. The method of embodiment 1. (Aspect 10) segmenting the motion data generated by the sensors located near the crosswalk based on corresponding zones in the vicinity of the crosswalk. 2. The method of embodiment 1. (Aspect 11) using the electronic sensors to generate motion-related data representative of a physical characteristic of the vulnerable roadway user; 2. The method of embodiment 1. (Aspect 12) deriving trajectory information about the vulnerable roadway users from the motion data generated by the sensors. 2. The method of embodiment 1. (Aspect 13) 1. An apparatus comprising equipment located at an at-grade intersection of a transportation network, the at-grade intersection includes a road intersection, a pedestrian crossing, and a railroad track; The device comprises: an input for receiving data from sensors oriented to monitor road vehicles and pedestrians at or near the grade crossing, and for receiving phase and timing data for signals on the road and signals on the railroad tracks; a wireless communication device that transmits a warning about a dangerous condition at or near the grade crossing to one of a ground transportation entity, a pedestrian, or a rail vehicle on the railroad track; a processor; Storage and Equipped with The storage storing a machine learning model capable of predicting behavior of ground traffic entities at or near the grade intersection at a current time, the machine learning model being based on training data regarding previous actions and associated behaviors of road vehicles and pedestrians at or near the intersection, and previous phase and timing data for signals on the road and signals on the railroad tracks; applying current motion data received from the sensors relating to road vehicles and pedestrians at or near the grade intersection and current phase and timing data for signals on the road and on the railway line to the machine learning model to predict impending behavior of the road vehicles and pedestrians; inferring an imminent danger situation for a rail vehicle on the railroad track at the intersection or a rail vehicle on the railroad track near the intersection from the predicted imminent behavior; causing the wireless communication device to transmit the warning regarding the dangerous situation to at least one of the road vehicle, the pedestrian, and the rail vehicle; the storage for instructions executable by the processor to Device. (Aspect 14) the warning is transmitted to an on-board device of the rail vehicle; 14. The apparatus of embodiment 13. (Aspect 15) the railroad tracks are on a separated railroad right-of-way; 14. The apparatus of embodiment 13. (Aspect 16) The railroad tracks are not located on a separate railroad right-of-way; 14. The apparatus of embodiment 13. (Aspect 17) the equipment includes roadside equipment; 14. The apparatus of embodiment 13. (Aspect 18) transmitting the warning by broadcasting the warning for reception by any of the ground transportation entities, the pedestrians, or the rail vehicles at or near the grade crossing; 14. The apparatus of embodiment 13. (Aspect 19) The imminent hazardous situation includes a collision or near miss. 14. The apparatus of embodiment 13. (Aspect 20) receiving data from infrastructure sensors representing the location and movement of an operating road vehicle or a walking pedestrian in a ground transportation network; receiving data in a basic safety message or a virtual basic safety message, a personal safety message, and a virtual personal safety message relating to the status of the road vehicle and the pedestrian, wherein the basic safety message, the virtual basic safety message, the personal safety message, and the virtual personal safety message include position, heading, and speed information relating to the road vehicle or the pedestrian, the basic safety message and the personal safety message being received from the road vehicle or the pedestrian, and the virtual basic safety message and the virtual personal safety message including position, heading, and speed information relating to the road vehicle and the pedestrian reconstructed from the data received from the infrastructure sensors; applying the data received from the infrastructure sensors and data from the basic safety message, the virtual basic safety message, the personal safety message, and the virtual personal safety message to a trained machine learning model to identify risky driving or walking behavior of one of the road vehicles or the pedestrians; automatically reporting said unsafe driving or walking behavior to authorities; A method comprising: (Aspect 21) identifying said road vehicle based on plate number recognition; 21. The method according to embodiment 20. (Aspect 22) identifying the pedestrian based on biometric recognition; 21. The method according to embodiment 20. (Aspect 23) identifying the road vehicle or the pedestrian based on social networking; 21. The method according to embodiment 20.

Claims

1. 1. An apparatus comprising equipment located at an at-grade intersection of a transportation network, the at-grade intersection includes a road intersection, a pedestrian crossing, and a railroad track; The device comprises: an input for receiving data from sensors oriented to monitor road vehicles and pedestrians at or near the grade crossing, and for receiving phase and timing data for signals on the road and signals on the railroad tracks; a wireless communication device that transmits a warning about a dangerous condition at or near the grade crossing to one of a ground transportation entity, a pedestrian, or a rail vehicle on the railroad track; a processor; Storage and Equipped with The storage storing a machine learning model capable of predicting behavior of ground traffic entities at or near the grade intersection at a current time, the machine learning model being based on training data regarding previous actions and associated behaviors of road vehicles and pedestrians at or near the intersection, and previous phase and timing data for signals on the road and signals on the railroad tracks; applying current motion data received from the sensors relating to road vehicles and pedestrians at or near the grade intersection and current phase and timing data for signals on the road and on the railway line to the machine learning model to predict impending behavior of the road vehicles and pedestrians; inferring an imminent danger situation for a rail vehicle on the railroad track at the intersection or a rail vehicle on the railroad track near the intersection from the predicted imminent behavior; causing the wireless communication device to transmit the warning regarding the dangerous situation to at least one of the road vehicle, the pedestrian, and the rail vehicle; the storage for instructions executable by the processor to Device.

2. the warning is transmitted to an on-board device of the rail vehicle; 10. The apparatus of claim 1.

3. the railroad tracks are on a separated railroad right-of-way; 10. The apparatus of claim 1.

4. The railroad tracks are not located on a separate railroad right-of-way; 10. The apparatus of claim 1.

5. the equipment includes roadside equipment; 10. The apparatus of claim 1.

6. transmitting the warning by broadcasting the warning for reception by any of the ground transportation entities, the pedestrians, or the rail vehicles at or near the grade crossing; 10. The apparatus of claim 1.

7. The imminent hazardous situation includes a collision or near miss.

10. The apparatus of claim 1.

Images