High-altitude vehicle-mounted machine

Abstract

Although other possibilities exist, the apparatus for use onboard a first ground transportation entity includes: (a) a receiver for information generated by a sensor of the first ground transportation entity's surrounding environment; (b) a processor; and (c) a memory storing instructions executable by the processor for generating and transmitting safety message information to a second ground transportation entity based on the information generated by the sensor.

Description

[Technical Field]

[0001] This application claims the benefit of the filing date of U.S. Provisional Patent Application No. 62 / 893,616, filed August 29, 2019, the entire contents of which are incorporated herein by reference.

[0002] US Pat. No. 10,235,882 is incorporated herein by reference.

[0003] This description relates to advanced vehicle equipment. [Background technology]

[0004] Collision avoidance systems are on the rise. King et al. (US Patent Application Publication No. 2007 / 0276600(A1), 2007) described, for example, placing sensors in front of an intersection and applying physics-based decision rules to predict whether two vehicles are about to collide at the intersection based on their heading and speed.

[0005] In Aoude et al. (U.S. Patent No. 9,129,519(B2), 2015, the entire contents of which are incorporated herein by reference), driver behavior is monitored and modeled to enable prediction and prevention of violations in traffic situations at intersections. Summary of the Invention [Problem to be solved by the invention]

[0006] Collision avoidance is the primary defense against injury, death, and property loss in ground transportation. Providing early warning of dangerous situations assists in collision avoidance. [Means for solving the problem]

[0007] In general, in one aspect, an apparatus for use onboard a first ground transportation entity includes a receiver (a) for information generated by a sensor of the first ground transportation entity's surrounding environment, a processor (b), and a memory (c) storing instructions executable by the processor for generating and transmitting safety message information to a second ground transportation entity based on the information generated by the sensor.

[0008] Implementations may include one or a combination of two or more of the following features: instructions executable by a processor to generate a prediction for use in generating safety message information; the prediction is generated by a predictive model; the predictive model is configured to predict a hazardous situation involving a first ground traffic entity, a second ground traffic entity, or another ground traffic entity; the hazardous situation involves the second ground traffic entity crossing a lane of a road; the second ground traffic entity includes a vehicle, and the hazardous situation includes a vehicle skidding across lanes; the second ground traffic entity includes a pedestrian or other vulnerable road user crossing the road; the vulnerable road user crosses the road at an intersection; the vulnerable road user crosses the road other than at an intersection; the predicted hazardous situation includes a predicted collision between a third ground traffic entity and the second ground traffic entity; the first ground traffic entity includes a vehicle, and the second ground traffic entity includes a pedestrian or other vulnerable road user. A third ground traffic entity follows the first ground traffic entity and the first ground traffic entity blocks the view of the third ground traffic entity. The third ground traffic entity is in a lane adjacent to the lane in which the first ground traffic entity is traveling. Instructions are executable by a processor to identify operating parameters of the third ground traffic entity. The second ground traffic entity has only an obstructed view of the third ground traffic entity. The second ground traffic entity comprises a pedestrian or other vulnerable road user. The safety message information transmitted by the processor comprises a basic safety message. The safety message information transmitted by the processor comprises a virtual basic safety message. The safety message information transmitted by the processor comprises a personal safety message. The safety message information transmitted by the processor comprises a virtual personal safety message. The safety message information transmitted by the processor comprises a virtual basic safety message transmitted on behalf of the third ground traffic entity.The third ground transportation entity includes an unconnected ground transportation entity. The safety message information transmitted by the processor includes a virtual personal safety message transmitted on behalf of the third ground transportation entity. The equipment includes a receiver (d) for information wirelessly transmitted from a source external to the first ground transportation entity. The apparatus of claim 1, including a first ground transportation entity. The safety message information includes a virtual intersection collision avoidance message (VICA). The safety message information includes an intersection collision avoidance message (ICA). The safety message information includes a virtual combined safety message (VCSM). The safety message information includes a combined safety message (CSM).

[0009] Generally, in one aspect, an apparatus for use onboard a first ground transportation entity includes a receiver (a) for first position fix information transmitted from a source external to the first ground transportation entity, a receiver (b) for information representing position or operational parameters of the first ground transportation entity, a processor (c), and a memory (d) storing instructions executable by the processor for generating updated position fix information based on the first position fix information and the information representing the operational parameters, and for transmitting a position fix message to another ground transportation entity based on the updated position fix information.

[0010] Implementations may include one or a combination of two or more of the following features: The position fix information transmitted from a source external to the first ground traffic entity includes a position fix message; The position fix information transmitted from a source external to the first ground traffic entity includes a radio technical commission for maritime (RTCM) correction message; The position fix information includes a GNSS position fix; The position or operating parameters include a current position of the first ground traffic entity; The source external to the first ground traffic entity includes an RSE or an external service configured to transmit the RTCM correction message over the Internet; The instructions are executable by a processor to ascertain a confidence level in the updated position fix information.

[0011] In general, in one aspect, information generated by a sensor onboard a first ground transportation entity regarding an environment surrounding the first ground transportation entity is received, and safety message information is generated based on the information generated by the sensor and transmitted to a second ground transportation entity.

[0012] Implementations may include one or a combination of two or more of the following features: A prediction is generated for use in generating safety message information. The prediction is generated by a predictive model. The predictive model is configured to predict a hazardous situation involving a first ground traffic entity, a second ground traffic entity, or another ground traffic entity. The hazardous situation involves the second ground traffic entity crossing a lane of a road. The second ground traffic entity includes a vehicle, and the hazardous situation includes a vehicle skidding across lanes. The second ground traffic entity includes a pedestrian or other vulnerable road user crossing a road. The vulnerable road user crosses a road at an intersection. The vulnerable road user crosses a road other than at an intersection. The hazardous situation includes a collision between a third ground traffic entity and the second ground traffic entity. The first ground traffic entity includes a vehicle, and the second ground traffic entity includes a pedestrian or other vulnerable road user. A third ground traffic entity follows the first ground traffic entity, and the first ground traffic entity blocks the view of the third ground traffic entity. The third ground traffic entity is in a lane adjacent to the lane in which the first ground traffic entity is traveling. Operational parameters of the third ground traffic entity are determined. The second ground traffic entity has a view of the third ground traffic only in an obstructed state. The second ground traffic entity comprises a pedestrian or other vulnerable road user. The safety message information comprises a basic safety message. The safety message information comprises a virtual basic safety message. The safety message information comprises a personal safety message. The safety message information comprises a virtual personal safety message. The safety message information comprises a virtual basic safety message transmitted on behalf of the third ground traffic entity. The safety message information transmitted by the processor comprises a virtual personal safety message transmitted on behalf of the third ground traffic. The third ground traffic entity comprises an unconnected ground traffic entity.Wirelessly transmitted information is received from a source external to the first ground transportation entity. The safety message information includes a virtual intersection collision avoidance message (VICA). The safety message information includes a virtual intersection collision avoidance message (ICA). The safety message information includes a virtual combined safety message (VCSM). The safety message information includes a combined safety message (CSM).

[0013] In general, in one aspect, first position correction information transmitted from a source external to a first ground transportation entity is received. Information representative of an operating parameter of the first ground transportation entity is received. Updated position correction information is generated based on the first position correction information and the information representative of the operating parameter. A position correction message is transmitted, and the position correction message is transmitted to another ground transportation entity based on the updated position correction information.

[0014] Implementations may include one or a combination of two or more of the following features: The position fix information transmitted from a source external to the first ground traffic entity includes a position fix message; The position fix information transmitted from a source external to the first ground traffic entity includes a Radio Technical Commission for Maritime (RTCM) fix message; The position fix information includes a GNSS position fix; The operating parameters include a current position of the first ground traffic entity; The source external to the first ground traffic entity includes an RSE or an external service configured to transmit the position fix message over the Internet; A confidence level in the updated position fix information is confirmed.

[0015] These and other aspects, features, and implementations may be expressed as methods, apparatus, systems, components, program products, ways of doing business, means or steps for performing a function, and other techniques.

[0016] These and other aspects, features, and embodiments will become apparent from the following description, including the claims. [Brief explanation of the drawings]

[0017] [Figure 1] FIG. 1 is a block diagram. [Figure 2] FIG. 2 is a block diagram. [Figure 3] FIG. 3 is a block diagram. [Figure 4] Figure 4 is a schematic diagram of the road network from above. [Figure 5] Figure 5 is a schematic diagram of the road network from above. [Figure 6] FIG. 6 is an annotated perspective view of an intersection. [Figure 7] FIG. 7 is an annotated perspective view of an intersection. [Figure 8] Figure 8 is a schematic diagram of the road network from above. [Figure 9] Figure 9 is a schematic diagram of the road network from above. [Figure 10] Figure 10 is a schematic diagram of the road network from above. [Figure 11] Figure 11 is a schematic diagram of the road network from above. [Figure 12] FIG. 12 is a schematic side view of a road network. [Figure 13] Figure 13 is a schematic diagram of the road network from above. [Figure 14] Figure 14 is a schematic diagram of the road network from above. [Figure 15] FIG. 15 is a block diagram. [Figure 16] FIG. 16 is a schematic perspective view of a road network. [Figure 17] Figure 17 is a schematic diagram of the road network from above. [Figure 18] Figure 18 is a schematic diagram of the road network from above. [Figure 19] Figure 19 is a schematic diagram of the road network from above. [Figure 20] Figure 20 is a schematic diagram of the road network from above. [Figure 21]Figure 21 is a schematic diagram of the road network from above. [Figure 22] Figure 22 is a schematic diagram of the road network from above. [Figure 23] Figure 23 is a schematic diagram of the road network from above. DETAILED DESCRIPTION OF THE INVENTION

[0018] With advances in sensor technology and computers, it has become feasible to predict dangerous situations (and provide early warning of dangerous situations) and thereby prevent collisions and near misses between ground transportation entities in ground transportation operations (i.e., enable collision avoidance).

[0019] The term "ground transportation" is used broadly herein to encompass any mode or medium of movement from place to place involving contact with land or water, e.g., on the surface of the Earth, such as walking or running (or performing other pedestrian behavior), non-motorized vehicles, motorized vehicles (autonomous, semi-autonomous, and non-autonomous), and rail vehicles.

[0020] This specification uses the term "ground transportation entity" (or sometimes simply "entity") broadly to encompass, for example, a person or individual motorized or non-motorized vehicle engaged in a mode of ground transportation, such as, among others, a pedestrian, bicycle rider, boat, car, truck, tram, streetcar, or train. In some cases, this specification uses the term "vehicle" or "road user" as a shorthand reference to a ground transportation entity.

[0021] The term "hazardous situation" is used broadly herein to encompass any event, occurrence, sequence, context, or other circumstance that may result in, or may be mitigated or avoided, for example, imminent property damage or personal injury or death. The term "hazard" is sometimes used interchangeably herein with "hazardous situation." The phrases "violation" or "violating" are sometimes used herein to refer to behavior of an entity that results in, may lead to, or will lead to a dangerous situation.

[0022] In some implementations of the technology discussed herein, the ground transportation network is used by a mixture of ground transportation entities that do not have or do not use transportation connectivity, and ground transportation entities that do have and use transportation connectivity.

[0023] The term "connectivity" is used broadly herein to encompass any capability of a ground transportation entity, for example, to (a) perceive and act on knowledge of its surroundings, other ground transportation entities in its vicinity, and traffic conditions associated with it, (b) broadcast or otherwise transmit data related to its status, or (c) both (a) and (b). The transmitted data may include its position, heading, speed, or the internal state of its components related to the traffic situation. In some examples, the ground transportation entity's awareness is based on wirelessly received data regarding other ground transportation entities or traffic conditions related to the ground transportation entity's operation. The received data may originate from other ground transportation entities, infrastructure devices, or both. Typically, connectivity involves transmitting or receiving data in real time, or substantially in real time, or as one or more of the ground transportation entities act on the data in the traffic situation.

[0024] In this specification, the term "traffic situation" is used broadly to encompass any situation in which two or more ground transportation entities operate in proximity to one another and in which the operation or status of each of the entities may affect or be related to the operation or status of the others.

[0025] As used herein, a ground transportation entity that does not have or does not use connectivity or connectivity aspects is referred to as a "disconnected ground transportation entity" or simply an "disconnected entity." As used herein, a ground transportation entity that includes and uses connectivity or connectivity aspects is referred to as a "connected ground transportation entity" or simply a "connected entity."

[0026] The term "associated entity" is sometimes used herein to refer to a ground transportation entity that broadcasts data to its surroundings, including, for example, location, heading, speed, or the status of onboard safety systems (e.g., brakes, lights, and wipers).

[0027] In some cases, the term "uncoordinated entity" is used herein to refer to a ground transportation entity that does not broadcast one or more types of data, such as its position, speed, heading, or status, to its surrounding environment.

[0028] In some cases, the term "neighborhood" of a ground transportation entity is used broadly herein to encompass, for example, the area in which a broadcast by the entity can be received by other ground transportation entities or infrastructure devices. In some instances, neighborhood varies with the entity's location and the number and characteristics of obstacles around the entity. An entity traveling on open roads in the desert has a very large neighborhood because there are no obstacles that would prevent a broadcast signal from the entity from reaching long distances. Conversely, neighborhood in an urban canyon is reduced by buildings around the entity. Furthermore, there may be electromagnetic noise sources that degrade the quality of the broadcast signal, thereby reducing the reception distance (neighborhood).

[0029] 14, the neighborhood of entity 7001 traveling along road 7005 may be represented by concentric circles, with outermost circle 7002 representing the outermost extent of the neighborhood. Any other entity located within circle 7002 is in the neighborhood of entity 7001. Any other entity located outside circle 7002 is outside the neighborhood of entity 7001 and cannot receive broadcasts by entity 7001. Entity 7001 is invisible to all entities and infrastructure devices outside its neighborhood.

[0030] Typically, coordinated entities continuously broadcast their status data. Connected entities in the vicinity of the broadcasting entity can receive these broadcasts, process the received data, and act based on the received data. For example, if a vulnerable road user is carrying a wearable device that can receive a broadcast from an entity such as an approaching truck, the wearable device can process the received data and inform the vulnerable user when it is safe to cross the road. This action occurs without regard to the location of the coordinated entities or the vulnerable user relative to the "smart" intersection, as long as the user's device can receive the broadcast, i.e., is within the vicinity of the coordinated entities.

[0031] The terms "vulnerable road user" or "vulnerable road user" are used broadly herein to encompass any user of a roadway or other feature of a road network who is not using, for example, a motorized vehicle. Vulnerable road users are generally not protected against injury or death or property damage if they are struck by a motorized vehicle. In some examples, a vulnerable road user may be a person walking, running, bicycling, or performing any type of activity that exposes that person to the risk of direct physical contact by a vehicle or other ground transportation entity in the event of a collision.

[0032] In some embodiments, the collision avoidance techniques and systems described herein (sometimes referred to herein simply as "systems") use sensors mounted on infrastructure equipment to monitor, track, detect, and predict the actions (e.g., speed, direction, and location), behavior (e.g., high speeds), and intentions (e.g., violating stop signs) of ground transportation entities and drivers and their operators. The information provided by the sensors ("sensor data") enables the systems to predict dangerous situations and provide early warnings to entities to increase the chances of collision avoidance.

[0033] The term "collision avoidance" is used broadly herein to encompass any situation in which a collision or near miss between two or more ground transportation entities, or between a ground transportation entity and another object in the surrounding environment, that may result from, for example, a dangerous situation, is prevented or the opportunity for such an interaction is reduced.

[0034] The term "early warning" is used broadly herein to encompass any notification, alarm, instruction, command, broadcast, transmission, or other transmission or receipt of information that, for example, identifies, suggests, or in any manner indicates a dangerous situation and is useful for collision avoidance.

[0035] Road intersections are prime locations where dangerous situations can occur. The technology described herein can equip intersections with infrastructure devices including sensors, computing hardware, and intelligence to enable simultaneous monitoring, detection, and prediction of dangerous situations. Data from these sensors is normalized to a single reference coordinate system and then processed. Artificial intelligence models of traffic flow along different approaches to the intersection are constructed. These models, for example, assist entities that are likely to violate traffic rules. The models are configured to detect dangerous situations before an actual violation and can therefore be considered predictive. Based on the prediction of a dangerous situation, an alert is sent from the infrastructure device at the intersection to all connected entities in the vicinity of the intersection. Each entity that receives the alert processes the data in the alert and performs alert filtering. Alert filtering is the process of discarding or ignoring alerts that are not useful to the entity. If the alert is considered useful (i.e., not ignored as a result of filtering), such as an alert of an impending collision, the entity can automatically react to the alert (e.g., by applying the brakes), or a notification can be presented to the driver, or both.

[0036] The system may be used on, but is not limited to, roadways, waterways, and railroads. These and other similar transportation contexts are sometimes referred to herein as "surface transportation networks."

[0037] Although the present specification often discusses the system in the context of intersections, it may also be applied in other contexts.

[0038] The term "intersection" is used broadly herein to encompass any real arrangement of roads, rails, bodies of water, or other paths of travel where, for example, two or more ground transportation entities traveling along a route in a ground transportation network may occupy the same location at some time and location and thus collide.

[0039] Ground transportation entities using the ground transportation network move at various speeds and may arrive at a given intersection at different speeds and times. If the entity's speed and distance from the intersection are known, dividing the distance by the speed (both described in the same unit system) gives the arrival time at the intersection. However, because the intended speed changes due to, for example, traffic conditions, speed limits on the route, traffic signals, and other factors, the expected arrival time at the intersection changes continuously. This dynamic change in the expected arrival time makes it impossible to predict the actual arrival time with 100% confidence.

[0040] Considering factors that influence an entity's motion requires applying multiple relationships between the entity's velocity and various influencing factors. The absolute state of an entity's motion can be observed by sensors that track the entity from the entity or from an external location. Data captured by these sensors can be used to model patterns of the entity's motion, behavior, and intent. Machine learning can be used to generate complex models from vast amounts of data. Patterns that cannot be directly modeled using the entity's kinematics can be captured using machine learning. A trained model can predict whether the entity is about to move or stop at a particular point by using the entity's tracking data from sensors that track the entity.

[0041] In other words, in addition to detecting information about ground transportation entities directly from sensor data, the system uses artificial intelligence and machine learning to process vast amounts of sensor data to learn patterns of ground transportation entity behavior, behavior, and intentions, for example, at intersections of the ground transportation network, on approaches to such intersections, and at crosswalks of the ground transportation network. Based on the direct use of current sensor data and the results of applying artificial intelligence and machine learning to the current sensor data, the system generates early warnings, such as alerts of dangerous situations, which in turn assist in collision avoidance. In connection with early warnings in the form of instructions or commands, the commands or instructions may be targeted to specific autonomous or human-driven entities to directly control the vehicle. For example, the instructions or commands may slow down or stop an entity driven by a malicious person that is determined to be running a red light with the intent of injuring someone.

[0042] The system can be tailored to make predictions for that particular intersection and send alerts to entities in the vicinity of the device broadcasting the alert. To this end, the system uses sensors to derive data about dangerous entities and passes current readings from the sensors through a trained model. The model's output can then predict a dangerous situation and broadcast a corresponding alert. The alert received by connected entities in the vicinity contains information about the dangerous entity, so the receiving entity can analyze the information to assess the threat posed to it by the dangerous entity. If a threat exists, the receiving entity can take action on its own (e.g., slow down) or notify the driver of the receiving entity using a human-machine interface based on visual, audio, tactile, or any kind of sensory stimulation. The autonomous entity can take action on its own to avoid the dangerous situation.

[0043] The alert may also be transmitted directly over a cellular or other network to a mobile phone or other device equipped to receive the alert and carried by the pedestrian. The system identifies potentially dangerous entities at an intersection and broadcasts (or transmits directly) an alert to the pedestrian's personal device that includes a communication unit. The alert may, for example, prevent the pedestrian from entering the crosswalk, thus avoiding a potential accident.

[0044] The system may further track pedestrians and broadcast information related to their status (location, speed, and other parameters) to other entities so that they may take action to avoid dangerous situations.

[0045] As shown in FIG. 1, the system includes at least the following types of components:

[0046] 1. Roadside Equipment (RSE) 10 includes or uses sensors 12 to monitor, track, detect, and predict the behavior (e.g., speed, direction, and location), behavior (e.g., high speeds), and intention (e.g., violating stop signs) of ground transportation entities 14. The RSE may further include or use data processing units 11 and data storage 18. Ground transportation entities exhibit a wide range of behaviors depending on the infrastructure of the ground transportation network, as well as the state of the entity itself, the driver, and other ground transportation entities. To capture the entity's behavior, the RSE collects information from sensors, other RSEs, OBEs, OPEs, local or central servers, and other data processing units. The RSE also stores data received by it and may store data processed during some or all of the steps in the pipeline.

[0047] The RSE may store data on a local storage device or remote storage. The collected data is processed in real time using predefined logic or logic based on dynamically collected data, which means the RSE may automatically update its own logic. The data may be processed in one processing unit or a cluster of processing units to obtain faster results. The data may be processed in a local or remote processing unit or a local or remote cluster of processing units. The RSE may use simple logic or advanced models trained on the collected data. The models may be trained locally or remotely.

[0048] The RSE may preprocess data before using the trained model to filter outliers. Outliers may appear due to noise, reflections in the sensor, or some other artifact. The resulting outliers may result in false alarms that can affect the performance of the RSE overall. The filtering method may be based on data collected by the RSE, OBE, OPE, or online resources. The RSE may interface with other controllers, such as traffic light controllers at intersections or other locations, to extract information for use in the data processing pipeline.

[0049] The RSE may also include or use communications equipment 20 to communicate wired or wirelessly with other RSEs, and with OBEs, OPEs, local or central servers, and other data processing units. The RSE may use any available standard for communicating with other equipment. The RSE may use a wired or wireless Internet connection to download and upload data to and from other equipment, a cellular network to send and receive messages from other cellular devices, and special wireless devices to communicate with infrastructure devices and other RSEs at intersections or other locations.

[0050] RSEs can be installed in close proximity to different types of intersections. For example, at signalized intersections (e.g., intersections where traffic is controlled by signal lights), the RSE 10 is installed near the traffic light controller 26, either in the same enclosure or in a nearby enclosure. Data (e.g., traffic light phase and timing) flows 28 between the traffic light controller and the RSE. At non-signalized intersections, the RSE 10 is typically located to simplify connecting it to sensors 12 used to monitor roads or other features of the ground transportation network in the vicinity of the intersection. The proximity of the RSE to the intersection helps maintain a low latency system, which is important for providing the receiving ground unit with the maximum time to respond to an alert.

[0051] 2. Onboard Equipment (OBE) 36 mounted on, supported by, or within the ground transportation entity 14, including sensors 38 that determine the entity's position and kinematics (operational data) as well as safety-related data about the entity. The OBE further includes a data processing unit 40, data storage 42, and communication equipment 44 that can wirelessly communicate with other OBEs, OPEs, RSEs, and possibly servers and computing units.

[0052] On Person Equipment (OPE) 46 may be, without limitation, a mobile phone, a wearable device, or any other device that can be worn by, held by, attached to, or otherwise connected to a person or animal. The OPE may include or be coupled to a data processing unit 48, data storage 50, and communication equipment 52, as needed. In some embodiments, the OPE functions as a special communication unit for potentially vulnerable road users who are not vehicles. In some examples, the OPE may also be used for other purposes. The OPE may include components that provide visual, audio, or tactile alerts to potentially vulnerable road users.

[0053] Potentially affected road users may include pedestrians, cyclists, road workers, people in wheelchairs, scooters, self-balancing devices, battery-powered personal mobility devices, animal-powered carriages, guide or police animals, livestock, herds, and pets.

[0054] Typically, an OPE is carried by a vulnerable road user and is capable of sending and receiving messages. An OPE may be attached to or integrated into a mobile phone, tablet, personal mobility device, bicycle, wearable device (e.g., watch, bracelet, anklet), or attached to a pet collar.

[0055] The message transmitted by the OPE may include kinematic information related to the potential road user, including but not limited to time, 3D location, heading, speed, and acceleration. The transmitted message may further convey data representing the potential road user's alertness level, current behavior, and future intentions, such as whether the potential road user is currently crossing a crosswalk, listening to music, or about to cross a crosswalk. Among other things, the message may convey the potential road user's blob size or data size, whether there is an external device (e.g., a stroller, cart, or other device) with the potential road user, whether the potential road user has a physical disability, or whether the potential road user is using any personal assistance. If the potential road user is a worker, the message may convey the worker's category and further describe the type of activity being performed by the worker. If a cluster of similar potential road users (e.g., a group of pedestrians) has similar characteristics, one message may be transmitted to avoid multiple message broadcasts.

[0056] Typically, messages received by the OPE are warning messages from roadside equipment or entities. The OPE may act on the received messages by issuing warnings to vulnerable road users. Warning messages convey data useful for providing customized warnings for vulnerable road users. For example, a warning to a vulnerable road user may indicate the type of dangerous situation and suggest possible actions to take. The OPE may apply warning filtering to all received messages and present only relevant messages to the vulnerable road user.

[0057] Alert filtering is based on the application of a learning algorithm to historical data associated with OPE, which allows alert filtering to be custom tailored to each vulnerable road user. The OPE learning algorithm tracks vulnerable road user responses to received alerts and tailors future alerts to elicit the best response times and attention from vulnerable road users. Learning algorithms may also be applied to data conveyed in transmitted messages.

[0058] 4. A data storage server 54, which may be, but is not limited to, cloud storage, local storage, or any other storage facility that allows for the storage and retrieval of data. The data storage server is accessible by the RSE, the computing unit, and possibly by the OBE, the OPE, and the data server, for example, to store data related to early warning and collision avoidance. The data storage server is accessible from the RSE, and possibly from the OBE, the OPE, and the data server, for fetching the stored data. The data may be raw sensor data, processed data by the processing unit, or any other information generated by the RSE, the OBE, and the OPE.

[0059] Sensors at intersections that continuously monitor ground traffic entities can generate large amounts of data daily. The volume of this data depends on the number and type of sensors. The data is processed in real time, for example, locally at the intersection, and stored for future analysis, which requires data storage units (e.g., hard disk drives, solid-state drives, and other mass storage devices). Local storage devices fill up after a period of time depending on their storage capacity, the volume of data generated, and the rate at which it is generated. To preserve the data for future use, the data is uploaded to a remote server with larger capacity. The remote server can upgrade its storage capacity on demand as needed. The remote server may use data storage devices similar to local storage (e.g., hard disk drives, solid-state drives, or other mass storage devices) accessible through a network connection.

[0060] Data stored locally and on a server for future analysis may include data broadcast by ground transportation entities and received by the RSE, saved for future analysis. Stored data may be downloaded from a server or other remote source for processing at the RSE. For example, a machine learning model of an intersection where the RSE is located may be stored on a server or other remote storage and downloaded by the RSE for use in analyzing current data received at the RSE from a local source.

[0061] 5. Computing unit 56, a powerful computing machine located in the cloud, locally (e.g., as part of the RSE), or a combination thereof. Among other functions, the computing unit processes available data to generate predictive, machine-learning-based models of the behavior, behavior, and intentions of vehicles, pedestrians, or other ground transportation entities using the transportation network. Each computing unit may include specialized hardware for processing the corresponding type of data (e.g., a graphics processing unit for processing images). Under heavy processing loads, the computing units in the RSE may become overloaded. This can occur, for example, when additional data-generating units (e.g., sensors) are added to the system, causing a computational overload. Overload can also occur if the logic running in the computing unit is replaced with more computationally intensive logic. Overload can occur due to an increase in the number of ground transportation entities being tracked. In the event of a local computational overload, the RSE may offload some of its tasks to another computing unit. The other computing unit may be near the RSE or remote (e.g., a server). Computational tasks may be prioritized, and non-time-critical tasks may be performed on other computing units, with results obtained by the local computing unit. For example, a computing unit in an RSE may request another computing unit to run a job to analyze stored data and use the data to train a model. The trained model is then downloaded by the computing unit in the RSE for storage and use therein.

[0062] A computing unit in an RSE may use other smaller computing units to perform computationally intensive jobs more efficiently and reduce the time required. Available computing units are used wisely to perform most tasks in the shortest time, for example, by dividing tasks between the RSE computing unit and other available computing units. Computing units may also be attached to the RSE as external devices to add more computing power to the computing units in the RSE. An externally attached computing unit may include the same or a different architecture than the computing units in the RSE. An externally attached computing unit may communicate with existing computing units using any available communication ports. An RSE computing unit may request more computing power from an external computing unit as needed.

[0063] The remainder of this document will, among other things, explain in detail the roles and functions of the above-mentioned components in the system.

[0064] Roadside Equipment (RSE)

[0065] As shown in FIG. 2, the RSE may include, but is not limited to, the following components:

[0066] 1. One or more communication units 103, 104 that enable the reception and / or transmission of motion data and other data related to ground transportation entities, and road safety data, from and to nearby vehicles or other ground transportation entities, infrastructure, and remote servers and data storage systems 130. In some examples, this type of communication is known as infrastructure-to-vehicles (I2V), infrastructure-to-pedestrians (I2P), infrastructure-to-infrastructure (I2I), and infrastructure-to-devices (I2D), and combinations thereof, known as infrastructure-to-everything (I2X). The communication may be wireless or wired and may conform to a wide variety of communication protocols.

[0067] 2. Communication unit 103 is used for communication with ground transportation entities, and unit 104 is used for communication with a remote server and data storage system 130 over the Internet.

[0068] 3. Local storage 106 for storing programs, intersection models, and behavior and traffic models. The local storage 106 may be used for temporary storage of data collected from the sensors 101.

[0069] 4. Sensors 101 and sensor controllers 107 that enable monitoring (e.g., generating data about) moving objects, such as ground traffic entities, typically near the RSE. Sensors may include, but are not limited to, cameras, radar, lidar, ultrasonic detectors, or any other hardware that can detect or infer from detected data, among other things, the distance to a ground traffic entity, or the speed, heading, or position of the ground traffic entity, or a combination thereof. Sensor fusion is performed using the aggregation or combination of data from two or more sensors 101.

[0070] 5. A position receiver (102) (e.g., a GPS receiver) that provides positioning data (e.g., coordinates of the RSE's location) and helps correct positioning errors in the location of ground transportation entities.

[0071] 6. A processing unit 105 that acquires and uses data generated from the sensors and incoming data from the communication units 103, 104. The processing unit processes and stores the data locally and, in some embodiments, transmits the data for remote storage and further processing. The processing unit also generates messages and alerts that are broadcast or otherwise transmitted over wireless communication facilities to nearby pedestrians, motor vehicles, or other ground transportation entities, and in some examples, to signs or other infrastructure indication devices. The processing unit also periodically reports the health and status of all RSE systems to a remote server for monitoring.

[0072] 7. Expansion connector 108 that allows control and communication between the RSE and other hardware or other components, such as temperature and humidity sensors, traffic light controllers, other computing units mentioned above, and other electronic components that may become available in the future.

[0073] Onboard equipment (OBE)

[0074] The on-board equipment is typically original equipment for the ground transportation entity or may be added to the entity by a third-party supplier. As shown in Figure 3, the OBE may include, but is not limited to, the following components:

[0075] 1. A communications unit 203 that enables transmission and / or reception of data to and from nearby vehicles, pedestrians, cyclists, or other ground transportation entities and infrastructure, and combinations thereof. The communications unit further enables transmission and / or reception of data between vehicles or other ground transportation entities and a local or remote server 212 for purposes of machine learning and for remote monitoring of the ground transportation entities by the server. In some examples, this type of communication is known as vehicle-to-vehicle (V2V), vehicle-to-pedestrian (V2P), vehicle-to-infrastructure (V2I), vehicle-to-device (V2D), and combinations thereof, known as vehicle-to-everything (V2X). The communication may be wireless or wired and may conform to a wide variety of communication protocols.

[0076] The communication unit 204 allows the OBE to communicate with remote servers over the Internet for program updates, data storage, and data processing.

[0077] 2. Local storage 206 for storing programs, intersection models, and traffic models. The local storage 206 may be used for temporary storage of data collected from the sensors 201.

[0078] 3. Sensors 201 and sensor control unit 207, which may include, but are not limited to, external cameras, lidar, radar, ultrasonic sensors, or any device that can be used to detect nearby objects or people or other ground traffic entities. Sensors 201 may further include additional kinematic sensors, global positioning receivers, and internal and local microphones and cameras.

[0079] 4. A location receiver 202 (eg, a GPS receiver) that provides location data (eg, coordinates of the location of a ground transportation entity).

[0080] 5. A processing unit 205 for acquiring, using, generating, and transmitting data, including consuming data from and transmitting data to a communication unit, as well as consuming data from sensors in or on the ground transportation entity.

[0081] 6. Expansion connector 208 to allow control and communication between the OBE and other hardware.

[0082] 7. An interface unit that may be retrofitted or integrated into the head unit, steering wheel, or driver mobile device in one or more ways, for example, using visual, audible, or tactile feedback.

[0083] Smart OBE (SOBE: Smart OBE)

[0084] In a world where all vehicles and other ground transportation entities are connected entities, each vehicle or other ground transportation entity could be a coordinated entity and could report its current location, safety status, intentions, and other information to others. Currently, nearly all vehicles are not connected entities, are unable to report such information to other ground transportation entities, and are operated by humans with different levels of skill, happiness, stress, and behavior. Without such connectivity and communication, it becomes difficult to predict the next move of a vehicle or ground transportation entity, further resulting in a lesser ability to implement collision avoidance and provide early warnings.

[0085] A smart OBE monitors the surrounding environment and the user or occupant of the ground transportation entity. It further monitors the health and status of the entity's different systems and subsystems. The SOBE monitors the outside world by listening, for example, to radio transmissions from emergency broadcasts, traffic and safety messages from nearby RSEs, and messages regarding safety, location, and other operational information from other connected vehicles or other ground transportation entities. The SOBE also interfaces with onboard sensors that can view road and driving conditions, such as cameras, distance sensors, vibration sensors, microphones, or any other sensors that enable such monitoring. The SOBE also monitors its immediate surroundings and generates a map of all stationary and moving objects.

[0086] The SOBE may further monitor the behavior of users or occupants of a vehicle or other ground transportation entity. The SOBE may use microphones to monitor conversation quality. It may further use other sensors, such as seat sensors, cameras, hydrocarbon sensors, and sensors for volatile organic compounds and other toxic substances. It may further use kinematic sensors to measure driver reactions and behavior and infer driving quality therefrom.

[0087] SOBE also receives vehicle-to-vehicle messages from other ground transportation entities (e.g., basic safety messages (BSM)) and vehicle-to-pedestrian messages from potentially vulnerable road users (e.g., personal safety messages (PSM)).

[0088] The SOBE then fuses data from this array of sensors, sources, and messages. The SOBE then applies the fused data to an artificial intelligence model that can predict not only the next action or reaction of the driver or user of a vehicle or other ground transportation entity or potential road user, but also the intentions and future trajectories and associated near-miss or collision risk due to other vehicles, ground transportation entities, and nearby potential road users. For example, the SOBE may use the BSM received from a nearby vehicle to predict that the nearby vehicle is about to enter a lane-changing maneuver that poses a risk to its own host vehicle, and may alert the driver of the impending risk. Risk is calculated by the SOBE based on the probability of various future predicted trajectories of the nearby vehicle (e.g., going straight, changing lanes to the right, changing lanes to the left) and the associated collision risk with the host vehicle for each of those trajectories. If the collision risk is higher than a certain threshold, a warning is displayed to the driver of the host vehicle.

[0089] Machine learning is typically required to predict intent and future trajectories due to the complexity of modeling human driver behavior, which is also influenced by external factors (e.g., changing surroundings and weather conditions).

[0090] SOBE is characterized by powerful computing power capable of processing multiple data feeds, some of which provide several megabytes of data per second. The amount of available data is, in turn, proportional to the level of detail required from each sensor.

[0091] SOBE also includes powerful signal processing equipment capable of extracting useful information from surrounding environments known to have high noise levels and low signal-to-noise ratios. SOBE also protects drivers from the overwhelming number of alerts their vehicles receive by providing smart alert filtering. Alert filtering is the result of machine learning models that can distinguish which alerts are significant given the current location, surrounding environmental conditions, driver behavior, vehicle health and status, and kinematics.

[0092] Smart SOBEs are important for collision avoidance and early warning, and for bringing about a safer transportation network for all users, not just the occupants or users of the vehicle containing the SOBE. SOBEs can detect and predict the movements of different entities on the road, thus assisting in collision avoidance.

[0093] Person equipment (OPE)

[0094] As described above, person-on-board equipment (OPE) includes any device that may be held by, attached to, or otherwise directly connected to a pedestrian, jogger, or other person who is a ground transportation entity or otherwise present on or using the ground transportation network. Such a person may be, for example, a road user who may be at risk of being struck by a vehicle. OPE may encompass, but is not limited to, mobile devices (e.g., smartphones, tablets, digital assistants), wearable devices (e.g., eyewear, watches, bracelets, anklets), and implants. Existing components and features of OPE may be used to track and report location, speed, and heading. OPE may be used to receive and process data and display alerts to the user through various modes (e.g., visual, audible, tactile).

[0095] Honda has developed a communication system and method for V2P applications that focuses on direct communication between vehicles and pedestrians using an OPE. In one example, a vehicle is equipped with an OBE for broadcasting messages to pedestrian OPEs in the surrounding environment. The messages convey the vehicle's current status, including, for example, vehicle parameters, speed, and direction of travel. For example, the messages can be basic safety messages (BSMs). When necessary, the OPEs present pedestrians with warnings about predicted dangerous situations tailored to the pedestrian's level of distraction to avoid collisions. In another example, the pedestrian OPEs broadcast messages (e.g., personal safety messages (PSMs)) to OBEs of vehicles in the surrounding environment where pedestrians may cross the vehicle's intended path. When necessary, the vehicle OBEs display warnings to the vehicle user about predicted dangers to avoid collisions. See Strickland, Richard Dean, et al., "Vehicle to pedestrian communication system and method." (U.S. Patent No. 9,421,909).

[0096] The system described herein uses an I2P or I2V approach that uses sensors external to vehicles and pedestrians (primarily in infrastructure) to track and collect data about pedestrians and other vulnerable road users. For example, sensors may track pedestrians crossing streets and vehicles operating at or near crossing locations. The collected data is then used to build predictive models of the intentions and behavior of pedestrians and vehicle drivers on roads using rule-based and machine learning methods. These models help analyze the collected data and make predictions of pedestrian and vehicle paths and intentions. If a danger is predicted, a message is broadcast from the RSE to the OBE and / or OPE to alert each entity in the other's intended path, allowing each to take preemptive action with enough time to avoid a collision.

[0097] Remote Computing (Cloud Computing and Storage)

[0098] Data collected from sensors connected to or embedded in RSEs, OBEs, and OPEs needs to be processed so that effective mathematical machine learning models can be generated. This processing requires a lot of data processing power to shorten the time required to generate each model. The required processing power is much more than what is typically available locally in the RSE. To solve this, data can be transmitted to remote computing facilities that provide the necessary capabilities and can be scaled on demand. Remote computing facilities are referred to herein as "remote servers," following the terminology used in the computing literature. In some instances, it may be possible to perform some or all of the processing in the RCEs by equipping them with high-powered computing capabilities.

[0099] Rule-Based Processing

[0100] Unlike artificial intelligence and machine learning technologies, rule-based processing can be applied at any time without requiring data collection, training, and model building. Rule-based processing can be deployed from the beginning of system operation and typically occurs until sufficient training data is acquired to generate a machine learning model. After a new installation, rules are set up to process incoming sensor data. This is not only useful for improving road safety, but also serves as a suitable test case to ensure all components of the system are operating as expected. Rule-based processing can be added and used later as an additional layer to capture rare cases where machine learning may not be able to make accurate predictions. The rule-based approach is based on simple associations between collected data parameters (e.g., speed, range, etc.). The rule-based approach can also provide a baseline for evaluating the performance of machine learning algorithms.

[0101] In rule-based processing, sensors monitor vehicles or other ground traffic entities traversing a portion of a ground transportation network. If their current speed and acceleration exceed thresholds that prevent them from stopping before a stop bar (line) on the road, an alert is generated, for example. A variable area is assigned to each vehicle or other ground traffic entity. The area is labeled as a dilemma zone, where the vehicle has not yet been labeled as a violating vehicle. If the vehicle crosses the dilemma zone and enters a danger zone because its speed or acceleration, or both, exceed predefined thresholds, the vehicle is labeled as a violating entity and an alert is generated. The thresholds for speed and acceleration are based on physics and kinematics and vary depending on each ground traffic entity approaching an intersection, for example.

[0102] Two traditional rule-based approaches are 1) static TTI (Time-To-Intersection) and 2) static RDP (Required Deceleration Parameter). See Aoude, Georges S. et al., "Driver behavior classification at intersections and validation on large naturalistic data set." IEEE Transactions on Intelligent Transportation Systems 13.2(2012):724-736.

[0103] Static TTI (Time to Intersection) uses the estimated time to reach an intersection as a classification criterion. In its simplest form, TTI is

number

[0104] The static RDP (requested deceleration parameter) calculates the deceleration required for a vehicle to come to a safe stop, given the vehicle's current speed and position on the road.

number

number

[0105] Similar to the static TTI algorithm, RDP alert The parameter reflects the conservatism level of the rule-based algorithm.

[0106] We use rule-based approaches as a baseline for evaluating the performance of the machine learning algorithms of the present disclosure, and in some instances, we run them in parallel with the machine learning algorithms to capture rare cases, i.e., rare cases that the machine learning may not be able to predict.

[0107] Machine Learning

[0108] Modeling driver behavior has been shown to be a complex task given the complexity of human behavior. See H.M. Mandalia and D.D. Dalvucci, "Using Support Vector Machines for Lane-Change Detection," Human Factors and Ergonomics Society Annual Meeting Proceedings, vol. 49, pp. 1965-1969, 2005. Machine learning techniques are well suited to modeling human behavior, but they require "learning" using training data to operate properly. To provide superior detection and prediction results, this specification uses machine learning to model traffic detected at intersections or other characteristics of the ground transportation network during a training period before alert processing is applied to current traffic during the deployment phase. Machine learning can further be used to model driver responses using in-vehicle data from on-board equipment (OBE), which can be further based on in-vehicle sensors and driving record history and preferences. This specification also uses machine learning models to detect and predict the trajectories, behaviors, and intentions of potentially vulnerable road users (e.g., pedestrians). Machine learning can also be used to model the response of potential road users from operator-on-board equipment (OPE). These models can include interactions between entities, potential road users, and between one or more entities and one or more potential road users.

[0109] Machine learning techniques may also be used to model the behavior of non-autonomous ground traffic entities, and to predict their intentions and communicate with them, and with other involved entities, when a near miss or accident or other dangerous situation is predicted by observing and / or communicating with the non-autonomous ground traffic entity.

[0110] The machine learning mechanism operates in two phases: 1) training and 2) deployment.

[0111] Training Phase

[0112] After installation, the RSE begins collecting data from sensors it can access. Because AI model training requires significant computing power, it is typically performed on a powerful server containing multiple parallel processing modules to speed up the training phase. For this reason, data acquired at the RSE's location in the ground transportation network can be packaged and sent to a remote, powerful server immediately after acquisition. This is done using an internet connection. The data is then prepared automatically or with the assistance of a data scientist. An AI model is then built to capture important characteristics of the traffic flow of vehicles and other ground transportation entities relative to that intersection or other aspects of the ground transportation network. The captured data features may include the position, direction, and movement of vehicles or other ground transportation entities, which can then be converted into intent and behavior. By knowing the intent, the AI ​​model can be used to predict, with high accuracy, the actions and future behavior of vehicles or other ground transportation entities approaching the traffic location. The trained AI model is then tested on a subset of data that was not included in the training phase. If the AI ​​model's performance meets expectations, training is considered complete. This phase is repeated repeatedly using different model parameters until a satisfactory performance of the model is achieved.

[0113] Deployment Phase

[0114] In some embodiments, the completed and tested AI model is then transmitted over the Internet to RSEs at traffic locations in the ground transportation network. The RSEs are then ready to process new sensor data and perform prediction and detection of dangerous situations, such as traffic light violations. If a dangerous situation is predicted, the RSE generates an appropriate warning message. A dangerous situation may be predicted, a warning message may be generated, and the warning message may be broadcast to and received by vehicles and other ground transportation entities in the vicinity of the RSE before the predicted dangerous situation occurs. This provides sufficient time for operators of vehicles or other ground transportation entities to react and take collision avoidance measures. The output of the AI ​​model from various intersections where the corresponding RSEs are located may be recorded and made available online in a dashboard that incorporates all data generated and displayed in an intuitive and user-friendly manner. Such a dashboard may be used as an interface with customers of the system (e.g., city traffic engineers or planners). An example of a dashboard is a map that includes markers showing the locations of monitored intersections, violations that have occurred, and statistical and analytical results based on AI predictions and actual outcomes.

[0115] Smart RSE (SRSE) and bridging connected and unconnected entities

[0116] As already alluded to, gaps exist between the capabilities and behaviors of connected and unconnected entities. For example, connected entities are typically coordinated entities that continuously advertise their location and safety system status, such as speed, heading, brake status, and headlight status, to the world. Unconnected entities are unable to coordinate and communicate in these ways. Thus, even connected entities are unaware of unconnected entities that are not in the connected entity's vicinity or that are outside their detection range due to interference, distance, or lack of a vantage point.

[0117] Including the appropriate equipment and configuration, RSEs can be enabled to detect all entities using the ground transportation network in their vicinity, including unconnected entities. Specialized sensors can be used to detect different types of entities. For example, radar is suitable for detecting moving metal objects, such as cars, buses, and trucks. Such road entities are most likely to move in one direction toward an intersection. Cameras are suitable for detecting vulnerable road users who may wander near an intersection waiting for a safe time to cross.

[0118] Placing sensors on components of the ground transportation network has at least the following advantages:

[0119] - Vantage points: Infrastructure poles, beams, and support cables typically include high vantage points. High vantage points allow for a more comprehensive view of the intersection. This is similar to a control tower at an airport, where air traffic controllers have a panoramic view of most of the critical and vulnerable users on the ground. For ground traffic entities, in contrast, the view from a sensor's (camera, lidar, radar, etc., or other) vantage point can be obstructed or obstructed by trucks in nearby lanes, direct sunlight, or other interference. Sensors at intersections can be selected to be resistant to or less susceptible to such interference. Radar, for example, is not affected by sunlight and remains effective during the evening commute. Thermal cameras are more likely to detect pedestrians in bright light conditions where an optical camera's view is obstructed.

[0120] Fixed position: A sensor located at an intersection can be adjusted and fixed to detect in a specific direction that may be optimal for detecting important targets. This helps the processing software to detect objects better. As an example, if a camera has a fixed view, the background information (of stationary objects and structures) in the fixed view can be easily detected and used to improve the identification and classification of relatively important moving entities.

[0121] Fixed sensor locations also allow for easier placement of each entity in a unified global view of the intersection. Because the sensor view is fixed, measurements from the sensors can be easily mapped to an integrated global position map of the intersection. Such an integrated map is useful when performing a global analysis of traffic movement from all directions to investigate the interactions and dependencies of one traffic flow on another. One example is detecting near misses (hazardous situations) before they occur. When two entities are traveling along intersecting paths, the global and integrated view of the intersection allows for the calculation of each entity's arrival time at the intersection on their respective paths. If the time is within a certain limit or tolerance, the near miss can be flagged (e.g., the subject of a warning message) before it occurs.

[0122] With the help of sensors installed on infrastructure components, smart RSE (SRSE) can bridge this gap and enable connected entities to be aware of "dark" or unconnected entities.

[0123] FIG. 8 shows a scenario illustrating how strategically placed sensors can help a connected entity identify the speed and location of an unconnected entity.

[0124] Connected entity 1001 travels along path 1007. Entity 1001 is presented with a green light 1010. Unconnected entity 1002 travels along path 1006. Entity 1002 is presented with a red light 1009 but intends to make a right turn on the red light along path 1006. This places entity 1002 directly in entity 1001's path. A dangerous situation is imminent because entity 1001 is unaware of entity 1002. Because entity 1002 is an unconnected entity, it cannot broadcast (e.g., advertise) its location and heading to other entities sharing the intersection. Furthermore, even if entity 1001 were connected, it would not be able to "see" entity 1002, who is obscured by building 1008. There is a risk that entity 1001 could continue straight through the intersection and collide with entity 1002.

[0125] If the intersection is configured as a smart intersection, radar 1004 mounted on a beam 1005 above the road at the intersection detects the entities 1002 and their speed and distance. This information can be relayed to the connected entities 1001 through the SRSE 1011, which acts as a bridge between the unconnected entities 1002 and the connected entities 1001.

[0126] Artificial Intelligence and Machine Learning

[0127] Smart RSEs also rely on learning traffic patterns and entity behavior to better predict and prevent dangerous situations and avoid collisions. As shown in FIG. 8 , radar 1004 constantly detects and provides data for each entity moving along the approach road 1012. This data is collected and transmitted to the cloud, either directly or through the RSE, for example, for analysis and to build and train a model that accurately represents traffic along the approach road 1012. When the model is complete, it is downloaded to the SRSE 1011. This model can then be applied to each entity moving along the approach road 1012. If the model classifies an entity as violating (or attempting to violate) a traffic rule, a warning (alert) can be broadcast by the SRSE to all connected entities in the vicinity. This warning, known as an intersection collision avoidance warning, can be received by the connected entities and used as a basis for considering dangerous situations and avoiding collisions. By using an appropriate traffic model, violating entities can be detected in advance, giving connected entities using the intersection sufficient time to react to and avoid dangerous situations.

[0128] With the help of multiple sensors (some mounted high on components of the ground transportation network's infrastructure), artificial intelligence models, and accurate traffic models, the SRSE can obtain a virtual overview of the ground transportation network and recognize each entity in its field of view, including unconnected entities in its field of view that are not "visible" to connected entities in its field of view. The SRSE can use this data to feed the AI ​​model and provide alerts to connected entities on behalf of unconnected entities, which would otherwise be unaware of the existence of unconnected entities sharing the road.

[0129] The SRSE includes high-power computing available at the SRSE's location within the same housing, or by connecting to a server to a nearby unit or over the internet. The SRSE can process data received directly from sensors or via broadcast from nearby SRSEs, emergency and weather information, and other data. The SRSE also includes mass storage to aid in data storage and processing. High-bandwidth connectivity is also required to aid in transmitting raw data and AI models between the SRSE and even more powerful remote servers. The SRSE uses AI to enhance other traffic hazard detection technologies to achieve high accuracy and provide additional time to react and avoid collisions.

[0130] SRSEs can still be compatible with current and new standardized communication protocols, so they can seamlessly interface with equipment already deployed in the field.

[0131] The SRSE may further reduce network congestion by sending messages only when necessary.

[0132] Global and integrated intersection topology

[0133] Effective traffic monitoring and control at an intersection benefits from a bird's-eye view of the intersection unobstructed by obstacles, lighting, or any other interference.

[0134] As mentioned above, different types of sensors can be used to detect different types of entities. The information from these sensors can differ, e.g., the location or motion parameters the data represents, the native format of the data, or both. For example, radar data typically includes speed, distance, and possibly additional information, e.g., the number of moving and stationary entities within the radar's field of view. Camera data, in contrast, can represent an image of the field of view at any point in time. Lidar data can provide the location of a point in 3D space corresponding to the reflection point of a laser beam emitted from a lidar at a specific time and direction of travel. Generally, each sensor provides data in a native format that accurately represents the physical quantity it measures.

[0135] To obtain a unified view (representation) of an intersection, it is useful to fuse data from different types of sensors. For the purpose of fusion, data from the various sensors is converted into a common (unified) format that is independent of the sensors used. The data contained in the unified format from all of the sensors includes the global position, speed, and heading of each entity using the intersection, regardless of how it was detected.

[0136] By using this integrated global data, the smart RSE can not only detect and predict the movements of entities, but also identify the relative positions and headings of different entities with respect to each other. Thus, the SRSE can achieve improved detection and prediction of dangerous situations.

[0137] For example, in the scenario shown in FIG. 9, motorized entity 2001 and vulnerable road user 2002 share the same crosswalk. Entity 2001 moves along road 2007 and is detected by radar 2003. Vulnerable road user 2002, walking along sidewalk 2006, is detected by camera 2004. Vulnerable road user 2002 may decide to cross road 2007 using crosswalk 2005. Doing so places road user 2002 in the path of entity 2001, creating a potentially dangerous situation. If the data 2003 and 2004 from each of the sensors were considered independently, and assuming no other information was taken into account, the dangerous situation would not be identified because each of the sensors would only detect entities in its respective field of view. Furthermore, each of the sensors may be unable to detect objects that they are not designed to detect. However, when a synthetic view is considered by the SRSE, the position and dynamics of the entity 2001 and of the potentially affected road user 2002 may be located in the same reference frame, i.e., a geographical coordinate system, e.g., a map projection, or other coordinate system. When considered in a common reference frame, the fused data from the sensors may be used to detect and predict potentially dangerous situations between the two entities 2001 and 2002. The following paragraphs explain the transformation between sensor space and synthetic space.

[0138] Conversion of radar data to a synthetic standard

[0139] As shown in FIG. 10 , radar 3001 is used to monitor road entities traveling along a road including two lanes 3005 and 3008, each including centerlines 3006 and 3007. Stop bars 3003 indicate the ends of lanes 3005 and 3008. T 3006 can be defined by a collection of markers 3003 and 3004. While FIG. 10 shows only two markers, the centerline is generally a piecewise linear function. The global positions of markers 3003 and 3004 (and other markers not shown) are predetermined by the roadway design and known to the system. The precise global position of radar 3001 can be further determined. Thus, distances 3009 and 3010 of markers 3003 and 3004 from radar 3001 can be calculated. Distance 3011 of entity 3002 from radar 3001 can be measured by radar 3001. Using simple geometry, the system can determine the location of entity 3002 using the measured distance 3011. The result is a global position because it is derived from the global positions of markers 3003, 3004 and radar 3001. Since each roadway can be approximated by a generalized piecewise linear function, the above method can be applied to any roadway that can be monitored by radar.

[0140] 11 shows a similar scenario on a curved road. Radar 4001 monitors an entity moving along road 4008. Markers 4003 and 4004 represent linear segments 4009 (of a piecewise linear function) of centerline 4007. Distances 4005 and 4006 represent the normal distances between the plane 4010 of radar 4001 and markers 4003 and 4004, respectively. Distance 4007 is the measured distance of entity 4002 from radar plane 4010. Following this, given the global positions of radar 4001 and markers 4003 and 4004, the global position of entity 4002 can be calculated using simple ratio arithmetic.

[0141] Transforming camera data into a synthetic reference

[0142] Knowing the camera's height, global position, orientation, tilt, and field of view makes it simple to calculate the global position of each pixel in the camera image using existing 3D geometric laws and transformations. As a result, when an object is identified in an image, its global position can be easily estimated by knowing the pixel it occupies. It is useful to note that the type of camera is irrelevant if its specifications, such as sensor size, focal length, or field of view, or a combination thereof, are known.

[0143] 12 shows a side view of camera 5001 looking at entity 5002. Height 5008 and tilt angle 5006 of camera 5001 can be determined at installation. Field of view 5007 can be obtained from the specifications of camera 5001. The global position of camera 5001 can also be determined at installation. From the known information, the system can determine the global positions of points 5003 and 5004. The distance between points 5003 and 5004 is further divided into pixels in the image generated by camera 5001. The number of pixels is known from the specifications of camera 5001. The pixel occupied by entity 5002 can be identified. Thus, distance 5005 can be calculated. The global position of entity 5002 can also be calculated.

[0144] A global, integrated view of any intersection can be pieced together by fusing information from various sensors. FIG. 13 shows a plan view of a four-way intersection. Each leg of the intersection is separated by a median strip 6003. The intersection in the figure is monitored by two different types of sensors, radar and camera, and the principles described herein can be generalized to other types of sensors. In this example, radar monitoring area 6001 overlaps camera monitoring area 6002. With a unified global view, each entity moving between areas continues to be tracked within the unified global view. This, for example, easily enables SRSE to identify relationships between the actions of different entities. Such information enables a truly universal, bird's-eye view of the intersection and roadways. The integrated data from the sensors can then be fed to an artificial intelligence program, as described in the following paragraphs.

[0145] FIG. 2 above illustrates the components of an RSE. Additionally, in an SRSE, the processing unit may further include one or more specialized processing units that can process data in parallel. An example of such a unit is a graphics processing unit, or GPU. With the assistance of a GPU or similar hardware, machine learning algorithms can operate much more efficiently in an SRSE and provide results in real time. Such a processing architecture enables real-time prediction of dangerous situations and thus allows for early warnings to be sent to provide entities with sufficient time to react and avoid collisions. Additionally, because the SRSE may perform processing that may use data from different sensors and different types of sensors, the SRSE may build an integrated view of an intersection that is useful for analyzing traffic flow and detecting and predicting dangerous situations.

[0146] Usage example

[0147] A wide variety of examples may benefit from the system and the early warning it may provide for collision avoidance. Examples are provided here.

[0148] Example 1: Vulnerable Ground Transportation Entities

[0149] As shown in FIG. 4 , a roadway crossing a typical intersection 409 may include crosswalks that include specific crossing areas 401, 402, 403, and 404 that pedestrians and other vulnerable road users (vulnerable road users) can use to walk across the roadway. Sensors suitable for detecting such crossings or other vulnerable road users are located at one or more vantage points that enable monitoring of the crosswalks and the crosswalk's surrounding environment. During the training phase, collected data may be used to train an artificial intelligence model to learn about the behavior of vulnerable road users at the intersection. During the deployment phase, the AI ​​model may then use current data about vulnerable road users, for example, to predict when a vulnerable road user intends to cross the roadway and to make that prediction before the vulnerable road user begins to cross. If the behavior and intentions of pedestrians and other vulnerable road users, drivers, vehicles, and other persons and ground traffic entities can be predicted in advance, early warnings (e.g., alerts) may be sent to any or all of them. Early warnings may allow vehicles to stop, slow down, reroute, or a combination thereof, and may allow vulnerable road users to refrain from crossing the road if a dangerous situation is predicted to be imminent.

[0150] Generally, sensors are used to monitor all areas near intersections for possible movement of vulnerable road users and vehicles. The type of sensor used depends on the type of object being monitored and tracked. Some sensors are better at tracking people and bicycles or other non-motorized vehicles. Some sensors are better at monitoring and tracking motorized vehicles. The solution described herein is sensor and hardware agnostic, as the type of sensor is irrelevant as long as the sensor provides appropriate data at a sufficient data rate, which may depend on the type of object being monitored and tracked. For example, Doppler radar is an appropriate sensor for monitoring and tracking vehicle speed and distance. The data rate, or sampling rate, is the rate at which the radar can provide successive new data values. The data rate must be fast enough to capture the dynamics of the behavior of the monitored and tracked object. The higher the sampling rate, the more detail is captured and the more robust and accurate the data's representation of behavior. If the sampling rate is too low and the vehicle travels a significant distance between two sample instances, it becomes difficult to model the behavior due to missed details during the intervals where no data is generated.

[0151] For crosswalks, sensors monitor the intersection and areas near the intersection for crossing pedestrians and other vulnerable road users (e.g., cyclists). Data from these sensors can be segmented to represent conditions with different virtual zones to aid in detection and localization. Zones can be selected to correspond to critical areas where dangerous situations may be expected, such as sidewalks, sidewalk entrances, and road approaches 405, 406, 407, 408 to the intersection. Activity and other conditions in each zone are recorded. Recordings can include, but are not limited to, kinematics (e.g., position, heading, speed, and acceleration) and facial and body features (e.g., eyes, posture).

[0152] The number of sensors, the number of zones, and the shape of the zones are specific to each intersection and to each approach to the intersection.

[0153] FIG. 5 shows a floor plan of a typical illustrative scene illustrating different zones used to monitor and track the movements and behavior of pedestrians or other vulnerable road users, and motorized and non-motorized vehicles and other ground transportation entities.

[0154] Sensors are configured to monitor crosswalks across roadways. Virtual zones (301, 302) may be located on the sidewalks and along the crosswalks. Other sensors are positioned to monitor vehicles and other ground traffic entities traveling on the road leading to the crosswalks, with virtual zones (303, 304) strategically placed to help detect, for example, incoming vehicles and other ground traffic entities, their distance from the crosswalk, and their speed.

[0155] The system (e.g., the RSE or SRSE associated with the sensors) collects streams of data from all sensors. When the system is first operational, an initial rule-based model may be deployed to aid in equipment calibration and function. Meanwhile, sensor data (e.g., speed and distance from a radar unit, images and video from a camera) is collected and stored locally on the RSE for preparation and, in some embodiments, transmission to a remote computer powerful enough to use this collected data to build AI models of the behavior of different entities at the intersection. In some examples, the RSE is an SRSE capable of generating the AI ​​models itself.

[0156] The data is then prepared and a trajectory is constructed for each ground traffic entity passing through the intersection. For example, a trajectory can be derived from radar data by connecting together points at different distances that belong to the same entity. Pedestrian trajectories and behavior can be derived, for example, from camera and video recordings. By implementing video and image processing techniques, pedestrian movements can be detected in images and videos and their respective trajectories can be estimated.

[0157] For human behavior, intelligent machine learning-based models typically outperform simple rules based on simple physics, as human intent is difficult to capture and large data sets are required to be able to detect patterns.

[0158] When the machine learning (AI) model is completed on the server, it is downloaded to the RSE, e.g., via the internet. The RSE then applies current data captured from the sensors to the AI ​​model, causing it to predict intent and behavior, identify when a dangerous situation is imminent, and trigger a corresponding alert that is disseminated (e.g., broadcast) to vehicles and other ground traffic entities and to potential road users and drivers as an early warning in time to enable them to take collision avoidance steps.

[0159] This exemplary setup can be combined with any other use case, for example traffic at a traffic lighted intersection or at grade intersection.

[0160] Example 2: Intersection with traffic lights

[0161] For signalized intersections (e.g., those controlled by traffic lights), the overall configuration of the system is as in Example 1. One difference may be the type of sensors used to monitor or track vehicle speed, direction, distance, and location. The crosswalk configuration in Example 1 may also be combined with the signalized intersection configuration for a more general solution.

[0162] The concept of operation for the traffic lighted intersection use case is to track road users around the intersection using external sensors that collect data about the users or data communicated by the users themselves, to predict their behavior, and generally to broadcast warnings through different communication means about approaching dangerous situations due to violation of traffic rules at the intersection, for example violating a red light.

[0163] Data about road users can be collected using (a) entity data broadcast by each entity itself regarding its current state, for example, through a BSM or PSM, and (b) sensors installed externally on the infrastructure or on vehicles, for example, Doppler radar, ultrasonic sensors, visual or thermal cameras, lidar, etc. As mentioned above, the type of sensor selected and its location and orientation at the intersection must provide the widest coverage of the intersection, or the portion of it under investigation, and the collected data about entities approaching the intersection will be the most accurate. The collected data therefore enables the reconstruction of the current state of road users and the generation of accurate, timely, and useful VBSMs (virtual basic safety messages) or VPSMs (virtual personal safety messages). The frequency with which data must be collected depends on the potential danger of each type of road user and the criticality of the potential violation. For example, a motorized vehicle moving at high speed at an intersection typically requires 10 data updates per second to achieve real-time collision avoidance, while a pedestrian crossing the intersection at a much slower speed may require as few as one data update per second.

[0164] As mentioned above, Figure 4 shows an example plan view of a signalized intersection including detection virtual zones. These zones may segment each approach to the intersection into separate lanes 410, 411, 412, 413, 405, 406, 407, and 408, and may further divide each lane into areas corresponding to typical distance ranges from the stop bar. The selection of these zones may generally be performed empirically to suit the characteristics of the particular approach and intersection. Segmenting the intersection allows for more accurate identification of the relative heading, speed, acceleration, and position of each road user, which in turn allows for a better assessment of the potential hazard that road user poses to other surface transportation entities.

[0165] To determine whether an observed traffic situation is unsafe, the system further needs to compare the results of the predicted situation with the status of traffic lights and consider local traffic rules (e.g., left-turn lanes, right turns on red, etc.). Therefore, collecting and using signal phase and timing (SPaT) information for an intersection is required. SPaT data can be collected by directly interfacing with traffic light controllers at the intersection, typically through a wired connection to read the data, or by interfacing with a traffic management system to receive requested data, for example, through an API. To ensure that road user status is always synchronized with traffic signal status, it is important to collect SPaT data at a rate as close as possible to the rate at which road user data is collected. A further complication to the need for knowledge of SPaT information is that modern traffic control techniques used to regulate traffic flow near intersections are not based on fixed timing but use algorithms that can dynamically adapt to real-time traffic conditions. Therefore, it is important to incorporate SPaT data prediction algorithms to ensure the highest accuracy in violation prediction. These SPaT data prediction algorithms can be built using rule-based or machine learning methods.

[0166] For each approach to an intersection, data is collected by the RSE (or SRSE), and a machine learning (AI) model is constructed to explain vehicle behavior corresponding to the collected data. Current data collected at the intersection is then applied to the AI ​​model to generate an early prediction of whether a vehicle or other ground traffic entity traveling on one of the approaches to the intersection is about to violate, for example, a traffic light. If a violation is imminent, a message is relayed (e.g., broadcast) from the RSE to nearby ground traffic entities. Vehicles (including the violating vehicle) and pedestrians or other potentially affected road users receive the message and have time to take appropriate preemptive measures to avoid a collision. The message may be delivered to the ground traffic entities by one or more of the following techniques: flashing lights, signs, or radio signals, among others.

[0167] If a vehicle or other entity approaching an intersection is equipped with an OBE or OPE, it can receive a message broadcast from the RSE about a potential hazard predicted at the intersection. This allows the user to be alerted and take appropriate preemptive measures to avoid a collision. If a user of a road violation at the intersection is also equipped with an OBE or OPE, the user will also receive the broadcast alert. An algorithm in the OBE or OPE can then tailor the message to include the user's violation behavior and alert the user appropriately.

[0168] The decision to send an alert depends not only on the vehicle's behavior as indicated by data collected by sensors at the intersection. While sensors play a large role in the decision, other inputs are also considered. These inputs may include, but are not limited to, information from nearby intersections (if a vehicle runs a red light at a nearby intersection, there is a higher probability that the vehicle will do the same at this intersection), information from other associated vehicles, or even information from the vehicle itself, for example, if the vehicle reports that it is experiencing an anomaly.

[0169] Example 3: Intersection without traffic lights

[0170] Controlled intersections without traffic lights, such as those controlled by stop signs or yield signs, can also be monitored. Sensors can be used to monitor approach paths controlled by traffic lights, and predictions can be made about incoming vehicles, similar to predictions about incoming vehicles at approaches to signalized intersections. The rules of the road at controlled intersections without traffic lights are typically clearly defined. Ground traffic entities at approach paths controlled by stop signs must come to a complete stop. At multi-way stop intersections, right-of-way is determined by the order in which ground traffic entities reach the intersection. Special cases can be considered using one-way stops. A set of sensors can similarly monitor approach paths without stop signs. Such a configuration can assist in stop sign gap negotiation. For intersections controlled by yield signs, ground traffic entities at approach paths controlled by yield signs must reduce their speed to grant right-of-way to other ground traffic entities at the intersection.

[0171] A major challenge is that due to internal factors (e.g., driver distraction) or external factors (e.g., lack of visibility), ground transportation entities violate the rules of the road and put other ground transportation entities at risk.

[0172] In the general example of a stop sign controlled intersection (i.e., each approach is controlled by a stop sign), the overall configuration of the system is as in Example 1. One difference may be the type of sensors used to monitor or track vehicle speed, direction, distance, and position. Another difference is that no traffic light control system is included, and road rules are indicated by road signs. The configuration for a pedestrian crossing in Example 1 can be further combined with a non-traffic light controlled intersection configuration for a more general solution.

[0173] It can be further appreciated that Figure 4 shows an example plan view of a four-way stop intersection including detection virtual zones. These zones may segment each approach to the intersection into separate lanes 410, 411, 412, 413, 405, 406, 407, 408, and may further divide each lane into areas corresponding to typical distance ranges from the stop bar. The selection of these zones may generally be done empirically to suit the characteristics of the particular approach and intersection.

[0174] In a similar manner to that described above for Figure 4, current data collected at an intersection is applied to an AI model to generate an early prediction of whether a vehicle or other ground traffic entity traveling on one of the approaches to the intersection is about to violate a stop sign. If a violation is imminent, the message may be treated similarly to the previous example involving a traffic light violation.

[0175] Further, as discussed above, the decision to send an alert may be based on the factors discussed above, as well as other information, such as whether the vehicle ran a stop sign at a nearby intersection, that suggests there is a higher probability that the vehicle will do the same at this intersection.

[0176] Figure 18 shows a use case for an intersection without controlled traffic lights. Figure 18 illustrates how an SRSE containing strategically placed sensors can alert connected entities to an imminent dangerous situation arising from an unconnected entity.

[0177] Connected entity 9106 is traveling along path 9109. Entity 9106 has the right-of-way. Unconnected entity 9107 is traveling along path 9110. Entity 9107 is presented with a yield sign 9104, which merges with path 9109 without giving entity 9106 the right-of-way, placing entity 9107 directly in entity 9106's path. A dangerous situation is imminent because entity 9106 is unaware of entity 9107. Because entity 9107 is an unconnected entity, it cannot advertise (broadcast) its position and heading to other entities sharing the intersection. Furthermore, entity 9106 may not be able to "see" entity 9107 that is not in its direct line of sight. As entity 9106 continues along its path, it may eventually collide with entity 9107.

[0178] Because the intersection is a smart intersection, radar 9111 mounted on a beam 9102 above the road detects entity 9107. Radar 9111 also detects the speed and distance of entity 9107. This information can be relayed as an alert to connected entities 9106 through SRSE 9101. SRSE 9101 contains a machine learning model for entities moving along approach road 9110. Entity 9107 is classified by the model as a potential violator of traffic rules, and a warning (alert) is broadcast to connected entities 9106. This warning is sent in advance to give entity 9106 enough time to react to and prevent a dangerous situation.

[0179] Example 4: At-grade intersection

[0180] At-grade intersections are dangerous because they can carry motorized vehicles, pedestrians, and rail vehicles. Often, the road leading to the at-grade intersection is in the blind spot of train or other rail vehicle operators (e.g., conductors). Because rail vehicle operators operate primarily based on line-of-sight information, this increases the likelihood of accidents when road users violate the rail vehicle's right-of-way and cross the at-grade intersection when they are not authorized to do so.

[0181] The behavior of the at-grade intersection use case is similar to that of a signalized intersection in the sense that at-grade intersections are often a collision point between road and rail traffic, regulated by traffic rules and signals. Therefore, this use case also requires collision avoidance warnings to increase safety near at-grade intersections. Rail traffic can have planned separated rail rights-of-way, such as high-speed rail, or no separated rail rights-of-way, such as light urban rail or tram. With light rail and tram, this use case becomes even more important, as these rail vehicles also operate on active roads and must follow the same traffic rules as road users.

[0182] Figure 6 shows a common use case for an at-grade intersection where a road and pedestrian crossing cross a railroad. Similar to the pedestrian crossing use case, sensors are deployed to collect data on pedestrian movements and intentions. Other sensors are used to monitor and predict the movements of road vehicles approaching the intersection. Data on road users can also be collected from road user broadcasts (e.g., BSM or PSM). Data from nearby intersections, vehicles, and remote command and control centers can be used to determine whether to trigger an alert.

[0183] Further data on SPaT for road and rail approaches is needed to properly assess potential violations.

[0184] Similar to the signalized intersection use case, the collected data allows for the generation of predictive models using rule-based and machine learning algorithms.

[0185] In this use case, rail vehicles are equipped with an OBE or OPE to receive collision avoidance warnings. If a violation of the rail vehicle's right-of-way is predicted, the RSE broadcasts a warning message, alerting the rail vehicle driver that a road user is present on its intended path and allowing the rail vehicle driver to take preemptive action with enough time to avoid a collision.

[0186] If the violation road user is further equipped with an OBE or an OPE, the message broadcast by the RSE is further received by the violation road user. An algorithm in the OBE or OPE can then adjust the received message to include the user's violation behavior and warn the user appropriately.

[0187] (Bridging the Gap) Virtual Connected Ground Transportation Environment

[0188] As mentioned above, a useful application of the system is to generate a virtual connected surrounding environment in place of disconnected ground transportation entities. An obstacle to the adoption of connected technologies is not only the lack of infrastructure installations, but also the near-existence of connected vehicles, connected vulnerable road users, and other connected ground transportation entities.

[0189] In connection with connected vehicles, in some regulatory regimes, such vehicles constantly transmit what are called basic safety messages (BSMs). BSMs include, among other information, the vehicle's location, direction of travel, speed, and future route. Other connected vehicles can pay attention to these messages and use them to generate a map of vehicles in their surrounding environment. By knowing where vehicles in the surrounding environment are, a vehicle, whether autonomous or not, contains useful information for maintaining a high level of safety. For example, an autonomous vehicle may avoid performing a maneuver if a connected vehicle is present in its path. Similarly, a driver may receive an alert if any other vehicles are present in the path they plan to follow, such as if there is a sudden lane change.

[0190] Until all ground transportation entities are equipped to transmit and receive traffic safety messages and information, some road entities will be "dark" or invisible to the rest of the road entities. Dark road entities pose a risk of dangerous situations.

[0191] Dark road entities do not advertise (e.g., broadcast) their locations and are therefore invisible to connected entities, which may assume that all road entities broadcast their information (i.e., are connected entities). While on-board sensors can detect obstacles and other road entities, the range of these sensors tends to be too short to be effective in preventing dangerous situations and collisions. Thus, a gap exists between the connectivity of connected vehicles and the lack of connectivity of unconnected vehicles. The technology described below aims to bridge this gap by using infrastructure intelligence that can detect all vehicles at intersections or other components of a ground transportation network and send messages on behalf of unconnected vehicles.

[0192] The system may establish a virtual connected ground traffic environment that may bridge the gap between a future in which most vehicles (and other ground traffic entities) are envisioned to be connected entities and the present in which most vehicles and other ground traffic entities lack connectivity, for example at intersections. In the virtual connected ground traffic environment, smart traffic lights and other infrastructure installations may use sensors to track all vehicles and other ground traffic entities (connected, disconnected, semi-autonomous, autonomous, non-autonomous) and (in the case of vehicles) generate virtual BSM messages (VBSM) on their behalf.

[0193] A VBSM message can be considered a subset of a BSM. It may not contain all fields required to generate a BSM, but it may contain all localization information, including position, heading, speed, and trajectory. Because V2X communications are standardized and anonymous, VBSM and BSM cannot be easily distinguished and follow the same message structure. The main difference between the two messages is the availability of the source of the information contained in these messages. A VBSM may not include data and information that is not easily generated by external sensors, such as steering angle, brake status, tire pressure, or wiper activation.

[0194] With appropriate sensors installed, an intersection containing a smart RSE can detect all road entities moving through the intersection. The RSE can then convert all data from multiple sensors into a global integrated coordinate system. This global integrated system is represented by the geographic location, speed, and direction of travel of each road entity. Each road entity, whether connected or not, is detected by intersection equipment, and a global integrated position is generated for it. Standard safety messages can therefore be broadcast on behalf of road entities. However, if the RSE broadcasts safety messages for all entities it detects, the RSE may send messages on behalf of connected road entities. To resolve conflicts, the RSE can filter connected road entities from its list of dark entities. This can be achieved because the RSE continuously receives safety messages from connected vehicles and the RSE sensors continuously detect road entities passing through the intersection. If the location of a detected road entity matches a location from which a safety message was received by the RSE receiver, the road entity is inferred to be connected, and no safety messages are broadcast by the RSE on its behalf. This is shown in Figure 15.

[0195] By bridging the gap between connected and non-connected vehicles, connected entities (including autonomous vehicles) may safely maneuver through intersections with full awareness of all nearby road entities.

[0196] This aspect of the technology is illustrated in FIG. 17. An intersection 9001 involves multiple road entities at a given time. Some of these entities 9004, 9006 are unconnected, while others 9005, 9007 are connected. Vulnerable road users 9004, 9007 are detected by a camera 9002. Motorized road entities 9005, 9006 are detected by a radar 9003. The location of each road entity is calculated. Broadcasts from the connected road entities are then received by an RSE 9008. The entity's location from which the message was received is compared to a location at which the entity was detected. If the two entities match within a predetermined tolerance, the entity at that location is considered connected, and no safety messages are sent on its behalf. The remaining road entities without matching receiving locations are considered dark, and safety messages are broadcast on their behalf.

[0197] Collision warning and intersection violation warning, which are an integrated part of the V2X protocol, require each entity to be connected for the system to be effective. That requirement is an obstacle to the deployment of V2X devices and systems. Intersections with smart RSEs address that concern by providing a virtual bridge between connected and unconnected vehicles.

[0198] The U.S. Department of Transportation (DOT) and National Highway Traffic Safety Administration (NHTSA) have identified many connected vehicle applications that use BSM and can help substantially reduce non-malfunction crashes and fatalities. These applications include, but are not limited to, Forward Collision Warning (FCW), Intersection Movement Assist (IMA), Left Turn Assist (LTA), Do Not Pass Warning (DNPW), and Blind Spot / Lane Change Warning (BS / LCW). The U.S. DOT and NHTSA have defined these applications as follows:

[0199] FCW resolves rear-end collisions, warning drivers of stopped, slowing, or slower vehicles ahead. IMA is designed to avoid intersection crossing collisions, warning drivers of vehicles approaching from the side at intersections and covering two main scenarios: same- or opposite-direction turn paths and straight-through intersection paths. LTA resolves collisions when one involved vehicle turns left at an intersection and another vehicle is traveling straight from the opposite direction, warning drivers of approaching opposite-direction traffic when attempting to make a left turn. DNPW assists drivers in avoiding opposite-direction collisions caused by overtaking maneuvers, warning drivers of approaching opposite-direction vehicles when attempting to overtake a slower vehicle on an undivided two-lane roadway. BS / LCW resolves collisions when vehicles perform lane change / merging maneuvers before a collision and alerts drivers of vehicles in their blind spots in approaching or adjacent lanes.

[0200] The V2X protocol specifies that these applications must be accomplished using vehicle-to-vehicle (V2V) communications, where one connected distant vehicle broadcasts basic safety messages to a connected host vehicle. The host vehicle's OBE then attempts to coordinate its own vehicle parameters, such as speed, heading, and trajectory, with these BSMs to determine whether there is a potential hazard or threat posed by the distant vehicle, as described thus far herein. Furthermore, autonomous vehicles particularly benefit from such applications because they allow vehicles in the surrounding environment to communicate their intent, an important piece of information not contained in the data collected from their onboard sensors.

[0201] However, current vehicles are not connected, and as noted above, it will take a very long time until there is a high percentage of connected vehicles for the BSM to function properly as described above. Thus, in an environment with a small percentage of connected vehicles, connected vehicles are not required to receive and analyze as many BSMs as they might receive if the environment had a high enough percentage of connected vehicles to enable the above-described applications and to fully benefit from V2X communications.

[0202] VBSMs can help bridge the gap between a current environment with mostly unconnected entities and a future environment with mostly connected entities, and enable the above-mentioned applications in the interim. In the technology described herein, a connected vehicle that receives a VBSM processes it as a regular BSM for the application. Because VBSMs and BSMs follow the same message structure and VBSMs contain substantially the same basic information as BSMs, such as speed, acceleration, heading, past trajectory, and predicted trajectory, the results of applying the message to a given application are substantially the same.

[0203] For example, consider an intersection with an unprotected left turn, where a connected host vehicle is attempting a left turn while an unconnected remote vehicle is traveling straight from the opposite direction with the right-of-way. This is a situation where the execution of the maneuver depends on the host vehicle's driver's judgment regarding the situation. An improper assessment of the situation can result in a conflict and a potential close encounter or collision. External sensors installed in the infrastructure of the surrounding environment can detect and track the remote vehicle or even both vehicles, collect basic information such as speed, acceleration, heading, and past trajectory, and transmit it to the RSE, which can then use rule-based algorithms and / or machine learning algorithms to construct a predicted trajectory for the remote vehicle, fill in the fields required for the VBSM, and broadcast it on behalf of the unconnected remote vehicle. The host vehicle's OBE receives the VBSM containing information about the remote vehicle and processes it in its LTA application to determine whether the driver's maneuver poses a potential hazard and whether the OBE should display a warning to the host vehicle's driver to take proactive or corrective action to avoid the collision. A similar result can also be achieved if a remote vehicle is connected and receives data from the RSE and sensors about an oncoming vehicle attempting to make a left turn that predicts a collision.

[0204] VBSM can also be used in lane-changing maneuvers. Such maneuvers can be dangerous if a vehicle changing lanes does not take the necessary steps to ensure the maneuver is safe, such as checking its rearview and side mirrors and blind spots. New advanced driver assistance systems, such as blind spot warnings using onboard ultrasonic sensors, have been developed to help prevent vehicles from making dangerous lane changes. However, these systems can have drawbacks if the sensors are dirty or their field of view is blocked. Furthermore, existing systems do not attempt to warn an endangered vehicle about another vehicle intending to change lanes. While V2X communication can help solve this problem through applications such as BS / LCW using V2X, the vehicle intending to change lanes may be an unconnected vehicle and therefore unable to communicate its intentions. VBSM can help achieve that goal. Similar to the LTA use case, external sensors installed in the surrounding environment's infrastructure may detect and track unconnected vehicles attempting to perform a lane-change maneuver, collecting basic information such as speed, acceleration, direction of travel, and past trajectory and transmitting it to the RSE. The RSE then uses rule-based and machine learning algorithms to construct a predicted trajectory for the lane-changing vehicle, fills in the required fields for the VBSM, and broadcasts it on behalf of the unconnected vehicle. The OBE of the at-risk vehicle then receives the VBSM containing information about vehicles attempting to merge into the same lane and processes it to determine whether the maneuver poses a potential hazard and whether it should display a lane-change warning to the vehicle's driver. If the lane-changing vehicle is a connected vehicle, its OBE may similarly receive VBSMs from the RSE regarding vehicles in its blind spot and determine whether the lane-change maneuver poses a potential hazard to traffic in the surrounding environment and whether it should display a blind-spot warning to the vehicle's driver.When both vehicles are connected, they can broadcast their BSM to each other, enabling BS / LCW applications, but these applications still benefit from applying the same rule-based and / or machine learning algorithms described above to the BSM data to predict the lane-changing vehicle's intent early and determine whether the OBE should display a warning.

[0205] Autonomous Vehicles

[0206] The lack of connectivity for unconnected road entities impacts autonomous vehicles. Sensors in autonomous vehicles have short ranges or narrow fields of view. They cannot detect vehicles approaching buildings, for example, on street corners. They also cannot detect vehicles that may be hidden behind delivery trucks. These hidden vehicles are invisible to autonomous vehicles if they are unconnected entities. These circumstances affect the ability of autonomous vehicle technology to achieve the level of safety necessary for mass adoption of the technology. Smart intersections can help mitigate this gap and aid in public acceptance of autonomous vehicles. Autonomous vehicles are only as good as their sensors. Intersections equipped with smart RSEs can extend the coverage of onboard sensors around blind corners or beyond large trucks. Such extensions allow autonomous and other connected entities to coexist with traditional, unconnected vehicles. Such coexistence can accelerate the adoption of autonomous vehicles and the benefits they bring.

[0207] The virtual connected ground transportation environment includes VBSM messages that enable the implementation of vehicle-to-vehicle (V2V), vehicle-to-pedestrian (V2P), and vehicle-to-device (V2D) applications that would otherwise be difficult to implement.

[0208] The system may use machine learning to quickly and accurately generate the fields of data required for various safety messages, pack them into VBSM message structures, and transmit the messages to nearby ground transportation entities using various mediums, such as, but not limited to, DSRC, Wi-Fi, cellular, or traditional road signs.

[0209] Virtual personal safety message (VPMS)

[0210] The ground traffic environment may include not only disconnected vehicles, but also disconnected people and other vulnerable road users.

[0211] In some regulatory regimes, connected vulnerable ground transportation entities continuously transmit personal safety messages (PSMs). PSMs include, among other information, the location, heading, speed, and future route of the vulnerable ground transportation entity. Connected vehicles and infrastructure can receive these messages and use them to generate maps that include vulnerable entities and increase the level of safety in the ground transportation network.

[0212] Thus, a hypothetical connected ground transportation environment may bridge the gap between a future in which most vulnerable ground transportation entities are assumed to be connected and a present in which most vulnerable ground transportation entities lack connectivity. In a hypothetical connected ground transportation environment, smart traffic lights and other infrastructure installations may use sensors (connected and unconnected) to track all vulnerable ground transportation entities and generate VPSMs on their behalf.

[0213] A VPSM message may be considered a subset of a PSM. A VPSM need not include all fields required to generate a PSM, but may include data necessary for safety assessment and prevention of unsafe situations, including location-specific information, including position, heading, speed, and trajectory. In some instances, non-standard PSM fields, such as driver intent, posture, or looking direction, may also be included in a VPSM.

[0214] The system may use machine learning to quickly and accurately generate these fields, pack them into a VPSM message structure, and transmit it to nearby ground transportation entities using various mediums, such as, but not limited to, DSRC, Wi-Fi, cellular, or traditional road signs.

[0215] VPSM messages enable the implementation of pedestrian-to-vehicle (P2V), pedestrian-to-infrastructure (P2I), pedestrian-to-device (P2D), vehicle-to-pedestrian (V2P), infrastructure-to-pedestrian (I2P), and device-to-pedestrian (D2P) applications that would be difficult to implement using other techniques.

[0216] FIG. 16 shows a pedestrian 8102 crossing a crosswalk 8103. The crosswalk 8103 may be at an intersection or at a crosswalk midway across a block spanning a stretch of road between intersections. A camera 8101 is used to monitor the sidewalk 8104. The global position of the boundary of the camera's 8101 field of view 8105 may be determined at installation. The field of view 8105 is covered by a predetermined number of pixels reflected by the camera's 8101 specifications. A road entity 8102 may be detected within the camera's field of view, and its global position may be calculated. The speed and direction of the road entity 8102 may further be determined from its movement at s points in time. The path of the road entity 8102 may be represented by a breadcrumb 8106, which is a sequence of locations crossed by the entity 8102. This data may be used to construct a virtual PSM message. The PSM message may then be broadcast to all entities near the intersection.

[0217] Traffic and behavior enforcement at unsignalized intersections

[0218] Another useful application of the system is traffic enforcement (eg, stop signs, yield signs) at intersections without traffic lights, and enforcement of good driving behavior anywhere on the surface transportation network.

[0219] As a by-product of generating VBSM and VPSM, the system may track and detect road users who do not comply with traffic laws and increase the probability of unsafe situations and collisions. Prediction of unsafe situations may be extended to include enforcement. Unsafe situations do not require collisions to occur. Near misses are common and can increase driver stress levels, leading to subsequent accidents. The frequency of near misses is positively correlated with lack of enforcement.

[0220] Additionally, using VBSM, the system can detect inappropriate driving behavior, such as abrupt lane changes and other forms of reckless driving. Data collected by the sensors can be used to train and enable machine learning models to flag ground traffic entities that are engaging in risky driving behavior.

[0221] Law enforcement authorities typically enforce the rules of the road against ground traffic entities that contain vulnerable road users, but the authorities need to be present near intersections to monitor, detect, and report violations. By using VBSM and VPSM to track unconnected ground traffic entities that contain vulnerable road users, a smart RSE can act as a law enforcement authority and enforce the rules of the road at intersections. For example, an unconnected vehicle tracked by a smart RSE can be detected, identified, and reported to authorities for violating a stop sign or a yield sign. Similarly, a vulnerable road user near an intersection tracked by a smart RSE can be detected, identified, and reported to authorities for illegally crossing the intersection.

[0222] For enforcement and other purposes, ground transportation entities may be identified using unique identification information, including but not limited to plate number recognition. Potentially victimized road users may be identified using biometric recognition, including but not limited to facial recognition, retinal recognition, and voice waveform recognition. In special cases involving civil investigations or criminal investigations, social media networks (e.g., Facebook, Instagram, Twitter) may further be used to support the identification of offending ground transportation entities or potentially victimized road users. An example of a social network being utilized is uploading captured photos of a violator to the social network and requesting social network users who recognize the violator to provide intelligence to law enforcement authorities that will help identify the violator.

[0223] Altitude SOBE

[0224] A smart RSE may use sensors and predictive models to predict dangerous situations and then send virtual safety messages (e.g., ICA, VBSM, VPSM, VICA, and VCSM (virtual combined safety message)) on behalf of unconnected road users, including potential victims. A smart OBE may use incoming virtual or standard safety messages (e.g., BSM, PSM, VBSM, VPSM, VICA, and VCSM), vehicle sensors, and predictive models to predict dangerous situations and alert the driver of the host vehicle. An advanced SOBE may do all of this and (a) (acting as a smart RSE) send virtual BSMs, virtual PSMs, virtual ICAs, and VCSMs, and standard messages, whenever applicable, on behalf of other road users, even those unconnected, and especially potential victims; (b) send advanced BSMs to its own ground traffic entities containing information based on its own behavior intent predictions; and (c) act as an RSE and send messages, such as GPS fixes.

[0225] This specification refers to BSM, PSM, ICA, VBSM, VPSM, and VICA. Other types of safety messages exist or may be developed, including the Coordinated Perception Message (CPM) under development, reported at https: / / www.sae.org / standards / content / j2945 / 8 / . References to BSM, PSM, ICA, VBSM, VPSM, and VICA are intended to further reference other existing and future safety messages, including CPMs. As suggested, for example, a CPM may contain data about multiple objects (and, in that sense, a container for these objects), such as VBSM, VPSM, and VICA. This specification sometimes refers to messages that may be containers for VBSM, VPSM, VICA, and other types of virtual safety messages, such as VCSMs. An ESOBE may generate VCSMs based on its detection of ground traffic entities and other objects using its host vehicle's sensors. VCSMs may be periodically broadcast by an ESOBE to make other ground traffic entities aware of disconnected or blocked ground traffic entities or other objects.

[0226] In this example, an on-board equipment (OBE) is described that may have advanced and additional capabilities in some respects beyond those of the OBEs and SOBEs previously described. In some implementations described herein, such an enhanced SOBE (ESOBE) goes beyond actually functioning as a smart RSE, for example, by enhancing sensors already present in the vehicle in which the ESOBE resides (the “host vehicle”) with predictive models. In some examples, the ESOBE may transmit (1) virtual safety messages on behalf of one or more other vehicles, and (2) advanced standard safety messages for the host ground traffic entity using intent predictions of its own behavior, along with (3) virtual safety messages it transmits operating in its role as an on-board RSE (e.g., GPS fixes), for example, to other vehicles or potentially affected road users. In some cases, an on-board RSE is referred to herein as an enhanced RSE (“ERSE”).

[0227] Below we describe scenarios and applications for Advanced SOBE ("ESOBE") technology, including:

[0228] 1. The ESOBE generates and broadcasts a virtual BSM on behalf of another vehicle that is, for example, skidding and is not connected. A third connected vehicle, which cannot see the skidding vehicle from its position due to an obstructed view, can then receive the virtual BSM and use it, for example, to generate an early warning of a dangerous situation to the driver of the third vehicle.

[0229] 2. The ESOBE generates and broadcasts a virtual PSM on behalf of a crossing pedestrian or other vulnerable road user, e.g., at a marked or designated crosswalk at an intersection. A third connected vehicle (including, e.g., a moving or stationary vehicle such as a public transport vehicle) from whose location the crossing vulnerable road user is not visible due to an obstructed view can then receive the virtual PSM and use it, e.g., to generate an early warning of a dangerous situation to the driver of the third vehicle.

[0230] 3. The ESOBE generates and broadcasts a virtual PSM on behalf of a crossing pedestrian or other vulnerable road user at a mid-block crosswalk rather than at an intersection. A third connected vehicle, whose position is not visible to the vulnerable crossing road user due to an obstructed view, can then receive the virtual PSM and use it, for example, to generate an early warning of a dangerous situation to the driver of the third vehicle.

[0231] 4. The ESOBE generates and broadcasts an enhanced standard BSM from its host vehicle, for example, by including in the BSM a prediction of a forward collision identified based on information received by the ESOBE from the host vehicle's onboard cameras and / or sensors. Vehicles following the host vehicle may use the enhanced BSM to generate an early warning to the driver of the following vehicle that the host vehicle may apply heavy braking within, for example, 1-2 seconds.

[0232] 5. The ESOBE generates and broadcasts GPS fixes to other ground traffic entities. In this mode, the ESOBE essentially operates as an RSE, helping to extend the coverage of RSE fixes by augmenting the OBE's own GPS. The OBE may also store and obtain the most recent GPS fixes it received from the nearest RSE it approached.

[0233] In all of these scenarios, as well as others described above and below, messages broadcast by ESOBE may include virtual messages bundled in VCSMs.

[0234] Obstructed skidding vehicles and angled collisions

[0235] FIG. 19 shows a vehicle or another ground traffic entity 1907 (e.g., not equipped with or otherwise connected to an OBE) traveling within lane 1914 and skidding across a multi-lane road 1912 into an adjacent lane 1916. The vehicle skidding may occur due to, for example, heavy braking, a slippery surface, or another reason. In this scenario, the vehicle 1906 is traveling within lane 1916 and is equipped with an ESOBE 1908 and a sensor 1918. The sensor 1918 may be, but is not limited to, a camera, radar, lidar, ultrasonic distance sensor, or the like, and combinations thereof. The ESOBE 1908 processes the data feed from the sensors in real time and generates periodic basic safety messages (BSMs) for the vehicle 1906.

[0236] A third vehicle 1902 is traveling in lane 1920. The skidding vehicle 1907, or anything in front of or partially to the side of vehicle 1906, is obscured or partially obscured from view by vehicle 1902.

[0237] Upon detecting a skidding vehicle 1907 using the vehicle's 1906 sensors 1918, the ESOBE 1908 of the vehicle 1906 determines measured parameters of the vehicle 1907 (assumed to be unaware of the OBE in this scenario). The measured parameters may include one or more of speed, heading, path history, path prediction, braking status, or others, or combinations thereof. Based on these measured parameters, the ESOBE generates and broadcasts a virtual BSM on behalf of the unaware vehicle 1907.

[0238] Based on the measured parameters determined by the ESOBE of vehicle 1906 and the resulting virtual BSM broadcast by the ESOBE of vehicle 1906, vehicle 1902 becomes aware of skidding vehicle 1907, even though neither the driver of vehicle 1902 nor on-board sensors in vehicle 1902 can see vehicle 1907 because vehicle 1907 is occluded by vehicle 106. Using this information available to the SOBE of vehicle 1902, along with its measured parameters of its own motion, including speed, direction of travel, etc., the SOBE of vehicle 1902 can predict a possible threat of collision with vehicle 1907 and alert its driver accordingly.

[0239] In addition to making it easier for vehicles in the surrounding environment (e.g., vehicle 1902 in this scenario) to become aware of unseen, unconnected vehicles, if the ESOBE of vehicle 1906 detects the possibility of an angled collision between vehicle 1907 and vehicle 1902 within lane 1920 by predicting a skid path for vehicle 1907, the ESOBE of vehicle 1906 may generate and transmit an Intersection Collision Avoidance Message (ICA). If vehicle 1902 is capable of receiving and handling ICA messages, vehicle 1902 may receive them, process them, and alert its driver of a possible angled (intersection) collision.

[0240] In typical DSRC (dedicated short range communication), by contrast, ICAs are generated and transmitted only at road intersections to warn drivers about angled collisions (e.g., when one vehicle is predicted to cross the path of another vehicle). In the technology described herein, because the ESOBE comprises a predictive algorithm, ICAs can be triggered, generated, and transmitted from the ESOBE of a moving vehicle even at locations other than road intersections. Such ICAs can be used to warn drivers and enable them to avoid even angled collisions that may occur on straight road segments, as described above.

[0241] Obstructed pedestrians: Crossing a crosswalk

[0242] In an implementation according to this scenario, the ESOBE may act as a broadcaster of the virtual PSM, for example, on behalf of pedestrians or other potentially vulnerable road users.

[0243] As shown in Figure 20, in this scenario, pedestrian 2000 is crossing a two-lane road 2012 at crosswalk 2010. Vehicle 2006 traveling in lane 2014 blocks (obstructs pedestrian 2000's view of) pedestrian 2000 from vehicle 2002 traveling in lane 2016. Zone 2004 indicates the range of awareness (visibility) of the driver and vehicle 2002's sensors. Vehicle 2006 limits zone 2004, blocking anything behind vehicle 2006 (relative to vehicle 2002) from being seen by vehicle 2002. If vehicle 2002 continues to move without awareness of pedestrian 2000's position, speed, and direction of travel, a serious accident could occur.

[0244] Bus 2006 includes ESOBE 2008 and sensor 2018. Sensor 2018 may be, but is not limited to, a camera, radar, lidar, ultrasonic distance sensor, or the like, and combinations thereof. ESOBE 2008 processes the simultaneous data feeds of sensor 2018 in real time. When ESOBE 2008 detects pedestrian 2000 using the sensor data, ESOBE 2008 automatically begins broadcasting a virtual PSM message on behalf of pedestrian 2000, which can be received by vehicle 2002. As a result, vehicle 2002 will recognize pedestrian 2000 crossing lane 2014 even if neither the driver nor the onboard sensors of vehicle 2002 can see (recognize) pedestrian 2000 on the other side of vehicle 2006.

[0245] Obstructed Pedestrian: Mid-Block Crosswalk

[0246] This scenario is similar to the previous one, except in this case the pedestrian is crossing a road mid-block, away from a crosswalk or other formal road intersection.

[0247] As shown in Figure 21, a pedestrian 2100 is crossing the road 2112 in front of a vehicle 2106. The pedestrian 2100 is blocked from the view of the driver and the sensors of the vehicle 2102. The recognition zone of the vehicle 2102 is indicated by the dotted area 2104.

[0248] The vehicle 2106 is equipped with an ESOBE 2108 that can transmit and receive V2X messages, such as PSM and BSM. The ESOBE 2108 is further capable of processing simultaneous data feeds from sensors 2118.

[0249] After being detected by ESOBE 2108, information including but not limited to the pedestrian's 2100 global position, speed, and heading is encoded into a virtual PSM message, and the virtual PSM message is broadcast. The vehicle 2102 receives the virtual PSM message and thus recognizes the pedestrian 2100. The algorithmic on-board vehicle 2102 can predict whether an imminent dangerous situation exists and take appropriate action, including but not limited to alerting the driver or automatically slowing or stopping the vehicle.

[0250] Obstructed Pedestrian: Forward Collision

[0251] 22 shows a pedestrian 2400 crossing a lane road 2412 at a crosswalk 24102. A larger vehicle (e.g., a truck) 2406 is traveling in lane 2414 and is equipped with an ESOBE 2408 and a sensor 2418. The sensor 2418 can be, but is not limited to, a camera, radar, lidar, ultrasonic distance sensor, etc., and combinations thereof. The ESOBE 2408 processes the data feed from the sensor in real time.

[0252] Another vehicle 2402 is traveling behind a larger vehicle 2406 in the same lane 2414. A pedestrian 2400 or other object in front of the vehicle 2406 is obstructing the view of the vehicle 2402.

[0253] If vehicle 2406 suddenly has to brake due to the presence of pedestrian 2400 (which vehicle 2402 is unaware of), vehicle 2402 may collide with the rear end of vehicle 2406 .

[0254] Upon detecting the pedestrian 2400 using the sensor 2418, the ESOBE 2408 of the vehicle 2406 immediately begins broadcasting a virtual PSM on behalf of the pedestrian 2400 in addition to the normal basic safety message (BSM) that it transmits for itself. As a result, together with knowledge of the presence of the vehicle 2406 (from the BSM), the vehicle 2402 is further aware of the pedestrian 2400 crossing the lane 2414 (from the virtual PSM), even if the driver and on-board sensors of the vehicle 2402 cannot see or detect the pedestrian 2400 beyond the vehicle 2406.

[0255] The ESOBE in vehicle 2406 implements artificial intelligence processes that learn from its driver's behavior in various situations. For example, upon detecting pedestrian 2400, ESOBE 2408 applies AI algorithms to predict whether vehicle driver 2406 is going to apply the brakes and to predict the future point in time at which this might occur. Based on these predictions, the ESOBE may decide to add braking information to the BSM message it broadcasts to other vehicles earlier than would occur under a typical system. This earlier delivery of the braking message may give other vehicles, such as vehicle 2402, more time to predict a collision in advance. In other words, other vehicles may benefit from the AI ​​capabilities in the ESOBE of the first vehicle.

[0256] ESOBE - Location Correction Service

[0257] The availability of high-precision GPS position data is important for DSRC-based V2X safety applications, or other applications that assume or require higher than typical GPS accuracy. ESOBE can provide GPS fix information to increase the accuracy of GPS position data for nearby ground traffic entities. As shown in Figure 23, four vehicles (2502, 2503, 2504, and 2505) traveling in a given direction on different lanes of a road may be able to use GPS fix data, such as in the scenario described below.

[0258] In this scenario, a vehicle (e.g., a truck or bus) 2506 is traveling in one lane 2516 and is equipped with an ESOBE 2508 capable of transmitting and receiving V2X messages. Although the vehicle 2506 is shown as a large vehicle, the vehicle 2506 can be any ground transportation entity of any size.

[0259] Assume that (a) there are no RSEs present in the vicinity of this area, (b) vehicle 2506 (including its ESOBE) has recently passed through an area where an RSE with differential GNSS (global navigation satellite system) broadcast capability via DSRC (e.g., using technologies such as RTK, DGPS, or Wide Area RTK) was present, (c) vehicle 2506 is no longer in the vicinity of the RSE, and (d) vehicle 2506 (and its ESOBE) has passed through an area where the particular RSE was present and where the particular RSE was transmitting periodic RTCM (Radio Technical Commission for Maritime Affairs) correction messages.

[0260] Upon receiving these RTCM correction messages, the ESOBE 2508 in the vehicle 2506 corrected its own position and stored the RTCM correction data for further use. While a particular RSE was within range, the ESOBE continued to correct its position using the received correction messages and update these messages for further use.

[0261] After an ESOBE moves out of the coverage area of ​​a particular RSE, it uses the stored correction data to construct a newer correction message based on its current location and provides a correction service (based on the newer correction message) to other vehicles (2502, 2503, 2504, 2505 in FIG. 23) similar to the correction service provided by the RSE if the RSE were within range. The ESOBE's distance from the particular RSE from which the correction data was collected and the ESOBE's current location information can be used to update the correction messages (e.g., generate newer correction messages) before rebroadcasting them as part of the correction service. In this manner, the ESOBE effectively operates as a base station providing accurate RTCM correction data for other road users in areas where no RSE is within range. Thus, the ESOBE not only forwards correction data from the RSE or other external sources, but also updates the correction data based on the host vehicle's current location and other factors mentioned in this example.

[0262] Additionally, ESOBE runs an algorithm to determine the accuracy of the reconstructed correction data before broadcasting it, and only broadcasts the correction data if the algorithm determines a very high level of confidence in the recovered correction data.

[0263] In some embodiments, the algorithm for retransmission of corrected data may take into account, among other things, the following information: The time period since ESOBE received correction data from the RSE, the distance traveled from the RSE, the direction of travel (heading) to determine whether the correction still applies at the current vehicle position, and the confidence level of the original correction data received from the RSE, or any combination thereof. Consider:

[0264] In some examples, another feature of ESOBE is that if ESOBE has direct access to an external service that transmits an RTCM correction feed over the Internet, ESOBE can generate DSRC RTCM correction messages independently using this feed. ESOBE may decide to switch to this mode in scenarios where no RSE is present or if the reliability level of the data received from the RSE is not adequate. In this example, ESOBE has the intelligence to select a better source of correction information.

[0265] Among the benefits of ESOBE, its ability to construct or reconstruct itself and transmit correction messages received from RSEs assists ground traffic entities with low-performance, inexpensive GPS devices in correcting their positions, helps ground traffic entities execute safety algorithms more reliably, extends the effective coverage area of ​​RSEs, and broadcasts RTCM corrections over DSRC networks or other short-range vehicular networks using V2X standard correction messages.

[0266] Other embodiments are within the scope of the following claims. (Additional note 1) 1. An apparatus comprising equipment for use onboard a first ground transportation entity, said equipment comprising: (a) a receiver for information generated by a sensor of the first ground transportation entity's surrounding environment; (b) a processor; (c) a memory storing instructions executable by the processor to generate and transmit safety message information to a second ground transportation entity based on the information generated by one or more of the sensors; Including, Device. (Additional note 2) the instructions are executable by the processor to generate a prediction for use in generating the safety message information. Item 1. The device of item 1. (Additional note 3) the prediction is generated by a predictive model; Item 2. The device according to item 2. (Additional note 4) the predictive model is configured to predict a hazardous situation involving one or more of the first ground transportation entities, one or more of the second ground transportation entities, or one or more other ground transportation entities; Item 3. The device according to item 3. (Additional note 5) the hazardous condition involves the crossing of a lane of a roadway by one or more of the second ground transportation entities; Item 4. The device according to item 4. (Additional note 6) the one or more of the second ground transportation entities include one or more vehicles, and the hazardous situation includes a skid across the lanes by the one or more vehicles; Item 5. The device according to item 5. (Additional note 7) the one or more second ground transportation entities include one or more pedestrians crossing a roadway or one or more other potentially injured road users; Item 5. The device according to item 5. (Additional note 8) one or more vulnerable road users crossing the road at an intersection; Item 7. The device according to item 7. (Additional note 9) the vulnerable road user crosses the road at a location other than an intersection; Item 7. The device according to item 7. (Additional note 10) the predicted dangerous situation includes a predicted collision between a third ground traffic entity and the second ground traffic entity; Item 4. The device according to item 4. (Additional note 11) the first ground traffic entity includes a vehicle and the second ground traffic entity includes a pedestrian or other vulnerable road user; Item 3. The device according to item 3. (Additional note 12) the third ground traffic entity follows the first ground traffic entity, and the view of the third ground traffic entity is obstructed by the first ground traffic entity; The device described in appended paragraph 10. (Additional note 13) the third ground transportation entity is in a lane adjacent to the lane in which the first ground transportation entity is traveling; The device described in appended paragraph 10. (Additional note 14) the instructions are executable by the processor to identify operational parameters of a third ground transportation entity. Item 1. The device of item 1. (Additional note 15) the second ground traffic entity has only an obstructed view of the third ground traffic entity; Item 3. The device according to item 3. (Additional note 16) the second ground transportation entity includes a pedestrian or other vulnerable road user; Item 3. The device according to item 3. (Additional note 17) the safety message information transmitted by the processor includes a basic safety message; Item 1. The device of item 1. (Additional note 18) the safety message information transmitted by the processor includes a virtual basic safety message; Item 1. The device of item 1. (Additional note 19) the safety message information transmitted by the processor includes a personal safety message; Item 1. The device of item 1. (Additional note 20) the safety message information transmitted by the processor includes a virtual personal safety message; Item 1. The device of item 1. (Additional note 21) the safety message information transmitted by the processor includes a virtual basic safety message transmitted on behalf of a third ground transportation entity; Item 1. The device of item 1. (Additional note 22) the third ground transportation entity comprises an unconnected ground transportation entity; 22. The device according to claim 21. (Additional note 23) the safety message information transmitted by the processor includes a virtual personal safety message transmitted on behalf of a third ground transportation entity; Item 1. The device of item 1. (Additional note 24) the equipment includes the receiver for information wirelessly transmitted from a source external to the first ground transportation entity; Item 1. The device of item 1. (Additional note 25) the first ground transportation entity; Item 1. The device of item 1. (Additional note 26) the safety message information includes a Virtual Intersection Collision Avoidance (VICA) message; Item 1. The device of item 1. (Additional note 27) the safety message information includes an intersection collision avoidance message (ICA); Item 1. The device of item 1. (Additional note 28) the safety message information includes a virtual combined safety message (VCSM); Item 1. The device of item 1. (Additional note 29) the safety message information includes a combined safety message (CSM); Item 1. The device of item 1. (Additional note 30) 1. An apparatus comprising equipment for use onboard a first ground transportation entity, said equipment comprising: (a) a receiver for first position fix information transmitted from a source external to the first ground transportation entity; (b) the receiver for information representing a location parameter or an operational parameter of the first ground transportation entity; (c) a processor; (d) a memory storing instructions executable by the processor to generate updated position correction information based on the first position correction information and the information representing the operating parameter, and to send a position correction message to another ground transportation entity based on the updated position correction information; Including, Device. (Additional note 31) the position fix information transmitted from the source external to the first ground transportation entity includes the position fix message; The device according to claim 30. (Additional note 32) the position correction information transmitted from the source external to the first ground transportation entity comprises a Radio Technical Commission for Maritime (RTCM) correction message; The device according to claim 30. (Additional note 33) the position fix information includes a GNSS position fix; The device according to claim 30. (Additional note 34) the location parameters or the operational parameters include a current location of the first ground transportation entity; The device according to claim 30. (Additional note 35) the source external to the first ground transportation entity includes a RSE or an external service configured to transmit the position fix message over the Internet; The device according to claim 30. (Additional note 36) the instructions are executable by the processor to verify a confidence level in the updated position correction information. The device according to claim 30. (Additional note 37) receiving information generated by a sensor on board a first ground transportation entity regarding an environment surrounding the first ground transportation entity; generating and transmitting safety message information to a second ground transportation entity based on the information generated by the sensor; A method comprising: (Additional note 38) generating a prediction for use in generating the safety message information. The method described in appended item 37. (Additional note 39) the prediction is generated by a predictive model; The method described in appended item 38. (Additional note 40) the predictive model is configured to predict a hazardous situation involving the first ground transportation entity, the second ground transportation entity, or another ground transportation entity; The method described in appended item 39. (Additional note 41) the hazardous condition involves the second ground transportation entity crossing a lane of a roadway; The method described in appended paragraph 40. (Additional note 42) the second ground transportation entity includes a vehicle, and the hazardous situation includes a cross-lane skid by the vehicle; The method described in appendix 41. (Additional note 43) the second ground transportation entity includes a pedestrian crossing a road or other vulnerable road user; The method described in appendix 41. (Additional note 44) the vulnerable road user crosses the road at an intersection; The method described in appendix 43. (Additional note 45) the vulnerable road user crosses the road other than at an intersection; The method described in appendix 43. (Additional note 46) the predicted dangerous situation includes a predicted collision between a third ground traffic entity and the second ground traffic entity; The method described in appended paragraph 40. (Additional note 47) the first ground traffic entity includes a vehicle and the second ground traffic entity includes a pedestrian or other vulnerable road user; The method described in appended paragraph 40. (Additional note 48) the third ground traffic entity follows the first ground traffic entity, and the first ground traffic entity blocks the view of the third ground traffic entity; The method described in appended item 46. (Additional note 49) the third ground transportation entity is in a lane adjacent to the lane in which the first ground transportation entity is traveling; The method described in appended item 46. (Additional note 50) identifying operational parameters of a third ground transportation entity; The method described in appended item 37. (Additional note 51) the second ground traffic entity has only an obstructed view of the third ground traffic entity; The method described in appended item 46. (Additional note 52) the second ground transportation entity includes a pedestrian or other vulnerable road user; The method described in appended item 37. (Additional note 53) the safety message information includes a basic safety message; The method described in appended item 37. (Additional note 54) the safety message information includes a virtual basic safety message; The method described in appended item 37. (Additional note 55) the safety message information includes a personalized safety message; The method described in appended item 37. (Additional note 56) the safety message information includes a virtual personal safety message; The method described in appended item 37. (Additional note 57) the safety message information includes a virtual basic safety message transmitted on behalf of a third ground transportation entity; The method described in appended item 37. (Additional note 58) the third ground transportation entity includes an unconnected ground transportation entity; The method described in appended item 53. (Additional note 59) the safety message information transmitted by the processor includes a virtual personal safety message transmitted on behalf of a third ground transportation entity; The method described in appended item 53. (Additional note 60) receiving wirelessly transmitted information from a source external to the first ground transportation entity; The method described in appended item 37. (Additional note 61) the safety message information includes a Virtual Intersection Collision Avoidance (VICA) message; The method described in appended item 37. (Additional note 62) the safety message information includes an intersection collision avoidance message (ICA); The method described in appended item 37. (Additional note 63) the safety message information includes a virtual combined safety message (VCSM); Item 38. The device described in appended item 37. (Additional note 64) the safety message information includes a combined safety message (CSM); Item 38. The device described in appended item 37. (Additional note 65) receiving first position fix information transmitted from a source external to the first ground transportation entity; receiving information representative of an operational parameter of the first ground transportation entity; generating updated position correction information based on the first position correction information and the information representing the operational parameter; transmitting a position correction message to another ground transportation entity based on the updated position correction information; and A method comprising: (Additional note 66) the position fix information transmitted from the source external to the first ground transportation entity includes the position fix message; The method described in appended paragraph 65. (Additional note 67) the position correction information transmitted from the source external to the first ground transportation entity comprises a Radio Technical Commission for Maritime (RTCM) correction message; The method described in appended paragraph 65. (Additional note 68) the position fix information includes a GNSS position fix; The method described in appended paragraph 65. (Additional note 69) the operational parameters include a current location of the first ground transportation entity; The method described in appended paragraph 65. (Additional note 70) the source external to the first ground transportation entity includes an RSE or an external service configured to transmit a position fix message over the Internet; The method described in appended paragraph 65. (Additional note 71) ascertaining a confidence level in the updated position fix information; The method described in appended paragraph 65.

Claims

1. 1. An apparatus comprising: equipment for use onboard a first connected ground transportation entity in a connected ground transportation environment, said equipment comprising: (a) a receiver for information generated by a sensor of the first connected ground transportation entity's surrounding environment; (b) a processor; (c) a memory, generating safety message information on behalf of a second, unconnected ground transportation entity in the surrounding environment of the first connected ground transportation entity, the safety message information including a virtual safety message reconstructed from the information generated by the sensors in the surrounding environment of the first connected ground transportation entity, the virtual safety message having the same message structure as a safety message broadcast between connected ground transportation entities in the connected ground transportation environment; transmitting the safety message information to a third connected ground transportation entity in the surrounding environment of the first connected ground transportation entity on behalf of the second unconnected ground transportation entity; detecting a fourth ground transportation entity in proximity to the first connected ground transportation entity based at least on the information generated by the sensor; generating, based on detecting the fourth ground transportation entity, (i) a prediction of a future change in behavior of the first connected ground transportation entity, and (ii) a prediction of a future time point at which the predicted future change in behavior will occur; generating a second safety message, the second safety message including a prediction of the future behavior change and a prediction of the future time point; transmitting the second safety message to the third connected ground transportation entity or to a fifth ground transportation entity; the memory storing instructions executable by the processor to perform operations including: Including, Device.

2. The operation is generating, based on the information generated by the sensors of the surrounding environment of the first connected ground transportation entity, a prediction of a hazardous situation in the surrounding environment of the first connected ground transportation entity for use in generating the safety message information; Further comprising:

10. The apparatus of claim 1.

3. the prediction of the hazardous situation is generated by a predictive model; 3. The apparatus of claim 2.

4. the hazardous situation involves one or more of the first connected ground transportation entity, the second unconnected ground transportation entity, or one or more other ground transportation entities, including the third connected ground transportation entity; 3. The apparatus of claim 2.

5. the hazardous condition involves the crossing of a lane of a roadway by the second unconnected ground transportation entity; 5. The apparatus of claim 4.

6. the second unconnected ground transportation entity includes a vehicle; the dangerous situation includes the vehicle skidding across the lane; 6. The apparatus of claim 5.

7. the second unconnected ground transportation entity includes a pedestrian crossing a road or other vulnerable road user; 6. The apparatus of claim 5.

8. the vulnerable road user crosses the road at an intersection; 8. The apparatus of claim 7.

9. the vulnerable road user crosses the road at a location other than an intersection; 8. The apparatus of claim 7.

10. the predicted dangerous situation includes a predicted collision between the third connected ground transportation entity and the second unconnected ground transportation entity; 10. The apparatus of claim 1.

11. the first connected ground transportation entity includes a vehicle; the second unconnected ground transportation entity includes a pedestrian or other vulnerable road user; 10. The apparatus of claim 1.

12. the third connected ground transportation entity follows the first connected ground transportation entity; the view of the third connected ground transportation entity is obstructed by the first connected ground transportation entity; 10. The apparatus of claim 1.

13. the third connected ground transportation entity is in a lane adjacent to the lane in which the first connected ground transportation entity is traveling; 10. The apparatus of claim 1.

14. the instructions are executable by the processor to identify operational parameters of the third connected ground transportation entity.

10. The apparatus of claim 1.

15. the second unconnected ground transportation entity has only an obstructed view of the third connected ground transportation entity; 10. The apparatus of claim 1.

16. the second unconnected ground transportation entity includes a pedestrian or other vulnerable road user; 10. The apparatus of claim 1.

17. the safety messages broadcast among connected ground transportation entities in the connected ground transportation environment include basic safety messages; The basic safety message is a vehicle-to-vehicle safety message broadcast from a connected remote vehicle to a connected host vehicle.

10. The apparatus of claim 1.

18. the safety message information transmitted by the processor includes a virtual basic safety message; the virtual basic safety message is a vehicle-to-vehicle safety message reconstructed from data received from the sensors; 10. The apparatus of claim 1.

19. the virtual basic safety message is a subset of the basic safety messages; the basic safety messages include vehicle-to-vehicle safety messages broadcast from connected remote vehicles to a connected host vehicle; 20. The apparatus of claim 18.

20. the virtual basic safety message includes location information including the position, heading, speed, and trajectory of the second unconnected ground transportation entity; 20. The apparatus of claim 19.

21. the safety messages broadcast among connected ground transportation entities in the connected ground transportation environment include personal safety messages; The personal safety message is a vehicle-to-pedestrian safety message broadcast between the vehicle and the pedestrian.

10. The apparatus of claim 1.

22. the safety message information transmitted by the processor includes a virtual personal safety message; the virtual personal safety message is a vehicle-to-pedestrian safety message reconstructed from data received from the sensors; 10. The apparatus of claim 1.

23. the safety message information is transmitted to the third connected ground transportation entity in the surrounding environment of the first connected ground transportation entity on behalf of the second unconnected ground transportation entity using a standardized and anonymized communication protocol; 10. The apparatus of claim 1.

24. the equipment including a receiver for information wirelessly transmitted from a source external to the first connected ground transportation entity; 10. The apparatus of claim 1.

25. the first connected ground transportation entity; 10. The apparatus of claim 1.

26. the safety message information includes a Virtual Intersection Collision Avoidance (VICA) message; 10. The apparatus of claim 1.

27. the safety message information includes an Intersection Collision Avoidance (ICA) message; 10. The apparatus of claim 1.

28. the safety message information includes a virtual combined safety message (VCSM); the virtual combined safety message comprises a bundle of safety messages reconstructed from data received from the sensors; 10. The apparatus of claim 1.

29. the safety message information includes a combined safety message (CSM); the combined safety message comprises a combination of safety messages; the safety messages include vehicle-to-vehicle safety messages broadcast between connected vehicles and vehicle-to-pedestrian safety messages broadcast between vehicles and pedestrians; 10. The apparatus of claim 1.

30. receiving information generated by a sensor onboard a first connected ground transportation entity regarding an environment surrounding the first connected ground transportation entity in the connected ground transportation environment; generating safety message information on behalf of a second, unconnected ground transportation entity in the surrounding environment of the first connected ground transportation entity, the safety message information including a virtual safety message reconstructed from the information generated by the sensors in the surrounding environment of the first connected ground transportation entity, the virtual safety message having the same message structure as a safety message broadcast between connected ground transportation entities in the connected ground transportation environment; transmitting the safety message information to a third connected ground transportation entity in the surrounding environment of the first connected ground transportation entity on behalf of the second unconnected ground transportation entity; detecting a fourth ground transportation entity in proximity to the first connected ground transportation entity based at least on the information generated by the sensor; generating, based on detecting the fourth ground transportation entity, (i) a prediction of a future change in behavior of the first connected ground transportation entity, and (ii) a prediction of a future time point at which the predicted future change in behavior will occur; generating a second safety message, the second safety message including a prediction of the future behavior change and a prediction of the future time point; transmitting the second safety message to the third connected ground transportation entity or to a fifth ground transportation entity; A method comprising:

31. generating, based on the information generated by the sensors of the surrounding environment of the first connected ground transportation entity, a prediction of a hazardous situation in the surrounding environment of the first connected ground transportation entity for use in generating the safety message information.

31. The method of claim 30.

32. the prediction of the hazardous situation is generated by a predictive model; 32. The method of claim 31 .

33. the hazardous situation involves one or more of the first connected ground transportation entity, the second unconnected ground transportation entity, or one or more other ground transportation entities, including the third connected ground transportation entity; 32. The method of claim 31 .

34. the hazardous condition involves the crossing of a lane of a roadway by the second unconnected ground transportation entity; 34. The method of claim 33.

35. the second unconnected ground transportation entity includes a vehicle; the dangerous situation includes the vehicle skidding across the lane; 35. The method of claim 34.

36. the second unconnected ground transportation entity includes a pedestrian crossing a road or other vulnerable road user; 35. The method of claim 34.

37. the vulnerable road user crosses the road at an intersection; 37. The method of claim 36.

38. the vulnerable road user crosses the road other than at an intersection; 37. The method of claim 36.

39. the predicted dangerous situation includes a predicted collision between the third connected ground transportation entity and the second unconnected ground transportation entity; 31. The method of claim 30.

40. the first connected ground transportation entity includes a vehicle; the second unconnected ground transportation entity includes a pedestrian or other vulnerable road user; 31. The method of claim 30.

41. the third connected ground transportation entity follows the first connected ground transportation entity; the first connected ground transportation entity obstructs the view of the third connected ground transportation entity; 31. The method of claim 30.

42. the third connected ground transportation entity is in a lane adjacent to the lane in which the first connected ground transportation entity is traveling; 31. The method of claim 30.

43. determining an operational parameter of the third connected ground transportation entity; 31. The method of claim 30.

44. the second unconnected ground transportation entity has only an obstructed view of the third connected ground transportation entity; 31. The method of claim 30.

45. the second unconnected ground transportation entity includes a pedestrian or other vulnerable road user; 31. The method of claim 30.

46. the safety messages broadcast among connected ground transportation entities in the connected ground transportation environment include basic safety messages; The basic safety message is a vehicle-to-vehicle safety message broadcast from a connected remote vehicle to a connected host vehicle.

31. The method of claim 30.

47. the safety message information includes a virtual basic safety message; the virtual basic safety message is a vehicle-to-vehicle safety message reconstructed from data received from the sensors; 31. The method of claim 30.

48. the virtual basic safety message is a subset of the basic safety messages; the basic safety messages include vehicle-to-vehicle safety messages broadcast from connected remote vehicles to a connected host vehicle; 48. The method of claim 47.

49. the virtual basic safety message includes location information including the position, heading, speed, and trajectory of the second unconnected ground transportation entity; 49. The method of claim 48.

50. the safety messages broadcast among connected ground transportation entities in the connected ground transportation environment include personal safety messages; The personal safety message is a vehicle-to-pedestrian safety message broadcast between the vehicle and the pedestrian.

31. The method of claim 30.

51. the safety message information includes a virtual personal safety message; the virtual personal safety message is a vehicle-to-pedestrian safety message reconstructed from data received from the sensors; 31. The method of claim 30.

52. the safety message information is transmitted to the third connected ground transportation entity in the surrounding environment of the first connected ground transportation entity on behalf of the second unconnected ground transportation entity using a standardized and anonymized communication protocol; 31. The method of claim 30.

53. receiving wirelessly transmitted information from a source external to the first connected ground transportation entity; 31. The method of claim 30.

54. the safety message information includes a Virtual Intersection Collision Avoidance (VICA) message; 31. The method of claim 30.

55. the safety message information includes an Intersection Collision Avoidance (ICA) message; 31. The method of claim 30.

56. the safety message information includes a virtual combined safety message (VCSM); the virtual combined safety message comprises a bundle of safety messages reconstructed from data received from the sensors; 31. The method of claim 30.

57. the safety message information includes a combined safety message (CSM); the combined safety message comprises a combination of safety messages; the safety messages include vehicle-to-vehicle safety messages broadcast between connected vehicles and vehicle-to-pedestrian safety messages broadcast between vehicles and pedestrians; 31. The method of claim 30.

Images