Method for a vehicle, vehicle and storage medium

Abstract

Methods, vehicles, and storage media for vehicles are provided. In addition, techniques for determining a priority order at a multi-lane stop are described. In embodiments, a track is assigned an identification and a new track is compared to an old track. The new track is matched to the old track based on one or more factors. The identification of the new track is reassigned to the identification of the old track, where the new track is determined to be matched to the old track based on the one or more factors. An earliest time for an agent to appear is determined based on the identification and taking into account a perception occlusion area. A priority order for navigating through the intersection is determined based on local rules, the identification, and the earliest time for the agent to appear, and the vehicle travels through the multi-lane stop intersection according to the priority order.

Description

Technical Field

[0001] This specification relates to priority determination. More specifically, this technology relates to priority determination at multi-way stops. Background Technology

[0002] Autonomous vehicles can be used to transport people and / or goods (e.g., packages, objects, or other items) from one location to another. For example, an autonomous vehicle can navigate to a person's location, wait for the person to board, and navigate to a designated destination (e.g., a location chosen by the person). To navigate in their environment, these autonomous vehicles are equipped with various types of sensors to detect objects in their vicinity. Summary of the Invention

[0003] According to one aspect of the invention, a method for a vehicle includes: using at least one processor to assign an identifier to a track, wherein the track corresponds to an agent observed by the vehicle when approaching a multi-stop intersection; using the at least one processor to compare a new track with an old track, wherein the new track is matched with the old track based on one or more factors; using the at least one processor to reassign an identifier of the new track to an identifier of the old track, wherein the match between the new track and the old track is determined based on the one or more factors; using the at least one processor... The processor determines the earliest time an agent appears based on an identifier and taking into account perceived occlusion areas; using the at least one processor, it determines a priority order for navigating through an intersection based on local rules, the identifier, and the earliest time an agent appears, wherein the agents are classified as one of a previous agent, a concurrent agent, and a subsequent agent related to the time the vehicle arrives at the stopping point; and using the at least one processor, it causes the vehicle to proceed through the multiple-stop intersection according to the priority order, wherein the local rules determine the ranks of agents passing through the multiple-stop intersection.

[0004] According to another aspect of the invention, a vehicle includes: at least one computer-readable medium storing computer-executable instructions; at least one processor communicatively coupled to at least one device and configured to execute the computer-executable instructions, the execution being performed in accordance with the method described above.

[0005] According to another aspect of the invention, at least one non-transitory storage medium stores instructions that, when executed by at least one processor, cause the at least one processor to perform the above-described method. Attached Figure Description

[0006] Figure 1An example of an autonomous vehicle (AV) with autonomous capabilities is shown.

[0007] Figure 2 An example "cloud" computing environment is shown.

[0008] Figure 3 The computer system is shown.

[0009] Figure 4 An example architecture for AV is shown.

[0010] Figure 5 Examples of inputs and outputs that a sensing system can use are shown.

[0011] Figure 6 An example of a LiDAR system is shown.

[0012] Figure 7 The image shows a LiDAR system in operation.

[0013] Figure 8 Additional details on the operation of the LiDAR system are shown.

[0014] Figure 9 A block diagram illustrating the relationship between the inputs and outputs of the planning system.

[0015] Figure 10 A block diagram showing the inputs and outputs of the control system is provided.

[0016] Figure 11 A block diagram showing the controller's inputs, outputs, and components is provided.

[0017] Figure 12A This is a diagram illustrating multi-path stopping.

[0018] Figure 12B This is a system block diagram that enables the re-identification of previously observed agents.

[0019] Figure 12C This is a system block diagram that enables the handling of rolling stops.

[0020] Figure 13 It is based on the timing diagram of this technology

[0021] Figure 14 This is a diagram illustrating the first scenario where the AV channel is nearing a multi-channel stop.

[0022] Figure 15 This is a diagram illustrating the second scenario where the AV is close to multiplex stop.

[0023] Figure 16 This is a diagram illustrating the third scenario where the AV input approaches a multi-channel stop.

[0024] Figure 17 This is a diagram illustrating the fourth scenario where the AV channel is nearing a multi-channel stop.

[0025] Figures 18A to 18C Example of a scenario where scrolling stops.

[0026] Figure 19 It is a process flow chart used to determine the priority of processes. Detailed Implementation

[0027] In the following description, numerous specific details are set forth for purposes of explanation in order to provide a thorough understanding of this disclosure. However, it will be apparent that this disclosure may be practiced without these specific details. In other instances, well-known constructions and apparatuses are shown in block diagram form to avoid unnecessarily obscuring this disclosure.

[0028] In the accompanying drawings, for ease of description, a specific arrangement or order of schematic elements (such as those representing devices, modules, systems, instruction blocks, and data elements) is shown. However, those skilled in the art will understand that the specific order or arrangement of the schematic elements in the drawings is not intended to imply a requirement for a particular processing order or sequence, or a separation of processing procedures. Furthermore, the inclusion of schematic elements in the drawings is not intended to imply that such elements are required in all embodiments, nor is it intended to imply that features represented by such elements cannot be included in some embodiments or cannot be combined with other elements in some embodiments.

[0029] Furthermore, in the accompanying drawings, connecting elements, such as solid or dashed lines or arrows, are used to illustrate connections, relationships, or associations between two or more other schematic elements. The absence of any such connecting element does not imply that connections, relationships, or associations cannot exist. In other words, connections, relationships, or associations between some elements are not shown in the drawings so as not to obscure the content of this disclosure. Additionally, for ease of illustration, a single connecting element is used to represent multiple connections, relationships, or associations between elements. For example, if a connecting element represents communication of signals, data, or instructions, those skilled in the art will understand that such an element represents one or more signal paths (e.g., a bus) that may be necessary to affect the communication.

[0030] Reference will now be made in detail to embodiments, examples of which are illustrated in the accompanying drawings. Numerous specific details are set forth in the following detailed description in order to provide a thorough understanding of the various embodiments described. However, it will be apparent to those skilled in the art that the various embodiments described can be practiced without these specific details. In other instances, well-known methods, procedures, components, circuits, and networks have not been described in detail so as not to unnecessarily obscure aspects of the embodiments.

[0031] The features described below can each be used independently of each other or in any combination with other features. However, any individual feature may not solve any of the problems discussed above, or may only solve one of the problems discussed above. Some of the problems discussed above may not be adequately solved by any of the features described herein. Although headings are provided, information relating to specific headings but not found in the sections bearing those headings can be found elsewhere in this specification. Embodiments are described herein based on the following summary:

[0032] 1. General Overview

[0033] 2. System Overview

[0034] 3. AV Architecture

[0035] 4. AV Input

[0036] 5. AV Planning

[0037] 6. AV Control

[0038] 7. Prioritize

[0039] 8. Markings after previous blockages

[0040] 9. Labeling, blocking, and relabeling

[0041] 10. Parse the intermediate ID

[0042] 11. Unresolved intermediate ID

[0043] 12. Has priority order for scrolling to stop.

[0044] 13. Processing flow for prioritization

[0045] General Overview

[0046] Navigation in a multi-stop environment involves determining the order in which agents arrive at an intersection. This order is called the arrival order. In the case of a multi-stop environment, the arrival order (collectively referred to as the priority order) can define which agents have the right of way relative to other agents to proceed through the intersection. Agents at the intersection (e.g., vehicles, cyclists) are sensed and a tracking identifier (ID) is assigned to new agents, while previously identified agents that have disappeared from perception (e.g., are no longer perceived) are re-identified when perception returns. The priority order can be derived, at least in part, based on the unique identifier assigned to the agents. When a vehicle has the highest priority, it is expected to arrive at the intersection before vehicles with lower priority, based on local traffic regulations (e.g., when the First-In-First-Out (FIFO) rule is applied as the highest priority determination rule, the first vehicle to arrive at the intersection has the right-of-way over the second arriving vehicle, the second arriving vehicle has the right-of-way over the third arriving vehicle, and so on; depending on local circumstances, other priority determination rules may be applied as a tie breaker, or as a higher priority determination rule that overturns the FIFO order).

[0047] Some advantages of these technologies include minimizing the impact of congestion on navigation through intersections. Furthermore, this technology is applicable to agents that do not come to a complete stop at intersections, such as vehicles that cross the stop line at an intersection, or vehicles that stop at a point further ahead than indicated by the infrastructure (painted lines and / or stop sign locations).

[0048] System Overview

[0049] Figure 1 An example of an autonomous vehicle 100 is shown.

[0050] As used herein, the term “autonomy” refers to a function, feature, or facility that enables a vehicle to operate partially or fully without real-time human intervention, including but not limited to full AV, high AV, and conditional AV.

[0051] As used in this article, an autonomous vehicle (AV) is a vehicle with autonomous capabilities.

[0052] As used in this article, "vehicle" includes any mode of transport for goods or people. Examples include cars, buses, trains, airplanes, drones, trucks, ships, vessels, submersibles, and spacecraft. Driverless cars are an example of vehicles.

[0053] As used herein, a “track” refers to a path or route that navigates an AV from a first spatiotemporal location to a second spatiotemporal location. In embodiments, the first spatiotemporal location is referred to as the initial location or starting point, and the second spatiotemporal location is referred to as the destination, final location, target, target location, or target position. In some examples, a track consists of one or more road segments (e.g., segments of a road), and each road segment consists of one or more blocks (e.g., a lane or part of an intersection). In embodiments, spatiotemporal locations correspond to real-world locations. For example, a spatiotemporal location is a pick-up or drop-off point for people or goods to board or alight.

[0054] As used herein, “(one or more) sensor” includes one or more hardware components for detecting information relating to the environment surrounding the sensor. Some hardware components may include sensing components (e.g., image sensors, biometric sensors), transmission and / or receiving components (e.g., laser or radio frequency wave transmitters and receivers), electronic components (such as analog-to-digital converters), data storage devices (such as RAM and / or non-volatile memory), software or firmware components, and data processing components (such as application-specific integrated circuits), microprocessors, and / or microcontrollers.

[0055] As used herein, a “scene description” is a data structure (e.g., a list) or data stream that includes one or more classified or labeled objects detected by one or more sensors on an AV vehicle, or one or more classified or labeled objects provided by a source outside the AV.

[0056] As used in this article, a "road" is a physical area that can be traversed by vehicles and can correspond to a named arterial road (e.g., a city street, an interstate highway, etc.) or an unnamed arterial road (e.g., a driveway within a house or office building, a section of a parking lot, a section of an vacant parking lot, a dirt road in a rural area, etc.). Because some vehicles (e.g., four-wheel drive pickup trucks, off-road vehicles, etc.) can traverse a variety of physical areas that are not particularly suitable for vehicle travel, a "road" can be any physical area that has not been formally defined as an arterial road by any municipality or other government or administrative agency.

[0057] As used herein, a “lane” is the portion of a road that can be traversed by vehicles. Sometimes lanes are identified based on lane markings. For example, a lane may correspond to most or all of the space between lane markings, or only a portion of the space between lane markings (e.g., less than 50%). For instance, a road with lane markings spaced far apart may accommodate two or more vehicles between the markings, allowing one vehicle to overtake another without crossing the lane markings; therefore, this could be interpreted as a lane being narrower than the space between lane markings, or as having two lanes between the markings. Lanes can also be interpreted in the absence of lane markings. For example, a lane may be defined based on the physical characteristics of the environment (e.g., rocks and trees along main roads in rural areas, or natural obstacles that should be avoided, for example, in underdeveloped areas). Lanes can also be interpreted independently of lane markings or physical characteristics. For example, a lane may be interpreted based on any unobstructed path in an area that would otherwise lack features that would be interpreted as lane boundaries. In the example scenario, an AV could be interpreted as a lane traversing an unobstructed portion of a field or open space. In another example scenario, an AV can interpret lanes that pass through a wide road (e.g., wide enough for two or more lanes) without lane markings. In this scenario, the AV can communicate lane-related information to other AVs, allowing them to coordinate route planning using the same lane information.

[0058] The term “over-the-air (OTA) client” includes any AV, or any electronic device (e.g., computer, controller, IoT device, electronic control unit (ECU)) embedded in, coupled to or communicating with an AV.

[0059] The term "over-the-air (OTA) update" means any update, alteration, deletion, or addition to software, firmware, data, or configuration settings, or any combination thereof, delivered to an OTA client using proprietary and / or standardized wireless communication technologies, including but not limited to: cellular mobile communications (e.g., 2G, 3G, 4G, 5G), radio local area networks (e.g., WiFi), and / or satellite Internet.

[0060] The term "edge node" refers to one or more edge devices coupled to a network that provide a portal for communicating with AV and can communicate with other edge nodes and cloud-based computing platforms to schedule OTA updates and deliver OTA updates to OTA clients.

[0061] The term "edge device" refers to a device that implements an edge node and provides a physical wireless access point (AP) to the core network of an enterprise or service provider (e.g., VERIZON, AT&T). Examples of edge devices include, but are not limited to: computers, controllers, transmitters, routers, routing switches, integrated access devices (IADs), multiplexers, metropolitan area network (MAN) and wide area network (WAN) access devices.

[0062] "One or more" includes functions performed by one element, functions performed by more than one element, such as in a distributed manner, several functions performed by one element, several functions performed by several elements, or any combination of the above.

[0063] It will also be understood that, although in some instances the terms “first,” “second,” etc., are used herein to describe various elements, these elements should not be limited by these terms. These terms are used only to distinguish one element from another. For example, without departing from the scope of the various described embodiments, a first contact may be referred to as a second contact, and similarly, a second contact may be referred to as a first contact. Both the first contact and the second contact are contacts, but they are not the same contact.

[0064] The terminology used in the description of the various embodiments described herein is for the purpose of describing particular embodiments only and is not intended to be limiting. As used in the description of the various embodiments described and the appended claims, the singular forms “a,” “an,” and “the” are also intended to include the plural forms, unless the context clearly indicates otherwise. It will also be understood that “and / or,” as used herein, refers to and covers any and all possible combinations of one or more related list items. It will also be understood that when the terms “comprising,” “including,” “possessing,” and / or “having” are used in this specification, they specifically indicate the presence of the stated features, integers, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0065] As used herein, depending on the context, the term "if" may optionally be understood as meaning "when" or "at that time" or "in response to being determined" or "in response to being detected." Similarly, depending on the context, the phrase "if determined" or "if [the stated condition or event] has been detected" may optionally be understood as meaning "when determined" or "in response to being determined" or "when [the stated condition or event] is detected" or "in response to being detected."

[0066] As used herein, an AV system refers to an AV and an array of hardware, software, stored data, and real-time generated data that support AV operation. In embodiments, the AV system is incorporated within an AV. In embodiments, the AV system is distributed across several locations. For example, some of the software of the AV system is similar to that described below. Figure 2 The cloud computing environment described is implemented on the cloud computing environment 200.

[0067] Generally, this document describes technologies applicable to any vehicle with one or more autonomous capabilities, including fully automated vehicles (AV), highly automated vehicles (AV), and conditionally automated vehicles (AV), such as so-called Level 5, Level 4, and Level 3 vehicles, respectively (see SAE International Standard J3016: Classification and Definition of Terms Related to Automated Driving Systems for Motor Vehicles on Roads, the entire contents of which are incorporated herein by reference for further details on vehicle autonomy levels). The technologies described in this document are also applicable to partially automated vehicles (AV) and driver-assisted vehicles, such as so-called Level 2 and Level 1 vehicles (see SAE International Standard J3016: Classification and Definition of Terms Related to Automated Driving Systems for Motor Vehicles on Roads). In embodiments, one or more Level 1, Level 2, Level 3, Level 4, and Level 5 vehicle systems may automatically perform certain vehicle operations (e.g., steering, braking, and map usage) under certain operating conditions based on the processing of sensor inputs. The technologies described in this document can benefit vehicles of any level, ranging from fully automated vehicles to human-operated vehicles.

[0068] AVs have advantages over vehicles that require human drivers. One advantage is safety. For example, in 2016, the U.S. experienced 6 million car accidents, 2.4 million injuries, 40,000 deaths, and 13 million vehicle collisions, with an estimated social cost of over $910 billion. From 1965 to 2015, the number of traffic fatalities per 100 million miles driven in the U.S. decreased from about 6 to about 1, partly due to additional safety features deployed in vehicles. For example, an extra half-second of warning associated with an impending collision is believed to mitigate 60% of front and rear collisions. However, passive safety features (such as seat belts and airbags) may have reached their limits in improving these figures. Therefore, active safety measures, such as automated vehicle controls, are a possible next step in improving these statistics. Since human drivers are considered to be responsible for serious pre-collision events in 95% of collisions, autonomous driving systems could potentially achieve better safety outcomes by: identifying and avoiding emergencies more reliably than humans; making better decisions, obeying traffic regulations better than humans, and predicting future events better than humans; and controlling vehicles more reliably than humans.

[0069] refer to Figure 1 The AV system 120 enables the vehicle 100 to operate along a trajectory 198, traversing the environment 190 to the destination 199 (sometimes referred to as the final location), while avoiding objects (e.g., natural obstacles 191, vehicles 193, pedestrians 192, cyclists and other obstacles) and complying with road rules (e.g., operating rules or driving preferences).

[0070] In an embodiment, the AV system 120 includes means 101 for receiving and operating operation commands from and on a computer processor 146. The term "operation command" is used to refer to executable instructions (or a set of instructions) that cause a vehicle to perform actions (e.g., driving maneuvers). Operation commands may, without limitation, include instructions for causing the vehicle to begin moving forward, stop moving forward, begin moving backward, stop moving backward, accelerate, decelerate, make a left turn, and make a right turn. In an embodiment, the computer processor 146 is referenced below. Figure 3 The processor 304 described is similar. Examples of the device 101 include a steering controller 102, a brake 103, a gear, an accelerator pedal or other acceleration control mechanism, a windshield wiper, a side door lock, a window controller, and a turn indicator.

[0071] In an embodiment, the AV system 120 includes sensors 121 for measuring or inferring attributes of the state or condition of the vehicle 100, such as the AV's position, linear and angular velocities and linear and angular accelerations, and heading (e.g., the orientation of the front of the vehicle 100). Examples of sensors 121 are GPS, inertial measurement units (IMUs) that measure both linear acceleration and angular rate of the vehicle, wheel rate sensors for measuring or estimating wheel slip ratios, wheel braking pressure or braking torque sensors, engine torque or wheel torque sensors, and steering angle and angular rate sensors.

[0072] In an embodiment, sensor 121 also includes sensors for sensing or measuring properties of the AV's environment. Examples include a monocular or stereo camera 122 with visible, infrared, or thermal (or both) spectra, a LiDAR 123, a RADAR, an ultrasonic sensor, a time-of-flight (TOF) depth sensor, a rate sensor, a temperature sensor, a humidity sensor, and a precipitation sensor.

[0073] In one embodiment, the AV system 120 includes a data storage unit 142 and a memory 144 for storing machine instructions associated with a computer processor 146 or data collected by the sensor 121. In another embodiment, the data storage unit 142 is associated with the following... Figure 3The described ROM 308 or storage device 310 is similar. In an embodiment, memory 144 is similar to main memory 306 described below. In an embodiment, data storage unit 142 and memory 144 store historical, real-time, and / or predictive information about environment 190. In an embodiment, the stored information includes maps, driving performance, traffic congestion updates, or weather conditions. In an embodiment, data related to environment 190 is transmitted from remote database 134 to vehicle 100 via a communication channel.

[0074] In an embodiment, AV system 120 includes communication devices 140 for communicating measured or inferred attributes of the state and conditions of other vehicles (such as position, linear velocity and angular velocity, linear acceleration and angular acceleration, and linear heading and angular heading) to vehicle 100. These devices include vehicle-to-vehicle (V2V) and vehicle-to-infrastructure (V2I) communication devices, as well as devices for wireless communication via point-to-point or ad hoc networks, or both. In an embodiment, communication device 140 communicates across the electromagnetic spectrum (including radio and optical communications) or other media (e.g., air and acoustic media). Combinations of vehicle-to-vehicle (V2V), vehicle-to-infrastructure (V2I) communication (and in some embodiments, one or more other types of communication) are sometimes referred to as vehicle-to-everything (V2X) communication. V2X communication typically conforms to one or more communication standards for communication with AVs, between AVs, and within AVs.

[0075] In an embodiment, the communication device 140 includes a communication interface. For example, this may be a wired, wireless, WiMAX, Wi-Fi, Bluetooth, satellite, cellular, optical, near-field, infrared, or radio interface. The communication interface transmits data from a remote database 134 to the AV system 120. In an embodiment, the remote database 134 is embedded in, for example... Figure 2 In the cloud computing environment 200 described herein, communication device 140 transmits data collected from sensor 121 or other data related to the operation of vehicle 100 to remote database 134. In some embodiments, communication device 140 transmits information related to teleoperation to vehicle 100. In some embodiments, vehicle 100 communicates with other remote (e.g., "cloud") servers 136.

[0076] In this embodiment, the remote database 134 also stores and transmits digital data (e.g., data such as road and street locations). This data is stored in memory 144 on the vehicle 100 or transmitted from the remote database 134 to the vehicle 100 via a communication channel.

[0077] In one embodiment, the remote database 134 stores and transmits historical information (e.g., rate and acceleration distribution) related to driving attributes of vehicles that previously traveled along trajectory 198 at similar times of day. In one implementation, such data may be stored in memory 144 on vehicle 100 or transmitted from the remote database 134 to vehicle 100 via a communication channel.

[0078] The computer processor 146 located on the vehicle 100 generates control actions in an algorithmic manner based on both real-time sensor data and prior information, allowing the AV system 120 to perform its autonomous driving capabilities.

[0079] In one embodiment, the AV system 120 includes a computer peripheral device 132 coupled to a computer processor 146 for providing information and alerts to a user of the vehicle 100 (e.g., a passenger or a remote user) and receiving input from that user. In another embodiment, the peripheral device 132 is similar to the one described in the following reference. Figure 3 The discussed display 312, input device 314, and cursor controller 316 are coupled wirelessly or wiredly. Any two or more interface devices can be integrated into a single device.

[0080] In one embodiment, the AV system 120 receives and enforces a privacy level for an occupant, such as one specified by the occupant or stored in a profile associated with the occupant. The occupant's privacy level determines how access is permitted to specific occupant-related information (e.g., occupant comfort data, biometric data, etc.) stored in the occupant profile and / or stored on cloud server 136 and associated with the occupant profile. In one embodiment, the privacy level specifies specific occupant-related information that is deleted once the ride is complete. In another embodiment, the privacy level specifies specific occupant-related information and identifies one or more entities authorized to access that information. Examples of the specified entities authorized to access the information may include other AVs, third-party AV systems, or any entity that could potentially access the information.

[0081] An occupant's privacy level can be specified at one or more granular levels. In an embodiment, the privacy level identifies specific information to be stored or shared. In an embodiment, the privacy level applies to all information associated with the occupant, allowing the occupant to specify that her personal information is not stored or shared. The designation of entities authorized to access specific information can also be specified at various granular levels. The various sets of entities authorized to access specific information may, for example, include other AVs, cloud server 136, specific third-party AV systems, etc.

[0082] In an embodiment, AV system 120 or cloud server 136 determines whether AV 100 or another entity can access certain information associated with an occupant. For example, a third-party AV system attempting to access occupant input related to a specific time and place must, for example, obtain authorization from AV system 120 or cloud server 136 to access information associated with the occupant. For example, AV system 120 uses a specified privacy level for the occupant to determine whether location- and time-related occupant input can be presented to a third-party AV system, AV 100, or another AV. This allows the occupant's privacy level to specify which other entities are allowed to receive data related to the occupant's actions or other data associated with the occupant.

[0083] Figure 2 This illustrates an example of a "cloud" computing environment. Cloud computing is a service delivery model that enables convenient, on-demand access over a network to a shared pool of configurable computing resources, such as networks, network bandwidth, servers, processing power, memory, storage, applications, virtual machines, and services. In a typical cloud computing system, one or more large cloud data centers house the machines used to deliver the services provided by the cloud. Now refer to... Figure 2 The cloud computing environment 200 includes cloud data centers 204a, 204b, and 204c interconnected via cloud 202. Data centers 204a, 204b, and 204c provide cloud computing services to computer systems 206a, 206b, 206c, 206d, 206e, and 206f connected to cloud 202.

[0084] A cloud computing environment 200 includes one or more cloud data centers. Generally speaking, a cloud data center (e.g.) Figure 2 The cloud data center 204a shown refers to the cloud (e.g., Figure 2 The physical arrangement of servers in cloud 202 (or a specific portion of the cloud) is illustrated. For example, servers are physically arranged in rooms, groups, rows, and racks within a cloud data center. A cloud data center has one or more regions, which include one or more server rooms. Each room has one or more rows of servers, and each row includes one or more racks. Each rack includes one or more individual server nodes. In some implementations, servers in regions, rooms, racks, and / or rows are arranged into groups based on the physical infrastructure requirements of the data center facility, including power, energy, heat, heat sources, and / or other requirements. In this embodiment, server nodes are similar to... Figure 3 The computer system described herein. Data center 204a has many computing systems distributed across multiple racks.

[0085] Cloud 202 includes cloud data centers 204a, 204b, and 204c, and networks and network resources (e.g., network devices, nodes, routers, switches, and network cables) for connecting cloud data centers 204a, 204b, and 204c and facilitating access to cloud computing services by computing systems 206a-206f. In embodiments, the network represents one or more local networks, wide area networks, or any combination of wired or wireless networks coupled using terrestrial or satellite connections. Data exchanged over the network is transmitted using various network layer protocols, such as Internet Protocol (IP), Multiprotocol Label Switching (MPLS), Asynchronous Transfer Mode (ATM), Frame Relay, etc. Furthermore, in embodiments where the network represents a combination of multiple subnetworks, different network layer protocols are used on each underlying subnetwork. In some embodiments, the network represents one or more interconnected internetworks (such as the public Internet).

[0086] Computing systems 206a-206f or cloud computing service consumers connect to cloud 202 via network links and network adapters. In embodiments, computing systems 206a-206f are implemented as various computing devices, such as servers, desktops, laptops, tablets, smartphones, Internet of Things (IoT) devices, AVs (including cars, drones, space shuttles, trains, buses, etc.), and consumer electronics. In embodiments, computing systems 206a-206f are implemented in other systems or as part of other systems.

[0087] Figure 3 A computer system 300 is illustrated. In an implementation, the computer system 300 is a dedicated computing device. The dedicated computing device is hardwired to execute these technologies, or includes a digital electronic device persistently programmed to execute the aforementioned technologies, such as one or more application-specific integrated circuits (ASICs) or field-programmable gate arrays (FPGAs), or may include one or more general-purpose hardware processors programmed to execute these technologies according to program instructions in firmware, memory, other memory, or a combination thereof. Such a dedicated computing device may also combine custom hardwired logic, ASICs, or FPGAs with custom programming to accomplish these technologies. In various embodiments, the dedicated computing device is a desktop computer system, a portable computer system, a handheld device, a network device, or any other device that includes hardwired and / or program logic to implement these technologies.

[0088] In an embodiment, computer system 300 includes a bus 302 or other communication mechanism for conveying information, and a processor 304 coupled to the bus 302 to process information. Processor 304 is, for example, a general-purpose microprocessor. Computer system 300 also includes main memory 306 (such as random access memory (RAM) or other dynamic storage devices) coupled to the bus 302 to store information and instructions executed by processor 304. In one implementation, main memory 306 is used to store temporary variables or other intermediate information during the execution of instructions to be executed by processor 304. When these instructions are stored in a non-transitory storage medium accessible to processor 304, computer system 300 becomes a dedicated machine customized to perform the operations specified in the instructions.

[0089] In an embodiment, the computer system 300 further includes a read-only memory (ROM) 308 or other static storage device coupled to the bus 302 for storing static information and instructions of the processor 304. A storage device 310, such as a disk, optical disk, solid-state drive, or three-dimensional cross-point memory, is provided and coupled to the bus 302 to store information and instructions.

[0090] In this embodiment, the computer system 300 is coupled via a bus 302 to a display 312, such as a cathode ray tube (CRT), liquid crystal display (LCD), plasma display, light-emitting diode (LED) display, or an organic light-emitting diode (OLED) display for displaying information to a computer user. An input device 314, including alphanumeric keys and other keys, is coupled to the bus 302 for communicating information and command selection to the processor 304. Another type of user input device is a cursor controller 316 (such as a mouse, trackball, touchscreen, or cursor arrow keys) for communicating directional information and command selection to the processor 304 and for controlling the movement of the cursor on the display 312. Such an input device typically has two degrees of freedom on two axes (a first axis (e.g., the x-axis) and a second axis (e.g., the y-axis)), which allow the device to specify a position in a plane.

[0091] According to one embodiment, the techniques described herein are executed by computer system 300 in response to processor 304 executing one or more sequences of one or more instructions contained in main memory 306. These instructions are read into main memory 306 from another storage medium, such as storage device 310. Executing the sequence of instructions contained in main memory 306 causes processor 304 to perform the process steps described herein. In alternative embodiments, hardwired circuitry is used instead of or in combination with software instructions.

[0092] As used herein, the term "storage medium" refers to any non-transitory medium that stores data and / or instructions that enable a machine to operate in a particular manner. Such storage media include non-volatile media and / or volatile media. Non-volatile media include, for example, optical discs, magnetic disks, solid-state drives, or three-dimensional cross-point memory such as storage device 310. Volatile media include dynamic memory (such as main memory 306). Common forms of storage media include, for example, floppy disks, floppy disks, hard disks, solid-state drives, magnetic tape or any other magnetic data storage media, CD-ROMs, any other optical data storage media, any physical media with perforations, RAM, PROMs and EPROMs, FLASH-EPROMs, NV-RAMs, or any other memory chips or memory cartridges.

[0093] Storage media differ from transmission media, but can be used in conjunction with them. Transmission media participate in the transfer of information between storage media. For example, transmission media include coaxial cables, copper wires, and optical fibers, which include wires with a bus 302. Transmission media can also take the form of sound waves or light waves, such as those generated during radio wave and infrared data communication.

[0094] In embodiments, various forms of media involve carrying one or more sequences of one or more instructions to processor 304 for execution. For example, these instructions may initially be executed on a disk or solid-state drive of a remote computer. The remote computer loads the instructions into its dynamic memory and transmits them over a telephone line using a modem. A local modem of computer system 300 receives data over the telephone line and converts the data into an infrared signal using an infrared transmitter. An infrared detector receives the data carried in the infrared signal, and appropriate circuitry places the data on bus 302. Bus 302 carries the data to main memory 306, from which processor 304 retrieves and executes the instructions. The instructions received by main memory 306 may optionally be stored on storage device 310 before or after execution by processor 304.

[0095] Computer system 300 also includes a communication interface 318 coupled to bus 302. Communication interface 318 provides bidirectional data communication coupled to network link 320 connected to local network 322. For example, communication interface 318 is an Integrated Services Digital Network (ISDN) card, a cable modem, a satellite modem, or a modem used to provide data communication connectivity with a corresponding type of telephone line. As another example, communication interface 318 is a Local Area Network (LAN) card used to provide data communication connectivity with a compatible LAN. In some implementations, a wireless link is also implemented. In any such implementation, communication interface 318 transmits and receives electrical, electromagnetic, or optical signals carrying digital data streams representing various types of information.

[0096] Network link 320 typically provides data communication to other data devices via one or more networks. For example, network link 320 provides connectivity to host computer 324 or to a cloud data center or device operated by Internet Service Provider (ISP) 326 via local network 322. ISP 326, in turn, provides data communication services via a worldwide packet data communication network now commonly referred to as the "Internet" 328. Both local network 322 and Internet 328 use electrical, electromagnetic, or optical signals that carry digital data streams. Signals through various networks and signals on network link 320 via communication interface 318 are example forms of transmission media carrying digital data entering and leaving computer system 300. In embodiments, network 320 includes the aforementioned cloud 202 or a portion of cloud 202.

[0097] Computer system 300 sends messages and receives data including program code via (one or more) networks, network links 320, and communication interfaces 318. In an embodiment, computer system 300 receives code for processing. The received code is executed by processor 304 upon receipt and / or stored in storage device 310, or in other non-volatile storage devices for later execution.

[0098] AV architecture

[0099] Figure 4 Showing for AV (e.g., Figure 1 The example architecture 400 of the vehicle 100 shown is illustrated. Architecture 400 includes a sensing system 402 (sometimes referred to as a sensing circuit), a planning system 404 (sometimes referred to as a planning circuit), a control system 406 (sometimes referred to as a control circuit), a positioning system 408 (sometimes referred to as a positioning circuit), and a database system 410 (sometimes referred to as a database circuit). Each system plays a role in the operation of the vehicle 100. Commonly, systems 402, 404, 406, 408, and 410 can be... Figure 1This is a portion of the AV system 120 shown. In some embodiments, any of systems 402, 404, 406, 408, and 410 is a combination of computer software (e.g., executable code stored on a computer-readable medium) and computer hardware (e.g., one or more microprocessors, microcontrollers, application-specific integrated circuits (ASICs), hardware memory devices, other types of integrated circuits, other types of computer hardware, or any or all combinations of these hardware). Systems 402, 404, 406, 408, and 410 are each sometimes referred to as processing circuitry (e.g., computer hardware, computer software, or a combination of both). Any or all combinations of systems 402, 404, 406, 408, and 410 are also examples of processing circuitry.

[0100] In use, the planning system 404 receives data representing the destination 412 and determines data representing the trajectory 414 (sometimes called a route) that the vehicle 100 can travel to reach (e.g., arrive at) the destination 412. In order for the planning system 404 to determine the data representing the trajectory 414, the planning system 404 receives data from the sensing system 402, the positioning system 408, and the database system 410.

[0101] The sensing system 402 uses, for example, as Figure 1 One or more sensors 121 are shown to identify nearby physical objects. The objects are classified (e.g., grouped into types such as pedestrians, bicycles, cars, traffic signs, etc.), and a scene description including the classified objects 416 is provided to the planning system 404.

[0102] The planning system 404 also receives data representing the location 418 of the AV from the positioning system 408. The positioning system 408 determines the location of the AV by calculating location using data from sensor 121 and data from database system 410 (e.g., geographic data). For example, the positioning system 408 uses data from GNSS (Global Navigation Satellite System) sensors and geographic data to calculate the longitude and latitude of the AV. In embodiments, the data used by the positioning system 408 includes high-precision maps with lane geometry properties, maps describing road network connectivity properties, maps describing lane physical properties (such as traffic speed, traffic volume, number of vehicle and bicycle lanes, lane width, lane traffic direction, or lane marking type and location, or combinations thereof), and maps describing the spatial locations of road features (such as intersections, traffic signs, or various types of other traffic signals). In embodiments, the high-precision map is constructed by adding data to a low-precision map via automatic or manual annotation.

[0103] The control system 406 receives data representing trajectory 414 and data representing AV position 418, and operates the AV control functions 420a-420c (e.g., steering, throttle, braking, ignition) in a manner that will cause the vehicle 100 to travel along trajectory 414 to reach destination 412. For example, if trajectory 414 includes a left turn, the control system 406 will operate the control functions 420a-420c in such a way that the steering angle of the steering function will cause the vehicle 100 to turn left, and the throttle and brake will cause the vehicle 100 to stop and wait for passing pedestrians or vehicles before making the turn.

[0104] AV input

[0105] Figure 5 The sensing system 402 is shown. Figure 4 The inputs used are 502a-502d (e.g., Figure 1 Examples of sensor 121 and outputs 504a-504d (e.g., sensor data) are shown. One input 502a is a LiDAR (light detection and ranging) system (e.g., Figure 1 The LiDAR system shown is 123. LiDAR is a technique that uses light (e.g., a beam of light such as infrared light) to obtain data related to physical objects in its line of sight. The LiDAR system produces LiDAR data as output 504a. For example, LiDAR data is a collection of 3D or 2D points (also called point clouds) used to construct a representation of environment 190.

[0106] Another input 502b is a RADAR (radar) system. RADAR is a technology that uses radio waves to acquire data related to nearby physical objects. RADAR can acquire data related to objects that are not within the line of sight of a LiDAR system. The RADAR system generates RADAR data as output 504b. For example, RADAR data is one or more radio frequency electromagnetic signals used to construct a representation of the environment 190.

[0107] Another input 502c is a camera system. The camera system uses one or more cameras (e.g., a digital camera using a light sensor such as a charge-coupled device [CCD]) to acquire information about nearby physical objects. The camera system produces camera data as output 504c. Camera data is typically in the form of image data (e.g., data in image data formats such as RAW, JPEG, PNG, etc.). In some examples, the camera system has multiple independent cameras, for example, for stereoscopic imaging (stereoscopic vision), which enables the camera system to perceive depth. Although the objects perceived by the camera system are described here as "nearby," this is relative to the AV (view of objects). In some embodiments, the camera system is configured to "see" distant objects (e.g., objects as far as 1 kilometer or more in front of the AV). Therefore, in some embodiments, the camera system has features such as sensors and lenses optimized for perceiving distant objects.

[0108] Another input 502d is a Traffic Light Detection (TLD) system. A TLD system uses one or more cameras to acquire information related to traffic lights, street signs, and other physical objects that provide visual navigation information. The TLD system produces TLD data as output 504d. TLD data is often in the form of image data (e.g., data in image data formats such as RAW, JPEG, PNG, etc.). The difference between a TLD system and a system that includes cameras is that a TLD system uses a camera with a wide field of view (e.g., using a wide-angle lens or fisheye lens) to acquire information related to as many physical objects as possible that provide visual navigation information, enabling the vehicle 100 to access all relevant navigation information provided by these objects. For example, the field of view of a TLD system is approximately 120 degrees or greater.

[0109] In some embodiments, sensor fusion technology is used to combine outputs 504a-504d. Thus, individual outputs 504a-504d are provided to other systems within the vehicle 100 (e.g., to systems such as...). Figure 4 The planning system 404 shown may provide combined outputs to other systems in the form of single or multiple combined outputs of the same type (e.g., using the same combination technique or combining the same outputs or both) or single or multiple combined outputs of different types (e.g., using different individual combination techniques or combining different individual outputs or both). In some embodiments, an early fusion technique is used. The early fusion technique is characterized by combining the outputs before applying one or more data processing steps to the combined outputs. In some embodiments, a late fusion technique is used. The late fusion technique is characterized by combining the outputs after applying one or more data processing steps to the individual outputs.

[0110] Figure 6 An example of a LiDAR system 602 is shown (e.g., Figure 5 The input 502a is shown. The LiDAR system 602 emits light 604a-604c from a emitter 606 (e.g., a laser emitter). The light emitted by the LiDAR system is typically not in the visible spectrum; for example, infrared light is often used. Some of the emitted light 604b encounters a physical object 608 (e.g., a vehicle) and is reflected back to the LiDAR system 602. (The light emitted from the LiDAR system typically does not penetrate the physical object, e.g., a solid physical object.) The LiDAR system 602 also has one or more photodetectors 610 for detecting the reflected light. In an embodiment, one or more data processing systems associated with the LiDAR system generate an image 612 representing the field of view 614 of the LiDAR system. Image 612 includes information representing the boundary 616 of the physical object 608. Thus, image 612 is used to determine the boundary 616 of one or more physical objects near the AV.

[0111] Figure 7 The diagram illustrates a LiDAR system 602 in operation. In the scenario shown, the vehicle 100 receives both a camera system output 504c in the form of an image 702 and a LiDAR system output 504a in the form of LiDAR data points 704. In use, the vehicle 100's data processing system compares the image 702 with the data points 704. Specifically, physical objects 706 identified in the image 702 are also identified in the data points 704. Thus, the vehicle 100 perceives the boundaries of physical objects based on the contours and density of the data points 704.

[0112] Figure 8 Additional details of the operation of the LiDAR system 602 are shown. As described above, the vehicle 100 detects the boundaries of physical objects based on the characteristics of the data points detected by the LiDAR system 602. Figure 8As shown, a flat object, such as ground 802, will reflect light 804a-804d emitted from LiDAR system 602 in a consistent manner. In other words, because LiDAR system 602 emits light at a consistent interval, ground 802 will reflect light back to LiDAR system 602 at the same consistent interval. When vehicle 100 travels on ground 802, LiDAR system 602 will continue to detect light reflected by the next effective surface point 806 if nothing obstructs its path. However, if object 808 obstructs its path, the light 804e-804f emitted by LiDAR system 602 will be reflected from points 810a-810b in a manner inconsistent with the expected consistency. Based on this information, vehicle 100 can determine the presence of object 808.

[0113] Path planning

[0114] Figure 9 Show (for example, as) Figure 4 The diagram 900 illustrates the relationship between the inputs and outputs of the planning system 404. Generally, the output of the planning system 404 is a route 902 from a starting point 904 (e.g., a source location or initial location) to an ending point 906 (e.g., a destination or final location). Route 902 is typically defined by one or more road segments. For example, a road segment refers to the distance to be traveled over at least a portion of a street, road, highway, driveway, or other physical area suitable for vehicle travel. In some examples, such as if the vehicle 100 is an off-road capable vehicle such as a four-wheel drive (4WD) or all-wheel drive (AWD) car, SUV, or pickup truck, route 902 includes “off-road” segments such as unpaved paths or open fields.

[0115] In addition to route 902, the planning system also outputs lane-level route planning data 908. Lane-level route planning data 908 is used to navigate segments of route 902 at specific times based on conditions. For example, if route 902 comprises a multi-lane highway, lane-level route planning data 908 includes trajectory planning data 910, which vehicle 100 can use to select a lane from the multiple lanes based on factors such as whether an exit is nearby, whether another vehicle is present in one or more of the multiple lanes, or other factors that change over a period of minutes or less. Similarly, in some implementations, lane-level route planning data 908 includes a speed constraint 912 specific to a segment of route 902. For example, if the segment includes pedestrians or unexpected traffic, speed constraint 912 can limit vehicle 100 to a slower speed than expected, such as a speed limit based on the segment's speed limit data.

[0116] In an embodiment, the inputs to the planning system 404 include (e.g., from...) Figure 4 The database system 410 shown contains database data 914 and current location data 916 (for example, Figure 4 The AV position shown is 418), (for example, for use with Figure 4 The destination data 918 and object data 920 shown for destination 412 (e.g., as shown) Figure 4 The perception system 402 shown perceives classified objects 416. In some embodiments, database data 914 includes rules used during planning. The rules are specified using a formal language (e.g., Boolean logic). At least some of these rules will apply to any given situation encountered by vehicle 100. A rule applies to a given situation if it has conditions satisfied based on information available to vehicle 100 (e.g., information about the surrounding environment). Rules can have priorities. For example, a rule "move to the leftmost lane if the road is a highway" can have a lower priority than "move to the rightmost lane if the exit is within a mile."

[0117] AV control

[0118] Figure 10 Show (for example, as) Figure 4 The diagram shows a block diagram 1000 of the inputs and outputs of the control system 406. The control system operates according to a controller 1002, which includes, for example, one or more processors similar to processor 304 (e.g., one or more computer processors such as microprocessors or microcontrollers or both); short-term and / or long-term data storage devices similar to main memory 306, ROM 308, and storage device 310 (e.g., memory, random access memory, or flash memory or both); and instructions stored in the memory that, when executed (e.g., by one or more processors), perform the operation of controller 1002.

[0119] In one embodiment, controller 1002 receives data representing a desired output 1004. The desired output 1004 typically includes speed, such as rate and heading. The desired output 1004 may be based, for example, from (e.g., as...) Figure 4The data received by the planning system 404 (as shown) is used as follows. Based on the desired output 1004, the controller 1002 generates data that can be used as throttle input 1006 and steering input 1008. Throttle input 1006 indicates the magnitude of the desired output 1004 by engaging the throttle of the vehicle 100 (e.g., acceleration control), for example, by engaging the steering pedal or another throttle control. In some examples, throttle input 1006 also includes data that can be used to engage the brakes of the vehicle 100 (e.g., deceleration control). Steering input 1008 indicates the steering angle, such as the steering control of the AV (e.g., steering wheel, steering angle actuator, or other function for controlling the steering angle), which should be positioned to achieve the desired output 1004.

[0120] In one embodiment, controller 1002 receives feedback used when adjusting inputs provided to throttle and steering. For example, if vehicle 100 encounters an obstacle 1010 such as a hill, the measured rate 1012 of vehicle 100 drops below the desired output rate. In another embodiment, any measured output 1014 is provided to controller 1002 such that necessary adjustments are made, for example, based on the difference 1013 between the measured rate and the desired output. The measured outputs 1014 include measured position 1016, measured speed 1018 (including rate and heading), measured acceleration 1020, and other outputs measurable by the sensors of vehicle 100.

[0121] In one embodiment, information related to the disturbance 1010 is detected in advance, for example, by a sensor such as a camera or a LiDAR sensor, and this information is provided to a predictive feedback system 1022. The predictive feedback system 1022 then provides information that the controller 1002 can use to make appropriate adjustments. For example, if the vehicle 100's sensors detect ("see") a hill, the controller 1002 can use this information to prepare to engage the throttle at an appropriate time to avoid significant deceleration.

[0122] Figure 11 A block diagram 1100 shows the inputs, outputs, and components of controller 1002. Controller 1002 has a rate analyzer 1102 that affects the operation of throttle / brake controller 1104. For example, the rate analyzer 1102 instructs throttle / brake controller 1104 to accelerate or decelerate using throttle / brake 1106 based on feedback received by, for example, controller 1002 and processed by the rate analyzer 1102.

[0123] The controller 1002 also has a lateral tracking controller 1108 that affects the operation of the steering wheel controller 1110. For example, the lateral tracking controller 1108 instructs the steering wheel controller 1110 to adjust the position of the steering angle actuator 1112 based on feedback received by the controller 1002 and processed by the lateral tracking controller 1108.

[0124] Controller 1002 receives several inputs for determining how to control the throttle / brake 1106 and the steering angle actuator 1112. Planning system 404 provides controller 1002 with information, for example, to select the heading of vehicle 100 at the start of operation and to determine which road segment vehicle 100 will cross when it reaches an intersection. Positioning system 408 provides controller 1002 with information describing the current location of vehicle 100, for example, so that controller 1002 can determine whether vehicle 100 is at the expected location based on the positive control of the throttle / brake 1106 and steering angle actuator 1112. In embodiments, controller 1002 receives information from other inputs 1114, such as information received from a database, computer network, etc.

[0125] Priority determination

[0126] This technology enables priority determination at multi-stop intersections. In embodiments, priority determination is adapted to consider agents that might roll past stop signs at multi-stop intersections (i.e., pass a stop sign before the location designated by the infrastructure without first stopping). Generally, priority refers to the order of passage of a sequence of right-of-way permissions based on observations made by vehicles, in conjunction with road regulations. In the example, vehicles observe their respective path intersections (such as at intersections) and rely on priority order to resolve conflicts and avoid collisions. Priority order determines the priority of each vehicle passing through the intersection. As used herein, vehicles with higher priority proceed through the intersection before vehicles with lower priority.

[0127] A multi-stop intersection is an intersection with at least some traffic controls enforced by signs (e.g., stop signs) or other passive traffic control measures (e.g., flashing red lights). For example, at a multi-stop intersection, stop signs are present on at least two roads that meet at the intersection. In embodiments, a multi-stop intersection is generated by an intersection of multiple roads, where traffic flow is controlled by multiple stop signs. To navigate at a multi-stop intersection, AVs and agents typically adhere to road rules (including traffic regulations for crossing intersections and other self-evident norms).

[0128] In the example, multi-stop refers to an intersection where there are no indicators for agents on each of the entry lanes indicating how and / or when to proceed through the intersection. Currently, unsignaled multi-stop intersections on roads require negotiation between agents, involving error-prone human analysis of the intersection's state and the states of all agents navigating through it. Additionally, vehicles or other agents at or near the intersection may periodically become congested. Therefore, in the example, vehicles or other agents at or near the intersection may not always be visible. Furthermore, agents may not adhere to applicable road rules. For example, a vehicle might perform a rolling stop at a stop sign at the intersection instead of coming to a complete stop as indicated by the stop sign.

[0129] Figure 12A This is a diagram of a multi-lane stop 1200A. Typically, a multi-lane stop is an intersection where multiple roads intersect, and traffic flow is controlled by at least one stop sign or other traffic control device. At multi-lane stop 1200A, roads 1202 and 1204 intersect to form multi-lane stop 1200A. For ease of description, multi-lane stop 1200 can be referred to as an intersection. Figure 12A In the example, roads 1202 and 1204 intersect at intersection 1206, where multiple stops 1200A are located.

[0130] As shown in the figure, multi-lane stop 1200A is a four-lane stop that applies stop lines 1208A, 1208B, 1208C, and 1208D (collectively referred to as stop lines 1208) to the road surface. Multi-lane stop 1200A also includes stop signs 1210A, 1210B, 1210C, and 1210D (collectively referred to as stop signs 1210). Therefore, as shown in the figure, multi-lane stop 1200A is a controlled stop, where all approaching roads to intersection 1206 are controlled by stop signs. Multi-lane stop according to this technology can also be an uncontrolled stop. At an uncontrolled stop or intersection, there are no stop lines or stop signs. Multi-lane stop also includes stop areas 1212A, 1212B, 1212C, and 1212D (collectively referred to as stop areas 1212). While multi-road stopping includes stop lines 1208, stop signs 1210, and stop areas 1212 for traffic control, these elements may not be present at the intersection. However, this technique is applicable to any intersection with multiple roads, and the priority order for crossing the intersection is based on an order agreed upon by the agents at the intersection. For ease of description, stop lines, stop signs, and stop areas may be used interchangeably to describe predetermined locations where an agent or AV should stop before navigating through the intersection.

[0131] An AV (e.g., vehicle 100) navigates along a road (e.g., roads 1202, 1204) and observes other agents (e.g., vehicles, pedestrians, bicycles, motorcyclists, or other entities that may travel through the intersection) at or near the intersection. When an agent is observed and associated with behavior or trajectory, the AV generates a track for the corresponding agent. In embodiments, a track is a continuous set of data corresponding to the agent's pose (location and orientation). For ease of description, the terms "agent," "track," and "vehicle" are used interchangeably to refer to entities ordered to travel through an intersection. Additionally, some examples described herein refer to vehicles ordered to travel through an intersection. However, this technique is applicable to any agent that can be legally allocated right-of-way along a lane and that approaches and travels through an intersection.

[0132] In embodiments, to enable priority determination at multiple stops (e.g., intersection 1200A), this technique assigns a unique identifier to the observed track and re-identifies previously observed tracks. As used herein, "unique" means that the assigned identifier is different from and independent of other assigned identifiers. In an example, this technique can determine whether a currently observed track is a re-observed track or a newly observed track. A re-observed track is associated with an agent that was previously observed to be blocked or obstructed from view in some way, and is re-observed after the blockage has been removed or the obstructed area has been resolved. A newly observed track is a track without prior detection history (e.g., no previous blockage). In an example, "obstructed from view" means loss of perception of the agent. For example, an agent may be obstructed from view due to perception failure (e.g., intermittent sensor failure or range limitations under inclement weather). An obstructed perception area is an area within the vehicle's field of view where perception data is unavailable (e.g., perception is blocked or data is corrupted). In the example, this technique is applied when the agent is blocked. Blockage means that perception is obstructed by other objects.

[0133] The identification (ID) and re-identification (re-ID) methods described herein enable the determination of priority. In embodiments, priority depends on determining, based on observations of the trail, that the agent may have arrived at the intersection before the current primary autonomous vehicle (AV). Additionally, this technique can determine priority when a vehicle fails to come to a complete stop at the stop line, stop sign, or stop area of ​​the intersection. Failing to come to a complete stop before entering an intersection where a stop is intended, but instead slowing down to a low or rolling rate, is referred to as a rolling stop. For example, during a rolling stop, when road rules or signs indicate a need to stop, the vehicle may reduce its speed to a slow, decelerating rate, but fail to come to a complete stop. As described herein, this technique determines when a vehicle will make such a rolling stop and when to yield to a vehicle making a rolling stop.

[0134] The order of events according to this technology can be determined based on a number of rules. For example, the rules include: (1) occupying an intersection, (2) First-In-First-Out (FIFO), (3) Yield to the Right (YTR), and (4) Go Straight, Near, Far, and U-Turn (SNFU). In embodiments, the rules are hierarchically arranged in the listed order, where the first rule determines the agent with the highest priority, the next rule determines the agent with the next highest priority, and so on. In embodiments, geographical differences may exclude and / or disrupt the priority order of these rules, or additional rules derived from arrival order and relative location may be used.

[0135] In the example, if a vehicle is occupying an intersection (e.g., the vehicle is currently inside the intersection), then the vehicle has priority. A vehicle is considered inside the intersection when its critical portion is located within the area of ​​multiple road intersections, such as intersection 1206 (e.g., distance beyond the stop line or entrance distance into an overlapping lane area). For example, the critical portion can be specified as extending one meter, two meters, three meters, etc., into the intersection. After assigning priority to agents occupying the intersection, a FIFO ranking is evaluated. Using the FIFO ranking, if a vehicle is the first to arrive at the intersection, that vehicle has priority over AV. Next, a YTR ranking is evaluated. In the YTR ranking, when agents (e.g., concurrent agents) arrive at the intersection at approximately the same time, the agent going to the right (counter-clockwise) has priority. In the straight, near, far, and U-turn (SNFU) ranking, priority is based on the agent's turning intention. For example, for agents traveling in opposite directions, an agent driving straight or turning at the nearest point (e.g., turning right for right-hand traffic and turning left for left-hand traffic) takes precedence over an agent turning further away (e.g., turning left for right-hand traffic and turning right for left-hand traffic) or making a U-turn. For ease of description, this example and technique are typically described using the priority order in right-hand traffic (e.g., driving on the right side of the road). However, this technique is also applicable to left-hand traffic.

[0136] In the example, occupying an intersection is a higher priority rule than the first-in, first-out (FIFO) order. For instance, if an agent is substantially in the intersection at the time it is observed, the AV will yield to the agent even if it was previously blocked. If any agent occupies the intersection, this technique assigns that agent the highest priority, regardless of whether it is on a path conflicting with the AV. However, if an agent is in the intersection but not on a path conflicting with the AV, the AV does not need to yield.

[0137] As described in the following scenario, the order in which unobserved vehicles appear in the AV field of view affects the priority for compliant passage through an intersection. Furthermore, when a non-compliant vehicle is observed performing a rolling stop through an intersection, the rules for determining the safety AV priority apply.

[0138] Figure 12B This is a block diagram of system 1200B, which enables the re-identification of previously observed agents. As shown, system 1200B includes a sensor hub / sensing 1220 (e.g., Figure 1 Sensor 121). In Figure 12B In the example, the AV (e.g., vehicle 100) uses camera 1222 (e.g., Figure 1 Camera 122), LiDAR 1224 (e.g., Figure 1 The LiDAR 123 and radar 1226 are used to sense the surrounding environment.

[0139] Perceived data is transmitted to a tracking and fusion hub 1226. The tracking and fusion hub 1226 includes a fusion system 1228 and a re-identification system 1230. The fusion system 1228 receives perceived data from the sensor hub 1220. A visual similarity detector 1232 obtains the measurement history of the tracks and monitors the visual similarity among all tracks. For example, the measurement history includes historical data from the sensor hub / perceiver 1220, where data from the sensor hub / perceiver 1220 is continuously observed and stored. Visually similar data from the sensor hub / perceiver 1220 is extracted from the data output by the sensor hub / perceiver 1220 and transmitted to a new and old track manager 1234 for managing the creation and updating of tracks. For example, the new and old track manager 1234 creates tracks based on visually similar perceived data and assigns an identifier to newly observed tracks. An old track is a track created by an agent whose track is no longer observable within the AV field of view. A new trace is a newly observed trace while the agent is present in the AV's field of view. Once a new trace matches an old trace, the new trace's identifier is replaced by the old trace's identifier, thereby assigning a current identifier to match the old trace with a previous identifier stored in memory as an intermediate identifier. In many cases, this re-identification by matching the new trace with the old trace is used to obtain an earlier observation time of the old trace's arrival at the intersection before the observation is lost. The time of arrival at the intersection may also occur at a time instance where the new trace has not yet matched the old trace, so by maintaining the record of the intermediate identifier, the AV will not lose the agent's intersection arrival time.

[0140] The re-identification system 1230 receives old traces 1236 and new traces 1238 created and updated by the fusion system 1228. Data association 1240 associates traces from the fusion system 1228 by matching traces belonging to the same agent. For example, matching is based on the first pose of the trace, the duration of the clogging, and the last pose of previously observed and identified traces to determine that a trace associated with an intermediate identifier may originate from a previously observed and identified trace. In an embodiment, data association determines the probability during the clogging time that the last pose of a previously observed and identified trace results in the first pose of a trace associated with an intermediate identifier. Additionally, data association can predict associations between traces from the fusion system 1228. In this example, old traces are continuously compared to new traces because the new traces are associated with agents that can be re-identified in cases previously observed in the AV field of view. Old traces are those belonging to agents previously observed in the AV field of view. Therefore, when a new trace is associated with an old trace, the new and old traces are merged at the trace reidentifier and trace merger 1242. In the example, the trace reidentifier and trace merger 1242 examines the old trace 1236 and determines whether the old trace 1236 and the new trace 1238 should be associated and identified as the same. If so, the traces are merged. Merging traces includes resolving the IDs of traces assigned (as determined by data association) to belong to the same agent. For example, when a new trace is associated with an old trace, the ID of the associated old trace is assigned to the new trace.

[0141] In the example, more than one intermediate ID is assigned to the new trace before it is re-identified and merged with the old trace. Depending on the performance of the visual similarity detector, the intermediate IDs assigned by the new and old trace managers 1234 can be changed when a similarity threshold is met. For example, if no visually similar data is detected, the portion of the perceived data used to generate the trace is unavailable. Additionally, measurement data (e.g., Figure 5 Noise in the outputs (504a-504d) may cause corruption or other errors in the data used to generate the trace.

[0142] During operation, when the AV (e.g., vehicle 100) approaches the intersection, such as Figure 12BThe identifiers are applied to other agents approaching or located at the intersection. Identifiers, re-identifiers, or missing identifiers associated with the tracks assigned to agents are used to determine the priority order for navigating through the intersection. Based on the identifiers, re-identifiers, or missing identifiers, agents approaching the intersection are placed into one or more categories. Specifically, the planning system (e.g., planning system 404) maintains a history of previous agents, concurrent agents, and subsequent agents. Previous agents are those that arrived at the intersection before AV. According to First-In-First-Out (FIFO) ordering, previous agents have priority over AV. Concurrent agents are those that arrive at the intersection at approximately the same arrival time as AV. Since the arrival time of concurrent agents is approximately the same as AV, FIFO ordering results in a tie. In this scenario, additional rules can be applied to determine the priority order. Subsequent agents are those that arrive at the intersection after AV. According to FIFO ordering, AV has priority over subsequent agents.

[0143] In the example, when a track associated with a vehicle is re-identified, a successful re-identification is used to place the agent corresponding to the re-identified track into any of the aforementioned categories (previous agent, concurrent agent, or subsequent agent). All IDs, including intermediate IDs, have category assignments. For example, if a track is placed in a first category, then disappears and is re-identified, the first category is associated with the re-identified track. If the track is a recently observed track, a timestamp associated with that track is determined to place the recently observed track into a category.

[0144] For example, a newly observed vehicle (e.g., a new track not associated with an old track) is classified as the previous agent if and only if the stop marker area of ​​origin is blocked from the time the AV first stops at the intersection until the time the vehicle is first observed. In other words, the state of the intersection where the vehicle is now located is unknown for the entire period from the time the AV arrives at the intersection until the vehicle is first observed. A reasonable worst-case assumption is that the vehicle arrives before the AV and the AV does not observe the vehicle. As used herein, "not observed" means missing data or perception. Otherwise, the newly observed vehicle is placed in the subsequent agent category.

[0145] In this embodiment, a slowing rate constraint is applied to assist in determining the priority of the AV. The slowing rate constraint further enables safe navigation through multiple stops. For example, when an AV is at an intersection and the intersection stop is blocked, the AV cannot determine whether a vehicle is present at the blocked stop due to a lack of perception data. Despite the lack of a higher priority assignment, the AV moves slowly forward under the slowing rate constraint in response to a hypothetical agent occupying the obstructed stop area (also known as slowly slowing into the intersection). In some embodiments, the slowing rate constraint is a maximum speed or maximum acceleration that governs the movement of the AV. In examples, the slowing rate could be 1, 2, 3, 4, or 5 miles per hour.

[0146] To navigate in the presence of congestion (and without effective constraints from observable agents), the AV can execute a low deceleration rate to move forward through the intersection. The deceleration rate is a slow forward movement to improve the visibility of the congested portion of the intersection. This may occur, for example, when stop signs or stop lines are positioned at a distance from the intersection / road crossing area. When stop signs or stop lines are positioned at a distance from the intersection / road crossing area, as per [the relevant information]... Figure 17 As mentioned above, vehicles traveling in the same direction as the AV may cause the AV's visibility to be limited by blocking another part of the intersection.

[0147] In this embodiment, the technique quantifies a risk threshold for slowing down from the stop line to the intersection boundary compared to proceeding into the intersection. For example, if the risk threshold is considered low, the AV may slowly proceed into the intersection even if an unobservable stop area exists. Otherwise, the technique may rely on remote vehicle assistance to overcome stops at the intersection due to congestion. Furthermore, the technique performs congestion prediction for future states to determine whether forward slowing down will improve visibility. For example, predicting congestion at future locations to determine the usefulness of forward slowing down increasing visibility. In this embodiment, the technique also enables backward slowing down to resolve congestion, provided there are no obstacles (e.g., objects and / or agents, etc.) behind the AV. In this way, congestion can be resolved without the need for remote vehicle assistance.

[0148] In addition to slowing down forward or backward to resolve congestion at intersections, it is assumed that agents are located in various obstructed stop areas. Agents are assumed to be classified as prior agents. In some examples, an identifier is assigned to the assumed agent classified as a prior agent. Agents are assumed to be associated with time windows, and priority for navigating multiple stops is determined based on the assumed trail being located in a stop area within the time window that would place the trail in the prior agent category. For example, a time window is a period of time that begins when the AV has stopped and ends when the AV has traveled through the intersection (e.g., ...). Figure 13 (Time periods 1310 and 1314). In some examples, it is assumed that the agent has a higher priority when classified as the previous agent, and considering local rules (e.g., legislative rules including First-In-First-Out (FIFO), Yield to the Right (YTR), and Straight, Nearby, Far, U-Turn (SNFU),) the assigned identifier, and the earliest time the agent appears, forward slowing may violate the priority order. Such a violation is permissible when the priority order includes the assumed agent. In some embodiments, the earliest time the agent appears in the perception-obscured area is determined by observing that the first agent arrives at the corresponding stop area of ​​the multi-stop intersection before the vehicle, and the stop area is continuously obscured during the time period from the first arrival of the vehicle at its corresponding stop area until a new trace associated with the first agent is observed.

[0149] Figure 12C The system 1200C enables priority determination during the execution of a rolling stop. The rolling stop handler 1250 includes a multi-way stop sign intersection handler 1258 and a rate constraint calculator 1262. The multi-way stop sign intersection handler 1258 determines the priority list 1260. For example, the multi-way stop sign intersection handler 1258 includes... Figure 12B All or part of the 1200B system. The priority list includes a list of observed agents, tracks, and vehicle categories.

[0150] The rolling stop handler 1250 takes traffic data 1252, digital semantic map 1254, and AV status 1256 as input. (See also: Regarding...) Figure 12C The priority determination is modified using traffic data 1252, a digital semantic map 1254, and AV status 1256. Traffic data 1252 includes track status and footprint. For example, track status could be speed, footprint, or turn signal status. AV status 1256 includes, for example, AV footprint, location, and speed. The semantic map 1254 is based on a positioning system (e.g., Figure 4 The location of the positioning system 408) is a map of the current location. Therefore, when the AV arrives at the intersection, the development includes, for example, regarding... Figure 12BThe discussed categories are priority lists of traces. Traffic data 1252, digital semantic map 1254, AV status 1256, and categories found in priority list 1260 are used to assign rate constraints to AVs at multi-way stops. Therefore, rate constraint calculator 1262 applies rate constraints to the agent at the intersection to detect the execution of rolling stops.

[0151] Once a scroll stop is detected, the priority list is modified according to the scroll stop handling, and the AV will yield if one or more conditions are met. As discussed above, in the priority list, each agent is assigned to a category such as previous agent, concurrent agent, or subsequent agent. The scroll stop handling logic evaluates additional or other conditions to apply the priority list assignment based on the current multiplexing stop. In an embodiment, the conditions associated with the scroll stop handling can be modified regarding... Figure 12B The established priority order. For example, conditions could include: occupied stop signs, intersection polygons, traffic speed, signed displacement traffic, and the relationship between AV (Area of ​​Accident) and actual stop signs, etc. Regarding... Figures 18A to 18C Further description of the rolling stop handling.

[0152] For example, the condition includes determining whether an agent is at a relevant stop region (e.g., occupying a stop marker). This is done by comparing the agent's footprint to the stop line region polygon, taking into account the relevant stop line. In an embodiment, the size of the stop line region polygon is configurable. Therefore, this condition determines whether other agents are at or near the corresponding stop region.

[0153] Another condition is determining whether the AV has stopped. The AV is considered stopped when its footprint is within a stopping region and its rate is below a certain threshold (e.g., less than 0.1 m / s). In an embodiment, since speed measurements may include noise that prevents the measurement of the AV's speed from being zero, a threshold is used to distinguish the stopping point from using a speed measurement of zero as the stopping point. In an embodiment, the AV threshold used for stopping differs from the threshold used for stopping other agents, which can account for different noise in the speed measurements of the AV compared to other agents and potentially different behavioral norms (e.g., human drivers may generally be more aggressive or non-compliant). Additionally, the threshold is independently configurable and based on the agent type. In the example, a timeout is applied to the speed measurement results to determine if the AV has stopped. If the AV is below a predetermined rate for a certain amount of time, the AV is considered to have stopped completely. For example, a timeout might take half a second at speeds below 0.1 m / s.

[0154] The condition used to apply the allocation based on the current priority list of multiple stops during multi-stop processing is determining whether the AV or other agent is "further ahead". For example, this technique compares the agent with the marked relative longitudinal distance of the AV relative to its associated stop line when the AV first stops. Additionally, another condition is the intersection state. In this example, the intersection state is a separate state associated with the agent and is determined as either the agent is inside the intersection or has left the intersection. Based on a configurable distance threshold, an agent can be considered inside the intersection to interpret typical stop line overshoot as traveling through the intersection compared to indicating ownership. For example, being inside the intersection means that another agent extends into the intersection by exceeding the stop line by 1 meter, 2 meters, or 3 meters. The entire footprint of the agent does not need to be inside the intersection to indicate ownership of the intersection. The agent has left the intersection when its previous intersection state was "previously fully entered the intersection" (configurable definition) and "not inside the intersection". When the intersection status is "left intersection", the intersection crossing is complete, and the agent is removed from the priority list.

[0155] Figure 13 This is according to the timing diagram 1300 of this technology. At time 1302, the AV is approaching the stop marker. This technology enables the continuous addition of detected traces to a priority list, and the AV yields according to the priority list. In an embodiment, the priority list is dynamic and continuously evolves as line-of-sight occlusion conditions occur. In an embodiment, it is based on a map (such as...) Figure 12C Maps such as 1254 are used to determine the approach to multiple stops. For example, lane boundaries, lane connections, and stop areas are all on the map (such as...). Figure 9 As specified in maps such as 914, the possible approach to the intersection is reflected in the lane connectivity, where the approach of the AV is based on route planning on the lane connectivity curve (such as...). Figure 9 Lane-level route planning data (908, etc.).

[0156] By incorporating all traces (including those generated after a blockage) into a priority list, this technique avoids yielding failures that could lead to conflicts (e.g., collisions) with agents at the intersection. Therefore, this technique determines whether the currently observed agent is a re-observed agent that has been intermittently blocked (re-identified), or a newly observed agent without prior detection history. This technique applies re-identification or missing identifiers to determine priority. In the example, priority primarily depends on the assumption that previously unseen agents might have arrived at the intersection before the AV. Therefore, this technique determines whether the AV stops at the blocked stop line before any other agent.

[0157] At time 1304, the AV initiates braking and begins deceleration. In this example, time 1304 is the moment the AV initiates braking as it approaches the intersection. Time 1304 is defined by the AV speed, which is less than the minimum stopping speed, a low-speed tolerance threshold chosen to mitigate uncertainty or noise in the AV speed measurement. In some cases, uncertainty or noise may prevent the AV speed from being measured exactly as zero. Therefore, taking into account such uncertainty and noise, the minimum stopping speed could be 0.1 m / s. When the AV initiates braking at time 1304, it initializes blocked stop signs during time period 1308. Blocked stop signs are initialized by identifying the location of the stop signs at the intersection, determining whether the stop signs are blocked, and updating the priority list using this information. For ease of description, stop signs are used to describe the location where the agent stops before crossing the intersection. However, these locations can be identified by stop lines, stop signs, stop areas, or any combination thereof.

[0158] In this embodiment, the minimum time to stop is the minimum number of times required for the AV to completely stop, or a lower time threshold. At time 1306, the AV is considered to have stopped. In some embodiments, when the control system (e.g., Figure 4 The control system 406) outputs zero rate (e.g., Figure 10 When the output is 1004, the AV stops. After time 1306, during time period 1310, the blocked stop lines are updated. For example, some stop lines may become blocked when approaching an intersection. Therefore, from time 1304 to time 1306 (e.g., time period 1308), the AV initializes the blocked stop lines at the intersection. After time 1306, when the AV is completely stopped, the blocked stop lines are updated during time period 1310 because further congestion may occur when the AV is completely stopped. In addition, some congestion is resolved when the AV is completely stopped. As shown, during time period 1312, new tracks with priority can be added to the priority list until the vehicle stops at time 1306. After the vehicle stops at time 1306, the priority list is updated during time period 1314. In the example, the priority list is initialized when the AV begins braking as it approaches the intersection. For example, locations with multiple stops are identified, and the agent is observed at each location. Identify stopping points with congestion and create a priority list for the intersection. Update the priority list during time periods 1312 and 1314. The priority list terminates when the AV has already passed through the intersection.

[0159] In the example, when the AV initiates braking at time 1304, any vehicle that appears in the previously congested stop line area after the AV has begun braking is classified as the previous vehicle in the priority list. As discussed above, the previous vehicle arrived at the intersection before the AV. Therefore, if other stop lines (e.g., stop lines on other roads intersecting the road the AV is currently traveling on) are congested for the entire time period from when the AV begins braking at time 1304 until it comes to a complete stop at time 1306 (e.g., time period 1308), the vehicle associated with the new track is added to the priority list as the previous vehicle. On the other hand, if other stop lines are not congested at any point after the AV initiates braking at time 1304 and comes to a complete stop at time 1306, and no agent arrives at any other stop line, then the AV is considered to have arrived at the intersection first and is the first in the priority list.

[0160] Such as about Figure 12A-13 The following scenario illustrates the determination of priority order. For ease of description, a stop area is used to describe the location where the agent stops before crossing the intersection. However, these locations can be identified by stop lines, stop signs, stop areas, or any combination thereof. In some cases, the priority order is stored as a priority list. The priority order can be modified when the agent performs a roll-stop to cross the intersection.

[0161] Markings after a blockage

[0162] Figure 14 This is an illustration of scenario 1400 where an AV approaches a multiplexer 1403. Specifically, when a vehicle approaches and navigates multiplexer 1403 through intersection 1404, multiple scenarios 1402A, 1402B, 1402C, and 1402D exemplify scenario 1400 with four sequential timestamps T0, T1, T2, and T3, respectively.

[0163] In the first scenario 1402A at timestamp T0, AV 1406 approaches intersection 1404. As shown in scenario 1402A, vehicles 1408 and 1410 also approach intersection 1404. Before AV 1406 reaches stop area 1412A, vehicle 1408 reaches its stop area 1412B. As shown, vehicle 1410 and AV 1406 are approaching intersection 1404 from the same direction and stopping at the same stop area 1412A in adjacent lanes (assuming vehicle 1410 complies with traffic rules). Vehicle 1408 approaches intersection 1404 and should stop at stop area 1412B according to traffic rules. As shown in scenario 1402A, vehicle 1410 can be observed by AV 1406 when AV 1406 approaches intersection 1404. During this approach, when AV 1406 approaches the multi-channel stop, AV 1406 can assign an ID to vehicle 1410. For example, vehicle 1410 can be assigned an ID of 2.

[0164] In the second scenario 1402B, vehicle 1410 arrives at stop area 1412A before AV 1406 arrives at stop area 1412A. Additionally, vehicle 1408 arrives at its corresponding stop area 1412B before AV 1406 arrives at its corresponding stop area 1412A. In this scenario 1400, vehicles 1408 and 1410 arrive at their respective stop areas before AV 1406 arrives at its stop area. As shown in scenario 1402B, vehicle 1410 obstructs, blocks, or prevents AV 1406 from observing vehicle 1408 at its stop area 1412B. Because the position of vehicle 1410 obstructs AV 1406's observation of stop area 1412B, AV 1406 cannot determine the presence of vehicle 1412B as it approaches intersection 1404. Therefore, vehicle 1408 is blocked by vehicle 1410. In this embodiment, AV 1406 monitors all blocked stop areas at the intersection. If a stop area is blocked during AV's approach to intersection 1404, the AV will prioritize vehicles when they become observable in the stop area during the approach to multiple stops. In this way, the AV continuously updates the status of blocked stop areas at intersection 1404.

[0165] In scenario 1402C, vehicle 1410 enters intersection 1404. As vehicle 1410 enters the intersection, vehicle 1408 becomes observable by AV 1406. Once vehicle 1408 becomes observable in the previously congested stop area, AV 1406 updates the priority list by adding a new track. In scenario 1400, since vehicle 1408 is not observable by AV 1406 at any time while approaching intersection 1404, the track is newly observed and not associated with any old track. In scenario 1402D, vehicle 1410 is entirely within the intersection of intersection 1404. In the example, the previously congested vehicle 1408 is assigned an ID of 1. Because stop area 1412B is blocked when AV 1406 arrives at intersection 1404, and as the blockage in stop area 1412B is cleared, vehicle 1408 becomes visible. Therefore, vehicle 1408 is classified as the preceding vehicle, and AV 1406 will give way to vehicle 1408. Thus, vehicle 1408 is assigned a higher priority than AV 1406. Vehicle 1408 proceeds through intersection 1404 before AV 1406.

[0166] Identification, blocking and re-identification

[0167] Figure 15 This is an illustration of scenario 1500 where an AV approaches a multi-way stop. Specifically, when a vehicle approaches and navigates multi-way stop 1503 through intersection 1504, multiple scenarios 1502A, 1502B, 1502C, and 1502D exemplify scenario 1500 with four sequential timestamps T0, T1, T2, and T3, respectively.

[0168] In the first scenario 1502A at timestamp T0, AV 1506 approaches the stop area 1512A of the multi-way stop 1503. Vehicle 1510 approaches the stop area 1512C of the multi-way stop 1503. Vehicle 1508 approaches the stop area 1512B of the multi-way stop 1503. Approaching the intersection 1504, AV 1506 can observe each of vehicles 1508 and 1510. In this scenario, priority is determined based on the prediction of which vehicle will arrive at its stop area first.

[0169] In the first scene 1502A at timestamp T0, vehicle 1510 arrives at its corresponding stop area 1512C before AV 1506 arrives at its corresponding stop area 1512A. Vehicle 1508 approaches its corresponding stop area 1512B and will arrive at its corresponding stop area 1512B after vehicle 1510 arrives at its stop area 1512C, but before AV 1506 arrives at its corresponding stop area 1512A.

[0170] Vehicle 1510 is the first to arrive at its stopping area 1512C. In scenario 1502B at timestamp T1, vehicle 1508 has stopped at its corresponding stopping area 1512B. In the second scenario 1502B, AV 1506 has not yet come to a complete stop at its corresponding stopping area 1512A at intersection 1504.

[0171] In scenario 1502C at timestamp T2, vehicle 1510 travels through intersection 1504 and blocks AV 1506 from observing vehicle 1508 when it stops in its corresponding stop area 1512B. In scenario 1500, vehicle 1508 was initially observed and identified by AV 1506 when it was leaving the intersection, and then blocked by vehicle 1510 as it left the intersection. Therefore, in scenario 1502C, AV 1506 may not be able to determine whether a vehicle is present in stop area 1512B. This blockage creates an old trace associated with vehicle 1508. In scenario 1502D at timestamp T3, vehicle 1510 has left the intersection of intersection 1504. AV 1506 again observes vehicle 1508 in stop area 1512B. When vehicle 1508 is observed in the stopped area after the congestion is resolved, the AV will create a new trace.

[0172] When the vehicle 1508 becomes observable after the blockage, such as Figure 12B The vehicle 1508 is re-identified by matching the latest tracking observation with previous tracking observations corresponding to the same agent, determined through data association. Because AV 1506 re-identifies vehicle 1508 (e.g., Figure 12B (Data association between new and old tracks) allows AV 1506 to know the initial priority applied to vehicle 1508. In this example, AV 1506 will give way to vehicle 1508. In scenario 1500, vehicle 1510 is the first to pass through the intersection, followed by vehicle 1508, and then AV 1506.

[0173] Parsing the intermediate ID

[0174] Figure 16 This is an illustration of scenario 1600 where an AV approaches a multiplexer 1603. Specifically, when a vehicle approaches and navigates multiplexer 1603 through intersection 1604, multiple scenarios 1602A, 1602B, 1602C, and 1602D exemplify scenario 1600 with four sequential timestamps T0, T1, T2, and T3, respectively.

[0175] In the first scene 1602A at timestamp T0, AV 1606 approaches the stop area 1612A of multiplexer 1603. Vehicle 1610 approaches the stop area 1612C of multiplexer 1603. Vehicle 1608 approaches the stop area 1612B of multiplexer 1603. Upon approaching the intersection, AV 1606 can observe each component of vehicle 1608 and vehicle 1610. Figure 16 In the example, an ID of 1 can be assigned to vehicle 1608. An ID of 2 can be assigned to vehicle 1610. As discussed above, when AV 1606 approaches the stopping area, AV 1606 assigns an identifier to the vehicle.

[0176] In the first scene 1602A at timestamp T0, vehicle 1610 arrives at its corresponding stop area 1612C before AV 1606 reaches its corresponding stop area 1612A and before AV 1606 comes to a complete stop. Before AV 1606 arrives at its corresponding stop area, but after vehicle 1610 has arrived at its corresponding stop area 1612C, vehicle 1608 then arrives at its corresponding stop area 1612B.

[0177] In the second scenario 1602B at timestamp T1, vehicle 1610 travels through the intersection before AV 1606 stops at its corresponding stopping area 1612A. As vehicle 1610 travels through intersection 1604, vehicle 1608 is blocked by vehicle 1610 before AV 1606 stops.

[0178] In the third scenario 1602C at timestamp T2, vehicle 1608 becomes observable by AV 1606 as vehicle 1610 is leaving the intersection. In the example, the track associated with vehicle 1608 is re-identified after a short time delay. In an embodiment, an intermediate ID is assigned to the newly created track. For example, when vehicle 1608 is first observed near stop area 1612B, a first ID is assigned to the track associated with vehicle 1608, where the first ID can be assigned as 3, then becomes the old ID during the congestion event, and when vehicle 1608 is observed after the congestion, a second ID of 4 can be assigned to the new track associated with vehicle 1608, which does not match the old track with ID 3. In an embodiment, when the new track is not yet associated with an old track (e.g., a previously observed track), an intermediate ID is applied to the new track during a short time window. This technique analyzes new traces to determine if they are associated with previously identified agents. In scenario 1600, based on a stronger visual similarity in a later observation of vehicle 1608, a second new trace with ID 4 matches an older trace with ID 3 after a short time delay. After matching and re-identifying to again assign ID 3 to the trace associated with vehicle 1608, AV 1606 can determine that traces 3 and 4 share the same priority. The priority of trace ID 3 is the same as the priority of the first intermediate ID. During the duration during which the match between trace IDs 3 and 4 is not resolved, trace 3 may be mistakenly considered to have arrived after AV, and therefore AV may begin to move forward. However, in the fourth scenario 1602D at timestamp T3, AV 1606 again yields to vehicle 1608, and assuming a relatively short duration with mismatched intermediate IDs, AV's rate will still be low, so it will only move a small distance forward from its initial stopping point. Because AV 1606 re-identifies vehicle 1608, AV 1606 knows the priority order applied to vehicle 1608. In this example, vehicle 1610 is the first to pass through the intersection, followed by vehicle 1608, and then AV 1606.

[0179] Slow maneuvering

[0180] Figure 17 This is an illustration of scenario 1700 where an AV approaches a multiplex stop 1703. Specifically, when a vehicle approaches and navigates a multiplex stop through intersection 1704, multiple scenarios 1702A, 1702B, 1702C, and 1702D exemplify scenario 1700 with four sequence timestamps T0, T1, T2, and T3, respectively.

[0181] In scenario 1702A at timestamp T0, AV 1706 approaches stop area 1712A of intersection 1704. Vehicles 1710 and 1711 approach the same stop area 1712A in adjacent lanes. Vehicle 1708 approaches the intersection towards stop area 1712B. As shown, vehicle 1708 is blocked by vehicles 1710 and 1711. AV 1706 can assign a flag to the observable vehicles 1710 and 1711. In the example of scenario 1700, vehicle 1708 is blocked and is not observed by AV 1706 when approaching intersection 1704.

[0182] In the second scenario 1702B at timestamp T1, vehicle 1708 arrives at the stopping area 1712B of intersection 1704. In the second scenario 1702B, AV 1706 has not yet come to a complete stop. However, vehicle 1708 is blocked by vehicles 1710 and 1711, both of which have been identified by AV 1706.

[0183] In the third scene 1702C at timestamp T2, vehicle 1710 travels through intersection 1704. As vehicle 1710 travels through the intersection, vehicle 1711 moves into the stop area, further obstructing stop area 1712B. Vehicle 1708 is initially obstructed, and becomes observable after AV 1706 slightly slows forward to improve visibility. Observe vehicle 1708 after AV 1706 performs a slowing maneuver. As shown, AV 1706 has moved slightly and slowly forward beyond its stop area 1712A to improve the visibility of the obstructed stop area 1712B. Therefore, AV 1706 uses a slowing maneuver to improve visibility. After the slowing maneuver, vehicle 1708 becomes observable and is added to the priority list maintained by AV 1706. Because vehicle 1708 is associated with a newly identified track following the previous congestion, AV 1706 will give way to the previously congested vehicle 1708. As shown in the fourth scenario 1702D at timestamp T3, vehicle 1710 is the first to proceed through the intersection, followed by vehicle 1708, then AV 1706 and vehicle 1711. In this example, vehicle 1711 is classified as either a concurrent or subsequent vehicle, and depending on lane connectivity at the intersection, vehicle 1711 may not potentially conflict with the AV via the intersecting path in any case.

[0184] Priority order for scrolling stop

[0185] Figures 18A to 18CExample of a rolling stop scenario. As used herein, a rolling stop is not a complete stop where regulations or other road rules indicate that a complete stop is appropriate. For example, when approaching a multi-lane stop, a vehicle typically slows to a very low speed at or near the stop area associated with the multi-lane stop before proceeding through the multi-lane stop intersection.

[0186] In scenario 1800A, intersection 1804 exemplifies a multi-lane stop 1803. Stop sign 1810A controls traffic on the road traveled by AV 1806. To navigate to intersection 1804, AV stops at stop area 1812A as indicated by stop sign 1810A. Similarly, stop sign 1810B controls traffic on the road traveled by vehicle 1808. To navigate to intersection 1804, vehicle 1808 must comply with road rules and stop at stop area 1812B as indicated by stop sign 1810B.

[0187] like Figure 18B As shown, AV 1806 comes to a complete stop before crossing intersection 1804. Vehicle 1808 fails to stop before crossing intersection 1804 and instead performs a rolling stop. For example, when reaching a multi-way stop, a vehicle typically slows down to a very low speed at or near the stop area associated with the multi-way stop before proceeding through the multi-way stop intersection.

[0188] exist Figures 18A to 18C In the example, regarding Figure 12CThe described rolling stop handling applies to the observed agent, not the blocked agent. Specifically, the rolling stop handling described herein improves upon existing multiplexing logic to account for rolling stops or incomplete stops at stop signs. In an embodiment, when a rolling stop is detected, the AV will yield to a vehicle that meets one or more conditions. For example, conditions could be: occupying a stop sign, an intersection polygon, traffic speed, marked displacement traffic, or the AV actually stopping at the stop sign. As shown in scenario 1800B, the vehicle has exceeded the stop area 1812B. As described above, the condition detected by the AV is determining when the vehicle has exceeded the intended stop area or when the vehicle is further into the intersection than the AV. Since vehicle 1808 has passed the intended stop area, the rolling processing logic is applied according to the above rolling stop conditions, and the AV yields to vehicle 1808. For example, as a conflict resolution method, an agent performing a rolling stop is defined as: located within a predetermined distance of the intersection, not yet reduced to a rate below a minimum stopping threshold, and having traveled further relative to its corresponding stopping area compared to the forward movement of the AV relative to its stopping area. Higher priority in the priority order is assigned to the non-compliant agent performing the rolling stop to enable conflict resolution. Figure 18C In scenario 1800C, AV 1806 yields to vehicle 1808 crossing the intersection. Vehicle 1808 completes its crossing of intersection 1804, allowing it to turn left after leaving the AV's path. As vehicle 1808 crosses intersection 1804, it is removed from the priority list, and the AV is then allowed to proceed through the intersection.

[0189] Processing for determining priority

[0190] Figure 19 This is flowchart 1900, used for determining priority.

[0191] In box 1902, identifiers are assigned to tracks. These tracks correspond to agents observed by the vehicle as it approaches a multi-stop intersection. In the example, an intersection is a place where multiple roads intersect, such as a multi-stop. At a multi-stop, the AV and the agent typically follow road rules to cross the intersection. In an embodiment, identifiers are assigned to tracks observed by at least one agent as it approaches the intersection. For example, tracks are created based on visually similar perceptual data, and identifiers are assigned to newly observed tracks. In an embodiment, this identifier is an intermediate identifier. In an embodiment, this technique generates multiple IDs as described below and / or some other representations for ID confusion.

[0192] In box 1904, the new track is compared with the old track. The new track is matched with the old track based on one or more factors. In some embodiments, factors include visual similarity, hypothesized agent dynamics, duration of loss of observation, or any combination thereof. Data captured by the AV (such as...) can be used. Figure 1 The data captured by sensor 121 shown Figure 5 The hypothetical agent dynamics are determined by inputs 502a-502d and any combination thereof. The captured data associated with the agent can be used to determine the agent dynamics, such as the agent's associated velocity, acceleration, deceleration, and steering angle.

[0193] In the example, a new track is a track associated with an agent that has a relatively short observation history. During the current priority determination at the intersection, the agent associated with the new track is not blocked or otherwise unobservable within the AV field of view. An old track is a track associated with an agent whose new observation has ceased (e.g., the agent can no longer be observed by the vehicle).

[0194] In box 1906, the identifier of the new trace is reassigned to the identifier of the old trace, wherein the match between the new trace and the old trace is determined based on one or more factors. In some embodiments, the new trace is determined to match the old trace in an absolute manner, such that the match between the new trace and the old trace is true or false (e.g., a Boolean value). In the example, if no match is found between the new trace and the old trace, the new trace is a new observation, and a new identifier is generated and assigned to the new trace.

[0195] In some embodiments, ID obfuscation relates to the uncertainty in the confidence level of a new trace matching an old trace (e.g., if any). Matching between a new trace and an old trace is described from the perspective of different confidence levels of matching among multiple traces. In the example of a new trace and multiple old traces, matching is described from the perspective of confidence levels indicating the probability of a match. For example, a confidence level could indicate a 30% confidence level that the new trace matches old trace A, a 50% confidence level that the new trace matches old trace B, a 0% confidence level that the new trace matches old trace C, and a 20% confidence level that the new trace is a newly observed trace. If a new trace is likely a new trace (e.g., the new trace corresponds to the highest confidence level for a newly observed agent), a unique identifier such as "D" can be assigned to the new trace. In this example, old trace B has the highest confidence level, and the new trace matches old trace B. Additionally or alternatively, confidence levels are used to determine the priority of navigation through intersections. For example, a predetermined threshold is set to associate a new track with multiple identifiers from older tracks. Older tracks with a confidence level higher than the predetermined threshold relative to the new track are selected. For example, a predetermined threshold greater than 30% is considered. In the example of older tracks A, B, and C, the new track is associated with both older tracks A and B.

[0196] In box 1908, the earliest time of agent arrival is determined based on the identifier and taking into account perceived occlusion areas. The identifier includes assigned identifiers and reassigned identifiers. In the example, the earliest possible time of agent arrival at the intersection is based on the overall intermediate identifier and the current identifier, also taking into account that a new trace emerging from a perceived occlusion area may have arrived at the intersection before the AV if and only if the stopping area has been continuously occluded for a duration from when the AV first comes to a complete stop at the intersection until the new trace appears. In some embodiments, such as when a confidence level is used to determine a match between a new trace and an old trace, the new trace is associated with multiple old traces. Historically, when a new trace is associated with multiple old traces, the earliest time of arrival is the earliest time of arrival observed for the multiple old traces. In the example of a new trace matching old traces A and B, the earliest time of arrival is the earliest time that old trace A or old trace B appeared.

[0197] In box 1910, the priority order for navigating through the intersection is determined based on local rules, assigned markers, and the earliest time an agent appears. These agents are categorized as either prior agents, concurrent agents, or subsequent agents, in relation to the time the vehicle arrives at its stopping point. Local rules include legislative rules such as First-In-First-Out (FIFO), Yield to the Right (YTR), and Straight, Nearby, Far, U-Turn (SNFU). Local rules may also include generally agreed-upon road rules.

[0198] In box 1912, a vehicle travels through a stop-at-multiplex intersection according to a priority order. Local rules determine the rows and columns of agents that pass through the stop-at-multiplex intersection. In the example, the AV is controlled to navigate through the intersection according to a priority order. In some embodiments, the priority order is updated iteratively until the AV crosses the intersection. In the example, in response to an agent being classified as a concurrent agent, the AV travels through the stop-at-multiplex intersection according to local rules applicable to concurrent arrivals at the stop-at-multiplex intersection. Additionally, in the example, enabling the vehicle to travel through the stop-at-multiplex intersection according to a priority order includes yielding to a hypothetical agent, which occupies a perceived occlusion area and is classified as the previous agent. The previous agent may navigate the intersection before the AV.

[0199] In the preceding description, embodiments of the invention have been described with reference to numerous specific details, which may vary from implementation to implementation. Therefore, the specification and drawings should be considered illustrative rather than restrictive. The sole and exclusive indication of the scope of the invention, and what the applicant expects to be the scope of the invention, is the literal and equivalent scope of the claims published from this application in the specific form of the claims, including any subsequent amendments. Any definitions of terms expressly set forth herein for inclusion in such claims should be taken as meaning as such terms are used in the claims. Furthermore, when the term “comprising” is used in the preceding specification or appended claims, what follows that phrase may be an additional step or entity, or a sub-step / sub-entity of a previously stated step or entity.

[0200] Cross-references to related applications

[0201] This application claims priority to U.S. Provisional Patent Application 63 / 292,850, filed December 22, 2021, the entire contents of which are incorporated herein by reference.

Claims

1. A method for a vehicle, comprising: Using at least one processor, an identifier is assigned to the track, wherein the track corresponds to an agent observed by the vehicle when it approaches a multi-stop intersection; When a new track matches an old track, the identifier of the new track is reassigned to the identifier of the old track, wherein the match between the new track and the old track is determined based on one or more factors. Using the at least one processor, the earliest time of the agent's appearance is determined based on the identifier and taking into account the perceived occlusion area, wherein the earliest time of the agent's appearance in the perceived occlusion area is determined by observing that the first agent arrives at the corresponding stop area of ​​the multi-way stop intersection before the vehicle, and the stop area is continuously occluded during the time period from the first arrival of the vehicle at the corresponding stop area until a new trace associated with the first agent is observed; Using the at least one processor, a priority order for navigating through the intersection is determined based on local rules, the identifier, and the earliest time the agent appeared, wherein the agents are classified as one of a previous agent, a concurrent agent, and a subsequent agent related to the time the vehicle arrived at the stopping point; and Using the at least one processor, the vehicle is made to travel through the multiple stop intersection according to the priority order, wherein the local rules determine the row and column of the agent passing through the multiple stop intersection.

2. The method according to claim 1, wherein, Factors evaluated to match the new trail with the old trail include visual similarity, hypothetical agent dynamics, duration of loss of observation, or any combination thereof.

3. The method according to claim 1 or 2, wherein, When compared with old tracks, the new tracks correspond to agents with a relatively short observation history of the vehicle, while the old tracks correspond to agents whose observations have ceased.

4. The method according to claim 1 or 2, comprising: In response to an agent being classified as a concurrent agent, the vehicle proceeds through the multiple stop intersection according to the local rules applicable to concurrent arrivals at the multiple stop intersection.

5. The method according to claim 1 or 2, wherein, Proceeding the vehicle through the multiple stop intersection according to the priority order includes: giving way to a hypothetical agent, wherein the hypothetical agent occupies a perception-obstructed area.

6. The method according to claim 1 or 2, wherein, Proceeding the vehicle through the multi-stop intersection according to the priority order includes: slowing forward to obtain additional data associated with the perceived obstruction area.

7. The method according to claim 1 or 2, comprising: The higher priority in the priority order is assigned to the non-compliant agent, wherein the non-compliant agent is the agent that performs the scroll stop.

8. The method according to claim 1 or 2, wherein, The perception occlusion area is the region within the field of view of the vehicle where perception data is unavailable.

9. A vehicle, comprising: At least one computer-readable medium storing computer-executable instructions; At least one processor communicatively coupled to at least one device and configured to execute the computer-executable instructions, the execution being performed in accordance with any one of claims 1 to 8.

10. At least one non-transitory storage medium storing instructions that, when executed by at least one processor, cause the at least one processor to perform the method according to any one of claims 1 to 8.

11. A computer program product comprising a computer program configured to perform the method according to any one of claims 1-8 when executed by a processor.

Images