SYSTEM AND METHOD FOR RIGHT-OF-WAY DETERMINATION BASED ON FLEET VEHICLE SENSOR DATA - Patent application

Abstract

The computing system can receive sensor data from a subset of human-driven vehicles operating through a road segment within a region. The system can process the sensor data to determine a set of right-of-way rules for autonomous vehicles traveling through the road segment. In a particular example, the system can obtain an autonomous driving map utilized by the autonomous vehicles to operate through the road segment and modify the autonomous driving map to include the set of right-of-way rules for the road segment.

Description

[Background technology]

[0001] Current autonomous driving implementations utilize highly accurate autonomous driving maps recorded by mapping vehicles and labeled based on signs and traffic signals. In some cases, labor-intensive manual labeling of the autonomous driving maps is performed, which can result in inaccuracies and other errors. In addition to basic geometric environment information such as lane markings and sign locations, the autonomous driving maps can also include semantic information such as right-of-way rules for specific road segments of the road network. In certain situations (e.g., due to region-specific rules or traffic sign occlusion), the right-of-way rules cannot be directly recognized by the mapping vehicle's on-board sensors, which can result in further technical challenges in providing highly accurate, up-to-date autonomous driving maps for autonomous and / or semi-autonomous vehicle operation on the road network. Summary of the Invention

[0002] According to example embodiments, a system, method, and computer program product are described for determining right-of-way rules for a road segment based on sensor data from a fleet of human-driven vehicles. The system may receive sensor data from a subset of human-driven vehicles operating through the road segment. The system may process the sensor data to determine a set of right-of-way rules for autonomous and / or semi-autonomous vehicles operating through the road segment. In particular examples, the system may obtain an autonomous driving map utilized by the autonomous vehicles to operate through the road segment and modify the autonomous driving map to include the set of right-of-way rules for the road segment. Additionally or alternatively, the system may automatically generate and / or label an autonomous driving map for a road network to include the right-of-way rules. In additional embodiments, the system may compare recently generated right-of-way labels in the autonomous driving map with previously generated right-of-way labels (e.g., to determine road segments for which traffic control elements have changed in the real world over multiple periods of time). The system may further be used to verify right-of-way rules and other traffic control elements in existing autonomous driving maps (e.g., to verify that they are correct and up-to-date).

[0003] In various examples, the sensor data may indicate vehicle trajectories of human-driven vehicles through a road segment. The vehicle trajectories may indicate driving behaviors corresponding to multiple behavior classes, including braking, accelerating, stopping, coasting, and turning classes, of a subset of human-driven vehicles through the road segment. In a particular example, a computing system may overlay the vehicle trajectories on map data to determine a set of right-of-way rules. The sensor data received from the subset of human-driven vehicles may be generated by a set of odometry sensors for each of the subset of human-driven vehicles. The set of odometry sensors may include one or more of a positioning system (e.g., GPS or GNSS such as GLONASS), a braking sensor, a steering input sensor, a wheel speed sensor, or an acceleration sensor. It is believed that global position information may be very important for aligning a particular vehicle trajectory with other vehicle trajectories and existing geometric maps in a global frame of reference. Global position information may also be very important for extracting movement patterns and driving behaviors in combination with other sensor information.

[0004] As described herein, the autonomous driving map may be generated based at least in part on map data obtained from one or more mapping vehicles operating through the road segment and / or labeled autonomous driving rules corresponding to at least one of signs or signals along the road segment.

[0005] In particular implementations, the computing system can process sensor data from human-driven vehicles operating throughout a region to determine right-of-way rules for each road segment of the road network in the region. In such an example, the computing system can generate a set of autonomous driving maps for the autonomous vehicles traveling throughout the road network based on the right-of-way rules determined for each road segment of the road network. The disclosure herein is shown by way of example, and not by way of limitation, in the accompanying drawings, in which like reference numerals refer to like elements and in which: [Brief explanation of the drawings]

[0006] [Figure 1] FIG. 2 is a block diagram illustrating an example computing system that performs right-of-way determinations based on sensor data from fleet vehicles, according to examples described herein. [Figure 2] FIG. 2 is a block diagram illustrating an exemplary computing system including dedicated modules for performing right-of-way determinations based on sensor data from fleet vehicles, according to examples described herein. [Figure 3A] 1 illustrates a mapped road segment with vehicle trajectories overlaid on map data based on sensor data from a fleet of vehicles, according to examples described herein. [Figure 3B] 1 illustrates a mapped road segment with acceleration values ​​for fleet vehicles overlaid on map data based on sensor data from the fleet vehicles, according to examples described herein. [Figure 4] 1 shows graphs illustrating the speed and acceleration of individual vehicle trajectories through a road segment, according to examples described herein. [Figure 5] 1 is a flowchart illustrating a method for determining right-of-way for a road segment based on sensor data from a fleet of vehicles, according to examples described herein. [Figure 6] 1 is a flowchart illustrating a method for determining right-of-way for an autonomous vehicle through a road segment by generating a vehicle trajectory based on sensor data from a fleet of vehicles, according to examples described herein. DETAILED DESCRIPTION OF THE INVENTION

[0007] Described herein is a computing system for generating right-of-way information for a road segment within an autonomous map (also referred to as an autonomous driving map) based on sensor data received from a human-driven fleet of vehicles operating through the particular road segment. As described herein, the fleet of vehicles may include a human-driven vehicle including a set of odometry sensors and a communication interface for transmitting sensor data generated by the odometry sensors to the computing system over one or more networks. The odometry sensors may include one or more of a positioning system (e.g., a global positioning system (GPS) or other global navigation satellite system (GNSS)), braking sensors, steering input sensors, wheel speed sensors, acceleration sensors (e.g., an inertial measurement unit), etc. In various implementations, the computing system includes a communication interface for communicating over one or more networks with the human-driven fleet of vehicles operating throughout a geographic area. The computing system may receive sensor data from a subset of human-driven vehicles operating through a particular road segment within a region and process the sensor data to determine a set of right-of-way rules for autonomous vehicles traveling through the road segment.

[0008] As used herein, a "network" or "one or more networks" may include any type of network or combination of networks that enables communication between devices. In one embodiment, the network may include one or more of a local area network, a wide area network, the Internet, a secure network, a cellular network, a mesh network, a peer-to-peer communication link, or a combination thereof, and may include any number of wired or wireless links. Communication over the network(s) may be achieved through a network interface using, for example, any type of protocol, protection scheme, encoding, formatting, packaging, etc.

[0009] As used herein, a "right-of-way rule" or "set of right-of-way rules" is defined as a set of priorities for multiple competing routes, which may include lanes (e.g., turn lanes, merge lanes, roundabouts, etc.), crosswalks, railroad crossings, etc. The set of priorities determines which competing routes must yield to other competing routes at any given time. In common practice, right-of-way is regulated by lane markings, traffic control signals, traffic signs, and local ordinances. It is contemplated that the right-of-way rule(s) for competing lanes may change based on the time of day. For example, in certain localities, traffic signals may be operational during the day and shut down during the late night hours. Thus, the right-of-way rule(s) for a given road segment may be different for traffic signals and fixed signs depending on the time of day.

[0010] In particular examples, the computing system can modify an autonomous driving map (also referred to as an autonomous map) that includes a road segment to include a set of right-of-way rules for the road segment. The computing system can include a memory and database that stores the autonomous driving map, or can remotely obtain the autonomous driving map and add or otherwise edit the associated autonomous driving map to include right-of-way rules for a particular road segment determined from sensor data from the fleet vehicles. A road segment can include one or more lanes that collide or conflict with one or more other lanes, such as turn lanes merging onto a roadway, intersection lanes, highway on- and off-ramps, and roundabouts.

[0011] As described herein, an “autonomous map” or “autonomous driving map” includes a ground truth map recorded by a mapping vehicle using various sensors (e.g., a LIDAR sensor and / or a camera suite or other imaging devices) and labeled to indicate traffic regulations and / or right-of-way regulations at any given location. For example, a given autonomous map may be labeled by a human based on observed traffic signs, traffic signals, lane markings, and local regulations in the ground truth map. In a further example, reference points or other points of interest may be further labeled in the autonomous map for additional assistance to the autonomous vehicle. An autonomous or self-driving vehicle can then utilize the labeled autonomous map to perform localization, attitude determination, change detection, and various other operations required for autonomous driving on public roads. For example, an autonomous vehicle can reference the autonomous map to determine traffic regulations (e.g., speed limits) at the vehicle's current location and can navigate safely along its current route by dynamically comparing live sensor data from its onboard sensor suite with the corresponding autonomous map.

[0012] In various implementations, the computing system can process sensor data from the fleet vehicles to generate vehicle trajectories for the fleet vehicles through a road segment. The vehicle trajectory can indicate driver behavior through the road segment, which can correspond to multiple behavior classes, including braking behavior, accelerating behavior, stopping behavior, coasting behavior, and / or turning behavior, as the vehicle trajectory trajectory passes through the road segment. The vehicle trajectory can also indicate a temporal set of acceleration values ​​(e.g., positive acceleration, deceleration, continuous speed values) for each vehicle through the road segment. In certain examples, the computing system can also receive sensor data indicating vehicle speed, acceleration, yaw rate, etc., as the vehicle trajectory passes through the road segment and include this information in the vehicle trajectory.

[0013] As described herein, a computing system can implement a learning-based approach to classify the driving behavior of a fleet of vehicles through a road segment. In one example, the computing system implements a recurrent neural network to encode the sequence of geographic locations of each vehicle through a road segment and derive a corresponding driving pattern to classify the vehicle's driving behavior. In such an example, the computing system can temporally and statistically aggregate the classified driving behavior for each lane segment by recording the driving behavior over a period of time. In a further example, the computing system can implement a learning-based approach (e.g., a multilayer perceptron) that obtains statistical values ​​of the aggregated driving behavior from competing lanes of a road segment to predict or otherwise determine a set of right-of-way rules for the competing lanes.

[0014] As described herein, a computing system may store or include one or more machine learning models. In one embodiment, the machine learning model may include an unsupervised learning model. In one embodiment, the machine learning model may include a neural network (e.g., a deep neural network) or other types of machine learning models, including nonlinear and / or linear models. The neural network may include a feedforward neural network, a recurrent neural network (e.g., a long-short-term memory recurrent neural network), a convolutional neural network, or other forms of neural network. Some exemplary machine learning models may utilize attention mechanisms, such as self-attention. For example, some exemplary machine learning models may include a multi-head self-attention model (e.g., a Transformer model).

[0015] According to examples described herein, an autonomous driving map can be generated based at least in part on map data obtained from one or more mapping vehicles operating through a road segment and / or labeled autonomous driving rules corresponding to at least one of signs or signals along the road segment. Additionally or alternatively, an autonomous driving map or specific elements of the autonomous driving map (e.g., those indicating lane geometry or sign locations) can be generated from a fleet of vehicles equipped with a set of sensors. Upon determining a set of right-of-way rules for competing lanes of a road segment, a computing system can obtain an associated autonomous map that includes the road segment and modify the autonomous map to include a set of right-of-way rules for autonomous driving along the road segment. As described herein, the autonomous map and / or right-of-way rules determined by the methods described herein can be used by semi-autonomous vehicles, fully autonomous vehicles, and / or as a safety feature for human-operated vehicles.

[0016] Additionally or alternatively, the computing system may process sensor data from human-driven vehicles operating throughout a region to determine right-of-way rules for each road segment of a road network in the region. Upon determining the right-of-way rules, the computing system may generate a set of autonomous driving maps for autonomous and / or semi-autonomous vehicles operating throughout the road network based on the right-of-way rules determined for each road segment of the road network. According to such an implementation, the road network does not require the use of mapping vehicles and human labeling to create autonomous maps, because the right-of-way rules for all operational roads can be determined by the methods described herein.

[0017] The examples described herein achieve, among other advantages, the technical effect of utilizing sensor data from a fleet of human-operated vehicles to determine human driving behavior for competing lanes of a road segment. The examples described herein can generate vehicle trajectories for vehicles operating through a road segment, classify driving behavior, and determine a set of right-of-way rules for the road segment. Upon determining the right-of-way rules, the examples described herein can modify or create an autonomous map of a road network that includes right-of-way rules for use by semi-autonomous and / or autonomous vehicles as they navigate throughout the road network. Such examples provide technical solutions to various technical limitations that exist in the field of autonomous vehicle navigation on public road networks. Specifically, relying on mapping vehicles to generate templates for autonomous maps can result in signs or traffic signals being difficult to see (obscured). Furthermore, relying on human labeling of these recorded autonomous maps can result in erroneous right-of-way determinations, which can result in autonomous vehicles being unable to move forward due to unclear right-of-way rules and becoming stranded.

[0018] One or more examples described herein provide that the methods, techniques, and operations performed by a computing device are programmatically performed or performed as a computer-implemented method. "Programmatically," as used herein, means through the use of code or computer-executable instructions. These instructions may be stored in one or more memory resources of the computing device. Programmatically performed steps may or may not be automatic.

[0019] One or more examples described herein can be implemented using programmatic modules, engines, or components. A programmatic module, engine, or component can include a program, subroutine, portion of a program, or software or hardware component capable of performing one or more specified tasks or functions. As used herein, a module or component can reside on a hardware component separate from other modules or components. Alternatively, a module or component can be a shared element or process of other modules, programs, or machines.

[0020] Some examples described herein may generally require the use of computing devices that include processing and memory resources. For example, one or more examples described herein may be implemented in whole or in part in a computing device such as a server and / or personal computer using network equipment (e.g., a router). Memory, processing, and network resources may be used in connection with the establishment, use, or implementation of any example described herein (including the implementation of any method or system).

[0021] Furthermore, one or more examples described herein may be implemented through the use of instructions executable by one or more processors. These instructions may be recorded on a non-transitory computer-readable medium. The machines illustrated in or described with the following figures provide examples of processing resources and computer-readable media capable of recording and / or executing instructions for implementing the examples disclosed herein. In particular, many machines illustrated with examples of the present invention include a processor and various forms of memory for retaining data and instructions. Examples of non-transitory computer-readable media include persistent memory storage devices such as hard drives in personal computers or servers. Other examples of computer storage media include portable storage units such as flash memory or magnetic memory. Computers, terminals, and network-enabled devices are all examples of machines and devices that utilize processors, memory, and instructions stored on a computer-readable medium. Additionally, these examples may be implemented in the form of a computer program or in the form of a computer-usable carrier medium capable of carrying such a program.

[0022] Exemplary Computing System 1 is a block diagram illustrating an exemplary computing system 100 that performs right-of-way decisions based on sensor data from a fleet of vehicles, according to examples described herein. In one embodiment, computing system 100 can include control circuitry 110, which may include one or more processors (e.g., microprocessors), one or more processing cores, programmable logic circuits (PLCs), or programmable logic arrays (PLAs) / programmable gate arrays (PGAs), field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), or any other control circuitry. In some implementations, control circuitry 110 and / or computing system 100 may be part of or form a vehicle control unit (also referred to as a vehicle controller) embedded in or otherwise disposed in a vehicle (e.g., a Mercedes-Benz® car or van). For example, the vehicle controller may be or include an infotainment system controller (e.g., an infotainment head unit), a telematics control unit (TCU), an electronic control unit (ECU), a central powertrain controller (CPC), a central exterior & interior controller (CEIC), a zone controller, or any other controller (the term "or" is used interchangeably with "and / or" herein).

[0023] In one embodiment, control circuitry 110 may be programmed by one or more computer-readable or computer-executable instructions stored on non-transitory computer-readable medium 120. Non-transitory computer-readable medium 120 may be a memory device, also referred to as a data storage device, and may include an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination thereof. The non-transitory computer-readable medium 120 may form, for example, a floppy disk, a hard disk drive (HDD), a solid state drive (SDD), a solid-state integrated memory, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a dynamic random access memory (DRAM), a portable compact disc read-only memory (CD-ROM), a digital versatile disk (DVD), and / or a memory stick. In some cases, the non-transitory computer-readable medium 120 may store computer-executable or computer-readable instructions, such as instructions for implementing the methods described below in connection with FIGS. 5 and 6.

[0024] In various embodiments, the terms "computer-readable instructions" and "computer-executable instructions" are used to describe software instructions or computer code configured to perform various tasks and operations. In various embodiments, when the computer-readable instructions or computer-executable instructions form a module, the term "module" broadly refers to a collection of software instructions or code configured to cause control circuitry 110 to perform one or more functional tasks. The modules and computer-readable / computer-executable instructions may be described as performing various operations or tasks when control circuitry 110 or other hardware components execute the modules or computer-readable instructions.

[0025] In further embodiments, computing system 100 may include a communication interface 140 for communicating over one or more networks 150 to enable the transmission and reception of data. In various examples, computing system 100 may use communication interface 140 to communicate with fleet vehicles over one or more networks, receive sensor data, and perform right-of-way determination methods described throughout this specification. In particular embodiments, communication interface 140 may be used to communicate with one or more other systems. Communication interface 140 may include any circuitry, components, software, etc. for communicating over one or more networks 150 (e.g., a local area network, a wide area network, the Internet, a secure network, a cellular network, a mesh network, and / or a peer-to-peer communication link). In some implementations, communication interface 140 may include, for example, one or more of a communication controller, a receiver, a transceiver, a transmitter, a port, conductors, software, and / or hardware for communicating data / information.

[0026] In an exemplary embodiment, computing system 100 can receive sensor data from fleet vehicles over one or more networks 150 using communications interface 140. Control circuitry 110 can process the sensor data by executing instructions accessed from non-transitory computer-readable medium 120 to generate a vehicle trajectory of the fleet vehicles through a particular road segment having competing lanes and determine a set of right-of-way rules for the road segment based on the vehicle trajectory. Control circuitry 110 may then use the right-of-way rules to automatically label an autonomous map for use by semi-autonomous or fully autonomous vehicles navigating through a road network. The functionality of computing system 100 is further described below.

[0027] System Description 2 is a block diagram illustrating an exemplary computing system 200 including functional modules for performing right-of-way determinations based on sensor data from fleet vehicles, according to examples described herein. As described herein, computing system 200 may include a communication interface 205 that communicates with human-driven fleet vehicles 250 via one or more networks 260 to receive sensor data indicative of each vehicle's movement and / or control inputs through a particular road segment. As further described herein, the sensor data may be received from a set of sensors housed in each vehicle, such as a positioning system (GPS or other GNSS) and / or one or more odometry sensors (e.g., wheel spin sensors, braking input sensors, steering input sensors, acceleration sensors). Thus, the sensor data may include real-time position data and / or odometry information (e.g., speed, acceleration, yaw rate) of fleet vehicles 250 as they traverse through a road segment.

[0028] In various implementations, computing system 200 can include trajectory generation module 210, right-of-way determination module 220, and mapping module 230. In a further example, computing system 200 can include database 240 that stores a set of autonomous maps 242 utilized by autonomous and / or semi-autonomous vehicles to operate throughout a region. Specifically, autonomous map 242 can be created based on a mapping vehicle that generates map data using a sensor suite (e.g., including LIDAR sensors, image sensors, etc.) as the mapping vehicle travels through a road network on which the autonomous vehicle operates. The map data can be enriched with one or more layers of labeled data indicating specified traffic rules (e.g., speed limits, signs, crosswalk information, traffic signals, etc.) for any given road segment. By continuously comparing real-time sensor data generated by the onboard sensor suite with the associated autonomous map, the autonomous vehicle can perform localization and attitude determination processes that assist the autonomous or semi-autonomous vehicle in safely operating through a road network.

[0029] In a particular example, trajectory generation module 210 may receive sensor data from a subset of fleet vehicles 250 operating through a particular road segment that includes competing lanes, such as a turn lane merging onto the road. Trajectory generation module 210 may generate vehicle trajectories for the subset of fleet vehicles 250 through the particular road segment. As described below in connection with Figures 3A and 3B, the vehicle trajectories may show the path of each vehicle through the road segment. In a further example, the vehicle trajectories may be overlaid on map data to show the temporal sequence of movement of the vehicles as they traverse the road segment, e.g., accelerating, coasting, decelerating, and turning trajectories.

[0030] In various implementations, the right-of-way determination module 220 can process vehicle trajectory and sensor data to classify the driving behavior of each vehicle through a road segment. In some examples, the right-of-way determination module 220 can implement a learning-based approach to process vehicle trajectory information and derive vehicle driving patterns through a road segment. For example, the right-of-way determination module 220 can execute an artificial neural network (e.g., a recurrent neural network and / or a multilayer perceptron) that processes the temporal trajectory of vehicle paths through a road segment as input and outputs a prediction of the right-of-way rule(s) for the road segment. For example, the right-of-way rule determination module (right-of-way determination module) 220 can execute an artificial neural network in which particular nodes of the neural network receive the temporal trajectory of vehicle paths as sequential inputs and output a prediction of the right-of-way rule for the road segment (e.g., label the determined right-of-way rule on an autonomous map). Once the right-of-way rule is determined (determined, predicted), the autonomous vehicle and / or advanced driver assistance system may apply the right-of-way rule in navigating the corresponding road segment.

[0031] In a further example, right-of-way determination module 220 can process a temporal and statistical aggregation of driving behavior on a road segment and its competing lane(s) over a period of time to output right-of-way rule(s). In yet another example, right-of-way determination module 220 can be trained using a ground truth map (e.g., recorded by a mapping vehicle and labeled based on signs and traffic signals). Thus, given a set of vehicle trajectories through any road segment of a road network on which an autonomous vehicle may travel, right-of-way determination module 220 can output predicted or actual right-of-way rules for each lane of the road segment.

[0032] According to examples described herein, mapping module 230 can use the right-of-way output from right-of-way determination module 220 to validate labels in an existing autonomous map, modify an existing autonomous map to include the right-of-way rule(s) for the road segment, or generate a new autonomous map for the road segment to include the right-of-way rule(s). In various examples, mapping module 230 can include an autonomous map validator that determines whether the right-of-way rules labeled in existing autonomous map 242 are accurate. For example, an autonomous map stored in database 240 or accessed remotely can include a road segment traveled by a subset of fleet vehicles 250. Once the right-of-way rule(s) for the road segment have been determined, mapping module 230 can perform a search of the associated autonomous map that includes the road segment and compare the labeled right-of-way rules (e.g., human-labeled right-of-way rules) with the right-of-way rules output by right-of-way determination module 220. In a further example, if the right-of-way rules do not match, mapping module 230 can automatically flag the discrepancy for further processing and labeling, or can automatically re-label the autonomy with the right-of-way rule determined by right-of-way determination module 220. In a further example, the right-of-way rule can be used to verify other traffic control elements in the autonomous map because the right-of-way rule is based on other traffic control elements in such autonomous map. For example, labels in the autonomous map that identify and / or classify traffic signals, traffic signs, stop lines, crosswalks, etc. can be verified by mapping module 230 based on the right-of-way rule determined by right-of-way determination module 220.

[0033] Additionally or alternatively, mapping module 230 can automatically label existing autonomous maps recorded by a mapping vehicle using the right-of-way rules output by right-of-way determination module 220. For example, mapping module 230 can replace certain human labeling functions in creating autonomous maps 242 for autonomous and / or semi-autonomous vehicles. In such an example, the mapping vehicle may further be utilized to record a ground truth map of a given road network, and the ground truth map may be automatically annotated by mapping module 230 with right-of-way information for each road segment that includes conflicting lanes. In yet another example, mapping module 230 can utilize the right-of-way output from right-of-way determination module 220 to generate an autonomous map using road network data (e.g., an existing ground truth map or a virtualized ground truth map based on road network information).

[0034] It is contemplated that the right-of-way determination methods described throughout this disclosure can be performed on any road segment that includes competing lanes and can be used to supplement or replace existing labeling capabilities (e.g., human labeling) of autonomous map 242. It is further contemplated that new autonomous maps can be generated by mapping module 230 using the methods described herein. For example, in addition to determining right-of-way rules, computing system 200 can further infer road sign rules, speed limits, traffic signal locations, crosswalk locations, and the like based on sensor data and vehicle trajectories of fleet vehicles 250 through a specified road segment. Thus, the use of mapping vehicles and human or automated labeling of ground truth maps may be supplemented or excluded using the methods described herein.

[0035] Additionally, it is contemplated that the right-of-way determination methods described herein may further utilize external sensor data from fixed-position sensors (e.g., image sensors, LIDAR sensors, radar, sonar, infrared, etc.) having a field of view that includes the road segment including the competing lanes. Thus, trajectory generation module 210 may supplement the sensor data received from fleet vehicles 250 with additional sensor information from fixed sensors installed near the road segment to determine right-of-way rules.

[0036] Vehicle trajectory 3A shows a mapped road segment 305 with vehicle trajectories 310 overlaid on map data 300 based on sensor data from fleet vehicles, according to an example described herein. In the example shown in FIG. 3A, road segment 305 merges into competing lanes 315, and vehicle trajectories 310 include the temporal positions of each fleet vehicle through the road segment. As described herein, vehicle trajectories 310 are matched to road segment 305 (e.g., overlaid on map data 300) to show the paths taken by fleet vehicles through road segment 305. As described below with reference to FIG. 3B, vehicle trajectories 310 may further include information corresponding to acceleration values ​​of fleet vehicles through road segment 305.

[0037] 3B illustrates a road segment 305 with fleet vehicle acceleration values ​​overlaid on map data 300 based on sensor data from the fleet vehicles, according to examples described herein. In the example shown in FIG. 3B, vehicle trajectory 310 in FIG. 3A can include additional sensor information (e.g., position data over time, wheel spin information, speed data, acceleration data, etc.) that computing system 200 can use to generate position acceleration values ​​360 and deceleration values ​​370 (and coasting trajectories) for the fleet vehicles as they move forward through road segment 305. As described herein, computing system 200 can determine right-of-way rules between road segment 305 and competing lane 315 by analyzing vehicle trajectory 310, acceleration values ​​360, and deceleration values ​​370. In particular, computing system 200 can determine or classify the driving behavior of the fleet vehicles by analyzing vehicle trajectory 310 and the sensor information encoded therein (e.g., acceleration values ​​360 and deceleration values ​​370). As shown in FIG. 3B, the acceleration values ​​360 and deceleration values ​​370 of the fleet vehicles traveling through road segment 305 indicate a braking pattern before accelerating to merge into the competing lane, suggesting that competing lane 315 has the right-of-way on the road segment.

[0038] FIG. 4 shows a graph 400 illustrating the speed 410 and acceleration 405 of each vehicle's trajectory through a road segment, according to examples described herein. The speed 410 and acceleration 405 lines shown in FIG. 4 may correspond to the particular vehicle trajectory 310 in FIG. 3A and the acceleration values ​​360 and deceleration values ​​370 of a particular fleet vehicle traveling through the road segment 305. As shown in FIG. 4, the information encoded in the vehicle trajectory 310 may include a temporal value of the vehicle's speed 410 traveling through the road segment 305, as well as a temporal value of the vehicle's acceleration 405. In the example shown in FIG. 4, the fleet vehicles decelerate and slow down as they move forward through the road segment 305, and then accelerate as the vehicle fleet leaves the road segment 305. As described herein, the computing system 200 can process this vehicle trajectory information from multiple fleet vehicles traveling through the same road segment 305 to determine right-of-way rules for the road segment 305 and any conflicting lanes 315.

[0039] methodology 5 and 6 are flowcharts illustrating a method for determining right-of-way for a road segment based on sensor data from a fleet of vehicles, according to examples described herein. In the following description of the method of FIGS. 5 and 6, reference may be made to reference symbols representing particular features described with respect to the system diagrams of FIGS. 1 and 2. Furthermore, steps described with reference to the flowcharts of FIGS. 5 and 6 may be performed by the computing systems 100, 200 shown and described with respect to FIGS. 1 and 2. Furthermore, particular steps described with reference to the flowcharts of FIGS. 5 and 6 may be performed before, concurrently with, or subsequent to any other steps, and need not be performed in the order shown.

[0040] Referring to FIG. 5 , in block 500, computing system 200 may receive sensor data from fleet vehicles 250 operating through a road segment. As described herein, the sensor data may include position data indicating the location of each fleet vehicle 250 in time as the fleet vehicle 250 traverses through the road segment. In a further example, the sensor data may include wheel spin information, speed and / or acceleration information, braking input data, yaw rate information, etc. In block 505, computing system 200 may process the sensor data to determine a set of one or more right-of-way rules for the road segment. As an example, at an intersection of two roads, one road may include a two-way stop, each controlled by a stop sign. Even without knowing this sign information, computing system 200 may infer the presence of a stop sign, and therefore right-of-way, on the associated road based on the sensor data from the fleet vehicles. Determining right-of-way for autonomous vehicles is discussed in more detail with reference to FIG. 6 .

[0041] In particular implementations, at block 510, the computing system may obtain an autonomous driving map 242 that the autonomous and / or semi-autonomous vehicle utilizes to navigate through the road segment. As described herein, the autonomous driving map 242 may include a ground truth map, which includes map data recorded by a mapping vehicle and labeled (e.g., human-labeled) to indicate traffic rules for the road segment (e.g., based on human-identified road signs and traffic signals). An autonomous, semi-autonomous, or other self-driving vehicle may utilize the labeled autonomous map to perform localization, attitude determination, change detection, and various other operations required for autonomous navigation through the road segment. As further described herein, the autonomous driving map 242 may be accessed remotely from a third-party database or retrieved locally from the autonomous driving map database 240.

[0042] At block 520, computing system 200 may then modify autonomous driving map 242 to include a set of one or more right-of-way rules for the road segment determined from the sensor data from fleet vehicles 250. In particular examples, modifying autonomous driving map 242 may include automatically labeling the autonomous driving map to include the right-of-way rules for the road segment. Additionally or alternatively, modifying autonomous driving map 242 may include automatically labeling a ground truth map including raw sensor data recorded by a mapping vehicle to include the right-of-way rules determined from the sensor data of fleet vehicles 250. In further examples, computing system 200 may verify the right-of-way labels in the autonomous driving map (e.g., entered by a human) and / or generate a new autonomous driving map using the determined right-of-way information and other known information of the road network (e.g., locations of signs, traffic lights, speed limits, etc.). In further implementations, the computing system 200 can replace an incorrect right-of-way rule (e.g., one entered by a human) or can input a right-of-way rule into an autonomous driving map 242 that does not already include a right-of-way rule.

[0043] 6 is a flowchart illustrating a method for determining right-of-way for autonomous vehicles through a road segment by generating vehicle trajectories based on sensor data from fleet vehicles, according to examples described herein. Referring to FIG. 6 , at block 600, computing system 200 may receive sensor data from fleet vehicles 250 operating through a particular road segment. At block 605, computing system 200 may further generate vehicle trajectories for each vehicle through the road segment on the map data. For example, the vehicle trajectories may indicate each vehicle's path through the road segment, as well as acceleration and deceleration values ​​(and freewheeling values) over time and position as it traverses the road segment. In a further example, computing system 200 may encode additional data, such as braking input information, yaw rate, and wheel speed information, into the vehicle trajectories.

[0044] In a further example, the vehicle trajectories may indicate driving behaviors corresponding to multiple behavior classes, including braking, accelerating, stopping, coasting, and turning classes, of a subset of human-driven vehicles through the road segment. Computing system 200 may process the vehicle trajectories to determine driving patterns for fleet vehicles 250 operating through the road segment and classify one or more driving behaviors that indicate one or more right-of-way rules for the road segment.

[0045] At block 610, computing system 200 may process the vehicle trajectories to determine a set of one or more right-of-way rules for the road segment. The set of right-of-way rules may indicate whether a road segment (e.g., a particular lane) has right-of-way over a competing road segment or lane. In a further example, a road segment may have multiple competing lanes, each with a different right-of-way. Thus, the set of right-of-way rules may indicate that a particular road segment has higher right-of-way than a first competing lane, but must yield right-of-way to a second competing lane. Thus, computing system 200 may determine, for each particular lane in a competing lane area, right-of-way rule(s) for each of the competing lanes based on the vehicle trajectories. In particular implementations, computing system 200 may generate time-specific right-of-way rules (e.g., if timestamp information is included in the sensor data). For example, computing system 200 can determine, for a particular road section, the times of day (e.g., daytime hours) when traffic signals control right-of-way rules and the times when traffic signals are turned off and static signs are used for right-of-way.

[0046] In various implementations, computing system 200 can operate as a label verifier for autonomous driving map 242 that has already been labeled with right-of-way rules (e.g., via human labeling). At block 615, computing system 200 verifies the right-of-way rule(s) labeled in existing autonomous driving map 242 for the road segment. Additionally or alternatively, at block 620, computing system 200 can edit or modify the autonomous driving map or unlabeled ground truth map to include the right-of-way rule(s) for the road segment.

[0047] In a further example, in block 625, computing system 200 may generate one or more autonomous driving maps including right-of-way rules for an entire road network based on the processes described herein. In such an example, computing system 200 may process sensor data from any number of road segments with conflicting right-of-way rules to determine right-of-way rules for the entire road network (e.g., an autonomous driving grid in which autonomous vehicles are permitted to operate). Computing system 200 may then complement or replace existing autonomous mapping and labeling methods currently used in the art.

[0048] It is contemplated that the examples described herein extend to the individual elements and concepts described herein, independent of other concepts, ideas, or systems, and that examples include combinations of elements described anywhere in this application. While examples have been described in detail herein with reference to the accompanying drawings, the concepts are not limited to those precise examples. Accordingly, many modifications and variations will be apparent to those skilled in the art. Accordingly, it is intended that the scope of the concepts be defined by the following claims and their equivalents. Furthermore, it is contemplated that features described individually or as part of an example may be combined with other individually described features and parts of other examples, even if the feature is not mentioned in those other features or examples. Thus, the absence of a description of a combination does not preclude the claim of such a combination.

Claims

1. In a computing system, a communications interface for communicating with human-operated vehicles operating throughout the region via one or more networks; one or more processors; a memory for storing instructions; The instructions, when executed by the one or more processors, cause the computing system to: receiving, via the one or more networks, sensor data from a subset of the human-operated vehicles operating through road segments within the region; processing the sensor data to determine a set of right-of-way rules for an autonomous vehicle traveling through the road segment; obtaining an autonomous driving map utilized by the autonomous vehicle to operate through the road segment; modifying the autonomous driving map to include the set of right-of-way rules for the road segment; 1. A computing system comprising:

2. the sensor data indicates vehicle trajectories of a subset of the human-driven vehicles through the road segment; the vehicle trajectories indicate driving behaviors of the subset of human-driven vehicles passing through the road segment corresponding to a plurality of behavior classes, including braking behavior, accelerating behavior, stopping behavior, coasting behavior, and turning behavior classes; The computing system of claim 1 .

3. The instructions, when executed, cause the computing system to overlay the vehicle trajectory onto map data and determine the set of right-of-way rules. The computing system of claim 1 .

4. the autonomous driving map is generated based at least in part on (i) map data obtained from one or more mapping vehicles operating through the road segment, and / or (ii) labeled autonomous driving rules corresponding to at least one of signs and / or signals along the road segment; The computing system of claim 1 .

5. The instructions, when executed, cause the computing system to process sensor data from the human-operated vehicles operating throughout the region to determine right-of-way rules for each road segment of the road network of the region; The instructions when executed further cause the computing system to: generating a set of autonomous driving maps for an autonomous vehicle traveling throughout the road network based on the right-of-way rules determined for each road segment of the road network; The computing system of claim 1 .

6. the sensor data received from the subset of human-operated vehicles is generated by a set of odometry sensors for each of the subset of human-operated vehicles; The computing system of claim 1 .

7. the set of odometry sensors includes one or more of a positioning system, a braking sensor, a steering input sensor, a wheel speed sensor, or an acceleration sensor; 7. The computing system of claim 6.

8. A non-transitory computer-readable medium storing instructions, comprising: The instructions, when executed by one or more processors of a computing system, cause the computing system to: receiving, via one or more networks, sensor data from a subset of human-operated vehicles operating through road segments within the region; processing the sensor data to determine a set of right-of-way rules for an autonomous vehicle traveling through the road segment; obtaining an autonomous driving map utilized by the autonomous vehicle to operate through the road segment; modifying the autonomous driving map to include the set of right-of-way rules for the road segment; 1. A non-transitory computer-readable medium comprising:

9. the sensor data indicates vehicle trajectories of a subset of the human-driven vehicles through the road segment; the vehicle trajectories exhibit driving behaviors corresponding to a plurality of behavior classes, including braking behavior, acceleration behavior, stopping behavior, coasting behavior, and turning behavior classes, of the subset of human-driven vehicles passing through the road segment; 9. The non-transitory computer-readable medium of claim 8.

10. The instructions, when executed, cause the computing system to overlay the vehicle trajectory onto map data and determine the set of right-of-way rules.

9. The non-transitory computer-readable medium of claim 8.

11. the autonomous driving map is generated based at least in part on (i) map data obtained from one or more mapping vehicles operating through the road segment, and / or (ii) labeled autonomous driving rules corresponding to at least one of signs or signals along the road segment; 9. The non-transitory computer-readable medium of claim 8.

12. The instructions, when executed, cause the computing system to process sensor data from human-operated vehicles operating throughout the region to determine right-of-way rules for each road segment of a road network in the region; The instructions when executed further cause the computing system to: generating a set of autonomous driving maps for an autonomous vehicle traveling throughout the road network based on the right-of-way rules determined for each road segment of the road network; 9. The non-transitory computer-readable medium of claim 8.

13. the sensor data received from the subset of human-operated vehicles is generated by a set of odometry sensors for each of the subset of human-operated vehicles; 9. The non-transitory computer-readable medium of claim 8.

14. the set of odometry sensors includes one or more of a positioning system, a braking sensor, a steering input sensor, or an acceleration sensor; 14. The non-transitory computer-readable medium of claim 13.

15. 1. A computer-implemented method performed by one or more processors, comprising: receiving, via one or more networks, sensor data from a subset of human-operated vehicles operating through road segments within the region; processing the sensor data to determine a set of right-of-way rules for an autonomous vehicle traveling through the road segment; obtaining an autonomous driving map utilized by an autonomous vehicle to operate through the road segment; modifying the autonomous driving map to include the set of right-of-way rules for the road segment; Including, 10. A computer-implemented method comprising:

16. the sensor data indicates vehicle trajectories of a subset of the human-driven vehicles through the road segment; the vehicle trajectories indicate driving behaviors of the subset of human-driven vehicles passing through the road segment corresponding to a plurality of behavior classes, including braking behavior, accelerating behavior, stopping behavior, coasting behavior, and turning behavior classes; 16. The computer-implemented method of claim 15.

17. the one or more processors overlay the vehicle trajectory onto map data to determine the set of right-of-way rules.

17. The computer-implemented method of claim 16.

18. the autonomous driving map is generated based at least in part on (i) map data obtained from one or more mapping vehicles operating through the road segment, and (ii) labeled autonomous driving rules corresponding to at least one of signs or signals along the road segment; 16. The computer-implemented method of claim 15.

19. The one or more processors process sensor data from the human operated vehicles operating throughout the region to determine right-of-way rules for each road segment of a road network in the region, the method further comprising: generating a set of autonomous driving maps for an autonomous vehicle traveling throughout the road network based on the right-of-way rules determined for each road segment of the road network; 16. The computer-implemented method of claim 15.

20. the sensor data received from the subset of human-operated vehicles is generated by a set of odometry sensors for each of the subset of human-operated vehicles; 16. The computer-implemented method of claim 15.

Images