System and method for determining ground based on sensor data from fout vehicle

Abstract

A computing system may receive sensor data from a set of human-driven vehicles operating on a road segment. The system may process the sensor data to determine a set of right-of-way rules for autonomous driving vehicles traveling on the road segment. In some examples, the system may obtain an autonomous driving map used by an autonomous driving vehicle for operation on the road segment, and modify the autonomous driving map to include the set of pass rules for the road segment.

Description

BACKGROUND OF THE INVENTION

[0001] Current autonomous driving implementations utilize high-precision autonomy maps recorded by mapping vehicles and marked based on signage and traffic lights. In some cases, labor-intensive manual marking is performed on the autonomy maps, which can lead to inaccuracies and other errors. In addition to basic geometric environment information (such as lane markings and sign positions), the autonomy maps may also include semantic information such as right-of-way rules for designated road segments of a road network. In some cases (e.g., due to region-specific rules or ambiguous traffic signs), the right-of-way rules cannot be directly sensed by the on-vehicle sensors of the mapping vehicle, which can lead to additional technical problems in generating a highly accurate and up-to-date autonomy map for use by autonomous and / or semi-autonomous vehicles operating in the road network. SUMMARY OF THE INVENTION

[0002] A system, method, and computer program product for determining right-of-way rules for a road segment based on sensor data from a fleet of human-driven vehicles are described according to example embodiments. The system can receive sensor data from a subgroup of human-driven vehicles operating on a road segment. The system can process the sensor data to determine a set of right-of-way rules for an autonomous and / or semi-autonomous vehicle traveling on the road segment. In some examples, the system can obtain an autonomous driving map used by the autonomous vehicle to operate on the road segment and modify the autonomous driving map to include the set of right-of-way rules for the road segment. Additionally or alternatively, the system can automatically generate and / or label an autonomous driving map of a road network to include the right-of-way rules. In further embodiments, the system can compare newly generated right-of-way labels on the autonomous driving map with previously generated right-of-way labels (e.g., to identify road segments where traffic control elements have actually changed between time periods). The system can also be used to verify right-of-way rules and other traffic control elements in an existing autonomous driving map (e.g., to verify that these elements are correct and up-to-date).

[0003] In various examples, the sensor data can indicate the vehicle trajectory of the human-driven vehicle on the road segment. The vehicle trajectory can indicate driving behaviors corresponding to multiple behavior categories, and the multiple behavior categories can include a braking behavior category, an acceleration behavior category, a stationary behavior category, a coasting behavior category, and a turning behavior category of the subgroup of human-driven vehicles on the road segment. In some examples, the computing system can overlay the vehicle trajectory on the map data to determine the set of right-of-way rules. The sensor data received from the subgroup of human-driven vehicles can be generated by a set of odometer sensors of each human-driven vehicle in the subgroup of human-driven vehicles. The set of odometer sensors includes one or more of a positioning system (e.g., a GNSS such as GPS or GLONASS), a braking sensor, a steering input sensor, a wheel speed sensor, or an acceleration sensor. It is contemplated that global position information may be crucial for aligning a given vehicle trajectory with other vehicle trajectories and a prior geometric map in the global reference system. The global position information may also be crucial for extracting motion patterns and driving behaviors in combination with other sensor information.

[0004] As provided herein, the autonomous driving map can be generated based at least in part on: map data obtained from one or more survey vehicles operating on the road segment, and / or labeled autonomous driving rules corresponding to at least one of the signs or signals along the road segment.

[0005] In certain implementations, the computing system can process sensor data from the human-driven vehicles operating throughout the region to determine right-of-way rules for each road segment of the road network of the region. In such examples, the computing system can generate a set of autonomous driving maps for autonomous vehicles traveling throughout the road network based on the right-of-way rules determined for each road segment of the road network. BRIEF DESCRIPTION OF THE DRAWINGS

[0006] The disclosure herein is illustrated by way of example and not limitation in the figures, and like reference numerals refer to like elements in the figures, and in which:

[0007] Figure 1 is a block diagram depicting an example computing system for implementing right-of-way determination based on sensor data from a fleet of vehicles according to examples described herein;

[0008] Figure 2 is a block diagram illustrating an example computing system according to examples described herein, the example computing system including a dedicated module for implementing right-of-way determination based on sensor data from a fleet of vehicles;

[0009] Figure 3ADepicts a mapped road segment according to an example described herein, where vehicle trajectories are overlaid on map data based on sensor data from a fleet of vehicles;

[0010] Figure 3B Depicts a mapped road segment according to an example described herein, where acceleration values of a fleet of vehicles are overlaid on map data based on sensor data from the fleet of vehicles;

[0011] Figure 4 Depicts a graph of speed and acceleration of corresponding vehicle trajectories described on a road segment according to an example described herein;

[0012] Figure 5 Is a flowchart depicting a method for determining the right of way of a road segment based on sensor data from a fleet of vehicles according to an example described herein; and

[0013] Figure 6 Is a flowchart depicting a method for determining the right of way of an autonomous vehicle on a road segment by generating vehicle trajectories based on sensor data from a fleet of vehicles according to an example described herein. Detailed Description

[0014] This document describes a computing system that provides right-of-way information for road segments in an autonomy map based on sensor data received from a fleet of human-driven vehicles operating on a specific road segment. As provided herein, the fleet of vehicles may include human-driven vehicles that include a set of odometer sensors and a communication interface for sending sensor data generated by the odometer sensors to the computing system via one or more networks. The odometer sensors may include one or more of a positioning system (e.g., Global Positioning System (GPS) or other Global Navigation Satellite System (GNSS)), a braking sensor, a steering input sensor, a wheel speed sensor, an acceleration sensor (e.g., an Inertial Measurement Unit), etc. In various embodiments, the computing system may include a communication interface for communicating with a fleet of human-driven vehicles operating throughout an area via one or more networks. The computing system may receive sensor data from a subgroup of the human-driven vehicles operating on a specific road segment within the area and process the sensor data to determine a set of right-of-way rules for autonomous vehicles traveling on the road segment.

[0015] As provided herein, a "network" or "one or more networks" can include any type of network or combination of networks that permits communication between devices. In one embodiment, the network can include one or more of a local area network, a wide area network, the Internet, a secure network, a cellular network, a mesh network, a peer-to-peer communication link, or some combination thereof, and can include any number of wired or wireless links. For example, communication over the network can be implemented using any type of protocol, security scheme, encoding, format, packetization, etc., via a network interface.

[0016] As provided herein, a "right-of-way rule" or "a set of right-of-way rules" is defined as a set of priorities for multiple competing paths, which can include road lanes (e.g., turning lanes, merging lanes, roundabouts, etc.), crosswalks, railroad crossings, etc. The set of priorities determines which competing paths must yield to other competing paths at any given time. Typically, the right-of-way is prescribed by lane markings, traffic control signals, traffic signs, and local regulations. It is contemplated that the right-of-way rules for competing lanes may change based on the time of day. For example, in some areas, traffic lights can be activated during the day and deactivated during late-night hours. Thus, the right-of-way rules for a given road segment may vary between traffic lights and static signs depending on the time of day.

[0017] In some examples, the computing system can modify an autonomous driving map (also referred to as an autonomy map) that includes the road segment to include the set of right-of-way rules for the road segment. The computing system can include a memory and database that stores the autonomous driving map, or can remotely obtain the autonomous driving map and append or otherwise edit the relevant autonomous driving map to include the right-of-way rules for a designated road segment determined from sensor data of a fleet of vehicles. A road segment can include one or more lanes that conflict or compete with one or more other lanes, such as turning lanes that merge onto a road, intersection lanes, highway on-ramps and off-ramps, roundabouts, etc.

[0018] As provided herein, an "autonomy map" or "self-driving map" includes a ground truth map that is recorded and marked by a survey vehicle using various sensors (e.g., LIDAR sensors and / or a set of cameras or other imaging devices) to indicate traffic and / or right-of-way rules at any given location. For example, a given autonomy map may be manually marked based on traffic signs, traffic lights, lane markings, and local regulations observed in the ground truth map. In a further example, reference points or other points of interest may be further marked on the autonomy map to provide additional assistance to the self-driving vehicle. The self-driving vehicle or autonomous vehicle may then utilize the marked autonomy map to perform localization, pose, change detection, and various other operations required for autonomous driving on public roads. For example, the self-driving vehicle may refer to the autonomy map to determine traffic rules (e.g., speed limits) at the current location of the vehicle, and may dynamically compare real-time sensor data from an on-vehicle sensor suite with the corresponding autonomy map to safely navigate along the current route.

[0019] In various embodiments, the computing system may process the sensor data from the fleet vehicles to generate vehicle trajectories of the fleet vehicles on the road segment. The vehicle trajectories may indicate driving behaviors of the drivers on the road segment, and the driving behaviors may correspond to multiple behavior categories, the multiple behavior categories including a braking behavior category, an accelerating behavior category, a stationary behavior category, a coasting behavior category, and / or a turning behavior category on the road segment. The vehicle trajectories may also indicate a set of time acceleration values (e.g., positive acceleration, deceleration, continuous speed values) of each vehicle on the road segment. In some examples, the computing system may also receive sensor data indicating vehicle speed, acceleration, yaw rate, etc. on the road segment and include this information in the vehicle trajectories.

[0020] As provided herein, the computing system may implement a learning-based method to classify the driving behaviors of the fleet vehicles on the road segment. In one example, the computing system implements a recurrent neural network to encode a sequence of geographical locations of each vehicle on the road segment, thereby deriving a corresponding driving pattern and classifying the driving behaviors of the vehicle. In such examples, the computing system may aggregate the classified driving behaviors of each lane segment temporally and statistically by counting the driving behaviors over a period of time. In a further example, the computing system may implement a learning-based method (e.g., a multi-layer perceptron) that obtains statistical values of the aggregated driving behaviors from competing lanes of the road segment and predicts or otherwise determines a set of right-of-way rules for the competing lanes.

[0021] As provided herein, the computing system may store or include one or more machine learning models. In one embodiment, the machine learning model may include an unsupervised learning model. In one embodiment, the machine learning model may include a neural network (e.g., a deep neural network) or other types of machine learning models, including non-linear models and / or linear models. The neural network may include a feedforward neural network, a recurrent neural network (e.g., a long short-term memory recurrent neural network), a convolutional neural network, or other forms of neural networks. Some example machine learning models may utilize an attention mechanism, such as self-attention. For example, some example machine learning models may include a multi-head self-attention model (e.g., a transformer model).

[0022] According to the examples described herein, an autonomous driving map may be generated at least in part based on: map data obtained from one or more survey vehicles operating on the road segment; and / or labeled autonomous driving rules corresponding to at least one of the signs or signals along the road segment. Additionally or alternatively, the autonomous driving map or certain elements of the autonomous driving map (e.g., indicating lane geometry or sign positions) may be generated based on a fleet of vehicles equipped with a set of sensors. After determining a set of right-of-way rules for the competing lanes of the road segment, the computing system may obtain the relevant autonomy map containing the road segment and modify the autonomy map to include a set of right-of-way rules for autonomous driving along the road segment. As provided herein, the autonomy map and / or right-of-way rules determined by the methods described herein may be used by semi-autonomous vehicles, fully autonomous vehicles, and / or used as a safety feature for human-driven vehicles.

[0023] Additionally or alternatively, the computing system may process sensor data from the human-driven vehicles operating throughout the area to determine right-of-way rules for each road segment of the road network in the area. After determining the right-of-way rules, the computing system may generate a set of autonomous driving maps for autonomous vehicles and / or semi-autonomous vehicles operating throughout the road network based on the right-of-way rules determined for each road segment of the road network. According to such embodiments, the road network does not need to use survey vehicles and manual markings to create an autonomy map because the right-of-way rules for all operable roads can be determined by the methods described herein.

[0024] Among other benefits, the examples described herein achieve the following technical effects, namely, using sensor data from vehicles in a human-driven vehicle fleet to determine the human driving behavior of competing lanes of a road segment. The examples described herein can generate a vehicle trajectory of the vehicle operating on the road segment, classify the driving behavior, and determine a set of right-of-way rules for the road segment. After determining the right-of-way rules, the examples described herein can modify or create an autonomy map of the road network, which includes right-of-way rules for semi-autonomous vehicles and / or autonomous vehicles to operate throughout the road network. Such examples provide technical solutions to various technical limitations existing in the field of autonomous vehicle navigation on public road networks. That is, relying on survey vehicles to generate an autonomy map template may result in blurred signs or traffic lights. In addition, relying on manual marking of these recorded autonomy maps may result in errors in determining the right-of-way, which may cause an autonomous vehicle to get stuck, where the vehicle cannot continue to drive due to unclear right-of-way rules.

[0025] One or more examples described herein provide methods, techniques, and actions performed by a computing device programmatically or as a computer-implemented method. As used herein, programmatically means by using code or computer-executable instructions. These instructions can be stored in one or more memory resources of the computing device. The steps performed programmatically may or may not be automatic.

[0026] One or more examples described herein can be implemented using programming modules, engines, or components. Programming modules, engines, or components can include programs, subroutines, parts of a program, or software components or hardware components capable of performing one or more of the tasks or functions described. As used herein, a module or component can exist on a hardware component independently of other modules or components. Alternatively, a module or component can be a shared element or process of other modules, programs, or machines.

[0027] Some examples described herein generally may require the use of a computing device, including processing resources and memory resources. For example, one or more examples described herein can be implemented in whole or in part on computing devices such as servers and / or personal computers using network equipment (e.g., routers). Memory resources, processing resources, and network resources can all be used in connection with the establishment, use, or execution of any example described herein (including in connection with the execution of any method or the implementation of any system).

[0028] In addition, one or more examples described herein can be implemented by using instructions executable by one or more processors. These instructions can be carried on a non-transitory computer-readable medium. The machines illustrated or described with the following figures provide examples of processing resources and computer-readable media on which the instructions for implementing the examples disclosed herein can be carried and / or executed. Specifically, many of the machines illustrated with examples of the present invention include a processor and various forms of memory for storing data and instructions. Examples of non-transitory computer-readable media include permanent memory storage devices, such as hard disk drives on a personal computer or server. Other examples of computer storage media include portable storage units, such as flash memory or magnetic memory. Computers, terminal devices, and network-enabled devices are examples of machines and devices that utilize a processor, memory, and instructions stored on a computer-readable medium. Additionally, the examples can be implemented in the form of a computer program or a computer-usable carrier medium capable of carrying such a program.

[0029] Example Computing System

[0030] Figure 1 is a block diagram depicting an example computing system 100 that implements right-of-way determination based on sensor data from fleet vehicles according to the examples described herein. In one implementation, the computing system 100 can include control circuitry 110, which can include one or more processors (e.g., microprocessors), one or more processing cores, programmable logic circuitry (PLC) or programmable logic array (PLA) / programmable gate array (PGA), field programmable gate array (FPGA), application specific integrated circuit (ASIC), or any other control circuitry. In some embodiments, the control circuitry 110 and / or the computing system 100 can be part of a vehicle control unit (also referred to as a vehicle controller) embedded or otherwise disposed in a vehicle (e.g., a Mercedes- Benz or a van), or can form the vehicle control unit. For example, the vehicle controller can be or can include an infotainment system controller (e.g., an infotainment host unit), a telematics control unit (TCU), an electronic control unit (ECU), a central powertrain controller (CPC), a central external and internal controller (CEIC), a zone controller, or any other controller (the term "or" is used interchangeably with "and / or" herein).

[0031] In one embodiment, the control circuit 110 may be programmed by one or more computer-readable or computer-executable instructions stored on a non-transitory computer-readable medium 120. The non-transitory computer-readable medium 120 may be a memory device (also referred to as a data storage device), which may include an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination thereof. The non-transitory computer-readable medium 120 may form, for example, a computer floppy disk, a hard disk drive (HDD), a solid-state drive (SDD), or a solid-state integrated memory, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a dynamic random access memory (DRAM), a portable compact disc read-only memory (CD-ROM), a digital versatile disc (DVD), and / or a memory stick. In some cases, the non-transitory computer-readable medium 120 may store computer-executable instructions or computer-readable instructions, such as instructions for performing the methods described in connection with Figure 5 and Figure 6 .

[0032] In various embodiments, the terms "computer-readable instructions" and "computer-executable instructions" are used to describe software instructions or computer code configured to perform various tasks and operations. In various embodiments, if the computer-readable or computer-executable instructions form a module, the term "module" broadly refers to a collection of software instructions or code configured to cause the control circuit 110 to perform one or more functional tasks. Modules and computer-readable / executable instructions may be described as performing various operations or tasks when the control circuit 110 or other hardware components are executing the module or computer-readable instructions.

[0033] In a further embodiment, the computing system 100 may include a communication interface 140 that enables communication via one or more networks 150 to send and receive data. In various examples, the computing system 100 may use the communication interface 140 to communicate with fleet vehicles via one or more networks to receive sensor data and implement the right-of-way determination methods described throughout this disclosure. In certain embodiments, the communication interface 140 may be used to communicate with one or more other systems. The communication interface 140 may include any circuitry, components, software, etc. for communicating via one or more networks 150 (e.g., a local area network, a wide area network, the Internet, a secure network, a cellular network, a mesh network, and / or a peer-to-peer communication link). In some specific implementations, the communication interface 140 may include, for example, one or more of a communication controller, a receiver, a transceiver, a transmitter, a port, a conductor, software, and / or hardware for conveying data / information.

[0034] As an example implementation, the computing system 100 can receive sensor data from a fleet of vehicles via one or more networks 150 using the communication interface 140. The control circuit 110 can process the sensor data by executing instructions accessed from the non-transitory computer-readable medium 120 to generate a vehicle trajectory of the fleet of vehicles on a particular road segment having competing lanes and determine a set of right-of-way rules for the road segment based on the vehicle trajectory. The control circuit 110 can then use the right-of-way rules to automatically label an autonomy map for use by semi-autonomous or fully autonomous vehicles when navigating through the road network. A further description of the functionality of the computing system 100 is provided below.

[0035] System Description

[0036] Figure 2 FIG. 7 is a block diagram of an example computing system 200 that illustrates an example according to the examples described herein, the example computing system including functional modules for implementing right-of-way determination based on sensor data from a fleet of vehicles. As provided herein, the computing system 200 can include a communication interface 205 that is configured to communicate with a fleet of human-driven vehicles 250 via one or more networks 260 to receive sensor data indicative of the movement and / or control inputs of each vehicle on a particular road segment. As further provided herein, the sensor data can be received from a set of sensors housed on each vehicle, such as a positioning system (e.g., GPS or other GNSS) and / or one or more odometer sensors (e.g., wheel rotation sensors, brake input sensors, steering input sensors, acceleration sensors). Accordingly, the sensor data can include real-time position data and / or odometer information (e.g., speed, acceleration, yaw rate) of the fleet of vehicles 250 as they traverse the road segment.

[0037] In various embodiments, computing system 200 may include a trajectory generator module 210, a right-of-way determination module 220, and a mapping module 230. In a further example, computing system 200 may include a database 240 that stores a set of autonomy maps 242 for use by autonomous and / or semi-autonomous vehicles to operate throughout an area. Specifically, autonomy maps 242 may be created based on a mapping vehicle that, as it travels on the road network on which autonomous vehicles operate, uses a sensor suite (e.g., including LIDAR sensors, image sensors, etc.) to generate map data. The map data may be appended with one or more layers of tagged data that indicate the designated traffic rules for any given road segment (e.g., speed limits, signage, crosswalk information, traffic signals, etc.). An autonomous vehicle may continuously compare real-time sensor data generated by an on-board sensor suite with the relevant autonomy map to perform a localization and pose process to assist the autonomous or semi-autonomous vehicle in safely operating within the road network.

[0038] In some examples, the trajectory generator module 210 may receive sensor data from a subgroup of fleet vehicles 250 that operate on a particular road segment involving competitive lanes (such as a turning lane merging onto a road). The trajectory generator module 210 may generate vehicle trajectories for the subgroup of fleet vehicles 250 on the particular road segment. As described in connection with Figure 3A and Figure 3B below, the vehicle trajectories may indicate the path of each vehicle on the road segment. In a further example, the vehicle trajectories may be overlaid on the map data to indicate a time series of the movement of each vehicle on the road segment, such as the trajectories of acceleration, coasting, deceleration, and turning as the vehicle traverses the road segment.

[0039] In various embodiments, the right-of-way determination module 220 may process vehicle trajectories and sensor data to classify the driving behavior of each vehicle on the road segment. In some examples, the right-of-way determination module 220 may implement a learning-based approach to process vehicle trajectory information and derive the driving patterns of vehicles on the road segment. For example, the right-of-way determination module 220 may execute an artificial neural network (e.g., a recurrent neural network and / or a multi-layer perceptron) that processes the temporal trajectory of the vehicle paths on the road segment as input and outputs a prediction of the right-of-way rules for the road segment. For example, the right-of-way rule determination module 220 may execute an artificial neural network, where the temporal trajectory of the vehicle path is received as sequential input by certain nodes of the neural network to output a prediction of the right-of-way rules for the road segment (e.g., using the determined right-of-way rules to label the autonomy map). After determining the right-of-way rules, the autonomous vehicle and / or the advanced driver assistance system may apply the right-of-way rules when navigating the corresponding road segment.

[0040] In a further example, the right-of-way determination module 220 may process the temporal and statistical aggregation of the driving behavior on the road segment and on the competing lanes of the road segment over a period of time to output the right-of-way rules. In an even further example, the right-of-way determination module 220 may be trained using a ground truth map (e.g., recorded by a survey vehicle and marked based on signs and traffic lights). Thus, given a set of vehicle trajectories on any road segment of the road network on which the autonomous vehicle will operate, the right-of-way determination module 220 may output a set of predicted or actual right-of-way rules for each lane of the road segment.

[0041] According to the examples described herein, the mapping module 230 may utilize the right of way output from the right of way determination module 220 to verify labels on an existing autonomy map, modify the existing autonomy map to include the right of way rules for the road segment, or generate a new autonomy map for the road segment to include the right of way rules. In various examples, the mapping module 230 may include an autonomy map validator that determines whether the right of way rules marked on the existing autonomy map 242 are accurate. For example, the autonomy map stored in the database 240 or accessed remotely may include the road segments on which the subgroup of fleet vehicles 250 travels. After determining the right of way rules for a road segment, the mapping module 230 may perform a lookup of the relevant autonomy map including the road segment and compare the marked right of way rules (e.g., as manually marked) with the right of way rules output by the right of way determination module 220. In a further example, if the right of way rules do not match, the mapping module 230 may automatically mark the differences for further processing and marking, or automatically re - mark the autonomy using the right of way rules determined by the right of way determination module 220. In a further example, the right of way rules may be used to verify other traffic control elements in the autonomy map since the right of way rules are based on such elements. For example, the mapping module 230 may verify the labels identifying and / or classifying traffic lights, traffic signs, stop line markings, crosswalks, etc. in the autonomy map based on the right of way rules determined by the right of way determination module 220.

[0042] Additionally or alternatively, the mapping module 230 may utilize the right of way rules output by the right of way determination module 220 to automatically mark an existing autonomy map recorded by a mapping vehicle. For example, the mapping module 230 may replace certain manual marking functions when creating the autonomy map 242 for an autonomous vehicle and / or a semi - autonomous vehicle. In such examples, the mapping vehicle may still be used to record the ground truth map of a given road network, and the mapping module 230 may automatically assign right of way information at each road segment involving competing lanes to the ground truth map. In a still further example, the mapping module 230 may utilize the right of way output from the right of way determination module 220 and use road network data (e.g., an existing ground truth map or a virtualized ground truth map based on road network information) to generate an autonomy map.

[0043] It is contemplated that the right-of-way determination method described throughout this disclosure may be performed for any road segment involving competing lanes and may be used to supplement existing marking functions or replace existing marking functions (e.g., manual marking) of the autonomy map 242. It is further contemplated that a new autonomy map may be generated by the mapping module 230 using the methods described herein. For example, in addition to determining right-of-way rules, the computing system 200 may also infer road sign rules, speed limits, traffic signal locations, crosswalk locations, etc. based on sensor data and vehicle trajectories of the platoon vehicles 250 on a designated road segment. Thus, the methods described herein may be used to supplement or eliminate the use of mapping vehicles and the manual or automatic marking of ground truth maps.

[0044] It is further contemplated that the right-of-way determination method described herein may also utilize external sensor data from sensors at fixed locations whose field of view includes a road segment having competing lanes (e.g., image sensors, LIDAR sensors, radar, sonar, infrared, etc.). Thus, the trajectory generator module 210 may utilize additional sensor information from fixed sensors located near the road segment to supplement the sensor data received from the platoon vehicles 250 to determine right-of-way rules.

[0045] Vehicle Trajectory

[0046] Figure 3A Depicted is a mapped road segment 305 according to an example described herein, where vehicle trajectories 310 are overlaid on map data 300 based on sensor data from platoon vehicles. In Figure 3A the example shown, the road segment 305 merges onto competing lanes 315, and the vehicle trajectories 310 include the temporal positions of each platoon vehicle on the road segment. As described herein, the vehicle trajectories 310 are matched to the road segment 305 (e.g., overlaid on map data 300) to indicate the paths taken by the platoon vehicles on the road segment 305. As described below with respect to Figure 3B what is described, the vehicle trajectories 310 may also include information corresponding to the acceleration values of the platoon vehicles on the road segment 305.

[0047] Figure 3B Depicted is a road segment 305 according to an example described herein, where the acceleration values of the platoon vehicles are overlaid on map data 300 based on sensor data from the platoon vehicles. In Figure 3B the example shown, Figure 3AThe vehicle trajectory 310 in [it] may include additional sensor information (e.g., position data varying over time, wheel rotation information, speed and acceleration data, etc.), and the computing system 200 may use this additional sensor information to generate a position acceleration value 360 and a deceleration value 370 (as well as a coasting trajectory) for the platoon vehicles as they travel on the road segment 305. As described herein, the computing system 200 may analyze the vehicle trajectory 310, the acceleration value 360, and the deceleration value 370 to determine the right-of-way rules between the road segment 305 and the competing lane 315. Specifically, the computing system 200 may analyze the vehicle trajectory 310 and the sensor information encoded therein (e.g., the acceleration value 360 and the deceleration value 370) to determine or classify the driving behavior of the platoon vehicles. As Figure 3B shown, the acceleration value 360 and the deceleration value 370 of the platoon vehicle on the road segment 305 indicate a braking pattern before accelerating to merge into the competing lane, which may indicate that the competing lane 315 has the right-of-way over this road segment.

[0048] Figure 4 Figure 400 depicts a graph of the speed 410 and acceleration 405 of the corresponding vehicle trajectories on the road segment according to the examples described herein. Figure 4 The speed 410 line and the acceleration 405 line shown may correspond to Figure 3A a specific vehicle trajectory 310, and the acceleration value 360 and the deceleration value 370 of a specific platoon vehicle traveling on the road segment 305. As Figure 4 shown, the information encoded in the vehicle trajectory 310 may include the time values of the speed 410 of the vehicle on the road segment 305, and the time values of the acceleration 405 of the vehicle. In Figure 4 the example shown, the platoon vehicle decelerates and reduces its speed while traveling on the road segment 305, and then accelerates as the platoon vehicle leaves the road segment 305. As provided herein, the computing system 200 may process this vehicle trajectory information from multiple platoon vehicles traveling on the same road segment 305 to determine the right-of-way rules for the road segment 305 and any competing lane 315.

[0049] Method

[0050] Figure 5 and Figure 6 are flowcharts describing methods for determining the right-of-way of a road segment based on sensor data from platoon vehicles according to the examples described herein. In the following discussion of the Figure 5 and Figure 6 methods, reference may be made to reference characters representing certain features described in the system diagrams regarding Figure 1 and Figure 2 . Additionally, reference is made to Figure 5 andFigure 6 The steps described by the flowchart of Figure 1 and Figure 2 can be performed by the computing systems 100, 200 shown and described with respect to Figure 5 and Figure 6 Certain steps described by the flowchart of

[0051] can be performed before, in combination with, or after any other step, without necessarily being performed in the corresponding sequence shown. Figure 5 Referring to Figure 6 more details are provided regarding the determination of the right of way for autonomous vehicles.

[0052] In certain embodiments, at block 510, the computing system may obtain an autonomous driving map 242 used by the autonomous vehicle and / or semi-autonomous vehicle for operation on the road segment. As described herein, the autonomous driving map 242 may include a ground truth map that includes map data recorded and marked (e.g., manually marked) by a survey vehicle to indicate traffic rules (e.g., based on road signs and manually identified traffic lights) of the road segment. Autonomous vehicles, semi-autonomous vehicles, or other self-driving vehicles may utilize the marked autonomy map to perform positioning, pose, change detection, and various other operations required for autonomous driving on the road segment. As further provided herein, the autonomous driving map 242 may be remotely accessed from a third-party database or locally obtained from an autonomy map database 240.

[0053] At block 520, computing system 200 may then modify the autonomous driving map 242 to include a set of one or more right-of-way rules for the road segment determined based on the sensor data from the fleet vehicles 250. In some examples, modifying the autonomous driving map 242 may include automatically tagging the autonomous driving map to include the right-of-way rules for the road segment. Additionally or alternatively, modifying the autonomous driving map 242 may involve automatically tagging a ground truth map (which includes the raw sensor data recorded by the mapping vehicle) to include the right-of-way rules determined based on the sensor data from the fleet vehicles 250. In further examples, computing system 200 may verify right-of-way tags on the autonomous driving map (e.g., as manually entered), and / or may use the determined right-of-way information and other known information about the road network (e.g., signage, signal locations, speed limits, etc.) to generate a new autonomous driving map. In further embodiments, computing system 200 may replace incorrect right-of-way rules (e.g., as manually entered), or may input right-of-way rules into the autonomous driving map 242 that does not yet include right-of-way rules.

[0054] Figure 6 is a flowchart depicting a method for determining the right-of-way of an autonomous vehicle on a road segment by generating a vehicle trajectory based on sensor data from fleet vehicles as described herein. Referring Figure 6 to block 600, computing system 200 may receive sensor data from fleet vehicles 250 operating on a particular road segment. At block 605, computing system 200 may also generate vehicle trajectories for each vehicle on the road segment on the map data. For example, the vehicle trajectories may indicate the path of each vehicle on the road segment, as well as the time and position acceleration values and deceleration values (and coasting values) of the vehicle as it traverses the road segment. In further examples, computing system 200 may encode additional data in the vehicle trajectories, such as brake input information, yaw rate, wheel speed information, etc.

[0055] In further examples, the vehicle trajectories may indicate driving behaviors corresponding to multiple behavior categories, the multiple behavior categories including a braking behavior category, an acceleration behavior category, a stationary behavior category, a coasting behavior category, and a turning behavior category of the subgroup of human-driven vehicles on the road segment. Computing system 200 may process the vehicle trajectories to determine the driving patterns of the fleet vehicles 250 operating on the road segment, and may classify one or more driving behaviors indicative of one or more right-of-way rules for the road segment.

[0056] At block 610, computing system 200 may process the vehicle trajectory to determine a set of one or more right-of-way rules for the road segment. The set of right-of-way rules may indicate whether the road segment (e.g., a particular lane) has the right-of-way over competing road segments or lanes. In a further example, the road segment may have multiple competing lanes with different right-of-way rules. Thus, the set of right-of-way rules may indicate that a particular road segment has the right-of-way over a first competing lane but must defer the right-of-way to a second competing lane. Accordingly, based on the vehicle trajectory, computing system 200 may determine the right-of-way rules for each particular lane in the competing lane region. In some implementations, computing system 200 may generate time-specific right-of-way rules (e.g., when timestamp information is included in the sensor data). For example, computing system 200 may determine time blocks during which traffic signals control the right-of-way rules for a particular road segment (e.g., during daytime hours), and time blocks during which the traffic signals are deactivated and the right-of-way depends on static signage.

[0057] In various implementations, computing system 200 may act as a label validator for an autonomous driving map 242 that has been marked with right-of-way rules (e.g., by manual marking). At block 615, computing system 200 validates the right-of-way rules for the road segment marked on the existing autonomous driving map 242. Additionally or alternatively, at block 620, computing system 200 may edit or modify the autonomous driving map or an unlabeled ground truth map to include the right-of-way rules for the road segment.

[0058] In a further example, at block 625, computing system 200 may generate one or more autonomous driving maps to include the right-of-way rules for an entire road network based on the processes described herein. In such examples, computing system 200 may process sensor data from any number of road segments with right-of-way conflicts to determine the right-of-way rules for the entire road network (e.g., an autonomous driving grid in which autonomous vehicles are permitted to operate). Computing system 200 may then supplement or replace existing autonomous driving mapping and labeling methods currently used in the art.

[0059] It is contemplated to extend the examples described herein to the individual elements and concepts described herein (independent of other concepts, ideas, or systems), and to extend the examples to include combinations of elements described anywhere in the present invention. Although the examples are described in detail herein with reference to the accompanying drawings, it should be understood that these concepts are not limited to those exact examples. Thus, many modifications and variations will be apparent to those skilled in the art. Accordingly, the scope of these concepts is intended to be defined by the following claims and their equivalents. Additionally, it is contemplated that specific features described separately or as part of an example may be combined with other separately described features or part of other examples, even if the other features and examples do not mention that specific feature. Thus, the lack of description of a combination should not preclude claiming such a combination.

Claims

1. A computing system, the computing system comprising: A communication interface configured to communicate with human-driven vehicles operating throughout an area via one or more networks; One or more processors; A memory storing instructions that, when executed by the one or more processors, cause the computing system to: Receive sensor data from a subgroup of the human-driven vehicles operating on a road segment within the area via the one or more networks; Process the sensor data to determine a set of right-of-way rules for an autonomous vehicle traveling on the road segment; Obtain an autonomous driving map for use by the autonomous vehicle to operate on the road segment; And Modify the autonomous driving map to include the set of right-of-way rules for the road segment.

2. The computing system according to claim 1, wherein the sensor data indicates vehicle trajectories of the subgroup of human-driven vehicles on the road segment, the vehicle trajectories indicating driving behaviors corresponding to a plurality of behavior categories, the plurality of behavior categories including a braking behavior category, an accelerating behavior category, a stationary behavior category, a coasting behavior category, and a turning behavior category of the subgroup of human-driven vehicles on the road segment.

3. The computing system according to claim 1, wherein the executed instructions cause the computing system to overlay the vehicle trajectories on map data to determine the set of right-of-way rules.

4. The computing system according to claim 1, wherein the autonomous driving map is generated at least in part based on: (i) map data obtained from one or more survey vehicles operating on the road segment, and / or (ii) labeled autonomous driving rules corresponding to at least one of signs or signals along the road segment.

5. The computing system according to claim 1, wherein the executed instructions cause the computing system to process sensor data from the human-driven vehicles operating throughout the area to determine right-of-way rules for each road segment of the road network of the area, and wherein the executed instructions further cause the computing system to: Generate a set of autonomous driving maps for autonomous vehicles traveling throughout the road network based on the right-of-way rules determined for each road segment of the road network.

6. The computing system according to claim 1, wherein the sensor data received from the subgroup of human-driven vehicles is generated by a set of odometer sensors of each human-driven vehicle in the subgroup of human-driven vehicles.

7. The computing system according to claim 6, wherein the set of odometer sensors includes one or more of a positioning system, a braking sensor, a steering input sensor, a wheel speed sensor, or an acceleration sensor.

8. A non-transitory computer-readable medium storing instructions that, when executed by one or more processors of a computing system, cause the computing system to: Receive sensor data from a subgroup of human-driven vehicles operating on a road segment within an area via one or more networks; Process the sensor data to determine a set of right-of-way rules for an autonomous vehicle traveling on the road segment; Obtain an autonomous driving map used by the autonomous vehicle to operate on the road segment; and Modify the autonomous driving map to include the set of right-of-way rules for the road segment.

9. The non-transitory computer-readable medium according to claim 8, wherein the sensor data indicates vehicle trajectories of the subgroup of human-driven vehicles on the road segment, and the vehicle trajectories indicate driving behaviors corresponding to a plurality of behavior categories, the plurality of behavior categories including a braking behavior category, an accelerating behavior category, a stationary behavior category, a coasting behavior category, and a turning behavior category of the subgroup of human-driven vehicles on the road segment.

10. The non-transitory computer-readable medium according to claim 8, wherein the executed instructions cause the computing system to overlay the vehicle trajectories on map data to determine the set of right-of-way rules.

11. The non-transitory computer-readable medium according to claim 8, wherein the autonomous driving map is at least partially generated based on: (i) map data obtained from one or more surveying vehicles operating on the road segment, and (ii) labeled autonomous driving rules corresponding to at least one of signs or signals along the road segment.

12. The non-transitory computer-readable medium according to claim 8, wherein the executed instructions cause the computing system to process sensor data from human-driven vehicles operating throughout the region to determine right-of-way rules for each road segment of the road network of the region, and wherein the executed instructions further cause the computing system to: Generate a set of autonomous driving maps for autonomous vehicles traveling throughout the road network based on the right-of-way rules determined for each road segment of the road network.

13. The non-transitory computer-readable medium according to claim 8, wherein the sensor data received from the subgroup of human-driven vehicles is generated by a set of odometer sensors of each human-driven vehicle in the subgroup of human-driven vehicles.

14. The non-transitory computer-readable medium according to claim 13, wherein the set of odometer sensors includes one or more of a positioning system, a braking sensor, a steering input sensor, or an acceleration sensor.

15. A computer-implemented method executed by one or more processors, the computer-implemented method comprising: Receiving, via one or more networks, sensor data from a subgroup of human-driven vehicles operating on a road segment within a region; Processing the sensor data to determine a set of right-of-way rules for an autonomous vehicle traveling on the road segment; Obtaining an autonomous driving map used by the autonomous vehicle to operate on the road segment; and Modifying the autonomous driving map to include the set of right-of-way rules for the road segment.

16. The computer-implemented method according to claim 15, wherein the sensor data indicates vehicle trajectories of the subgroup of human-driven vehicles on the road segment, the vehicle trajectories indicating driving behaviors corresponding to a plurality of behavior categories, the plurality of behavior categories including a braking behavior category, an acceleration behavior category, a stationary behavior category, a coasting behavior category, and a turning behavior category of the subgroup of human-driven vehicles on the road segment.

17. The computer-implemented method according to claim 16, wherein the one or more processors superimpose the vehicle trajectories on map data to determine the set of right-of-way rules.

18. The computer-implemented method according to claim 15, wherein the autonomous driving map is generated at least in part based on: (i) map data obtained from one or more survey vehicles operating on the road segment, and (ii) labeled autonomous driving rules corresponding to at least one of the signs or signals along the road segment.

19. The computer-implemented method according to claim 15, wherein the one or more processors process sensor data from human-driven vehicles operating throughout the area to determine right-of-way rules for each road segment of the road network of the area, the method further comprising: generating a set of autonomous driving maps for autonomous vehicles traveling throughout the road network based on the right-of-way rules determined for each road segment of the road network.

20. The computer-implemented method according to claim 15, wherein the sensor data received from the subgroup of human-driven vehicles is generated by a set of odometer sensors of each human-driven vehicle in the subgroup of human-driven vehicles.

Images