Dynamic Allocation of Communication Network Resources for Autonomous Vehicles

Abstract

Various embodiments of the present invention provide methods, apparatus, systems, computing devices, computing entities, and / or the like for dynamic allocation of communication network resources by autonomous vehicles. For example, certain embodiments of the present invention utilize systems, methods, and computer program products that determine a data transmission prioritization policy and cause ancillary data objects to be provided based, at least in part, on the data transmission prioritization policy.

Description

[Technical field]

[0001] (CROSS REFERENCE TO RELATED APPLICATIONS) This application claims the benefit of U.S. Provisional Application No. 63 / 269,679, filed March 21, 2022, the entire contents of which are incorporated herein by reference.

[0002] The present disclosure relates generally to autonomous vehicles and, more particularly, to dynamic allocation of communication network resources by autonomous vehicles (AVs). [Background technology]

[0003] One goal of autonomous vehicle technology is to provide a vehicle that can safely navigate to a destination with limited or no driver assistance. Safe navigation of an AV from one point to another may include the ability to signal other vehicles, navigate around other vehicles on shoulders or emergency lanes, change lanes, bias appropriately in lanes, and navigate all portions or types of highway lanes. Autonomous vehicle technology may enable an AV to operate without requiring extensive learning or training by surrounding drivers by ensuring that the AV can operate safely in a manner that is clear, logical, or familiar to surrounding drivers and pedestrians.

[0004] During operation, an AV may be equipped with a local on-board control computer capable of determining one or more AV operations. The AV may also provide data to an operations server, which in turn may provide instructions and / or supplemental instructions to the AV. Summary of the Invention [Means for solving the problem]

[0005] Generally, embodiments of the present invention provide methods, apparatus, systems, computing devices, computing entities, and / or the like that enable an AV to dynamically allocate communication network resources. For example, certain embodiments of the present invention utilize systems, methods, and computer program products that determine a data transmission prioritization policy for transmission of auxiliary data, which in turn triggers transmission of auxiliary data objects of the auxiliary data.

[0006] According to one aspect, a control subsystem for an autonomous vehicle is provided. The control subsystem may comprise processing circuitry configured to determine a measure of auxiliary communication network resource availability associated with the auxiliary data. The processing circuitry may be further configured to determine a data transmission prioritization policy that a) describes one or more policies regarding transmission of the auxiliary data and b) is determined, at least in part, based on one or more operating conditions associated with the AV. The processing circuitry may be further configured to cause transmission of auxiliary data objects of the auxiliary data from the AV, the auxiliary data objects being generated, at least in part, based on the data transmission prioritization policy.

[0007] In some embodiments, the processing circuitry may be further configured to determine a measure of overall communications network resource availability available to the AV. The processing circuitry may be further configured to determine a measure of required communications network resource availability associated with the required data. The processing circuitry may be further configured to determine a measure of auxiliary communications network resource availability associated with the auxiliary data based, at least in part, on the measure of overall communications network resource availability and the measure of required communications network resource availability.

[0008] In some embodiments, the processing circuitry may be further configured to receive one or more signals from one or more sensors carried by the AV. The processing circuitry may be further configured to determine one or more operating states for the AV based, at least in part, on the one or more received signals. The processing circuitry may be further configured to update the data transmission prioritization policy in instances where the one or more determined operating states for the AV differ from one or more preceding operating states.

[0009] In some embodiments, the processing circuitry may be further configured to receive one or more transmission type requests comprising one or more requests for auxiliary data. The processing circuitry may be further configured to update a data transmission prioritization policy based, at least in part, on the one or more transmission type requests.

[0010] According to another aspect, a computer program product is provided. The computer program product may comprise at least one non-transitory computer-readable storage medium having computer-executable program code instructions stored therein, the computer-executable program code instructions being configured to determine a measure of auxiliary communication network resource availability associated with auxiliary data. The computer-executable program code instructions comprising the program code instructions may be further configured to a) describe one or more policies regarding transmission of the auxiliary data, and b) determine a data transmission prioritization policy determined, at least in part, based on one or more operating conditions associated with the AV. The computer-executable program code instructions comprising the program code instructions may be further configured to cause an auxiliary data object of the auxiliary data to be transmitted from the AV, the auxiliary data object being generated, at least in part, based on the data transmission prioritization policy.

[0011] In some embodiments, the computer-executable program code instructions comprising the program code instructions may be further configured to determine a measure of overall communications network resource availability available to the AV. The computer-executable program code instructions comprising the program code instructions may be further configured to determine a measure of required communications network resource availability associated with the required data. The computer-executable program code instructions comprising the program code instructions may be further configured to determine a measure of auxiliary communications network resource availability associated with the auxiliary data based at least in part on the overall measure of communications network resource availability and the measure of required communications network resource availability.

[0012] In some embodiments, the computer-executable program code instructions comprising the program code instructions may be further configured to receive one or more signals from one or more sensors carried by the AV. The computer-executable program code instructions comprising the program code instructions may be further configured to determine one or more operational states for the AV based, at least in part, on the one or more received signals. The computer-executable program code instructions comprising the program code instructions may be further configured to update a data transmission prioritization policy in instances where the one or more determined operational states for the AV differ from one or more preceding operational states.

[0013] In some embodiments, the computer-executable program code instructions comprising the program code instructions may be further configured to receive one or more transmission type requests comprising one or more requests for auxiliary data. The computer-executable program code instructions comprising the program code instructions may be further configured to update a data transmission prioritization policy based, at least in part, on the one or more transmission type requests.

[0014] According to another aspect, a method is provided. The method may include determining a measure of auxiliary communication network resource availability associated with auxiliary data. The method may further include a) describing one or more policies regarding transmission of the auxiliary data, and b) determining a data transmission prioritization policy determined at least in part based on one or more operating conditions associated with the AV. The method may further include causing an auxiliary data object of the auxiliary data to be transmitted from the AV, the auxiliary data object being generated at least in part based on the data transmission prioritization policy.

[0015] In some embodiments, the method may further include determining a measure of overall communications network resource availability available to the AV. The method may further include determining a measure of required communications network resource availability associated with the required data. The method may further include determining a measure of auxiliary communications network resource availability associated with the auxiliary data based at least in part on the measure of overall communications network resource availability and the measure of required communications network resource availability.

[0016] In some embodiments, the method may further include receiving one or more signals from one or more sensors carried by the AV. The method may further include determining one or more operational states for the AV based, at least in part, on the one or more received signals. The method may further include updating a data transmission prioritization policy in instances where the one or more determined operational states for the AV differ from one or more preceding operational states.

[0017] In some embodiments, the method may further include receiving one or more transmission type requests comprising one or more requests for auxiliary data. The method may further include updating a data transmission prioritization policy based, at least in part, on the one or more transmission type requests.

[0018] According to another aspect, an apparatus is provided. The apparatus may be configured with means for determining a measure of auxiliary communication network resource availability associated with auxiliary data. The apparatus may be configured with means for determining a data transmission prioritization policy that a) describes one or more policies regarding transmission of the auxiliary data, and b) is determined at least in part based on one or more operating conditions associated with the AV. The apparatus may be configured with means for causing an auxiliary data object of the auxiliary data to be transmitted from the AV, the auxiliary data object being generated at least in part based on the data transmission prioritization policy.

[0019] In some embodiments, the apparatus may be configured with means for determining a measure of overall communications network resource availability available to the AV. The apparatus may be configured with means for determining a measure of required communications network resource availability associated with the required data. The apparatus may be configured with means for determining a measure of auxiliary communications network resource availability associated with the auxiliary data based at least in part on the measure of overall communications network resource availability and the measure of required communications network resource availability.

[0020] In some embodiments, the apparatus may be configured with means for receiving one or more signals from one or more sensors carried by the AV. The apparatus may be configured with means for determining one or more operational states for the AV based, at least in part, on the one or more received signals. The apparatus may be configured with means for updating a data transmission prioritization policy in instances where the one or more determined operational states for the AV differ from one or more preceding operational states.

[0021] In some embodiments, the apparatus may be configured with means for receiving one or more transmission type requests comprising one or more requests for auxiliary data. The apparatus may be configured with means for updating a data transmission prioritization policy based, at least in part, on the one or more transmission type requests.

[0022] In some embodiments, the data transmission prioritization policy describes at least a data pre-processing policy that indicates how data should be processed prior to being transmitted from the AV. In some embodiments, the data transmission prioritization policy describes at least one or more data types associated with auxiliary data to be transmitted from the AV. In some embodiments, the data transmission prioritization policy describes at least a frequency at which auxiliary data should be transmitted from the AV. In some embodiments, the one or more operating conditions describe one or more of environmental operating conditions, driving events, AV operating conditions, or navigation operating conditions. In some embodiments, a data transmission prioritization machine learning model is used to determine the data transmission prioritization policy. [Brief description of the drawings]

[0023] For a more complete understanding of the present disclosure, reference is now made to the following brief description taken in conjunction with the accompanying drawings and detailed description, in which like reference numerals represent like parts.

[0024] [Figure 1] FIG. 1 illustrates an exemplary communications system that can be used to practice embodiments of the present invention.

[0025] [Diagram 2] FIG. 2 illustrates a block diagram of a control subsystem according to certain embodiments discussed herein.

[0026] [Diagram 3] FIG. 3 illustrates a block diagram of an operations server according to some embodiments discussed herein.

[0027] [Figure 4] FIG. 4 illustrates a schematic diagram of an example autonomous vehicle configured to implement autonomous driving operations according to certain embodiments discussed herein.

[0028] [Diagram 5] FIG. 5 illustrates a schematic diagram of an on-board control computer included within an AV, according to some embodiments discussed herein.

[0029] [Figure 6] FIG. 6 is a flowchart diagram of an example process for allocating communication network resources according to certain embodiments discussed herein.

[0030] [Figure 7] FIG. 7 is a flowchart diagram of an example process for updating a data transmission prioritization policy according to certain embodiments discussed herein. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0031] Detailed Description Traditionally, data transmission may be determined at least in part based on quality of service (QoS) parameters, which may be determined at least in part based on packet loss, bit rate, throughput, transmission delay, availability, jitter, etc. associated with the data transmission. However, embodiments of the present invention contemplate a data transmission decision based at least in part on overall communication network resource availability (e.g., bandwidth) without the need to consider the QoS associated with the data transmission. This allows for a simpler decision regarding the transmission of data, and thus allows for reduced latency and overall bandwidth usage when transmitting data. In particular, embodiments of the present invention contemplate the transmission of essential data and auxiliary data using a data transmission prioritization policy based at least in part on overall communication network resource availability. The data transmission prioritization policy, which may define one or more policies regarding the transmission of auxiliary data, may be determined at least in part based on one or more operating conditions associated with the AV. In some embodiments, a data transmission prioritization machine learning model may be used to determine the data transmission prioritization policy, such as by using one or more weighted combinations of one or more operating conditions corresponding to the AV. The auxiliary data objects of the auxiliary data may then be transmitted from the AV based, at least in part, on the data transmission prioritization policy. Thus, the AV may take advantage of auxiliary bandwidth network resource availability and provide, to one or more computing entities, auxiliary data that is determined to have the highest inferred importance based, at least in part, on one or more current operating conditions of the AV.

[0032] FIG. 1 illustrates an exemplary system 100 including one or more AVs 105, an operations server 1500 with a human operator 355, and an interface 362 for third party 360 interaction. The human operator 355 may also be known as a remote center operator (RCO). Communication between the AVs 105, the operations server 1500, and the user interface 362 occurs via a network 370. In some cases, if not all of the AVs 105 in a fleet are capable of communicating with the operations server 1500, the AVs 105 may communicate with each other via the network 370 or directly. Each AV 105 may include a vehicle control computer (VCU) that enables modules for network communication. In some embodiments, the AVs 105 may be a truck trailer tractor unit that is mounted on a trailer and transports cargo or loads. However, as would be understood by one of ordinary skill in the art, the AVs 105 may be embodied by any autonomous vehicle, such as a car, truck, drone, watercraft, and / or the like.

[0033] The autonomous vehicles may communicate with an operations server, which may serve many purposes, including tracking the progress of one or more autonomous vehicles (e.g., autonomous trucks), tracking the progress of a fleet of autonomous vehicles, sending steering commands to one or more autonomous vehicles, monitoring the health of the autonomous vehicles, monitoring the status of the cargo of each autonomous vehicle in contact with the operations server, facilitating communications between third parties (e.g., law enforcement, a client whose cargo is being carried) and each autonomous vehicle or a specific autonomous vehicle, enabling tracking of a specific autonomous truck in communication with the operations server (e.g., third-party tracking of a subset of vehicles in a fleet), arranging maintenance services for the autonomous vehicles (e.g., oil changes, refueling, maintaining other fluid levels), alerting affected autonomous vehicles of traffic or weather changes that may adversely affect routes or delivery plans, pushing updates wirelessly to autonomous trucks to keep all components up to date, and other purposes or functions that improve safety for the autonomous vehicles, their cargo, and / or their surroundings. The operations server may also determine performance parameters of the autonomous vehicle or truck, including any of the following: data logging frequency, compression rate, location, data type, communication prioritization, how often to service the autonomous vehicle (e.g., miles between service), when to perform Minimum Risk Conditions (MRC) maneuvers while monitoring the vehicle's progress during a maneuver, when to hand over control of the autonomous vehicle to a human driver (e.g., at the destination yard), ensuring that the autonomous vehicle passes a pre-trip inspection, ensuring that the autonomous vehicle implements or obeys legal requirements at checkpoints and weight check stations, ensuring that the autonomous vehicle implements or obeys commands from humans at road blocks, crosswalks, intersections, construction, or accident points, and / or the like.

[0034] To enable communication between the autonomous vehicles in the fleet and an operations server or command center, each autonomous vehicle may be equipped with a communications gateway. The communications gateway may have the capability to do any of the following: enable communications from AV to operations server (i.e., V2C) and from operations server to AV (C2V), enable AV to AV communications (V2V) within the fleet, convey availability or status of the communications gateway, acknowledge received communications, ensure security around remote commands between AVs and the operations server, reliably communicate location of AVs at set time intervals, enable operations server to ping AVs regarding location and vehicle health status, enable streaming of various sensor data directly to commands or operations server, enable automated alerts between AVs and operations server, comply with the International Organization for Standardization (ISO) 21434 standard, and the like.

[0035] The operations server or command center may be operated by one or more humans, also known as an operator and / or a remote center operator. The operators may set thresholds for autonomous vehicle health parameters such that preventative action can be taken if the autonomous vehicle meets or exceeds the thresholds. Examples of vehicle health parameters for which thresholds may be established by an operator may include fuel level, oil level, miles traveled since last maintenance, detected low tire pressure, cleaning fluid level, brake fluid level, steering and braking subsystem responsiveness, diesel exhaust fluid (DEF) level, communication capabilities (e.g., lack of responsiveness), positioning sensor capabilities (e.g., GPS, IMU malfunction), impact detection (e.g., vehicle crash), perception sensor capabilities (e.g., camera, LIDAR, radar, microphone array malfunction), computing resource capabilities (e.g., VCU or ECU malfunction or lack of responsiveness, temperature anomaly in computing unit), angle between tractor and trailer in towing situations (e.g., tractor trailer, 18-wheeler, or semi-truck), unauthorized access by a living entity (e.g., person or animal) to the interior of the autonomous truck, and any of the like. Preventive measures may include performing a minimum risk condition (MRC) maneuver, calling for service, or exiting a highway, or other such route changes that may be less taxing to the autonomous vehicle. An autonomous vehicle whose system health data meets or exceeds thresholds set in the operations server or by an operator may receive instructions automatically transmitted from the operations server to implement preventive measures.

[0036] The operator may be made aware of situations that affect one or more autonomous vehicles communicating with or monitored by the operations server that the affected autonomous vehicles may not be aware of. Such situations may include irregular or sudden changes in traffic flow (e.g., traffic jams or accidents), sudden weather changes, sudden changes in visibility, emergency conditions (e.g., fires, sinkholes, bridge failures), power outages affecting traffic lights, unexpected road construction, large or obscured road debris (e.g., objects unidentifiable by the autonomous vehicle), law enforcement activity on the roadway (e.g., car chases or road cleaning activities), and the like. These types of situations that may not be detectable by the autonomous vehicles may be brought to the attention of the operations server operator through traffic reports, law enforcement communications, data from other vehicles communicating with the operations server, reports from drivers of other vehicles in the area, and similar broadcast information venues. The autonomous vehicles may not be able to detect such situations due to limitations in their sensor systems or lack of access to information broadcast means (e.g., no direct communication with weather agencies). An operator at the operations server may push such information to affected autonomous vehicles in communication with the operations server. The affected autonomous vehicles may proceed to alter their route, trajectory, or speed in response to the information pushed from the operations server. In some cases, the information received by the operations server may trigger a threshold condition indicating that an MRC (minimum risk condition) maneuver is justified, or alternatively or in addition, the operator may evaluate the situation and determine that the affected autonomous vehicles should perform an MRC maneuver, and subsequently transmit such instructions to the affected vehicles. In these cases, each autonomous vehicle that receives either the information or instructions from the operations server or the operations server operator uses its on-board computing unit (i.e., VCU) to determine how to proceed safely, including performing an MRC maneuver, including pulling over or stopping.

[0037] The operations server or command center may allow third parties to interact with the operations server operator, with the autonomous truck, or with both the human system operator and the autonomous truck. The third party may be a customer whose goods are being transported, a law enforcement or emergency service provider, or a person assisting the autonomous truck when service is needed. In its interaction with the third party, the operations server may recognize different levels of access, such that a customer interested in the timing or progress of a shipment may only be allowed to view status updates regarding the autonomous truck, or may be able to view the status and provide input to the operations server regarding which parameters should be prioritized (e.g., speed, economy, staying on the originally planned route). By providing input to the operations server regarding parameter prioritization, the customer can influence the route and / or operating parameters of the autonomous truck.

[0038] Exemplary Control Subsystem 2 illustrates an embodiment of a control subsystem 1400. The control subsystem 1400 includes at least one processor 1402, at least one memory 1404, and at least one network interface 1406. The control subsystem 1400 may be configured as shown or in any other suitable configuration.

[0039] The processor 1402 may comprise one or more processors operatively coupled to the memory 1404. The processor 1402 is any electronic circuitry including, but not limited to, a state machine, one or more central processing unit (CPU) chips, logic units, cores (e.g., multi-core processors), field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), or digital signal processors (DSPs). The processor 1402 may be a programmable logic device, a microcontroller, a microprocessor, or any suitable combination of the foregoing. The processor 1402 is communicatively coupled to and in signal communication with the memory 1404 and the network interface 1406. The one or more processors are configured to process data and may be implemented in hardware or software. For example, the processor 1402 may be 8-bit, 16-bit, 32-bit, 64-bit, or any other suitable architecture. The processor 1402 may include an arithmetic logic unit (ALU) for performing arithmetic and logical operations, processor registers for providing operands to the ALU and storing results of ALU operations, and a control unit for fetching instructions from memory and executing them by directing the cooperative operation of the ALU, registers, and other components. The one or more processors are configured to perform various instructions. For example, the one or more processors are configured to execute instructions to implement functions disclosed herein, such as some or all of those described with respect to Figures 1-13. In some embodiments, the functions described herein are implemented using a logic unit, an FPGA, an ASIC, a DSP, or any other suitable hardware or electronic circuitry.

[0040] The memory 1404 is operable to store any of the information described below along with any other data, instructions, logic, rules, or code that, when executed by the processor 1402, is operable to implement the functions described herein. For example, the memory 1404 may store software instructions 1408, sensor data 1412 received from the sensors 1646, driving subsystem 1642, and / or control subsystem 1648 of the AV 105, map data 1510, route designation plans 1514, driving instructions 1518, and / or any other data / instructions described herein. The memory 1404 may further store resource allocation instructions 1450 that may be configured to determine mandatory data transmission instructions 1452, auxiliary data transmission instructions 1454, data transmission prioritization policy instructions 1456. The memory 1404 may comprise one or more disks, tape drives, or solid state drives, and may be used as an overflow data storage device to store programs when such programs are selected for execution, and to store instructions and data read during program execution. The memory 1404 may be volatile or non-volatile, and may comprise read only memory (ROM), random access memory (RAM), ternary content addressable memory (TCAM), dynamic random access memory (DRAM), and static random access memory (SRAM).

[0041] The network interface 1406 is configured to enable wired and / or wireless communication. The network interface 1406 is configured to communicate data between the control subsystem 1400 and other network devices, systems, or domains. For example, the network interface 1406 may comprise a WIFI interface, a local area network (LAN) interface, a wide area network (WAN) interface, a modem, a switch, or a router. The processor 1402 is configured to transmit and receive data using the network interface 1406. The network interface 1406 may be configured to use any suitable type of communication protocol.

[0042] In one embodiment, the control subsystem 1400 may be a subsystem of the on-board control computer system 1650 (see FIG. 4). The control subsystem 1400 may be implemented by one or more computing devices that may be responsible for determining data transmission prioritization policies and / or controlling transmission of essential data and / or auxiliary data. The control subsystem 1400 is in signal communication with the on-board control computer system 1650 (and its components) and the operations server 1500.

[0043] In some embodiments, the required data transmission instructions 1452 may be configured to provide instructions regarding the transmission of required data from the AV 105. The instructions may indicate the required data to transmit, a data pre-processing policy indicating how the required data should be processed prior to being transmitted from the AV 105, the frequency with which the required data should be transmitted from the AV 105, a receiving computing entity (e.g., the operations server 1500), and / or the like. The required data transmission instructions 1454 may be configured to cause the transmission of required data objects of the required data from the AV 105 to one or more computing entities, such as the operations server 1500.

[0044] In some embodiments, the auxiliary data transmission instructions 1454 may be configured to provide instructions regarding the transmission of auxiliary data from the AV 105. The instructions may indicate which auxiliary data to transmit, a data pre-processing policy indicating how the auxiliary data should be processed prior to being transmitted from the AV 105, the frequency with which the auxiliary data should be transmitted from the AV 105, a receiving computing entity (e.g., the operations server 1500), a measurement of auxiliary communication network resource availability, and / or the like. The auxiliary data transmission instructions 1454 may be configured to cause auxiliary data objects of the auxiliary data to be transmitted from the AV 105 to one or more computing entities, such as the operations server 1500.

[0045] In some embodiments, the data transmission prioritization policy instructions 1456 may be configured to provide instructions regarding the determination of a data transmission prioritization policy. For example, the data transmission prioritization policy instructions 1456 may provide instructions regarding the determination of one or more operating states associated with the AV 105, the generation of a data transmission prioritization policy, the updating of a transmission prioritization policy, and / or the like. The data transmission prioritization policy may be used by the auxiliary data transmission instructions 1454 for the transmission of auxiliary data.

[0046] Exemplary Operations Server 1500 3 illustrates an embodiment of an operations server 1500. The operations server 1500 includes at least one processor 1502, at least one memory 1504, and at least one network interface 1506. The operations server 1500 may be configured as shown or in any other suitable configuration.

[0047] The processor 1502 comprises one or more processors operatively coupled to the memory 1504. The processor 1502 is any electronic circuitry including, but not limited to, a state machine, one or more central processing unit (CPU) chips, logic units, cores (e.g., multi-core processors), field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), or digital signal processors (DSPs). The processor 1502 may be a programmable logic device, a microcontroller, a microprocessor, or any suitable combination of the foregoing. The processor 1502 is communicatively coupled to and in signal communication with the memory 1504, the network interface 1506, and the user interface 1528. The one or more processors are configured to process data and may be implemented in hardware or software. For example, the processor 1502 may be 8-bit, 16-bit, 32-bit, 64-bit, or any other suitable architecture. The processor 1502 may include an arithmetic logic unit (ALU) for performing arithmetic and logical operations, processor registers for providing operands to the ALU and storing results of ALU operations, and a control unit for fetching instructions from memory and executing them by directing the cooperative operation of the ALU, registers, and other components. The one or more processors are configured to perform various instructions. For example, the one or more processors are configured to execute instructions to implement functions disclosed herein, such as some or all of those described with respect to FIGS. 1-13. In some embodiments, the functions described herein are implemented using a logic unit, an FPGA, an ASIC, a DSP, or any other suitable hardware or electronic circuitry.

[0048] The memory 1504 is operable to store any of the information described below along with any other data, instructions, logic, rules, or code that, when executed by the processor 1502, is operable to implement the functions described herein. For example, the memory 1504 may store software instructions 1508, map data 1510, reroute plans 1520, routing plans 1514, training data sets 1522, driving instructions 1518, sensor data 1412 received from the AV 1602, traffic data 1524, a map building module 1526, and / or any other data / instructions. The memory 1504 may comprise one or more disks, tape drives, or solid state drives, and may be used as an overflow data storage device to store programs when such programs are selected for execution, and to store instructions and data read during program execution. The memory 1504 may be volatile or non-volatile and may comprise read-only memory (ROM), random access memory (RAM), ternary content addressable memory (TCAM), dynamic random access memory (DRAM), and static random access memory (SRAM).

[0049] The network interface 1506 is configured to enable wired and / or wireless communication. The network interface 1506 is configured to communicate data between the control subsystem 1400 and other network devices, systems, or domains. For example, the network interface 1506 may comprise a WIFI interface, a local area network (LAN) interface, a wide area network (WAN) interface, a modem, a switch, or a router. The processor 1502 is configured to transmit and receive data using the network interface 1506. The network interface 1506 may be configured to use any suitable type of communication protocol.

[0050] In one embodiment, the operations server 1500 may be implemented by a cluster of computing devices that may be responsible for overseeing the operation of the AV 105. For example, the operations server 1500 may be implemented by multiple computing devices using a distributed computing and / or cloud computing system. In another example, the operations server 1500 may be implemented by multiple computing devices in one or more data centers. The operations server 1500 is in signal communication with the AV 105 and its components (e.g., the on-board control computer 1650). In one embodiment, the operations server 1500 is configured to determine a specific routing plan 1514 for each AV 105. For example, the operations server 1500 may determine a specific routing plan 1514 for a given AV 105 that leads to reduced driving time and a safer driving experience to reach the destination of the AV 105. In one embodiment, the operations server 1500 may capture and record the navigation plan set by the user 1536 in each situation and use it in similar situations.

[0051] In one embodiment, the operations server 1500 may transmit the sensor data 1412 to the application server 1532 to be reviewed by the user 1536. The application server 1532 is generally any computing device configured to communicate with other devices, such as other servers (e.g., the operations server 1500), the AV 105, databases, etc., via a network interface. The application server 1532 is configured to perform specific functions described herein and to interact with the user 1536, for example, via its user interface 1534. Examples of the application server 1532 include, but are not limited to, a desktop computer, a laptop computer, a server, etc. In one embodiment, the application server 1532 may act as a presentation layer that the user 1536 accesses to review the sensor data 1412. Thus, the operations server 1500 may transmit the sensor data 1412 to the application server 1532. A user 1536 may review the sensor data 1412 from the user interface 1534 as described above to confirm, modify, and / or override the navigation solution for the AV 105 determined by the control subsystem 1400 and / or the operations server 1500.

[0052] Map data 1510 may include a virtual map of a city, including routes. In some embodiments, map data 1510 may include one or more map databases. Map data 1510 may include drivable areas, such as roads, routes, highways, and non-drivable areas, such as terrain. Map data 1510 may define location coordinates of road signs, lanes, lane markings, lane boundaries, road boundaries, traffic lights, and the like.

[0053] The map data 1510 may also define connections between lanes (e.g., a lane may flow into another adjacent lane). The map data 1510 may define information indicating the type of road lanes (e.g., travel lanes, passing lanes, emergency lanes, turn lanes, bus lanes, etc.), the type of lane boundaries (e.g., white lines, yellow lines, other road surface markings and / or mechanical markings, etc.), the type of road boundaries (e.g., regular curbs, red curbs, sidewalks, guard rails, other barriers, etc.), road intersections, one or more obstacles ahead of the autonomous vehicle, and other information about the road or an area adjacent to the road. The map data 1510 may also define road bumps such as curves, hills, valleys, road hazards such as speed bumps, potholes, road segments such as school zones, railroad crossings, etc.

[0054] The map construction module 1526 may be implemented by the processor 1502 executing the software instructions 1508 and is configured to construct the map data 1510. In one embodiment, the map construction module 1526 may construct the map data 1510 from sensor data received from one or more mapping vehicles. In one example, the mapping vehicle may be the AV 105. In another example, the mapping vehicle may be a non-autonomous vehicle connected or integrated with sensors 1646 operated by the AV 105 or a driver. In some cases, one or more mapping vehicles may be dispatched to observe a situation detected by the AV 105 and transmit more sensor data related to the situation, such as via mandatory and / or auxiliary data transmitted from the AV 105.

[0055] The map construction module 1526 is configured to use the sensor data to determine the portions of the map data 1510 with which they are associated. The map construction module 1526 may dynamically construct each section of the map data 1510 by merging different sensor data associated with each section of the map data 1510. The map construction module 1526 also uses the sensor data to find overlapping portions of the map data 1510 (e.g., by matching corresponding images, videos, LiDAR data, radar data, etc., observing the same portion of the map data 1510). The map construction module 1526 then connects the portions of the map data 1510 with their corresponding adjacent portions. In other words, the map construction module 1526 finds adjacent portions of the map data 1510 and stitches them together to build the map data 1510. The map construction module 1526 is also configured to update portions of the map data 1510 that are based, at least in part, on the received sensor data that needs to be updated.

[0056] For example, in the case of an unknown object on the roadway, once the operations server 1500 receives sensor data from the AV 105 via mandatory and / or auxiliary data transmissions, and once it is determined that an unknown object is in fact present at a particular location coordinate, the map construction module 1526 updates portions of the map data 1510 to reflect the presence of the unknown object. Similarly, the map construction module 1526 updates different portions of the map data 1510 based, at least in part, on sensor data associated with different instances in which an unexpected situation is encountered.

[0057] The map construction module 1526 is also configured to facilitate shadow testing of the routing plan 1514 using a new layout of one or more roads in the updated map data 1510. For example, as the map data 1510 is updated, the map construction module 1526 may perform one or more simulations of autonomous driving through updates to the map data 1510 for the AV 105. For example, the one or more simulations of autonomous driving may involve the AV 105 navigating around objects, changing lanes, pulling over, rerouting, etc.

[0058] The map building module 1526 selects, for each AV105, a particular course of autonomous driving from one or more simulations of autonomous driving that leads to a navigation solution that is more efficient, safe, and reliable compared to the other simulations.

[0059] The route plan 1514 is a plan for traveling from a starting location (e.g., a first AV origin / destination) to a destination (e.g., a second AV origin / destination). For example, the route plan 1514 of the AV 105 may define one or more street / road / highway combinations in a specific order from the starting location to the destination. The route plan 1514 of the AV 105 may define stages including a first stage (e.g., moving out from the starting location), multiple intermediate stages (e.g., proceeding along specific lanes of one or more specific streets / roads / highways), and a final stage (e.g., entering the destination). The route plan 1514 may include other information about the route from the starting location to the destination, such as road / traffic signs within the route plan 1514.

[0060] Driving instructions 1518 may include instructions and rules for adapting the autonomous driving of AV 105 in accordance with the driving rules for each stage of route planning 1514. For example, driving instructions 1518 may include instructions for staying within a speed range for roads traversed by AV 105, adapting the speed of AV 105 to changes observed by sensors 1646, such as the speed of surrounding vehicles, objects within the sensor's detection area, etc.

[0061] The object detection machine learning module 1520 may be implemented by the processor 1502 executing the software instructions 1508 and is generally configured to detect objects from the sensor data 1412. The object detection machine learning module 1520 may be implemented using neural networks and / or machine learning algorithms to detect objects from any type of data, such as images, videos, infrared images, point clouds, radar data, etc.

[0062] In one embodiment, the object detection machine learning module 1520 may be implemented using machine learning algorithms such as support vector machines (SVM), naive Bayes, logistic regression, k-nearest neighbors, decision trees, or the like. In one embodiment, the object detection machine learning module 1520 may utilize multiple neural network layers, convolutional neural network layers, and / or the like, in which the weights and biases of the perceptrons of these layers are optimized in the training process of the object detection machine learning module 1520. The object detection machine learning module 1520 may be trained by a training dataset 1522 that includes samples of a data type labeled with one or more objects in each sample. For example, the training dataset 1522 may include sample images of objects (e.g., vehicles, lane markings, pedestrians, road signs, etc.) labeled with the objects in each sample image. Similarly, the training dataset 1522 may include samples of other data types, such as video, infrared imagery, point clouds, radar data, etc., labeled with the objects in each sample data. The object detection machine learning module 1520 may be trained, tested, and refined by the training dataset 1522 and the sensor data 1412. The object detection machine learning module 1520 uses the sensor data 1412 (not labeled with objects) to increase the accuracy of their predictions when detecting objects. For example, supervised and / or unsupervised machine learning algorithms may be used to validate the predictions of the object detection machine learning module 1520 when detecting objects in the sensor data 1412.

[0063] The traffic data 1524 may include traffic data 1524 for roads / streets / highways in the map data 1510. The operations server 1500 may use traffic data 1524 collected by one or more mapping vehicles. The operations server 1500 may use traffic data 1524 captured from any source, such as crowd-sourced traffic data 1524 captured from external sources, such as, for example, Waze and Google Maps, live traffic reports, etc.

[0064] An exemplary AV105 and its operation FIG. 4 illustrates a block diagram of an example vehicle ecosystem 1600 in which autonomous driving operations may be determined. As illustrated in FIG. 16, the AV 105 may be a semi-trailer truck. The vehicle ecosystem 1600 includes several systems and components that may generate and / or deliver one or more sources of information / data and related services to an on-board control computer 1650, which may be located within the AV 105. The on-board control computer 1650 may be in data communication with multiple vehicle subsystems 1640, all of which may reside within the AV 105. A vehicle subsystem interface 1660 is provided to facilitate data communication between the on-board control computer 1650 and the multiple vehicle subsystems 1640. In some embodiments, the vehicle subsystem interface 1660 may include a controller area network (CAN) controller and communicate with devices within the vehicle subsystem 1640.

[0065] The AV 105 may include various vehicle subsystems that support the operation of the AV 105. The vehicle subsystems may include a control subsystem 1400, a vehicle driving subsystem 1642, a vehicle sensor subsystem 1644, and / or a vehicle control subsystem 1648. The components or devices of the vehicle driving subsystem 1642, the vehicle sensor subsystem 1644, and the vehicle control subsystem 1648 shown in FIG. 4 are examples. The vehicle driving subsystem 1642 may include components operable to provide powered motion for the AV 105. In an exemplary embodiment, the vehicle driving subsystem 1642 may include an engine / motor 1642a, wheels / tires 1642b, a transmission 1642c, an electrical subsystem 1642d, and a power source 1642e.

[0066] The vehicle sensor subsystem 1644 may include a number of sensors 1646 configured to sense information about the environment or conditions of the AV 105. The vehicle sensor subsystem 1644 may include one or more cameras 1646a or image capture devices, which may be equipped with microphones, radar units 1646b, one or more temperature sensors 1646c, wireless communication units 1646d (e.g., cellular communication transceivers), inertial measurement units (IMUs) 1646e, laser range finders / LiDAR units 1646f, global positioning system (GPS) transceivers 1646g, wiper control systems 1646h, microphones 1646i, and / or tire pressure sensors 1646j. The vehicle sensor subsystem 1644 may also include sensors configured to monitor internal systems of the AV 105 (e.g., O2 monitors, fuel gauges, engine oil temperature, etc.).

[0067] The IMU 1646e may include any combination of sensors (e.g., accelerometers and gyroscopes) configured to sense position and orientation changes of the AV 105 based, at least in part, on inertial acceleration. The GPS transceiver 1646g may be any sensor configured to estimate the geographic location of the AV 105. For this purpose, the GPS transceiver 1646g may include a receiver / transmitter operable to provide information regarding the position of the AV 105 relative to the Earth. The radar unit 1646b may represent a system that utilizes radio signals to sense objects in the local environment of the AV 105. In some embodiments, in addition to sensing objects, the radar unit 1646b may additionally be configured to sense the speed and heading of objects in the vicinity of the AV 105. The laser range finder or LiDAR unit 1646f may be any sensor configured to sense objects in the environment in which the AV 105 is located using a laser. The camera 1646a may include one or more devices configured to capture multiple images of the environment of the AV 105. The camera 1646a may be a still image camera or a motion video camera. In some embodiments, the camera 1646a may be configured with one or more microphones to capture audio data. Additionally or alternatively, one or more microphones 1646i may be configured to capture audio data of and / or related to the environment of the AV 105.

[0068] The vehicle control subsystem 1648 may be configured to control the operation of the AV 105 and its components. Thus, the vehicle control subsystem 1648 may include various elements such as a throttle and gear 1648a, a brake unit 1648b, a navigation unit 1648c, a steering system 1648d, and / or an autonomous control unit 1648e. The throttle 1648a may be configured to control, for example, the operating speed of the engine and thus the speed of the AV 105. The gear 1648a may be configured to control the gear selection of the transmission. The brake unit 1648b may include any combination of mechanisms configured to slow down the AV 105. The brake unit 1648b may use friction to slow down the wheels in a standard manner. The brake unit 1648b may include an anti-lock braking system (ABS) that may prevent the brakes from locking when applied. The navigation unit 1648c may be any system configured to determine a driving path or route for the AV 105. The navigation unit 1648c may additionally be configured to dynamically update the driving path while the AV 105 is in operation. In some embodiments, the navigation unit 1648c may be configured to incorporate data from the GPS transceiver 1646g and one or more predefined maps to determine a driving path for the AV 105. The steering system 1648d may represent any combination of mechanisms that may be operable to adjust the heading of the AV 105 in an autonomous mode or a driver-controlled mode.

[0069] The autonomous control unit 1648e may represent a control system configured to identify, evaluate, and avoid or otherwise navigate potential obstacles or obstructions in the environment of the AV 105. In general, the autonomous control unit 1648e may be configured to control the AV 105 for driverless operation or to provide driver assistance in controlling the AV 105. In some embodiments, the autonomous control unit 1648e may be configured to incorporate data from the GPS transceiver 1646g, the radar 1646b, the LiDAR unit 1646f, the camera 1646a, and / or other vehicle subsystems to determine a driving path or trajectory for the AV 105.

[0070] Many or all of the functions of the AV 105 can be controlled by an on-board control computer 1650. The on-board control computer 1650 may include at least one data processor 1670 (which may include at least one microprocessor) that executes processing instructions 1680 stored in a non-transitory computer-readable medium such as a data storage device 1690 or memory. The on-board control computer 1650 may also represent multiple computing devices that may be responsible for controlling individual components or subsystems of the AV 105 in a distributed manner. In some embodiments, the data storage device 1690 may contain processing instructions 1680 (e.g., program logic) executable by the data processor 1670 to implement various methods and / or functions of the AV 105.

[0071] The data storage device 1690 may contain additional instructions as well, including instructions for transmitting data to, receiving data from, interacting with, or controlling one or more of the vehicle driving subsystem 1642, the vehicle sensor subsystem 1644, and the vehicle control subsystem 1648. The on-board control computer 1650 can be configured to include a data processor 1670 and a data storage device 1090. The on-board control computer 1650 may control the function of the AV 105 based, at least in part, on inputs received from the various vehicle subsystems (e.g., the vehicle driving subsystem 1642, the vehicle sensor subsystem 1644, and the vehicle control subsystem 1648).

[0072] FIG. 5 illustrates an exemplary block diagram of an on-board control computer 1650 included within the AV 105. The on-board control computer 1650 includes at least one processor 1804 and a memory 1802 having instructions stored thereon (e.g., software instructions 1408, 1508, and process instructions 1680 of FIGS. 2, 3, and 4, respectively). The instructions, upon execution by the processor 1804, configure the on-board control computer 1650 and / or various modules of the on-board control computer 1650 to perform the operations described in FIGS. 2-4. The transmitter 1806 transmits or transmits information or data to one or more devices within the autonomous vehicle. For example, the transmitter 1806 can transmit instructions to one or more motors of a steering wheel to steer the autonomous vehicle. The receiver 1808 receives information or data transmitted or transmitted by the one or more devices. For example, the receiver 1808 receives the status of the current speed from an odometer sensor or the current transmission gear from the transmission. The transmitter 1806 and the receiver 1808 are also configured to communicate with the control subsystem 1400.

[0073] The on-board control computer 1650 may include a sensor fusion module 1702, which may perform image or signal processing operations. The sensor fusion module 1702 may acquire images from cameras located on the autonomous vehicle, perform image segmentation, and detect the presence of moving objects (e.g., other vehicles, pedestrians, etc.) and / or static obstacles (e.g., stop signs, speed bumps, terrain, etc.) located in the vicinity of the autonomous vehicle. The sensor fusion module 1702 may acquire LiDAR point cloud data items from LiDAR sensors located on the autonomous vehicle, perform LiDAR segmentation, and detect the presence of objects and / or obstacles located in the vicinity of the autonomous vehicle.

[0074] The sensor fusion module 1702 may perform instance segmentation on the images and / or point cloud data items to identify contours (e.g., boxes) around objects and / or obstacles located around the autonomous vehicle. The sensor fusion module 1702 may perform temporal fusion, in which objects and / or obstacles from one image and / or frame of a point cloud data item are correlated or associated with objects and / or obstacles from one or more images or frames received consecutively in time.

[0075] The sensor fusion module 1702 can fuse objects and / or obstacles from images acquired from the cameras and / or point cloud data items acquired from the LiDAR sensor. For example, the sensor fusion module 1702 can determine that an image from one of the cameras, with 1 / 2 of the vehicle located in front of the autonomous vehicle, is the same vehicle captured and located by another camera based on the location of the two cameras. The sensor fusion module 1702 transmits the fused object information to the interference module 1746 and the fused obstacle information to the occupancy grid module 1760. The on-board control computer can include the occupancy grid module 1760 and retrieve landmarks from a map database 1758 stored in the on-board control computer. The occupancy grid module 1760 can determine a drivable area and / or obstacles from the fused obstacles acquired from the sensor fusion module 1702 and the landmarks stored in the map database 1758. For example, the occupancy grid module 1760 can determine that the drivable area can include a speed bump obstacle.

[0076] The on-board control computer 1650 may also include a LiDAR-based object detection module 1712 that may perform object detection 1716 based on point cloud data items acquired from a LiDAR sensor located on the AV 105. The object detection techniques may provide the location of an object (e.g., in 3D world coordinates) from the point cloud data items. Below the LiDAR-based object detection module 1712, the on-board control computer may include an image-based object detection module that may perform object detection based on images acquired from a camera located on the autonomous vehicle. The object detection techniques may employ deep machine learning techniques to provide the location of an object (e.g., in 3D world coordinates) from images provided by the camera.

[0077] A radar on the autonomous vehicle may scan the area in front of the autonomous vehicle or the area towards which the autonomous vehicle is driven. The radar data is transmitted to a sensor fusion module 1702, which may use the radar data to correlate objects and / or obstacles detected by the radar with objects and / or obstacles detected from both the LiDAR point cloud data items and the camera imagery.

[0078] The planning module 1762 may perform a navigation plan 1764 to determine a set of trajectories on which the AV 105 may be driven. The set of trajectories may be determined based on the drivable area information, the location of obstacles, and the drivable area information. In some embodiments, the navigation plan may include determining areas adjacent to a road where the autonomous vehicle may be safely parked in case of an emergency. The planning module 1762 may include behavior decision-making to determine driving actions (e.g., steering, braking, throttling) in response to determining changing conditions on the road (e.g., a traffic light has turned yellow or the autonomous vehicle is in an unsafe driving condition because another vehicle has cut in front of the autonomous vehicle and entered an area within a predetermined safe distance from the location of the autonomous vehicle). The planning module 1762 performs trajectory generation to select a trajectory from the set of trajectories determined by the navigation planning operation. The selected trajectory information is transmitted by the planning module 1762 to the control module 1770.

[0079] The on-board control computer includes a control module 1770 that receives the proposed trajectory from the planning module 1762 and the autonomous vehicle location and attitude from the fusion localization module 1726. The control module 1770 includes a system identifier. The control module 1770 can perform model-based trajectory refinement to refine the proposed trajectory. For example, the control module 1770 can apply filtering (e.g., a Kalman filter) to smooth the proposed trajectory data and / or minimize noise. The control module 1770 can perform robust control by determining the amount of brake pressure, steering angle, throttle amount to control the vehicle's speed, and / or transmission gear to apply based on the refined proposed trajectory information and the current location and / or attitude of the autonomous vehicle. The control module 1770 can transmit the determined brake pressure, steering angle, throttle amount, and / or transmission gear to one or more devices in the autonomous vehicle to control and facilitate precise driving behavior of the autonomous vehicle.

[0080] Exemplary Dynamic Allocation of Communication Network Resources 6 illustrates a flow chart of an example method 600 for dynamically allocating communication network resources. One or more of the steps / operations depicted in FIG. 6 may be implemented, at least in part, in the form of software instructions 1408, 1508, and 1680 stored on a non-transitory tangible machine-readable medium (e.g., memory 1404, 1504, 1690, and 1802) that, when executed by one or more processors (e.g., processors 1402, 1502, 1670, and 1804), may cause the one or more processors to perform steps 601-605. In some embodiments, the method 600 may be performed by the control subsystem 1400, the operation server 1500, and the on-board control computer 1650 from FIGS. 2-5, respectively.

[0081] As shown in step / operation 601, an apparatus such as control subsystem 1400 may include means such as processor 1402, memory 1404, network interface 1406, and / or the like for determining a measure of overall communications network resource availability. As described above, AV 105 may be communicatively coupled to one or more other computing entities (e.g., operations server 1500, one or more other AVs, one or more third parties 360, etc.) via network 370. In some embodiments, the AV 105 may be communicatively coupled to one or more other computing entities using any suitable technology, including, but not limited to, a radio access network, Long Term Evolution Advanced (LTE-Advanced, LTE-A), Universal Mobile Telecommunications System (UMTS) Radio Access Network (UTRAN or E-UTRAN), Wireless Local Area Network (WLAN or WiFi), Worldwide Interoperability for Microwave Access (WiMAX), Bluetooth®, Personal Communications Services (PCS), ZigBee®, Wideband Code Division Multiple Access (WCDMA®), systems using Ultra Wideband (UWB) technology, sensor networks, Mobile Ad Hoc Networks (MANETs), and Internet Protocol Multimedia Subsystems (IMS), or any combination thereof.

[0082] Regardless of the connection type, the processor 1402 may determine a measure of overall communication network resource availability for the current network connection type. For example, the measure of overall communication network resource availability may be a measure of bandwidth available to the control subsystem 1400. In some embodiments, the measure of overall communication network resource availability may refer specifically to a measure of upload bandwidth availability.

[0083] Illustrative embodiments will be described herein in the context of the Open Systems Interconnection Model (OSI model), which is a model that conceptually characterizes the communication functions of a communication system, such as, for example, an LTE network. The OSI model is typically conceptualized as a hierarchical stack, with a given layer providing services to the layer above and being served by the layer below. Typically, the OSI model comprises seven layers, with the top layer of the stack being the application layer (layer 7), followed by the presentation layer (layer 6), the session layer (layer 5), the transport layer (layer 4), the network layer (layer 3), the data link layer (layer 2), and the physical layer (layer 1). Those skilled in the art will understand the functions and interoperation of the various layers, and therefore further details of each layer will not be described herein. However, it should be understood that, although the illustrative embodiments are well suited for implementation utilizing the OSI model, alternative embodiments are not necessarily limited to any particular communication function model.

[0084] Scheduling of overall communications network resource availability (e.g., available bandwidth) available to the control sub-system 1400 may occur within the application layer of the network 370. In some embodiments, the control sub-system 1400 may request an indication of overall communications network resource availability via the network interface 1406. In some embodiments, the control sub-system 1400 may determine the overall communications network resource availability locally. The overall communications network resource availability may be based, at least in part, on one or more determining factors such as a current network traffic load and / or a maximum resource allocation value as determined by a network subscription associated with the control sub-system 1400.

[0085] In step / operation 602, an apparatus such as the control subsystem 1400 may include means such as a processor 1402, memory 1404, and / or the like for determining a measure of essential communications network resource availability. The measure of essential communications network resource availability may be based, at least in part, on associated essential data. Essential data may refer to data required to be transmitted during AV105 operation. For example, essential data may include certain sensor data (e.g., speed, location coordinates, etc.) associated with the AV105. Essential data may be, for example, data associated with a basic operation of the AV105. Essential data may thus be essential for safe or effective operation of the AV105, such as data essential to perform a predetermined set of operations of the AV105. The predetermined set of operations may thus be a set of operations essential to achieve a certain safety or operational standard. Other examples of required data include data related to the overall health of a vehicle communication system (e.g., data to ensure that such communication system is functioning properly) or data related to the overall health of the mechanical aspects of the vehicle while the vehicle is in motion (e.g., data to ensure that there are no structural defects or problems). Indeed, the determination of which data is required may depend on the current status, operation, or desired operation of AV 105.

[0086] In some embodiments, the control subsystem 1400 may be configured with a data categorization set that may define one or more data types to be associated with a data category. The data categorization set may be a mapping of data categories to data types associated with the AV 105 associated with the control subsystem 1400, or vice versa. For example, the one or more candidate data categories may include a required data category and an auxiliary data category. The data type of "image data from camera 1" may be defined as required according to the data categorization set. Thus, any data associated with the data type "image data from camera 1" is determined to be required data. In some embodiments, the categorization set may be determined and initialized, at least in part, based on user input, such as the RCO 355.

[0087] The control subsystem 1400 may then determine a measure of required communications network availability based, at least in part, on the aggregation of the required data to be transmitted. For example, the control subsystem 1400 may determine an associated data packet size for the data packets containing the aggregated required data. In some embodiments, the measure of required communications network resource availability may refer to a measure of the upload bandwidth dedicated to the transmission of the required data. The measure of required communications network resource availability may be considered a measure of the communications network resources required for the AV 105 to perform a given set of operations.

[0088] In some embodiments, the control subsystem 1400 may be configured to preprocess the required data according to a required data preprocessing policy. The required data preprocessing policy may define a process for preprocessing each required data type in a particular manner prior to providing the required data to one or more additional computing entities. In some embodiments, the control subsystem 1400 may include a processor 1402 for performing one or more required data preprocessing routines as defined by the required data preprocessing policy such that the required data may be properly formatted for transmission. For example, a preprocessing routine for any image data may convert a raw image into one or more compressed images. Thus, the data size associated with the required data may be reduced.

[0089] In step / operation 603, an apparatus such as the control subsystem 1400 may include means such as a processor 1402, memory 1404, and / or the like for determining a measure of auxiliary communication network resource availability. The measure of auxiliary communication network resource availability may be based, at least in part, on the overall communication network resource availability and the essential communication network resource availability as determined in steps / operations 601 and 602, respectively. Auxiliary data may refer to data that is optional for transmission during AV105 operation. For example, auxiliary data may include sensor data associated with the AV105 that is not classified as essential (e.g., speed, location coordinates, etc.). Such optional auxiliary data may thus be data that is not essential for the safe operation of the AV105. In other words, a given set of operations may still be performed by the AV105 in the absence of auxiliary data. However, such auxiliary data may nevertheless improve the performance of the AV105 and further increase safety, efficiency, responsiveness, or another property of AV performance.

[0090] As described above, in some embodiments, the control subsystem 1400 may be configured with a data categorization set that may define one or more data types as associated with a data category. The data categorization set defines the data types that are classified as auxiliary data. For example, the one or more candidate data categories may include a required data category and an auxiliary data category. The data type of "image data from camera 1" may be defined as required according to the data categorization set, while the data type of "image data from camera 3" may be defined as auxiliary according to the data categorization set. Here, image data from camera 3 may be classified as auxiliary data because the fields of view of camera 1 and camera 3 overlap significantly, and therefore it may not be necessary to always have image data from camera 3. Thus, any data associated with the data type "image data from camera 3" is determined to be auxiliary data.

[0091] The control subsystem 1400 may then determine a measure of auxiliary communications network availability based, at least in part, on the overall communications network resource availability and the required communications network resource availability. For example, if the measure of overall network resource availability is 5 Megabits per second (Mbps) and the measure of required communications network resource availability is 3 Mbps, then the measure of auxiliary communications network resource availability may be determined to be 2 Mbps. In some embodiments, the measure of auxiliary communications network resource availability may refer to a measure of upload bandwidth dedicated to auxiliary data.

[0092] In step / operation 604, an apparatus such as the control subsystem 1400 may include means such as the processor 1402, memory 1404, and / or the like for determining a data transmission prioritization policy. The data transmission prioritization policy may describe one or more policies regarding the transmission of auxiliary data. In particular, the data transmission prioritization policy may describe a priority of the auxiliary data, which indicates a prioritized order in which data associated with the auxiliary data should be transmitted. The priority of the data may be based, at least in part, on an inferred importance associated with a particular data associated with the auxiliary data. For example, a data type of "sound signal" may be determined to be more important relative to a data type of "image data from camera 8", where camera 8 may cover an area that is already significantly covered by the required data. Thus, the "sound signal" data type may be assigned a higher priority than "image data from camera 8". In other words, the priority of the auxiliary data may be based on at least one of the degree of overlap between the auxiliary data and the required data, the additional functionality provided by the auxiliary data compared to the required data, and the degree of AV performance improvement provided by the auxiliary data. Auxiliary data having less overlap, more additional functionality, or a higher degree of AV performance improvement may therefore be ranked higher than auxiliary data provided, for example, with more overlap, less additional functionality, or a lower degree of AV performance improvement.

[0093] In some embodiments, the data transmission prioritization policy may also describe one or more of a data pre-processing policy indicating how the auxiliary data should be processed prior to being transmitted from the AV 105 and / or the frequency with which the auxiliary data should be transmitted from the AV 105. For example, a pre-processing routine on any image data may convert raw images into one or more compressed images. As another example, the frequency for the auxiliary data may be determined based on a data packet size associated with the auxiliary data. In instances where the auxiliary data size is large, the auxiliary data may be transmitted less frequently so as not to exceed a distributed measure of the auxiliary communication network availability.

[0094] Further, the data transmission prioritization policy may be determined, at least in part, based on one or more operating states associated with the AV 105. The one or more operating states for the AV 105 may be determined, at least in part, based on one or more signals received from one or more sensors carried by the AV, such as one or more of the sensors included in the vehicle sensor subsystem 1644. The control subsystem 1400 may evaluate one or more signal characteristics from the one or more received signals and determine one or more operating states for the AV 105. In some embodiments, the control subsystem may determine an AV operating condition operating state, an environmental operating state, a driving event operating state, and / or a navigation operating state. The determination of a particular operating state may affect the priority of the auxiliary data. For example, the receipt of a signal related to a particular event may result in the determination of an associated operating state of the AV 105. In this case, the priority of the auxiliary data associated with the detection of such an event may be adjusted such that such auxiliary data is ranked higher. Such priority adjustment may occur regardless of overlap between the auxiliary data and the required data. Beneficially, adjusting the priority of the assistance data in response to determining a particular operating state ensures that the assistance data most relevant to the current operating state is prioritized. This provides dynamic adjustment of the assistance data priority, balancing communication network resource availability with data priority, maximizing safe and efficient operation of the AV105 within the constraints of communication network resource availability. Another example scenario is camera operation when the autonomous vehicle is moving forward versus being stopped in place for a long period of time (e.g., in minimum risk conditions). When moving forward, the system may assign a higher priority to camera operation that provides a view in the direction of travel, typically a forward view. Conversely, when pulled over or stopped, camera operation that provides a view from all of the autonomous vehicle's surrounding environment may be prioritized (e.g., providing a full 360-degree view).In other words, auxiliary data related to a view in the direction of travel may be prioritized during movement in that direction, while auxiliary data related to a view of the surrounding environment may be prioritized when the vehicle is not moving.

[0095] The AV operating condition operational state may describe the status of the AV 105 with respect to one or more of the sensor subsystem 1644, the driving subsystem 1642, and / or the control subsystem 1648. The AV operating condition operational state may indicate whether a subsystem of the AV 105 is operating normally or abnormally. For example, if a signal from a tire pressure sensor 1646j of the sensor subsystem 1644 indicates a tire pressure value that does not meet one or more tire pressure thresholds, the AV operating condition operational state may be assigned an "abnormal" value. Furthermore, in some embodiments, the AV operating condition operational state may indicate the associated sensor and / or subsystem that caused the "abnormal" value to be determined. As a continuing example, the AV operating condition operational state may be assigned values ​​of "abnormal" and "tire pressure low". In other words, receipt of a signal indicating an "abnormal" value in one or more of the subsystems of the AV 105 may result in a determination of an AV operating condition operational state. Such a determination may therefore cause an adjustment of the priority of the auxiliary data. In this example, such an adjustment may be the prioritization of auxiliary data related to tire pressure, an increase in subsystems related to tire pressure, or an increase in any auxiliary data related to detecting normal / abnormal operation of the AV. In this manner, the AV may operate more safely and efficiently because the additional data related to the identified problem is prioritized.

[0096] The environmental operating state may describe the environmental conditions in which the AV 105 is operating. For example, the environmental operating state may indicate the weather conditions and / or temperature of the surrounding environment of the AV 105. In some embodiments, one or more signals from one or more sensors of the sensor subsystem 1644 (e.g., temperature sensor 1646c, camera 1646a, wiper control system 1646h, etc.) may be used to determine, either directly or indirectly, the weather conditions and / or temperature of the surrounding environment. For example, a signal from the temperature sensor 1646c may be used directly to determine the temperature of the environment of the AV 105. As another example, a signal from the camera 1646a may be used indirectly to determine the weather conditions. In particular, one or more images obtained from the one or more cameras 1646a may be analyzed to determine the presence of precipitation. The temperature as determined using the temperature sensor 1646c may be used in combination with one or more images to determine the type of precipitation. For example, it may be difficult to distinguish whether precipitation determined from one or more images is snow or rain. However, if the temperature value is above 5 degrees Celsius, the precipitation is classified as rain, if the temperature is below -5 degrees Celsius, the precipitation is classified as snow, and if the temperature is between -5 and 5 degrees Celsius, the precipitation is classified as freezing rain. Additionally or alternatively, the environmental operating conditions may indicate the presence of a weather event, such as hail, tornadoes, floods, hurricanes, thunderstorms, dust storms, windstorms, and / or the like. In other words, receipt of a signal indicative of a weather event may result in a determination of the environmental operating conditions. Such a determination may therefore cause an adjustment of the priority of the auxiliary data. In this embodiment, such an adjustment may be an increase in the ranking of the auxiliary data associated with detecting the weather event. In this manner, the AV may operate more safely and efficiently because the additional data associated with the identified weather event is prioritized.

[0097] The driving event operating state may describe one or more driving events related to the road segment on which the AV 105 is operating. For example, the driving event operating state may indicate one or more driving events such as detection of an emergency vehicle, a construction zone, a vehicle accident, a traffic jam, an unexpected object on the road segment, a discrepancy between an expected object from a map and a detected object, and / or the like. In some embodiments, one or more signals from one or more sensors of the sensor subsystem 1644 (e.g., camera 1646a, radar 1646b, IMU 1646e, LIDAR 1646f, etc.) may be used to determine the one or more driving events. For example, one or more images obtained from one or more cameras 1646a may be analyzed to determine the presence of an unexpected object on the road, a vehicle accident, and / or a discrepancy between an expected object and a detected object. As another example, one or more signals obtained from a camera (configured with a microphone) may be analyzed to determine the presence of sounds characteristic of a driving event, such as a siren, braking sounds, a horn, and / or the like. In other words, receipt of a signal indicative of a driving event may result in a determination of a driving event operating state. Such a determination may therefore cause an adjustment in the priority of the auxiliary data. In this example, such an adjustment may be an increase in the ranking of the auxiliary data associated with detecting the driving event. In this manner, the AV may operate more safely and efficiently as the additional data associated with the identified driving event is prioritized.

[0098] The navigation operational state may describe the current operation of the AV with respect to one or more routes. For example, the navigation operational state may indicate the current operating status of the AV 105 (e.g., moving, stationary, etc.), the operating speed, the operating direction, the acceleration pattern over a time window, the deceleration pattern over a time window, a set of coordinates indicating the current location of the AV, and / or the like. In some embodiments, one or more signals from one or more sensors of the sensor subsystem 1644 (e.g., camera 1646a, radar 1646b, IMU 1646e, LIDAR 1646f, global positioning 1646g, etc.) may be used to determine the navigation operational state. For example, the global positioning sensor 1646g may indicate a set of coordinates associated with the location of the AV 105, such as a set of GPS coordinates and / or a set of coordinates relative to a particular location (e.g., from a start point, a destination point, a waypoint, etc.). As another example, a signal from the IMU sensor 1646e may be used to determine whether the AV is moving or stationary. The signals from the IMU sensor 1646e may further be used to determine the motion velocity, acceleration profile, and / or deceleration profile over a time window (eg, over a 5 second period).

[0099] In some embodiments, a data transmission prioritization machine learning model is used to determine the data transmission prioritization policy. In some embodiments, the data transmission prioritization machine learning model may comprise a neural network. An output of the data transmission prioritization machine learning model may comprise a vector, where each value of the vector describes an ancillary data type, and the order of the values ​​in the vector indicates a priority order. For example, the first value in the vector may be associated with the highest priority ancillary data type, and the last value in the vector may be associated with the lowest priority ancillary data type. In some embodiments, the data transmission prioritization machine learning model may be trained using historical operating condition data associated with previous AVs. In some embodiments, the data transmission prioritization machine learning model performs one or more operations to generate a priority score for each ancillary data type based, at least in part, on the one or more operating conditions. In some embodiments, the priority score for each ancillary data type is determined, at least in part, based on one or more weighted combinations of the one or more operating conditions corresponding to the AV. Based, at least in part, on the priority score for each ancillary data type, the data transmission prioritization machine learning model may output a vector including a list of ancillary data types ordered according to their inferred priority and / or importance. In some embodiments, the data transmission prioritization machine learning model is a rule-based model configured to perform one or more operations that are independent of any trained parameters, such as one or more operations of a defined equation as described in the preceding sentence.In some embodiments, the data transmission prioritization machine learning model is configured to process n operating states associated with AV105 using one or more trained parameters (e.g., one or more trained parameters of a feedforward neural network machine learning model, such as a fully-connected feedforward neural network machine learning model) to generate a list of auxiliary data types ordered according to the inferred priority.

[0100] In some embodiments, the data transmission prioritization machine learning model comprises an ensemble machine learning model framework, in which the data transmission prioritization machine learning model comprises one or more individual operating state machine learning models. Each operating state machine learning model may be configured to process one or more operating states of a particular operating state type. For example, the one or more individual operating state machine learning models may include an AV operating condition operating state machine learning model, an environmental operating state machine learning model, a driving event operating state machine learning model, and / or a navigation operating state machine learning model. Each operating state machine learning model may be configured to generate an individual ranked list of auxiliary data types ordered according to the inferred priority and provide the list to the data transmission prioritization machine learning model, which may be configured to generate a final individual ranked list of auxiliary data types according to the inferred priority. Advantageously, the use of individual operating state machine learning models allows the operating states of the AV105 to be processed in parallel, thus reducing the overall computation time for generating the ranked list of auxiliary data types.

[0101] In some embodiments, the operations server 1500 may also request one or more auxiliary data types from the AV 105. In response, the control subsystem 1400 may allocate a portion of the measurement of auxiliary communications network resource availability to the requested auxiliary data types. Ancillary data types requested by the operations server 1500 may be automatically assigned a priority to the top of an ordered list such that requested auxiliary data is transmitted regardless of its associated priority score.

[0102] In some embodiments, operations server 1500 may request one or more types of auxiliary data based, at least in part, on required data and / or auxiliary data received from one or more other AVs, one or more AVs not associated with operations server 1500, and / or one or more third party sources. For example, if an AV provides data indicating a thunderstorm at a particular location, the operations server may request auxiliary data from AV 105 that accompanies a route that includes the particular location. The requested data may provide information about the current weather the AV 105 is experiencing (e.g., temperature sensor data, image data, etc.). Additionally or alternatively, operations server 1500 may receive weather data indicating a thunderstorm for a geographic region from a third party weather source. Thus, operations server 1500 may request auxiliary data from AVs 105 that are within or expected to be within the geographic region. Thus, operations server 1500, which may have access to information that the AV 105 does not know, may request auxiliary data that is determined to be relevant to the operating conditions of the AV 105.

[0103] In some embodiments, step / operation 604 may be performed according to the process depicted in FIG. 7, which is an exemplary process for updating a data transmission policy. FIG. 7 illustrates a flow chart of an exemplary method 700 for updating a data transmission prioritization policy. One or more of the steps / operations depicted in FIG. 7 may be implemented, at least in part, in the form of software instructions 1408, 1508, and 1680 stored on a non-transitory tangible machine-readable medium (e.g., memory 1404, 1504, 1690, and 1802) that, when executed by one or more processors (e.g., processors 1402, 1502, 1670, and 1804), may cause the one or more processors to perform steps 601-605. In some embodiments, method 600 may be performed by control subsystem 1400, operation server 1500, and on-board control computer 1650 from FIGS. 2-5, respectively.

[0104] Beginning at step / operation 701, an apparatus such as control subsystem 1400 may include means such as processor 1402, memory 1404, and / or the like for receiving one or more signals from one or more sensors. One or more signals from one or more sensors associated with AV 105 may be received at time t, where t may be defined by t0+Δt, where t0 indicates an initial time at which the one or more signals were received (e.g., when AV 105 began operation, an earlier timestamp, etc.), and Δt indicates the elapsed time since t0. The one or more received signals may be received from any of sensors in vehicle driving subsystem 1642, vehicle sensor subsystem 1646, and / or vehicle control subsystem 1648.

[0105] In step / operation 702, an apparatus such as control subsystem 1400 may include means, such as processor 1402, memory 1404, and / or the like, for determining one or more operating states. As described above, the one or more operating states for the AV 105 may be determined based, at least in part, on one or more signals received from one or more sensors carried by the AV. The control subsystem 1400 may evaluate one or more signal characteristics from the one or more received signals and determine one or more operating states for the AV 105. In some embodiments, the control subsystem may determine AV operating condition operating states, environmental operating states, driving event operating states, and / or navigation operating states, as described above.

[0106] In step / operation 703, an apparatus such as the control subsystem 1400 may include means such as a processor 1402, memory 1404, and / or the like for determining whether one or more operating states differ from one or more prior operating states. For example, the one or more prior operating states may correspond to a signal acquired at time t0, while the one or more operating states may correspond to a signal acquired at time t. In some embodiments, the one or more operating states may be determined at periodic or semi-periodic intervals (e.g., every 10 seconds) and used to define Δt. In some embodiments, the operating state may be compared to an operating state immediately preceding the current operating state. For example, if t0 corresponds to a time of 9:00:00 and Δt corresponds to a value of 10 seconds, then t may correspond to a value of 9:00:10.

[0107] In cases where one or more operating conditions are not different from one or more preceding operating conditions, the control subsystem 1400 may return to step / operation 701 and continue to monitor for changes in the one or more operating conditions.

[0108] In cases where one or more operating conditions differ from one or more preceding operating conditions, the control subsystem 1400 may proceed to step / operation 704. In step / operation 704, an apparatus such as the control subsystem 1400 may include means, such as a processor 1402, memory 1404, and / or the like, for updating a data transmission prioritization policy. The control subsystem may update the data transmission prioritization policy based, at least in part, on the one or more operating conditions.

[0109] As described above with respect to step / operation 604, the data transmission prioritization policy may describe one or more policies regarding the transmission of auxiliary data. These one or more policies may be updated to reflect one or more current operating conditions of the AV105, such as by using a data transmission prioritization machine learning model. The data transmission prioritization machine learning model may process one or more operating conditions associated with the AV105 and generate a new data transmission prioritization policy, which may replace a previous data transmission prioritization policy. In some embodiments, only those operating conditions that are determined to be different from a preceding operating condition are processed by the data transmission prioritization machine learning model. Alternatively, each operating condition associated with the AV105 is processed to generate a data transmission prioritization policy.

[0110] In this manner, the control subsystem 1400 may selectively update the data transmission prioritization policy only when needed, such as when the operational state of one or more of the AVs 105 changes. This advantageously allows the control subsystem 1400 to conserve computing resources while still effectively prioritizing the transmission of auxiliary data.

[0111] Turning again now to FIG. 6, in step / operation 605, an apparatus such as the control subsystem 1400 may include means such as a processor 1402, memory 1404, and / or the like for transmitting an auxiliary data object of auxiliary data. The auxiliary data object may be generated based at least in part on a data transmission prioritization policy. For example, the auxiliary data object may be generated to include the top n auxiliary data types according to the data transmission prioritization policy until a measure of auxiliary communication network resource availability is met. In some embodiments, if the inclusion of n auxiliary data types would cause the auxiliary data object to exceed the measure of auxiliary communication network resource availability, the control subsystem 1400 may include the top n-1 auxiliary data types in the auxiliary data object. Thus, the auxiliary data object may include the greatest amount of auxiliary data determined to be most relevant to the operations server 1500 and / or other computing entities.

[0112] Once the auxiliary data object is generated, the control subsystem 1400 may transmit the auxiliary data object to one or more other computing entities, such as the operations server 1500 and / or one or more AVs. In some embodiments, the auxiliary data object may be transmitted simultaneously with the required data object that includes the required data. In some embodiments, the required data may be included within the auxiliary data object. In some embodiments, the auxiliary data object is transmitted separately from the required data object. In some embodiments, the AV 105 may be equipped with two or more transmitters, each transmitter specialized for transmitting a particular type of data object. For example, the AV 105 may have two transmitters, a first transmitter configured to transmit the required data object and a second transmitter configured to transmit the auxiliary data object. Alternatively, one or more transmitters may be used to transmit both the required data object and the auxiliary data object.

[0113] Thus, the methods described above enable transmission of essential and auxiliary data based on a measure of overall communication network resource availability. Particular auxiliary data types transmitted by the AV105 may be prioritized according to an inferred importance based, at least in part, on one or more operational conditions of the AV105. A receiving computing entity, such as the operations server 1500, may use the received essential and / or auxiliary data and perform one or more actions as described above. For example, the operations server 1500 may determine to update routes associated with the AV105 and / or other AVs, inform the AV105 and / or other AVs to perform one or more minimum-risk MRC maneuvers, dispatch one or more other vehicles (e.g., mapping vehicles) to generate and / or update maps, train one or more models (e.g., data transmission prioritization machine learning models), and / or the like. In some embodiments, the operations server 1500 may provide an update message to the AV105 indicating one or more update conditions. For example, the one or more update conditions may indicate an updated route, instructions to perform an MRC maneuver, and / or the like.

[0114] Advantageously, embodiments of the present invention contemplate a data transmission decision based at least in part on overall communication network resource availability (e.g., bandwidth) without the need to consider QoS associated with the data transmission. This allows for a simpler decision regarding the transmission of data, and thus reduced latency and overall bandwidth usage when transmitting data. As described above, embodiments of the present invention contemplate the transmission of essential data and auxiliary data using a data transmission prioritization policy based at least in part on overall communication network resource availability. The data transmission prioritization policy, which may define one or more policies for the transmission of auxiliary data, may be determined at least in part based on one or more operating conditions associated with the AV. In some embodiments, a data transmission prioritization machine learning model may be used to determine the data transmission prioritization policy, such as by using one or more weighted combinations of one or more operating conditions corresponding to the AV. An auxiliary data object of the auxiliary data may then be transmitted from the AV based at least in part on the data transmission prioritization policy. Thus, the AV may take advantage of auxiliary bandwidth network resource availability and provide one or more computing entities with auxiliary data that is determined to have the highest inferred importance based, at least in part, on one or more current operating conditions of the AV.

[0115] Although several embodiments are provided in this disclosure, it should be understood that the disclosed systems and methods may be embodied in many other specific forms without departing from the spirit or scope of the disclosure. The examples are to be considered illustrative rather than restrictive, and the intention is not to be limited to the details given herein. For example, various elements or components may be combined or integrated into another system, or certain features may be omitted or not implemented.

[0116] Additionally, techniques, systems, subsystems, and methods described and illustrated in various embodiments as discrete or separate may be combined or integrated with other systems, modules, techniques, or methods without departing from the scope of the present disclosure. Other items shown or discussed as coupled or directly coupled or in communication with one another may be indirectly coupled or in communication through some interface, device, or intermediate component, whether electrically, mechanically, or otherwise. Other examples of changes, substitutions, and alterations will be ascertainable by those skilled in the art and may be made without departing from the spirit and scope disclosed herein.

[0117] In order to assist the Patent Office, and any reader of any patent issued on this application, in interpreting the claims appended hereto, Applicant emphasizes that none of the appended claims are intended to invoke 35 U.S.C. §112(f), as in existence as of the filing date of this application, unless the words "means for" or "step for" are expressly used in a particular claim.

[0118] Also disclosed herein are the following numbered appendices: 1. A control subsystem for an autonomous vehicle (AV), the control subsystem comprising processing circuitry, the processing circuitry comprising: determining a measure of auxiliary communications network resource availability associated with the auxiliary data; a) describing one or more policies regarding transmission of auxiliary data; and b) determining a data transmission prioritization policy determined at least in part based on one or more operating conditions associated with the AV; transmitting an auxiliary data object of the auxiliary data from the AV, the auxiliary data object being generated based at least in part on a data transmission prioritization policy; a control subsystem configured to: 2. The processing circuitry further comprises, when determining an auxiliary communications network resource availability measure associated with the auxiliary data, determining a measure of overall communications network resource availability available to the AV; and determining a measure of essential communications network resource availability associated with the essential data; determining an auxiliary communications network resource availability measure associated with the auxiliary data based at least in part on the overall communications network resource availability measure and the required communications network resource availability measure; 2. The control subsystem of claim 1, configured to: 3. The processing circuitry further comprises: receiving one or more signals from one or more sensors carried by the AV; determining one or more operational states for the AV based at least in part on the one or more received signals; 5. The control subsystem of any preceding claim, configured to: 4. The control subsystem of claim 3, wherein the processing circuitry is further configured to update the data transmission prioritization policy in instances where one or more determined operating states for the AV differ from one or more preceding operating states. 5. A control subsystem as described in any of the preceding appendices, wherein the data transmission prioritization policy describes at least a data pre-processing policy indicating how data should be processed prior to being transmitted from the AV. 6. The control subsystem of any of the preceding appendices, further comprising one or more transmitters, the one or more transmitters including a first transmitter configured to cause transmission of at least an auxiliary data object from the AV. 7. The control subsystem of claim 6, wherein the one or more transmitters include a second transmitter configured to cause transmission of the required data object from the AV. 8. A method comprising: determining a measure of auxiliary communications network resource availability associated with the auxiliary data; a) describing one or more policies regarding transmission of auxiliary data; and b) determining a data transmission prioritization policy determined at least in part based on one or more operating conditions associated with the AV; transmitting an auxiliary data object of the auxiliary data from the AV, the auxiliary data object being generated based at least in part on a data transmission prioritization policy; A method comprising: 9. Determining a measure of auxiliary communications network resource availability associated with the auxiliary data further comprises: determining a measure of overall communications network resource availability available to the AV; and determining a measure of essential communications network resource availability associated with the essential data; determining an auxiliary communications network resource availability measure associated with the auxiliary data based at least in part on the overall communications network resource availability measure and the required communications network resource availability measure; 9. The method of claim 8, comprising: 10. The method further comprises: receiving one or more signals from one or more sensors carried by the AV; determining one or more operational states for the AV based at least in part on the one or more received signals; 10. The method of claim 8 or 9, comprising: 11. The method of claim 10, further comprising updating a data transmission prioritization policy in cases where one or more determined operating states for the AV differ from one or more preceding operating states. 12. The method of any of appendixes 8-11, wherein the data transmission prioritization policy describes at least a data preprocessing policy indicating how data should be processed prior to being transmitted from the AV. 13. The method of any of Clauses 8-12, wherein the one or more operational states describe one or more of an environmental operating condition, a driving event, an AV operating condition, or a navigation operating condition. 14. The method further comprises: receiving one or more transmission type requests comprising one or more requests for auxiliary data; updating a data transmission prioritization policy based at least in part on the one or more transmission type requests; The method according to any one of claims 8-13, comprising: 15. A computer program product, the computer program product comprising at least one non-transitory computer readable storage medium having computer executable program code instructions stored therein, the computer executable program code instructions comprising: determining a measure of auxiliary communications network resource availability associated with the auxiliary data; a) describing one or more policies regarding transmission of auxiliary data; and b) determining a data transmission prioritization policy determined at least in part based on one or more operating conditions associated with the AV; transmitting an auxiliary data object of the auxiliary data from the AV, the auxiliary data object being generated based at least in part on a data transmission prioritization policy; 23. A computer program product comprising program code instructions configured to cause a computer to: 16. The program code instructions further include: determining a measure of overall communications network resource availability available to the AV; and determining a measure of essential communications network resource availability associated with the essential data; determining an auxiliary communications network resource availability measure associated with the auxiliary data based at least in part on the overall communications network resource availability measure and the required communications network resource availability measure; 16. The computer program product of claim 15, configured to: 17. The program code instructions further include: receiving one or more signals from one or more sensors carried by the AV; determining one or more operational states for the AV based at least in part on the one or more received signals; 17. The computer program product of claim 15 or 16, configured to cause 18. The computer program product of any of claims 15-17, wherein a data transmission prioritization machine learning model is used to determine a data transmission prioritization policy. 19. The computer program product of any of Appendices 15-18, wherein the data transmission prioritization policy describes at least one or more data types associated with the auxiliary data to be transmitted from the AV. 20. The computer program product of any of Appendixes 15-19, wherein the data transmission prioritization policy describes at least how frequently auxiliary data should be transmitted from the AV.

Claims

1. 1. A control subsystem for an autonomous vehicle (AV), the control subsystem comprising processing circuitry, the processing circuitry comprising: determining a measure of auxiliary communications network resource availability associated with the auxiliary data; a) describing one or more policies regarding transmission of the auxiliary data; and b) determining a data transmission prioritization policy determined based, at least in part, on one or more operating conditions associated with the AV; transmitting an auxiliary data object of the auxiliary data from the AV, the auxiliary data object being generated based at least in part on the data transmission prioritization policy; a control subsystem configured to:

2. The processing circuitry further comprises, when determining the auxiliary communications network resource availability measure associated with the auxiliary data, determining a measure of overall communications network resource availability available to said AV; determining a measure of essential communications network resource availability associated with the essential data; determining an auxiliary communications network resource availability measure associated with the auxiliary data based at least in part on the overall communications network resource availability measure and the required communications network resource availability measure; The control subsystem of claim 1 configured to:

3. The processing circuitry further comprises: receiving one or more signals from one or more sensors carried by the AV; determining one or more operating states for the AV based at least in part on the one or more received signals; 3. The control subsystem of claim 1 or 2, configured to:

4. 4. The control subsystem of claim 3, wherein the processing circuitry is further configured to update the data transmission prioritization policy in instances where the one or more determined operating states for the AV differ from one or more preceding operating states.

5. 2. A control subsystem as claimed in any preceding claim, wherein the data transmission prioritization policy describes at least a data pre-processing policy indicating how data should be processed prior to being transmitted from the AV.

6. the control subsystem further comprises one or more transmitters; A control subsystem according to any preceding claim, wherein the one or more transmitters include a first transmitter configured to cause at least the auxiliary data object to be transmitted from the AV.

7. The control subsystem of claim 6 , wherein the one or more transmitters include a second transmitter configured to cause transmission of required data objects from the AV.

8. 1. A method comprising: determining a measure of auxiliary communications network resource availability associated with the auxiliary data; a) describing one or more policies regarding transmission of the auxiliary data; and b) determining a data transmission prioritization policy determined based, at least in part, on one or more operating conditions associated with the AV; transmitting an auxiliary data object of the auxiliary data from the AV, the auxiliary data object being generated based at least in part on the data transmission prioritization policy; A method comprising:

9. Determining the measure of auxiliary communications network resource availability associated with the auxiliary data further comprises: determining a measure of overall communications network resource availability available to said AV; determining a measure of essential communications network resource availability associated with the essential data; determining an auxiliary communications network resource availability measure associated with the auxiliary data based at least in part on the overall communications network resource availability measure and the required communications network resource availability measure; The method of claim 8 , comprising:

10. The method further comprises: receiving one or more signals from one or more sensors carried by the AV; determining one or more operating states for the AV based at least in part on the one or more received signals; The method of claim 8 or 9, comprising:

11. 11. The method of claim 10, further comprising updating the data transmission prioritization policy in cases where the one or more determined operating states for the AV differ from one or more preceding operating states.

12. A method according to any of claims 8-11, wherein said data transmission prioritization policy describes at least a data pre-processing policy indicating how data should be processed prior to being transmitted from said AV.

13. The method of any of claims 8-12, wherein the one or more operating conditions describe one or more of an environmental operating condition, a driving event, an AV operating condition, or a navigation operating condition.

14. The method further comprises: receiving one or more transmission type requests comprising one or more requests for auxiliary data; updating the data transmission prioritization policy based at least in part on the one or more transmission type requests; The method according to any one of claims 8 to 13, comprising:

15. 1. A computer program product comprising at least one non-transitory computer readable storage medium having computer executable program code instructions stored therein, the computer executable program code instructions comprising: determining a measure of auxiliary communications network resource availability associated with the auxiliary data; a) describing one or more policies regarding transmission of the auxiliary data; and b) determining a data transmission prioritization policy determined based, at least in part, on one or more operating conditions associated with the AV; transmitting an auxiliary data object of the auxiliary data from the AV, the auxiliary data object being generated based at least in part on the data transmission prioritization policy; 23. A computer program product comprising program code instructions configured to cause a computer to:

16. The program code instructions further include: determining a measure of overall communications network resource availability available to said AV; determining a measure of essential communications network resource availability associated with the essential data; determining an auxiliary communications network resource availability measure associated with the auxiliary data based at least in part on the overall communications network resource availability measure and the required communications network resource availability measure; 20. The computer program product of claim 15, configured to:

17. The program code instructions further include: receiving one or more signals from one or more sensors carried by the AV; determining one or more operating states for the AV based at least in part on the one or more received signals; 17. A computer program product according to claim 15 or 16, configured to:

18. A computer program product according to any of claims 15-17, wherein a data transmission prioritization machine learning model is used to determine the data transmission prioritization policy.

19. A computer program product according to any of claims 15-18, wherein the data transmission prioritization policy describes at least one or more data types associated with the auxiliary data to be transmitted from the AV.

20. A computer program product according to any of claims 15-19, wherein the data transmission prioritization policy describes at least how frequently the auxiliary data should be transmitted from the AV.

Images