Dynamic allocation of communication network resources for autonomous vehicles

By configuring processing circuit systems and data transmission priority machine learning models in autonomous driving vehicles, dynamically allocating communication network resources, the problems of low data transmission delay and bandwidth usage efficiency in autonomous driving vehicles are solved, and efficient and secure communication is achieved.

CN118901273AInactive Publication Date: 2024-11-05TUSIMPLE INC
View PDF 0 Cites 2 Cited by

Patent Information

Application Number
CN202380028766.8
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Priority Date
2022-03-21
Filing Date
2023-03-21
Publication Date
2024-11-05
Estimated Expiration
Not applicable · inactive patent

AI Technical Summary

Technical Problem

When autonomous vehicles dynamically allocate communication network resources, the prior art is difficult to effectively utilize the overall communication network resource availability, resulting in low data transmission delay and bandwidth usage efficiency.

Method used

By configuring a processing circuit system in an autonomous vehicle, determining the communication network resource availability of auxiliary data and mandatory data, and updating the data transmission priority strategy based on the overall communication network resource availability and operational status, leveraging data transmission priority machine learning The model prioritizes the transmission of high-important auxiliary data.

Benefits of technology

It improves the efficiency and security of data transmission, reduces latency and bandwidth usage, and ensures efficient communication of autonomous vehicles under different operating states.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118901273A_ABST
    Figure CN118901273A_ABST
Patent Text Reader

Abstract

Various embodiments of the present invention provide methods, apparatuses, systems, computing devices, computing entities, and the like for dynamically allocating communication network resources by autonomous vehicles. For example, certain embodiments of the invention utilize systems, methods, and computer program products that determine a data transfer priority policy and cause secondary data objects to be provided based at least in part on the data transfer priority policy.
Need to check novelty before this filing date? Find Prior Art

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims the benefit of U.S. Provisional Application No. 63 / 269,679, filed on March 21, 2022, the entire contents of which are incorporated herein by reference. Technical Field

[0003] The present disclosure relates generally to autonomous vehicles and more particularly to dynamic allocation of communication network resources by an autonomous vehicle (AV). Background Art

[0004] One goal of autonomous vehicle technology is to provide a vehicle that can safely navigate toward a destination with limited or no driver assistance. Safe navigation of an AV from one location to another may include the ability to signal other vehicles, navigate around other vehicles on the shoulder or emergency lane, change lanes, appropriately veer in lane, and navigate all sections or types of highway lanes. By ensuring that the AV can safely operate in a manner that is obvious, logical, or familiar to surrounding drivers and pedestrians, autonomous vehicle technology may enable the AV to operate without requiring extensive learning or training from surrounding drivers.

[0005] During operation, the AV may be equipped with a local onboard control computer capable of determining one or more AV operations. The AV may also provide data to an operations server, which in turn may provide instructions and / or supplemental instructions to the AV. Summary of the invention

[0006] In general, embodiments of the present invention provide methods, apparatuses, systems, computing devices, computing entities, etc. that allow AVs to dynamically allocate communication network resources. For example, certain embodiments of the present invention utilize systems, methods, and computer program products for determining a data transmission priority policy regarding the transmission of auxiliary data and thereby causing the transmission of auxiliary data objects of the auxiliary data.

[0007] According to one aspect, a control subsystem for an autonomous vehicle is provided. The control subsystem may include processing circuitry configured to determine a measure of an auxiliary communication network resource availability associated with auxiliary data. The processing circuitry may also be configured to determine a data transmission priority policy that: a) describes one or more policies regarding transmission of the auxiliary data, and b) is determined based at least in part on one or more operating states associated with the AV. The processing circuitry may also be configured to cause an auxiliary data object of the auxiliary data to be transmitted from the AV, wherein the auxiliary data object is generated based at least in part on the data transmission priority policy.

[0008] In some embodiments, the processing circuit system may be further configured to determine a measure of overall communication network resource availability available to the AV. The processing circuit system may be further configured to determine a measure of mandatory communication network resource availability associated with mandatory data. The processing circuit system may be further configured to determine a measure of auxiliary communication network resource availability associated with auxiliary data based at least in part on the measure of overall communication network resource availability and the measure of mandatory communication network resource availability.

[0009] In some embodiments, the processing circuit system may be further configured to receive one or more signals from one or more sensors carried by the AV. The processing circuit system may be further configured to determine one or more operating states of the AV based at least in part on the received one or more signals. The processing circuit system may be further configured to update the data transmission priority policy if the determined one or more operating states of the AV are different from one or more previous operating states.

[0010] In some embodiments, the processing circuitry may be further configured to receive one or more transmission type requests including one or more requests for auxiliary data. The processing circuitry may be further configured to update the data transmission priority policy based at least in part on the one or more transmission type requests.

[0011] According to another aspect, a computer program product is provided. The computer program product may include at least one non-transitory computer-readable storage medium having computer-executable program code instructions stored therein, the computer-executable program code instructions including program code instructions configured to determine a measure of availability of auxiliary communication network resources associated with auxiliary data. The computer-executable program code instructions including the program code instructions may also be configured to determine a data transmission priority policy, the data transmission priority policy: a) describing one or more policies regarding transmission of auxiliary data, and b) determined at least in part based on one or more operating states associated with the AV. The computer-executable program code instructions including the program code instructions may also be configured to cause an auxiliary data object of the auxiliary data to be transmitted from the AV, wherein the auxiliary data object is generated at least in part based on the data transmission priority policy.

[0012] In some embodiments, the computer executable program code instructions including the program code instructions may also be configured to determine a measure of overall communication network resource availability available to the AV. The computer executable program code instructions including the program code instructions may also be configured to determine a measure of mandatory communication network resource availability associated with mandatory data. The computer executable program code instructions including the program code instructions may also be configured to determine a measure of auxiliary communication network resource availability associated with auxiliary data based at least in part on the measure of overall communication network resource availability and the measure of mandatory communication network resource availability.

[0013] In some embodiments, the computer executable program code instructions including the program code instructions may also be configured to receive one or more signals from one or more sensors carried by the AV. The computer executable program code instructions including the program code instructions may also be configured to determine one or more operating states of the AV based at least in part on the received one or more signals. The computer executable program code instructions including the program code instructions may also be configured to update the data transmission priority policy if the determined one or more operating states of the AV are different from one or more previous operating states.

[0014] In some embodiments, the computer executable program code instructions including the program code instructions may also be configured to receive one or more transmission type requests, the transmission type requests including one or more requests for auxiliary data. The computer executable program code instructions including the program code instructions may also be configured to update the data transmission priority policy based at least in part on the one or more transmission type requests.

[0015] According to another aspect, a method is provided. The method may include determining a measure of availability of an auxiliary communication network resource associated with auxiliary data. The method may also include determining a data transmission priority policy that: a) describes one or more policies regarding transmission of the auxiliary data, and b) is determined based at least in part on one or more operating states associated with the AV. The method may also include causing an auxiliary data object of the auxiliary data to be transmitted from the AV, wherein the auxiliary data object is generated based at least in part on the data transmission priority policy.

[0016] In some embodiments, the method may further include determining a measure of overall communication network resource availability available to the AV. The method may further include determining a measure of mandatory communication network resource availability associated with mandatory data. The method may further include determining a measure of auxiliary communication network resource availability associated with auxiliary data based at least in part on the measure of overall communication network resource availability and the measure of mandatory communication network resource availability.

[0017] In some embodiments, the method may further include receiving one or more signals from one or more sensors carried by the AV. The method may further include determining one or more operating states of the AV based at least in part on the received one or more signals. In one example, the method may further include updating the data transmission priority policy if the determined one or more operating states of the AV are different from one or more previous operating states.

[0018] In some embodiments, the method may further include receiving one or more transmission type requests, the transmission type requests including one or more requests for assistance data. The method may further include updating the data transmission priority policy based at least in part on the one or more transmission type requests.

[0019] According to another aspect, an apparatus is provided. The apparatus may be configured with means for determining a measure of availability of an auxiliary communication network resource associated with auxiliary data. The apparatus may be configured with means for determining a data transmission priority policy that: a) describes one or more policies regarding transmission of the auxiliary data, and b) is determined based at least in part on one or more operating states associated with the AV. The apparatus may be configured with means for causing an auxiliary data object of the auxiliary data to be transmitted from the AV, wherein the auxiliary data object is generated based at least in part on the data transmission priority policy.

[0020] In some embodiments, the apparatus may be configured with means for determining a measure of overall communication network resource availability available to the AV. The apparatus may be configured with means for determining a measure of mandatory communication network resource availability associated with mandatory data. The apparatus may be configured with means for determining a measure of auxiliary communication network resource availability associated with auxiliary data based at least in part on the measure of overall communication network resource availability and the measure of mandatory communication network resource availability.

[0021] In some embodiments, the apparatus may be configured with means for receiving one or more signals from one or more sensors carried by the AV. The apparatus may be configured with means for determining one or more operating states of the AV based at least in part on the received one or more signals. The apparatus may be configured with means for updating a data transmission priority policy if the determined one or more operating states of the AV are different from one or more previous operating states.

[0022] In some embodiments, the apparatus may be configured with means for receiving one or more transmission type requests including one or more requests for auxiliary data. The apparatus may be configured with means for updating a data transmission priority policy based at least in part on the one or more transmission type requests.

[0023] In some embodiments, the data transmission priority policy describes at least a data pre-processing policy that indicates how data should be processed before being transmitted from the AV. In some embodiments, the data transmission priority policy describes at least one or more data types associated with auxiliary data to be transmitted from the AV. In some embodiments, the data transmission priority policy describes at least a frequency at which auxiliary data is to be transmitted from the AV. In some embodiments, the one or more operating states describe one or more of an environmental operating condition, a driving event, an AV operating condition, or a navigation operating condition. In some embodiments, a data transmission priority machine learning model is used to determine the data transmission priority policy. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] For a more complete understanding of the present disclosure, reference is now made to the following brief description taken in conjunction with the accompanying drawings and detailed description, wherein like reference numerals represent like parts.

[0025] Figure 1 An example communication system is illustrated that can be used to practice embodiments of the invention.

[0026] Figure 2 illustrates a block diagram of a control subsystem according to some embodiments discussed herein;

[0027] Figure 3 illustrates a block diagram of an operating server according to some embodiments discussed herein;

[0028] Figure 4 illustrates a diagram of an example autonomous driving vehicle configured to implement autonomous driving operations according to some embodiments discussed herein;

[0029] Figure 5 illustrates a diagram of an onboard control computer included in an AV according to some embodiments discussed herein;

[0030] Figure 6 is a flow chart of an example process for allocating communication network resources according to some embodiments discussed herein; and

[0031] Figure 7 is a flow chart of an example process for updating a data transmission priority policy according to some embodiments discussed herein. DETAILED DESCRIPTION

[0032] Traditionally, data transmission can be determined at least in part based on quality of service (QoS) parameters, which can be determined at least in part based on packet loss, bit rate, throughput, transmission delay, availability, jitter, etc. associated with the data transmission. However, embodiments of the present invention contemplate determining data transmission based at least in part on overall communication network resource availability (e.g., bandwidth) without considering the QoS associated with the above-mentioned data transmission. This allows for a simpler determination of data transmission, and thereby allows for a reduction in delay and overall bandwidth usage when transmitting data. In particular, embodiments of the present invention contemplate transmitting mandatory data and auxiliary data based at least in part on overall communication network resource availability and using a data transmission priority policy. The data transmission priority policy (which can define one or more policies regarding the transmission of auxiliary data) can be determined at least in part based on one or more operating states associated with the AV. In some embodiments, a data transmission priority machine learning model can be used to determine a data transmission priority policy, such as by using one or more weighted combinations of one or more operating states corresponding to the AV. Then, an auxiliary data object of the auxiliary data can be transmitted from the AV based at least in part on the data transmission priority policy. Therefore, the AV can utilize the auxiliary bandwidth network resource availability to provide one or more computing entities with auxiliary data that is determined to have the highest inferred importance based at least in part on one or more current operating states of the AV.

[0033] Figure 1 An example system 100 is illustrated, which includes one or more AVs 105, an operation server 1500 with a human operator 355, and an interface 362 for third party 360 interaction. The human operator 355 may also be referred to as a remote center operator (RCO). Communication between the AV 105, the operation server 1500, and the user interface 362 is performed through a network 370. In some cases, when not all AVs 105 in the fleet are able to communicate with the operation server 1500, the AVs 105 may communicate with each other through the network 370 or directly. Each AV 105 may include a vehicle control computer (VCU) that enables a module for network communication. In some embodiments, the AV 105 may be a semi-truck tractor unit attached to a trailer to transport cargo or goods. However, as will be appreciated by those skilled in the art, the AV 105 may be embodied by any autonomous vehicle, such as a car, a truck, a drone, a watercraft, etc.

[0034] The autonomous vehicles may communicate with the operations server. The operations server may be used for a variety of purposes, including: tracking the progress of one or more autonomous vehicles (e.g., autonomous trucks); tracking the progress of a fleet of autonomous vehicles; sending maneuver instructions to one or more autonomous vehicles; monitoring the health of the autonomous vehicle(s); monitoring the status of the cargo of each autonomous vehicle in contact with the operations server; facilitating communications between third parties (e.g., law enforcement, cargo transportation customers) and each or specific autonomous vehicle; allowing tracking of specific autonomous trucks that communicate with the operations server (e.g., third-party tracking of a subset of vehicles in a fleet); arranging maintenance services for autonomous vehicles (e.g., oil changes, refueling, maintaining levels of other fluids); alerting affected autonomous vehicles of traffic or weather changes that may adversely affect routes or delivery schedules; facilitating over-the-air updates of autonomous trucks to keep all components up to date; and other purposes or functions that improve the safety of autonomous vehicles, their cargo, and / or their surroundings. The operations server may also determine performance parameters of the autonomous vehicle or autonomous truck, including any of the following: data logging frequency, compression rate, location, data type; communication priority; maintenance frequency of the autonomous vehicle (e.g., mileage between maintenance); when a minimum risk condition (MRC) maneuver should be performed while monitoring the vehicle's progress during the maneuver; when control of the autonomous vehicle should be handed over to a human driver (e.g., at a destination parking lot); ensuring that the autonomous vehicle passes a pre-trip inspection; ensuring that the autonomous vehicle executes or complies with legal requirements at checkpoints and weigh stations; ensuring that the autonomous vehicle executes or complies with human instructions at roadblocks, crosswalks, intersections, construction, or accident sites; and so on.

[0035] To allow communication between autonomous vehicles in a fleet and an operation server or command center, each autonomous vehicle may be equipped with a communication gateway. The communication gateway may have the ability to do any of the following: allow AV to operation server communication (i.e., V2C) and operation server to AV communication (C2V); allow AV to AV communication (V2V) within a fleet; transmit the availability or status of the communication gateway; confirm received communications; ensure the security of remote commands between AV and the operation server; reliably communicate the location of the AV at set time intervals; enable the operation server to ping the AV to obtain location and vehicle health; allow various sensor data to be streamed directly to the command or operation server; allow automatic alerts between AV and the operation server; comply with the International Organization for Standardization (ISO) 21434 standard; and so on.

[0036] The operation server or command center can be operated by one or more persons, also referred to as operators and / or remote center operators. The operator can set thresholds for autonomous vehicle health parameters so that when the autonomous vehicle reaches or exceeds the threshold, preventive measures can be taken. Examples of vehicle health parameters for which thresholds can be established by the operator can include any of the following: fuel level; oil level; miles driven since the last maintenance; low tire pressure detected; washer fluid level; brake fluid level; responsiveness of the steering and braking subsystems; diesel exhaust fluid (DEF) level; communication capabilities (e.g., lack of responsiveness); positioning sensor capabilities (e.g., GPS, IMU failure); collision detection (e.g., vehicle collision); perception sensor capabilities (e.g., camera, LIDAR, radar, microphone array failure); computing resource capabilities (e.g., VCU or ECU failure or lack of responsiveness, temperature anomalies in the computing unit); the angle between the tractor and the trailer in a towing situation (e.g., tractor trailer, 18-wheeler, or semi-truck); unauthorized entry of a living being (e.g., a person or animal) into the interior of an autonomous truck; and the like. Preventive measures may include performing a minimum risk condition (MRC) maneuver, seeking service, or exiting a highway or other such diversion that is less taxing on the autonomous vehicle. Autonomous vehicles whose system health data meets or exceeds a threshold set at the operations server or a threshold set by an operator may receive instructions automatically sent from the operations server to perform preventive measures.

[0037] An operator may be made aware of conditions that affect one or more autonomous vehicles in communication with a supervisory system or monitored by an operations server, while the affected autonomous vehicle(s) may not be aware of them. Such conditions may include: irregular or sudden changes in traffic flow (e.g., traffic jams or accidents); sudden changes in weather; sudden changes in visibility; emergency situations (e.g., fires, sinkholes, bridge failures); power outages that affect signal lights; unexpected road works; large or obscure road debris (e.g., objects that cannot be identified by the autonomous vehicle); law enforcement activities on the road (e.g., car chases or road clearing activities); and the like. These types of conditions that may not be detected by the autonomous vehicle may be brought to the attention of the operations server operator through traffic reports, law enforcement communications, data from other vehicles in communication with the operations server, reports from drivers of other vehicles in the area, and similar distributed information venues. The autonomous vehicle may not be able to detect such conditions due to limitations of the sensor system or lack of access to information distribution components (e.g., no direct communication with a meteorological agency). The operator of the operations server may push such information to the affected autonomous vehicles in communication with the operations server. The affected autonomous vehicles may change their routes, trajectories, or speeds in response to the information pushed from the operations server. In some cases, information received by the operating server may trigger a threshold condition indicating that an MRC (Minimum Risk Condition) maneuver is warranted; alternatively or additionally, an operator may assess the situation and determine that the affected autonomous vehicle should perform an MRC maneuver, and then send such instructions to the affected vehicle. In these cases, each autonomous vehicle that receives information or instructions from the operating server or operating server operator uses its onboard computing unit (i.e., VCU) to determine how to safely travel, including performing an MRC maneuver, including pulling over or stopping.

[0038] The operations server or command center may allow third parties to interact with the operations server operator, the self-driving truck, or both the manual system operator and the self-driving truck. The third party may be a customer whose freight is being transported, a law enforcement or emergency service provider, or a person assisting the self-driving truck when service is needed. In interacting with third parties, the operations server may identify different levels of access so that only customers who are concerned about the timing or progress of a shipment are allowed to view status updates for the self-driving truck, or may be able to view the status and provide input to the operations server about which parameters to prioritize (e.g., speed, economy, maintaining the originally planned route). By providing input to the operations server about parameter priorities, customers can influence the route and / or operating parameters of the self-driving truck.

[0039] Exemplary Control Subsystems

[0040] Figure 2One embodiment of a control subsystem 1400 is illustrated. The control subsystem 1400 includes at least one processor 1402, at least one memory 1404, and at least one network interface 1406. The control subsystem 1400 may be configured as shown or in any other suitable configuration.

[0041] Processor 1402 may include one or more processors operably coupled to memory 1404. Processor 1402 is any electronic circuit system, including but not limited to a state machine, one or more central processing unit (CPU) chips, a logic unit, a core (e.g., a multi-core processor), a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), or a digital signal processor (DSP). Processor 1402 may be a programmable logic device, a microcontroller, a microprocessor, or any suitable combination thereof. Processor 1402 is communicatively coupled to memory 1404 and network interface 1406 and performs signal communication with them. One or more processors are configured to process data and may be implemented in hardware or software. For example, processor 1402 may be 8-bit, 16-bit, 32-bit, 64-bit, or any other suitable architecture. Processor 1402 may include an arithmetic logic unit (ALU) for performing arithmetic and logic operations, a processor register that provides operands to the ALU and stores the results of the ALU operations, and a control unit that retrieves instructions from the memory and executes instructions by directing the coordinated operation of the ALU, registers, and other components. One or more processors are configured to implement various instructions. For example, one or more processors are configured to execute instructions to implement the functions disclosed herein, such as Figure 1-Figure 1 3. In some embodiments, the functions described herein are implemented using a logic unit, an FPGA, an ASIC, a DSP, or any other suitable hardware or electronic circuit system.

[0042] The memory 1404 is operable to store any information described below, and any other data, instructions, logic, rules, or code that when executed by the processor 1402 is operable to implement the function(s) described herein. For example, the memory 1404 may store software instructions 1408, sensor data 1412 (received from the sensors 1646, drive subsystem 1642, and / or control subsystem 1648 of the AV 105), map data 1510, route plans 1514, driving instructions 1518, and / or any other data / instructions described herein. The memory 1404 may also store resource allocation instructions 1450, which may be configured to determine mandatory data transfer instructions 1452, auxiliary data transfer instructions 1455, and data transfer priority policy instructions 1456. The memory 1404 includes one or more disks, tape drives, or solid-state drives, and may be used as an overflow data storage device to store such programs when such programs are selected for execution, and to store instructions and data read during program execution. Memory 1404 may be volatile or nonvolatile and may include read-only memory (ROM), random access memory (RAM), ternary content addressable memory (TCAM), dynamic random access memory (DRAM), and static random access memory (SRAM).

[0043] The network interface 1406 is configured to enable wired and / or wireless communications. The network interface 1406 is configured to transmit data between the control subsystem 1400 and other network devices, systems, or (multiple) domains. For example, the network interface 1406 may include a WIFI interface, a local area network (LAN) interface, a wide area network (WAN) interface, a modem, a switch, or a router. The processor 1402 is configured to send and receive data using the network interface 1406. The network interface 1406 can be configured to use any suitable type of communication protocol.

[0044] In one embodiment, the control subsystem 1400 may be a subsystem of the vehicle control computer system 1650 (see Figure 4 ). The control subsystem 1400 may be implemented by one or more computing devices that may be used to determine a data transmission priority strategy and / or control the transmission of mandatory data and / or auxiliary data. The control subsystem 1400 communicates signals with the onboard control computer system 1650 (and its components) and the operation server 1500.

[0045] In some embodiments, mandatory data transfer instructions 1452 may be configured to provide instructions regarding the transmission of mandatory data from AV 105. The instructions may indicate which mandatory data is to be transmitted, a data pre-processing policy indicating how the mandatory data should be processed before being transmitted from AV 105, the frequency with which the mandatory data is to be transmitted from AV 105, the receiving computing entity (e.g., operating server 1500), etc. Mandatory data transfer instructions 1454 may be configured to cause mandatory data objects of the mandatory data to be transmitted from AV 105 to one or more computing entities (such as operating server 1500).

[0046] In some embodiments, the auxiliary data transmission instructions 1454 may be configured to provide instructions regarding transmission of auxiliary data from the AV 105. The instructions may indicate which auxiliary data to transmit, a data pre-processing strategy indicating how the auxiliary data should be processed before being transmitted from the AV 105, the frequency at which the auxiliary data is to be transmitted from the AV 105, a receiving computing entity (e.g., the operating server 1500), a measure of auxiliary communication network resource availability, etc. The auxiliary data transmission instructions 1454 may be configured to cause auxiliary data objects of the auxiliary data to be transmitted from the AV 105 to one or more computing entities (such as the operating server 1500).

[0047] In some embodiments, data transmission priority policy instructions 1456 may be configured to provide instructions regarding determining a data transmission priority policy. For example, data transmission priority policy instructions 1456 may provide instructions regarding determining one or more operating states associated with AV 105, generating a data transmission priority policy, updating a data transmission priority policy, etc. The data transmission priority policy may be used by auxiliary data transmission instructions 1454 regarding transmission of auxiliary data.

[0048] Example Operation Server 1500

[0049] Figure 3 One embodiment of an operations server 1500 is shown. The operations server 1500 includes at least one processor 1502, at least one memory 1504, and at least one network interface 1506. The operations server 1500 may be configured as shown or in any other suitable configuration.

[0050] Processor 1502 includes one or more processors operably coupled to memory 1504. Processor 1502 is any electronic circuit system, including but not limited to a state machine, one or more central processing unit (CPU) chips, logic units, cores (e.g., multi-core processors), field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), or digital signal processors (DSPs). Processor 1502 may be a programmable logic device, a microcontroller, a microprocessor, or any suitable combination thereof. Processor 1502 is communicatively coupled to memory 1504, network interface 1506, and user interface 1528 and communicates signals with them. One or more processors are configured to process data and may be implemented in hardware or software. For example, processor 1502 may be 8-bit, 16-bit, 32-bit, 64-bit, or any other suitable architecture. Processor 1502 may include an arithmetic logic unit (ALU) for performing arithmetic and logic operations, processor registers that provide operands to the ALU and store the results of the ALU operations, and a control unit that retrieves instructions from memory and executes instructions by directing the coordinated operation of the ALU, registers, and other components. One or more processors are configured to implement various instructions. For example, one or more processors are configured to execute instructions to implement the functions disclosed herein, such as Figure 1-Figure 1 3. In some embodiments, the functions described herein are implemented using a logic unit, an FPGA, an ASIC, a DSP, or any other suitable hardware or electronic circuit system.

[0051] The memory 1504 is operable to store any of the information described below, and any other data, instructions, logic, rules, or code that, when executed by the processor 1502, is operable to implement the function(s) described herein. For example, the memory 1504 may store software instructions 1508, map data 1510, rerouting plans 1520, routing plans 1514, training data sets 1522, driving instructions 1518, sensor data 1412 received from the AV 1602, traffic data 1524, map building modules 1526, and / or any other data / instructions. The memory 1504 includes one or more disks, tape drives, or solid-state drives, and may be used as an overflow data storage device to store programs when such programs are selected for execution, and to store instructions and data read during program execution. The memory 1504 may be volatile or non-volatile, and may include read-only memory (ROM), random access memory (RAM), ternary content addressable memory (TCAM), dynamic random access memory (DRAM), and static random access memory (SRAM).

[0052] The network interface 1506 is configured to enable wired and / or wireless communications. The network interface 1506 is configured to transmit data between the control subsystem 1400 and other network devices, systems, or (multiple) domains. For example, the network interface 1506 may include a WIFI interface, a local area network (LAN) interface, a wide area network (WAN) interface, a modem, a switch, or a router. The processor 1502 is configured to send and receive data using the network interface 1506. The network interface 1506 can be configured to use any suitable type of communication protocol.

[0053] In one embodiment, the operation server 1500 may be implemented by a cluster of computing devices that may be used to oversee the operation of the AV 105. For example, the operation server 1500 may be implemented by multiple computing devices using a distributed computing and / or cloud computing system. In another example, the operation server 1500 may be implemented by multiple computing devices in one or more data centers. The operation server 1500 communicates signals with the AV 105 and its components (e.g., the onboard control computer 1650). In one embodiment, the operation server 1500 is configured to determine a specific routing plan 1514 for each AV 105. For example, the operation server 1500 may determine a specific routing plan 1514 for the AV 105 that may reduce driving time and provide a safer driving experience for reaching the destination of the AV 105. In one embodiment, the operation server 1500 may capture and record the navigation plan set by the user 1536 in each case and use it for similar cases.

[0054] In one embodiment, the operation server 1500 may send the sensor data 1412 to the application server 1532 for viewing by the user 1536. The application server 1532 is generally any computing device configured to communicate with other devices via a network interface, such as other servers (e.g., the operation server 1500), the AV 105, a database, etc. The application server 1532 is configured to perform the specific functions described herein and interact with the user 1536, for example, via its user interface 1534. Examples of application servers 1532 include, but are not limited to, desktop computers, laptop computers, servers, etc. In one example, the application server 1533 may act as a presentation layer in which the user 1536 may access to view the sensor data 1412. Thus, the operation server 1500 may send the sensor data 1412 to the application server 1532. The user 1536 may view the sensor data 1412 from the user interface 1534 and confirm, modify, and / or override the navigation solution for the AV 105 determined by the control subsystem 1400 and / or the operation server 1500, as described above.

[0055] Map data 1510 may include a virtual map of a city including routes. In some examples, map data 1510 may include one or more map databases. Map data 1510 may include drivable areas such as roads, trails, highways, and non-drivable areas such as terrain. Map data 1510 may specify location coordinates of road signs, lanes, lane markings, lane boundaries, road boundaries, traffic lights, and the like.

[0056] The map data 1510 may also specify connections between lanes (e.g., which lanes can merge into other adjacent lanes). The map data 1510 may specify information indicating road lane types (e.g., travel lanes, passing lanes, emergency lanes, turn lanes, bus lanes, etc.), lane boundary types (e.g., white lines, yellow lines, other road surface markings and / or mechanical markings, etc.), road boundary types (e.g., regular curbs, red curbs, sidewalks, guardrails, other obstacles, etc.), intersections, one or more obstacles in front of the autonomous vehicle, and other information about the road or areas near the road. The map data 1510 may also specify the elevation of the road, such as curves, hills, valleys; road hazards, such as speed bumps, potholes; road sections, such as highway school zones, railroad crossings, etc.

[0057] The map building module 1526 can be implemented by the processor 1502 executing the software instructions 1508, and is configured to build the map data 1510. In one embodiment, the map building module 1526 can build the map data 1510 based on the sensor data received from one or more mapping vehicles. In one example, the mapping vehicle can be an AV 105. In another example, the mapping vehicle can be an AV 105 or a non-autonomous driving vehicle connected or integrated with a sensor 1646 operated by a driver. In some cases, one or more mapping vehicles can be dispatched to observe a situation detected by the AV 105, such as via mandatory and / or auxiliary data transmitted from the AV 105, and send more sensor data related to the situation.

[0058] The map building module 1526 is configured to use the sensor data to determine which portion of the map data 1510 they are associated with. The map building module 1526 can dynamically build each portion of the map data 1510 by merging the different sensor data associated with each portion of the map data 1510. The map building module 1526 also uses the sensor data to find overlapping portions of the map data 1510 (e.g., by matching corresponding images, videos, LiDAR data, radar data, etc., observing the same portion of the map data 1510). The map building module 1526 then connects the portions of the map data 1510 with their corresponding adjacent portions. In other words, the map building module 1526 finds adjacent portions of the map data 1510, stitches them together, and builds the map data 1510. The map building module 1526 is also configured to update a portion of the map data 1510 that is based at least in part on the received sensor data that needs to be updated.

[0059] For example, in the case where an unknown object exists in the road, when the operation server 1500 receives sensor data from the AV 105 via mandatory data transmission and / or auxiliary data transmission, once it is confirmed that the unknown object does exist at a specific location coordinate, the map construction module 1526 updates a portion of the map data 1510 to reflect the presence of the unknown object. Similarly, the map construction module 1526 updates different portions of the map data 1510 based at least in part on sensor data associated with different situations in which an unexpected situation is encountered.

[0060] The map building module 1526 is also configured to facilitate shadow testing of the route plan 1514 using the new layout of one or more roads in the updated map data 1510. For example, when the map data 1510 is updated, the map building module 1526 can run one or more simulations of autonomous driving with the updated map data 1510 of the AV 105. For example, the one or more simulations of autonomous driving can relate to the AV 105 navigating around objects, changing lanes, pulling over, rerouting, etc.

[0061] The map building module 1526 selects a particular autonomous driving process from one or more simulations of autonomous driving that provides a more efficient, safe, and reliable navigation solution for each AV 105 compared to other simulations.

[0062] The route plan 1514 is a plan for traveling from a starting location (e.g., a first AV launch pad / landing pad) to a destination (e.g., a second AV launch pad or landing pad). For example, the route plan 1514 of the AV 105 may specify a combination of one or more streets / roads / highways in a specific order from the starting location to the destination. The route plan 1514 of the AV 105 may specify stages, including a first stage (e.g., moving out of the starting location), multiple intermediate stages (e.g., traveling along specific lanes of one or more specific streets / roads / highways), and a final stage (e.g., entering the destination). The route plan 1514 may include other information about the route from the starting location to the destination, such as road / traffic signs in the route plan 1514, etc.

[0063] The driving instructions 1518 may include instructions and rules for adapting the autonomous driving of the AV 105 according to the driving rules for each stage of the route plan 1514. For example, the driving instructions 1518 may include instructions for staying within a speed range for the road on which the AV 105 is traveling, adapting the speed of the AV 105 according to changes observed by the sensors 1646 (such as the speed of surrounding vehicles, objects within the detection area of ​​the sensors), and the like.

[0064] The object detection machine learning module 1520 may be implemented by the processor 1502 executing software instructions 1508, and is generally configured to detect objects from the sensor data 1412. The object detection machine learning module 1520 may be implemented using a neural network and / or machine learning algorithm for detecting objects from any data type, such as images, videos, infrared images, point clouds, radar data, etc.

[0065] In one embodiment, the object detection machine learning module 1520 may be implemented using a machine learning algorithm, such as a support vector machine (SVM), naive Bayes, logistic regression, k-nearest neighbor, decision tree, etc. In one embodiment, the object detection machine learning module 1520 may utilize multiple neural network layers, convolutional neural network layers, etc., wherein the weights and biases of the perceptrons of these layers are optimized during the training process of the object detection machine learning module 1520. The object detection machine learning module 1520 may be trained by a training data set 1522, which includes samples of data types labeled with one or more objects in each sample. For example, the training data set 1522 may include sample images of objects (e.g., vehicles, lane markings, pedestrians, road signs, etc.) labeled with (multiple) objects in each sample image. Similarly, the training data set 1522 may include samples of other data types labeled with (multiple) objects in each sample data, such as videos, infrared images, point clouds, radar data, etc. The object detection machine learning module 1520 may be trained, tested, and refined using the training data set 1522 and the sensor data 1412. The object detection machine learning module 1520 uses the sensor data 1412 (not labeled with objects) to improve the accuracy of their predictions when detecting objects. For example, when detecting objects in the sensor data 1412, supervised and / or unsupervised machine learning algorithms can be used to verify the predictions of the object detection machine learning module 1520.

[0066] The traffic data 1524 may include traffic data 1524 for roads / streets / highways in the map data 1510. The operation server 1500 may use traffic data 1524 collected by one or more map vehicles. The operation server 1500 may use traffic data 1524 captured from any source, such as crowdsourced traffic data 1524 captured from external sources (e.g., Waze and Google Maps, real-time traffic reports, etc.).

[0067] Example AV 105 and its operation

[0068] Figure 4A block diagram of an example vehicle ecosystem 1600 in which autonomous driving operations may be determined is shown. As shown in FIG. 16 , the AV 105 may be a semi-trailer. The vehicle ecosystem 1600 includes several systems and components that may generate and / or deliver one or more information / data sources and related services to an onboard control computer 1650 that may be located in the AV 105. The onboard control computer 1650 may be in data communication with a plurality of vehicle subsystems 1640, all of which may reside in the AV 105. A vehicle subsystem interface 1660 is provided to facilitate data communication between the onboard control computer 1650 and the plurality of vehicle subsystems 1640. In some embodiments, the vehicle subsystem interface 1660 may include a controller area network (CAN) controller for communicating with devices in the vehicle subsystems 1640.

[0069] The AV 105 may include various vehicle subsystems that support the operation of the AV 105. The vehicle subsystems may include a control subsystem 1400, a vehicle drive subsystem 1642, a vehicle sensor subsystem 1644, and / or a vehicle control subsystem 1648. Figure 4 The components or devices shown of the vehicle drive subsystem 1642, vehicle sensor subsystem 1644, and vehicle control subsystem 1648 are examples. The vehicle drive subsystem 1642 may include components operable to provide powered motion to the AV 105. In an example embodiment, the vehicle drive subsystem 1642 may include an engine / motor 1642a, wheels / tires 1642b, a transmission 1642c, an electrical subsystem 1642d, and a power source 1642e.

[0070] The vehicle sensor subsystem 1644 may include a plurality of sensors 1646 configured to sense information about the environment or condition of the AV 105. The vehicle sensor subsystem 1644 may include one or more cameras 1646a (which may be equipped with a microphone or image capture device), a radar unit 1646b, one or more temperature sensors 1646c, a wireless communication unit 1646d (e.g., a cellular communication transceiver), an inertial measurement unit (IMU) 1646e, a laser rangefinder / LDAR unit 1646f, a global positioning system (GPS) transceiver 1646g, a wiper control system 1646h, a microphone 1646i, and / or tire pressure sensor(s) 1646j. The vehicle sensor subsystem 1644 may also include sensors configured to monitor internal systems of the AV 105 (e.g., an O2 monitor, a fuel gauge, an engine oil temperature, etc.).

[0071] The IMU 1646e may include any combination of sensors (e.g., accelerometers and gyroscopes) configured to sense changes in the position and orientation of the AV 105 based at least in part on inertial acceleration. The GPS transceiver 1646g may be any sensor configured to estimate the geographic location of the AV 105. To this end, the GPS transceiver 1646g may include a receiver / transmitter operable to provide information about the location of the AV 105 relative to the earth. The radar unit 1646b may represent a system that utilizes radio signals to sense objects within the local environment of the AV 105. In some embodiments, in addition to sensing objects, the radar unit 1646b may also be configured to sense the speed and heading of objects close to the AV 105. The laser rangefinder or LiDAR unit 1646f may be any sensor configured to use lasers to sense objects in the environment in which the AV 105 is located. The camera 1646a may include one or more devices configured to capture multiple images of the environment of the AV 105. The camera 1646a may be a still image camera or a motion video camera. In some embodiments, the camera 1646a may be configured with one or more microphones to capture audio data. Additionally or alternatively, one or more microphones 1646i may be configured to capture audio data of the environment of the AV 105 and / or audio data related to the AV 105.

[0072] The vehicle control subsystem 1648 may be configured to control the operation of the AV 105 and its components. Thus, the vehicle control subsystem 1648 may include various elements, such as a throttle and gear 1648a, a brake unit 1648b, a navigation unit 1648c, a steering system 1648d, and / or an autonomous control unit 1648e. The throttle 1648a may be configured to control, for example, the operating speed of the engine, and thereby control the speed of the AV 105. The gear 1648a may be configured to control the gear selection of the transmission. The brake unit 1648b may include any combination of mechanisms configured to decelerate the AV 105. The brake unit 1648b may use friction to decelerate the wheels in a standard manner. The brake unit 1648b may include an anti-lock braking system (ABS), which may prevent the brakes from locking when the brakes are applied. The navigation unit 1648c may be any system configured to determine the driving path or route of the AV 105. The navigation 1648c unit may also be configured to dynamically update the driving path while the AV 105 is operating. In some embodiments, the navigation unit 1648c may be configured to combine data from the GPS transceiver 1646g, and one or more predetermined maps, in order to determine the driving path of the AV 105. The steering system 1648d may represent any combination of mechanisms that may be operable to adjust the heading of the AV 105 in either an autonomous mode or a pilot-controlled mode.

[0073] The autonomous control unit 1648e may represent a control system configured to identify, evaluate, and avoid or otherwise negotiate potential obstacles or obstructions in the environment of the AV 105. In general, the autonomous control unit 1648e may be configured to control the AV 105 to operate without a driver, or to provide driver assistance in controlling the AV 105. In some embodiments, the autonomous control unit 1648e may be configured to combine data from the GPS transceiver 1646g, radar 1646b, LiDAR unit 1646f, camera 1646a, and / or other vehicle subsystems to determine a travel path or trajectory for the AV 105.

[0074] Many or all of the functions of the AV 105 may be controlled by an onboard control computer 1650. The onboard control computer 1650 may include at least one data processor 1670 (which may include at least one microprocessor) that executes processing instructions 1680 stored in a non-transitory computer-readable medium (such as a data storage device 1690 or a memory). The onboard control computer 1650 may also represent a plurality of computing devices that may be used to control individual components or subsystems of the AV 105 in a distributed manner. In some embodiments, the data storage device 1690 may contain processing instructions 1680 (e.g., program logic) executable by the data processor 1670 to perform various methods and / or functions of the AV 105.

[0075] The data storage device 1690 may also contain additional instructions, including instructions for transmitting data to, receiving data from, interacting with, or controlling one or more of the vehicle drive subsystem 1642, the vehicle sensor subsystem 1644, and the vehicle control subsystem 1648. The onboard control computer 1650 may be configured to include the data processor 1670 and the data storage device 1090. The onboard control computer 1650 may control the functions of the AV 105 based at least in part on input received from various vehicle subsystems (e.g., the vehicle drive subsystem 1642, the vehicle sensor subsystem 1644, and the vehicle control subsystem 1648).

[0076] Figure 5 1 shows an exemplary block diagram of an onboard control computer 1650 included in the AV 105. The onboard control computer 1650 includes at least one processor 1804 and instructions (e.g., Figure 2 , Figure 3 and Figure 4 The instructions, when executed by the processor 1804, configure the vehicle control computer 1650 and / or the various modules of the vehicle control computer 1650 to perform Figure 2-Figure 4. Transmitter 1806 transmits or sends information or data to one or more devices in the autonomous vehicle. For example, transmitter 1806 can send instructions for manipulating the autonomous vehicle to one or more motors of the steering wheel. Receiver 1808 receives information or data transmitted or sent by one or more devices. For example, receiver 1808 receives the state of the current speed from the odometer sensor, or receives the state of the current transmission gear from the transmission. Transmitter 1806 and receiver 1808 are also configured to communicate with control subsystem 1400.

[0077] The onboard control computer 1650 may include a sensor fusion module 1702 that may perform image or signal processing operations. The sensor fusion module 1702 may acquire images from a camera located on the autonomous vehicle to perform image segmentation to detect the presence of moving objects (e.g., other vehicles, pedestrians, etc.) and / or static obstacles (e.g., stop signs, speed bumps, terrain, etc.) located around the autonomous vehicle. The sensor fusion module 1702 may acquire LiDAR point cloud data items from a LiDAR sensor located on the autonomous vehicle to perform LiDAR segmentation to detect the presence of objects and / or obstacles located around the autonomous vehicle.

[0078] The sensor fusion module 1702 can perform instance segmentation on the image and / or point cloud data items to identify contours (e.g., boxes) around objects and / or obstacles located around the autonomous vehicle. The sensor fusion module 1702 can perform temporal fusion, in which objects and / or obstacles from one image and / or one frame of point cloud data items are associated with objects and / or obstacles in one or more images or frames subsequently received in time.

[0079] The sensor fusion module 1702 can fuse objects and / or obstacles based on images acquired from the camera and / or point cloud data items acquired from the LiDAR sensor. For example, the sensor fusion module 1702 can determine that the image from one of the cameras (which includes half of the vehicle located in front of the autonomous driving vehicle) is the same as the vehicle captured by the other camera based on the positions of the two cameras. The sensor fusion module 1702 sends the fused object information to the interference module 1746 and sends the fused obstacle information to the occupancy grid module 1760. The onboard control computer includes an occupancy grid module 1760, which can retrieve landmarks from the map database 1758 stored in the onboard control computer. The occupancy grid module 1760 can determine the drivable area and / or obstacles based on the fused obstacles acquired from the sensor fusion module 1702 and the landmarks stored in the map database 1758. For example, the occupancy grid module 1760 can determine that the drivable area may include a speed bump obstacle.

[0080] The onboard control computer 1650 may also include a LiDAR-based object detection module 1712 that may perform object detection 1716 based on point cloud data items acquired from a LiDAR sensor located on the AV 105. The object detection technology may provide the location of the object (e.g., in 3D world coordinates) from the point cloud data items. Below the LiDAR-based object detection module 1712, the onboard control computer may include an image-based object detection module that may perform object detection based on images acquired from a camera located on the autonomous vehicle. The object detection technology may employ deep machine learning techniques to provide the location of the object (e.g., in 3D world coordinates) from the images provided by the camera.

[0081] The radar on the autonomous vehicle can scan the area in front of the autonomous vehicle or the area towards which the autonomous vehicle is traveling. The radar data is sent to the sensor fusion module 1702, which can use the radar data to correlate objects and / or obstacles detected by the radar with objects and / or obstacles detected from the LiDAR point cloud data items and camera images.

[0082] The planning module 1762 may perform navigation planning 1764 to determine a set of trajectories on which the AV 105 may be driven. The set of trajectories may be determined based on the drivable area information, the location of obstacles, and the drivable area information. In some embodiments, navigation planning may include determining an area next to a road where the autonomous vehicle can be safely parked in an emergency. The planning module 1762 may include a behavioral decision for determining a driving action (e.g., steering, braking, throttle) in response to determining a changing condition on the road (e.g., a traffic light turns yellow, or the autonomous vehicle is in an unsafe driving condition because another vehicle is in front of the autonomous vehicle and is traveling within a predetermined safe distance of the location of the autonomous vehicle). The planning module 1762 performs trajectory generation and selects a trajectory from the set of trajectories determined by the navigation planning operation. The selected trajectory information is sent by the planning module 1762 to the control module 1770.

[0083] The onboard control computer includes a control module 1770 that receives the proposed trajectory from the planning module 1762 and receives the autonomous vehicle position and attitude from the fused positioning module 1726. The control module 1770 includes a system identifier. The control module 1770 can perform model-based trajectory refinement to refine the proposed trajectory. For example, the control module 1770 can apply filtering (e.g., a Kalman filter) to smooth the proposed trajectory data and / or minimize noise. The control module 1770 can perform robust control by determining the amount of brake pressure to be applied, the steering angle, the throttle amount for controlling the vehicle speed, and / or the transmission gear based on the refined proposed trajectory information and the current position and / or attitude of the autonomous vehicle. The control module 1770 can send the determined brake pressure, steering angle, throttle amount, and / or transmission gear to one or more devices in the autonomous vehicle to control and facilitate the precise driving operation of the autonomous vehicle.

[0084] Example Dynamic Allocation of Communications Network Resources

[0085] Figure 6 A flow diagram of an example method 600 for dynamically allocating communication network resources is illustrated. Figure 6 One or more steps / operations depicted in the method 600 may be implemented at least in part in the form of software instructions 1408, 1508, and 1680 stored on a non-transitory tangible machine-readable medium (e.g., memory 1404, 1504, 1690, and 1802), which when executed by one or more processors (e.g., processors 1402, 1502, 1670, and 1804) may cause one or more processors to perform steps 601-605. In some embodiments, the method 600 may be respectively Figure 2-Figure 5 The control subsystem 1400, operation server 1500 and on-board control computer 1650 are executed.

[0086] As shown in step / operation 601, an apparatus such as control subsystem 1400 may include components for determining a measure of overall communication network resource availability, such as processor 1402, memory 1404, network interface 1406, etc. As described above, AV 105 may be communicatively coupled to one or more other computing entities (e.g., operation server 1500, one or more other AVs, one or more third parties 360, etc.) via network 370. In some embodiments, AV 105 may be communicatively coupled to one or more other computing entities using any suitable technology, including but not limited to a radio access network, advanced long term evolution (advanced LTE, i.e., LTE-A), universal mobile telecommunications system (UMTS) radio access network (UTRAN or E-UTRAN), wireless local area network (WLAN or WiFi), world interoperability for microwave access (WiMAX), Personal Communications Service (PCS), Wideband Code Division Multiple Access (WCDMA), systems using Ultra-Wideband (UWB) technology, sensor networks, Mobile Ad Hoc Networks (MANETs), and Internet Protocol Multimedia Subsystem (IMS), or any combination thereof.

[0087] Regardless of the connection type, the processor 1402 can determine a measure of overall communication network resource availability for the current network connection type. For example, the measure of overall communication network resource availability can be a measure of bandwidth available for the control subsystem 1400. In some embodiments, the measure of overall communication network resource availability can specifically refer to a measure of upload bandwidth availability.

[0088] The illustrative embodiments will be explained herein in the context of the open system interconnection model (OSI model), which is a model that conceptually characterizes the communication functions of a communication system (such as, for example, an LTE network). The OSI model is generally conceptualized as a layered stack in which a given layer serves a service and is served by a lower layer. Typically, the OSI model includes seven layers, with the top layer of the stack being the application layer (layer 7), followed by the presentation layer (layer 6), the session layer (layer 5), the transport layer (layer 4), the network layer (layer 3), the data link layer (layer 2), and the physical layer (layer 1). Those of ordinary skill in the art will understand the functions and interactions of the various layers, and therefore, further details of each layer are not described herein. However, it should be understood that while the illustrative embodiments are well suited for implementation using the OSI model, alternative embodiments are not necessarily limited to any particular communication function model.

[0089] The scheduling of overall communication network resource availability (e.g., available bandwidth) available to the control subsystem 1400 may be performed within the application layer of the network 370. In some embodiments, the control subsystem 1400 may request an indication of overall communication network resource availability via the network interface 1406. In some embodiments, the control subsystem 1400 may locally determine the overall communication network resource availability. The overall communication network resource availability may be based at least in part on one or more determining factors, such as the current network traffic load and / or a maximum resource allocation value determined by a network subscription associated with the control subsystem 1400.

[0090] At step / operation 602, an apparatus such as the control subsystem 1400 may include components for determining a measure of mandatory communication network resource availability, such as a processor 1402, a memory 1404, etc. The measure of mandatory communication network resource availability may be based at least in part on relevant mandatory data. Mandatory data may refer to data that needs to be transmitted during the operation of the AV 105. For example, the mandatory data may include certain sensor data associated with the AV 105 (e.g., speed, position coordinates, etc.). For example, the mandatory data may be data associated with the basic operation of the AV 105. Therefore, the mandatory data may be critical to the safe or efficient operation of the AV 105, such as data that is critical to performing a predetermined set of operations of the AV 105. Therefore, the predetermined set of operations may be a set of operations necessary to achieve a specific safety or operating standard. Other examples of mandatory data include data related to the overall health of the vehicle communication system (e.g., data used to confirm that such a communication system is working properly), or data related to the overall health of the mechanical aspects of the vehicle during vehicle transportation (e.g., data used to confirm that there are no structural defects or problems). In fact, the determination of which data is mandatory may depend on the current state, operation, or desired operation of the AV 105 .

[0091] In some embodiments, the control subsystem 1400 may be configured with a data classification set that may define one or more data types associated with a data category. The data classification set may be a mapping of data categories to data types associated with the AV 105 associated with the control subsystem 1400, and vice versa. For example, one or more candidate data categories may include mandatory data categories and auxiliary data categories. According to the data classification set, a data type of "image data from camera 1" may be defined as mandatory. Thus, any data associated with the data type "image data from camera 1" is determined to be mandatory data. In some embodiments, the classification set may be determined and initialized based at least in part on a user input, such as the RCO 355.

[0092] The control subsystem 1400 may then determine a measure of mandatory communication network resource availability based at least in part on the aggregation of mandatory data to be transmitted. For example, the control subsystem 1400 may determine the relative data packet size of the data packets containing the aggregated mandatory data. In some embodiments, the measure of mandatory communication network resource availability may refer to a measure of the upload bandwidth dedicated to the transmission of mandatory data. The measure of mandatory communication network resource availability may be considered a measure of the communication network resources required by the AV 105 to perform a predetermined set of operations.

[0093] In some embodiments, the control subsystem 1400 can be configured to pre-process the mandatory data according to a mandatory data pre-processing strategy. The mandatory data pre-processing strategy can define a process for pre-processing each mandatory data type in a specific manner before causing the mandatory data to be provided to one or more additional computing entities. In some embodiments, the control subsystem 1400 can cause the processor 1402 to execute one or more mandatory data pre-processor routines defined by the mandatory data pre-processing strategy so that the mandatory data can be properly formatted for transmission. For example, any pre-processing routine for image data can convert the original image into one or more compressed images. Therefore, the data size associated with the mandatory data can be reduced.

[0094] At step / operation 603, an apparatus such as the control subsystem 1400 may include a component, such as a processor 1402, a memory 1404, etc., for determining a measure of the availability of auxiliary communication network resources. The measure of the availability of auxiliary communication network resources may be based at least in part on the availability of the overall communication network resources and the availability of the mandatory communication network resources determined at steps / operations 601 and 602, respectively. Auxiliary data may refer to optional data for transmission during the operation of the AV 105. For example, the auxiliary data may include sensor data associated with the AV 105 (e.g., speed, position coordinates, etc.) that is not classified as mandatory. Therefore, such optional auxiliary data may not be data required for the safe operation of the AV 105. In other words, the AV 105 may still perform a predetermined set of operations without the auxiliary data. However, such auxiliary data may still improve the performance of the AV 105 and may further improve safety, efficiency, responsiveness, or another attribute of AV performance.

[0095] As described above, in some embodiments, the control subsystem 1400 may be configured with a data classification set that may define one or more data types associated with a data category. The data classification set defines data types that are classified as auxiliary data. For example, one or more candidate data categories may include mandatory data categories and auxiliary data categories. Although the data type of "image data from camera 1" may be defined as mandatory according to the data classification set, the data type of "image data from camera 3" may be defined as auxiliary according to the data classification set. Here, the image data from camera 3 may be classified as auxiliary data because the fields of view of camera 1 and camera 3 overlap significantly, so it may not be necessary to always have image data from camera 3. Therefore, any data associated with the data type "image data from camera 3" is determined to be auxiliary data.

[0096] The control subsystem 1400 may then determine a measure of the auxiliary communication network availability based at least in part on the overall communication network resource availability and the mandatory communication network resource availability. For example, if the measure of the overall network resource availability is 5 megabits per second (Mbps) and the measure of the mandatory communication network resource availability is 3 Mbps, the measure of the auxiliary communication network resource availability may be determined to be 2 Mbps. In some embodiments, the measure of the auxiliary communication network resource availability may refer to a measure of the upload bandwidth dedicated to the auxiliary data.

[0097] At step / operation 604, a device such as the control subsystem 1400 may include a device for determining a data transmission priority strategy, such as a processor 1402, a memory 1404, etc. The data transmission priority strategy may describe one or more strategies for the transmission of auxiliary data. In particular, the data transmission priority strategy may describe a priority ranking of auxiliary data, wherein the priority ranking of auxiliary data indicates the priority order of transmitting data associated with the auxiliary data. The priority ranking of data may be based at least in part on the inferred importance associated with specific data associated with the auxiliary data. For example, relative to the data type of "image data from camera 8", it may be determined that the "sound signal" data category is more important, where camera 8 may cover an area that has been significantly covered by mandatory data. Therefore, the "sound signal" data type may be assigned a higher priority than the "image data from camera 8" ranking. In other words, the priority ranking of auxiliary data may be based on at least one of the following: the degree of overlap of auxiliary data with mandatory data, the additional functionality provided by auxiliary data compared to mandatory data, and the degree of AV performance improvement provided by auxiliary data. Thus, auxiliary data with less overlap, more additional functionality, or a higher degree of AV performance improvement may be ranked higher than auxiliary data that provides more overlap, less additional functionality, or a lower degree of AV performance improvement.

[0098] In some embodiments, the data transmission priority policy may also describe one or more of the data pre-processing policies that indicate how the auxiliary data should be processed before being transmitted from the AV 105 and / or the frequency with which the auxiliary data is to be transmitted from the AV 105. For example, any pre-processing routine for image data may convert a raw image into one or more compressed images. As another example, the frequency of the auxiliary data may be determined based on the size of the data packets associated with the auxiliary data. In the case of a large size of the auxiliary data, the transmission frequency of the auxiliary data may be reduced so as not to exceed the allocated measure of the auxiliary communication network availability.

[0099] In addition, the data transmission priority strategy can be determined at least in part based on one or more operating states associated with the AV 105. The one or more operating states of the AV 105 can be determined at least in part based on one or more signals received from one or more sensors carried by the AV (such as one or more sensors included in the vehicle sensor subsystem 1644). The control subsystem 1400 can evaluate one or more signal characteristics based on the one or more signals received to determine one or more operating states of the AV 105. In some embodiments, the control subsystem can determine the AV operating condition operating state, the environmental operating state, the driving event operating state, and / or the navigation operating state. The determination of a specific operating state can affect the priority ranking of the auxiliary data. For example, the reception of a signal related to a specific event may result in the determination of the relevant operating state of the AV 105. In this case, the priority ranking of the auxiliary data related to the detection of such an event can be adjusted so that such auxiliary data is ranked higher. Whether the auxiliary data partially overlaps with the mandatory data, adjusting the priority ranking in this way can occur. Beneficially, adjusting the priority ranking of the auxiliary data in response to determining a specific operating state can ensure that the auxiliary data most relevant to the current operating state is processed first. This provides dynamic adjustment of auxiliary data priority, balancing communication network resource availability and data priority to maximize safe and efficient operation of the AV 105 within the constraints of communication network resource availability. Another example scenario is camera operation when the autonomous vehicle is moving forward compared to when it is stopped at a certain location for a long time (e.g., under minimal risk conditions). While driving, the system can assign higher priority to camera operations that provide a view in the direction of travel, typically a forward view. Conversely, when pulling over or stopping, camera operations that provide a view of all of the surroundings of the autonomous vehicle (e.g., providing a 360-degree panoramic view) can be prioritized. In other words, auxiliary data related to the view in the direction of travel can be prioritized during movement in that direction, while auxiliary data related to the view of the surroundings can be prioritized when the vehicle is not moving.

[0100] The AV operating condition operation state may describe the state of the AV 105 relative to one or more sensor subsystems 1644, drive subsystems 1642, and / or control subsystems 1648. The AV operating condition operation state may indicate whether a subsystem of the AV 105 is operating normally or abnormally. For example, if a signal from a tire pressure sensor 1646j of the sensor subsystem 1644 indicates that a tire pressure value does not meet one or more tire pressure thresholds, the AV operating condition operation state may be assigned an "abnormal" value. In addition, in some embodiments, the AV operating condition operation state may indicate the associated sensors and / or subsystems that led to the determination of the "abnormal" value. As a continuing example, the AV operating condition operation state may be assigned values ​​of "abnormal" and "low tire pressure". In other words, receiving a signal indicating an "abnormal" value in one or more of the subsystems of the AV 105 may result in the determination of the AV operating condition operation state. Therefore, such a determination may result in an adjustment in the priority ranking of auxiliary data. In this example, such an adjustment may be an increase in the ranking of auxiliary data related to tire pressure, an increase in the ranking of a subsystem related to tire pressure, or an increase in any auxiliary data related to detecting normal / abnormal operation of the AV. In this way, AVs can operate more safely and efficiently when additional data related to identified issues is prioritized.

[0101] The environmental operating state may describe the environmental conditions in which the AV 105 operates. For example, the environmental operating state may indicate the weather conditions and / or temperature of the surrounding environment of the AV 105. In some embodiments, one or more signals from one or more sensors of the sensor subsystem 1644 (e.g., temperature sensor 1646c, camera 1646a, wiper control system 1646h, etc.) may be used to directly or indirectly determine the weather conditions and / or temperature of the surrounding environment. For example, a signal from the temperature sensor 1646c may be used directly to determine the ambient temperature of the AV 105. As another example, a signal from the camera 1646a may be used indirectly to determine the weather conditions. In particular, one or more images acquired from one or more cameras 1646a may be analyzed to determine the presence of precipitation. The temperature determined using the temperature sensor 1646c may be used in conjunction with one or more images to determine the type of precipitation. For example, it may be difficult to discern whether the precipitation determined from one or more images is snow or rain. However, if the temperature value is above 5 degrees Celsius, the precipitation is classified as rain, if the temperature is below -5 degrees Celsius, the precipitation is classified as snow, and if the temperature is between -5 and 5 degrees Celsius, the precipitation is classified as sleet. Additionally or alternatively, the environmental operating state may indicate the presence of a weather event such as hail, tornado, flood, hurricane, thunderstorm, sandstorm, windstorm, etc. In other words, the receipt of a signal indicating a weather event may result in a determination of the environmental operating state. Therefore, such a determination may result in an adjustment of the priority ranking of the auxiliary data. In this example, such an adjustment may be an increase in the ranking of the auxiliary data related to the detected weather event. In this way, when the additional data related to the identified weather event is prioritized, the AV can operate more safely and efficiently.

[0102] The driving event operation state may describe one or more driving events associated with the road segment on which the AV 105 is traveling. For example, the driving event operation state may indicate one or more driving events, such as detection of an emergency vehicle, a construction zone, a vehicle accident, a traffic jam, an unexpected object on a road segment, a difference between an expected object on a map and a detected object, etc. In some embodiments, one or more signals from one or more sensors of the sensor subsystem 1644 (e.g., camera 1646a, radar 1646b, IMU 1646e, LIDAR 1646f, etc.) may be used to determine one or more driving events. For example, one or more images acquired from one or more cameras 1646a may be analyzed to determine whether there are unexpected objects, vehicle accidents, and / or differences between expected objects and detected objects in the road. As another example, one or more signals acquired from a camera (configured with a microphone) may be analyzed to determine the presence of sound characteristics of driving events such as sirens, brake whistles, horn sounds, etc. In other words, the receipt of a signal indicating a driving event may result in determining a driving event operation state. Therefore, such a determination may result in the adjustment of the priority ranking of auxiliary data. In this example, such an adjustment can be an increase in the ranking of auxiliary data related to the detected driving event. In this way, the AV can operate more safely and efficiently when additional data related to the identified driving event is prioritized.

[0103] The navigation operation state may describe the current operation of the AV with respect to one or more routes. For example, the navigation operation state may indicate the current operation state of the AV 105 (e.g., moving, stationary, etc.), the operation speed, the operation direction, the acceleration mode within the time window, the deceleration mode within the time window, a set of coordinates indicating the current position of the AV, etc. In some embodiments, one or more signals from one or more sensors of the sensor subsystem 1644 (e.g., camera 1646a, radar 1646b, IMU 1646e, LIDAR 1646f, global positioning 1646g, etc.) may be used to determine the navigation operation state. For example, the global positioning sensor 1646g may indicate a set of coordinates associated with the position of the AV 105, such as a set of GPS coordinates and / or a set of coordinates relative to a specific location (e.g., from an origin, a destination, an intermediate point, etc.). As another example, a signal from an IMU sensor 1646e may be used to determine whether the AV is moving or stationary. The signal from the IMU sensor 1646e may also be used to determine the operation speed, acceleration profile, and / or deceleration profile within a time window (e.g., within a period of 5 seconds).

[0104] In some embodiments, a data transmission priority machine learning model is used to determine a data transmission priority policy. In some embodiments, the data transmission priority machine learning model may include a neural network. The output of the data transmission priority machine learning model may include a vector, wherein each value of the vector describes an auxiliary data type, and the order of the values ​​within the vector indicates the priority order. For example, the first value in the vector may be associated with the highest priority auxiliary data type, and the last value in the vector may be associated with the lowest priority auxiliary data type. In some embodiments, the data transmission priority machine learning model may be trained using historical operating state data associated with a previous AV. In some embodiments, the data transmission priority machine learning model performs one or more operations to generate a priority score for each auxiliary data type based at least in part on one or more operating states. In some embodiments, the priority score of each auxiliary data type is determined at least in part based on one or more weighted combinations of one or more operating states corresponding to the AV. Based at least in part on the priority score of each auxiliary data type, the data transmission priority machine learning model may output a vector including a list of auxiliary data types sorted according to inferred priority and / or importance. In some embodiments, the data transmission priority machine learning model is a rule-based model that is configured to perform one or more operations that do not depend on any training parameters, such as one or more operations of the defined equation described in the preceding sentence. In some embodiments, the data transfer priority machine learning model is configured to process n operating states associated with the AV 105 using one or more training parameters (e.g., one or more training parameters of a feedforward neural network machine learning model, such as a fully connected feedforward neural network machine learning model) to generate a list of auxiliary data types ordered according to inferred priority.

[0105] In some embodiments, the data transmission priority machine learning model includes an integrated machine learning model framework such that the data transmission priority machine learning model includes one or more individual operating state machine learning models. Each operating state machine learning model can be configured to process one or more operating states of a specific operating state type. For example, one or more individual operating state machine learning models can include an AV operating condition operating state machine learning model, an environmental operating state machine learning model, a driving event operating state machine learning model, and / or a navigation operating state machine learning model. Each operating state machine learning model can be configured to generate a ranked list of individual auxiliary data types sorted according to the inferred priority, and provide the list to the data transmission priority machine learning model, which can be configured to generate a final ranked list of individual auxiliary data types according to the inferred priority. Advantageously, the use of individual operating state machine learning models allows the operating states of the AV 105 to be processed in parallel, thereby reducing the overall computation time for generating the ranked list of auxiliary data types.

[0106] In some embodiments, the operations server 1500 may also request one or more auxiliary data types from the AV 105. In response, the control subsystem 1400 may assign a portion of the measure of auxiliary communication network resource availability to the requested auxiliary data type. The auxiliary data type requested by the operations server 1500 may be automatically assigned a priority ranking at the head of the sorted list so that the requested auxiliary data may be transmitted regardless of its associated priority score.

[0107] In some embodiments, the operation server 1500 may request one or more auxiliary data types based at least in part on mandatory data and / or auxiliary data received from one or more other AVs, one or more AVs not associated with the operation server 1500, and / or one or more third-party sources. For example, if the AV provides data indicating a thunderstorm at a particular location, the operation server may request auxiliary data from the AV 105 that includes a route to the particular location. The requested data may be information about the current weather that the AV 105 is experiencing (e.g., temperature sensor data, image data, etc.). Additionally or alternatively, the operation server 1500 may receive weather data indicating thunderstorms in a geographic area from a third-party weather source. Thus, the operation server 1500 may request auxiliary data from an AV 105 that is located or expected to be located in the geographic area. Thus, the operation server 1500, which may know information that the AV 105 does not know, may request auxiliary data that is determined to be relevant to the operating conditions of the AV 105.

[0108] In some embodiments, step / operation 604 may be performed according to Figure 7 The process is performed by the process depicted in, which is an example process for updating a data transfer policy. Figure 7 A flow chart of an example method 700 for updating a data transmission priority policy is illustrated. Figure 7 One or more of the steps / operations depicted in the method 600 may be implemented at least in part in the form of software instructions 1408, 1508, and 1680 stored on a non-transitory tangible machine-readable medium (e.g., memory 1404, 1504, 1690, and 1802), which when executed by one or more processors (e.g., processors 1402, 1502, 1670, and 1804) may cause one or more processors to perform steps 601-605. In some embodiments, the method 600 may be respectively Figure 2-Figure 5 The control subsystem 1400, operation server 1500 and on-board control computer 1650 are executed.

[0109] Beginning with step / operation 701, an apparatus such as the control subsystem 1400 may include components for receiving one or more signals from one or more sensors, such as a processor 1402, a memory 1404, etc. One or more signals from one or more sensors associated with the AV 105 may be received at time t, where t may be defined by t0+Δt, and t0 indicates an initial time at which the one or more signals are received (e.g., when the AV 105 begins operation, a previous time step, etc.), and Δt indicates the time elapsed since t0. The one or more signals received may be received from any of the sensors of the vehicle drive subsystem 1642, the vehicle sensor subsystem 1646, and / or the vehicle control subsystem 1648.

[0110] At step / operation 702, an apparatus such as the control subsystem 1400 may include components for determining one or more operating states, such as a processor 1402, a memory 1404, etc. As described above, one or more operating states of the AV 105 may be determined based at least in part on one or more signals received from one or more sensors carried by the AV. The control subsystem 1400 may evaluate one or more signal characteristics based on the received one or more signals to determine one or more operating states of the AV 105. In some embodiments, the control subsystem may determine an AV operating condition operating state, an environmental operating state, a driving event operating state, and / or a navigation operating state, as described above.

[0111] At step / operation 703, an apparatus such as control subsystem 1400 may include components such as processor 1402, memory 1404, etc. for determining whether one or more operating states are different from one or more previous operating states. For example, one or more previous operating states may correspond to a signal acquired at time t0, and one or more operating states may correspond to a signal acquired at time t. In some embodiments, one or more operating states may be determined at periodic or semi-periodic intervals (e.g., every 10 seconds) and may be used to define Δt. In some embodiments, the operating state may be compared to the operating state immediately preceding the current operating state. For example, if t0 corresponds to time 9:00:00, and Δt corresponds to a value of 10 seconds, then t may correspond to a value of 9:00:10.

[0112] In one example, if the one or more operating states are not different from the one or more previous operating states, the control subsystem 1400 may return to step / operation 701 and continue to monitor for changes in the one or more operating states.

[0113] In one example, the one or more operating states are different than the one or more previous operating states, and the control subsystem 1400 may proceed to step / operation 704. At step / operation 704, an apparatus such as the control subsystem 1400 may include components for updating a data transmission priority policy, such as the processor 1402, the memory 1404, etc. The control subsystem may update the data transmission priority policy based at least in part on the one or more operating states.

[0114] As described above with respect to step / operation 604, the data transmission priority policy may describe one or more policies regarding the transmission of auxiliary data. These one or more policies may be updated to reflect one or more current operating states of the AV 105, such as by using a data transmission priority machine learning model. The data transmission priority machine learning model may process one or more operating states associated with the AV 105 to generate a new data transmission priority policy, which may replace a previous data transmission priority policy. In some embodiments, the data transmission priority machine learning model only processes operating states that are determined to be different from a previous operating state. Alternatively, each operating state associated with the AV 105 is processed to generate a data transmission priority policy.

[0115] In this manner, the control subsystem 1400 can selectively update the data transmission priority policy only when needed, such as when one or more operating states of the AV 105 change. This advantageously allows the control subsystem 1400 to conserve computing resources while still effectively prioritizing the transmission of auxiliary data.

[0116] Now return to Figure 6 , at step / operation 605, an apparatus such as the control subsystem 1400 may include components for causing an auxiliary data object of the auxiliary data to be transmitted, such as a processor 1402, a memory 1404, etc. The auxiliary data object may be generated based at least in part on a data transmission priority policy. For example, the auxiliary data object may be generated according to the data transmission priority policy to include the first n auxiliary data types until the measure of the auxiliary communication network resource availability is satisfied. In some embodiments, if the inclusion of the n auxiliary data types causes the auxiliary data object to exceed the measure of the auxiliary communication network resource availability, the control subsystem 1400 may include the first n-1 auxiliary data types in the auxiliary data object. Thus, the auxiliary data object may include the maximum number of auxiliary data determined to be most relevant to the operation server 1500 and / or other computing entities.

[0117] Once the auxiliary data object is generated, the control subsystem 1400 can cause the auxiliary data object to be transmitted to one or more other computing entities, such as the operation server 1500 and / or one or more AVs. In some embodiments, the auxiliary data object can be transmitted simultaneously with the mandatory data object including the mandatory data. In some embodiments, the mandatory data can be included in the auxiliary data object. In some embodiments, the auxiliary data object is transmitted separately from the mandatory data object. In some embodiments, the AV 105 can be equipped with two or more transmitters, each of which is dedicated to transmitting a specific type of data object. For example, the AV 105 can have two transmitters, wherein the first transmitter is configured to transmit the mandatory data object and the second transmitter is configured to transmit the auxiliary data object. Alternatively, one or more transmitters can be used to transmit both the mandatory data object and the auxiliary data object.

[0118] Thus, the above method allows for the transmission of mandatory data and auxiliary data based on a measure of overall communication network resource availability. Specific auxiliary data types transmitted by the AV 105 can be prioritized based on an inferred importance based at least in part on one or more operating states of the AV 105. A receiving computing entity such as the operation server 1500 can use the received mandatory and / or auxiliary data to perform one or more operations as described above. For example, the operation server 1500 can determine to update a route associated with the AV 105 and / or other AVs, notify the AV 105 and / or other AVs to perform one or more minimum risk MRC maneuvers, schedule one or more other vehicles (e.g., mapping vehicles) to generate and / or update maps, train one or more models (e.g., data transmission priority machine learning models), and the like. In some embodiments, the operation server 1500 can provide an update message indicating one or more update conditions to the AV 105. For example, the one or more update conditions can indicate an updated route, instructions for performing an MRC maneuver, and the like.

[0119] Advantageously, embodiments of the present invention contemplate determining data transmission based at least in part on overall communication network resource availability (e.g., bandwidth) without considering the QoS associated with the above-mentioned data transmission. This allows for a simpler determination of data transmission, and thereby allows for a reduction in delay and overall bandwidth usage when transmitting data. As described above, embodiments of the present invention contemplate transmitting mandatory data and auxiliary data based at least in part on overall communication network resource availability and using a data transmission priority policy. The data transmission priority policy (which may define one or more policies regarding the transmission of auxiliary data) may be determined based at least in part on one or more operating states associated with the AV. In some embodiments, a data transmission priority machine learning model may be used to determine a data transmission priority policy, such as by using one or more weighted combinations of one or more operating states corresponding to the AV. Then, an auxiliary data object of the auxiliary data may be transmitted from the AV based at least in part on the data transmission priority policy. Thus, the AV may utilize the auxiliary bandwidth network resource availability to provide one or more computing entities with auxiliary data that is determined to have the highest inferred importance based at least in part on one or more current operating states of the AV.

[0120] Although several embodiments are provided in the present disclosure, it should be understood that the disclosed systems and methods may be embodied in many other specific forms without departing from the spirit or scope of the present disclosure. The present embodiments are considered to be illustrative rather than restrictive, and are not intended to be limited to the details given herein. For example, various elements or components may be combined or integrated into another system, or certain features may be omitted or not implemented.

[0121] In addition, without departing from the scope of the present disclosure, the techniques, systems, subsystems and methods described and illustrated as discrete or separate in various embodiments may be combined or integrated with other systems, modules, techniques or methods. Other items shown or discussed as coupled or directly coupled or communicating with each other may be indirectly coupled or communicated through some interface, device or intermediate component, whether electrical, mechanical or other. Other examples of changes, substitutions and variations are within the ability of those skilled in the art to determine and may be made without departing from the spirit and scope disclosed herein.

[0122] To assist the Patent Office and any reader of any patent issuing based upon this application in interpreting the claims appended hereto, applicants note that unless “means” or “step” is expressly used in a particular claim, no invocation is intended to invoke 35 U.S.C. § 112(f), as it existed on the date of filing this application.

[0123] The following numbered clauses are also disclosed herein:

[0124] 1. A control subsystem for an autonomous vehicle (AV), the control subsystem comprising a processing circuit system, the processing circuit system being configured to:

[0125] determining a measure of availability of an auxiliary communication network resource associated with the auxiliary data;

[0126] determining a data transmission priority policy, the data transmission priority policy: a) describing one or more policies regarding transmission of auxiliary data, and b) determined based at least in part on one or more operating states associated with the AV; and

[0127] An auxiliary data object causing the auxiliary data to be transmitted from the AV, wherein the auxiliary data object is generated based at least in part on the data transmission priority policy.

[0128] 2. A control subsystem according to clause 1, wherein in determining the measure of availability of auxiliary communication network resources associated with the auxiliary data, the processing circuit system is further configured to:

[0129] Determining a measure of overall communication network resource availability available to the AV;

[0130] determining a measure of mandatory communication network resource availability associated with the mandatory data; and

[0131] A measure of auxiliary communication network resource availability associated with the auxiliary data is determined based at least in part on the measure of overall communication network resource availability and the measure of mandatory communication network resource availability.

[0132] 3. A control subsystem according to any of the preceding clauses, wherein the processing circuit system is further configured to:

[0133] receiving one or more signals from one or more sensors carried by the AV; and

[0134] One or more operating states of the AV are determined based at least in part on the received one or more signals.

[0135] 4. The control subsystem of clause 3, wherein the processing circuitry is further configured to update the data transfer priority policy if the determined one or more operating states of the AV are different from one or more previous operating states.

[0136] 5. A control subsystem according to any of the preceding clauses, wherein the data transmission priority policy describes at least a data pre-processing policy indicating how data should be processed before being transmitted from the AV.

[0137] 6. The control subsystem according to any of the preceding clauses, wherein the control subsystem further comprises one or more transmitters, wherein the one or more transmitters comprises a first transmitter configured to cause at least the auxiliary data object to be transmitted from the AV.

[0138] 7. The control subsystem of clause 6, wherein the one or more transmitters include a second transmitter configured to cause the mandatory data object to be transmitted from the AV.

[0139] 8. A method comprising:

[0140] determining a measure of availability of an auxiliary communication network resource associated with the auxiliary data;

[0141] determining a data transmission priority policy, the data transmission priority policy: a) describing one or more policies regarding transmission of auxiliary data, and b) determined based at least in part on one or more operating states associated with the AV; and

[0142] An auxiliary data object causing the auxiliary data to be transmitted from the AV, wherein the auxiliary data object is generated based at least in part on the data transmission priority policy.

[0143] 9. The method of clause 8, wherein determining a measure of the availability of the auxiliary communication network resources associated with the auxiliary data further comprises:

[0144] Determining a measure of overall communication network resource availability available to the AV;

[0145] determining a measure of mandatory communication network resource availability associated with the mandatory data; and

[0146] A measure of auxiliary communication network resource availability associated with the auxiliary data is determined based at least in part on the measure of overall communication network resource availability and the measure of mandatory communication network resource availability.

[0147] 10. The method according to clause 8 or 9, further comprising:

[0148] receiving one or more signals from one or more sensors carried by the AV; and

[0149] One or more operating states of the AV are determined based at least in part on the received one or more signals.

[0150] 11. The method of clause 10, further comprising updating a data transmission priority policy if the determined one or more operating states of the AV are different from one or more previous operating states.

[0151] 12. A method according to any of clauses 8 to 11, wherein the data transmission priority policy describes at least a data pre-processing policy indicating how data should be processed before being transmitted from the AV.

[0152] 13. A method according to any of clauses 8 to 12, wherein the one or more operating states describe one or more of an environmental operating condition, a driving event, an AV operating condition, or a navigation operating condition.

[0153] 14. A method according to any one of clauses 8 to 13, further comprising:

[0154] receiving one or more transmission type requests, the one or more transmission type requests comprising one or more requests for assistance data; and

[0155] The data transmission priority policy is updated based at least in part on the one or more transmission type requests.

[0156] 15. A computer program product comprising at least one non-transitory computer-readable storage medium having computer-executable program code instructions stored therein, the computer-executable program code instructions comprising program code instructions configured to:

[0157] determining a measure of availability of an auxiliary communication network resource associated with the auxiliary data;

[0158] determining a data transmission priority policy, the data transmission priority policy: a) describing one or more policies regarding transmission of auxiliary data, and b) determined based at least in part on one or more operating states associated with the AV; and

[0159] An auxiliary data object causing the auxiliary data to be transmitted from the AV, wherein the auxiliary data object is generated based at least in part on the data transmission priority policy.

[0160] 16. The computer program product according to clause 15, wherein the program code instructions are further configured to:

[0161] Determining a measure of overall communication network resource availability available to the AV;

[0162] determining a measure of mandatory communication network resource availability associated with the mandatory data; and

[0163] A measure of auxiliary communication network resource availability associated with the auxiliary data is determined based at least in part on the measure of overall communication network resource availability and the measure of mandatory communication network resource availability.

[0164] 17. A computer program product according to clause 15 or 16, wherein the program code instructions are further configured to:

[0165] receiving one or more signals from one or more sensors carried by the AV; and

[0166] One or more operating states of the AV are determined based at least in part on the received one or more signals.

[0167] 18. A computer program product according to any one of clauses 15 to 17, wherein a data transmission priority machine learning model is used to determine a data transmission priority strategy.

[0168] 19. A computer program product according to any of clauses 15 to 18, wherein the data transmission priority policy describes at least one or more data types associated with auxiliary data to be transmitted from the AV.

[0169] 20. The computer program product according to any of clauses 15 to 19, wherein the data transmission priority policy describes at least the frequency with which auxiliary data is to be transmitted from the AV.

Claims

1. A control subsystem for an autonomous vehicle (AV), the control subsystem comprising a processing circuit system, the processing circuit system being configured to: determining a measure of availability of an auxiliary communication network resource associated with the auxiliary data; determining a data transmission priority policy, the data transmission priority policy: a) describing one or more policies regarding the transmission of the auxiliary data, and b) determined based at least in part on one or more operating states associated with the AV; as well as An auxiliary data object of the auxiliary data is caused to be transmitted from the AV, wherein the auxiliary data object is generated based at least in part on the data transmission priority policy.

2. The control subsystem of claim 1 , wherein in determining the measure of the auxiliary communication network resource availability associated with the auxiliary data, the processing circuit system is further configured to: determining a measure of overall communication network resource availability available to the AV; determining a measure of availability of mandatory communication network resources associated with mandatory data; as well as The measure of the auxiliary communication network resource availability associated with the auxiliary data is determined based at least in part on the measure of the overall communication network resource availability and the measure of the mandatory communication network resource availability.

3. The control subsystem of claim 1 or 2, wherein the processing circuit system is further configured to: receiving one or more signals from one or more sensors carried by the AV; and One or more operating states of the AV are determined based at least in part on the received one or more signals.

4. The control subsystem of claim 3, wherein the processing circuit system is further configured to update the data transfer priority policy if the determined one or more operating states of the AV are different from one or more previous operating states.

5. A control subsystem according to any one of the preceding claims, wherein the data transmission priority policy describes at least a data pre-processing policy indicating how data should be processed before being transmitted from the AV.

6. A control subsystem according to any of the preceding claims, wherein the control subsystem further comprises one or more transmitters, wherein the one or more transmitters comprise a first transmitter configured to cause at least the auxiliary data object to be transmitted from the AV.

7. The control subsystem of claim 6, wherein the one or more transmitters include a second transmitter configured to cause a mandatory data object to be transmitted from the AV.

8. A method comprising: determining a measure of availability of an auxiliary communication network resource associated with the auxiliary data; determining a data transmission priority policy, the data transmission priority policy: a) describing one or more policies regarding the transmission of the auxiliary data, and b) determined based at least in part on one or more operating states associated with the AV; as well as An auxiliary data object of the auxiliary data is caused to be transmitted from the AV, wherein the auxiliary data object is generated based at least in part on the data transmission priority policy.

9. The method of claim 8, wherein determining a measure of the auxiliary communication network resource availability associated with the auxiliary data further comprises: determining a measure of overall communication network resource availability available to the AV; determining a measure of availability of mandatory communication network resources associated with mandatory data; as well as The measure of the auxiliary communication network resource availability associated with the auxiliary data is determined based at least in part on the measure of the overall communication network resource availability and the measure of the mandatory communication network resource availability.

10. The method according to claim 8 or 9, further comprising: receiving one or more signals from one or more sensors carried by the AV; as well as One or more operating states of the AV are determined based at least in part on the received one or more signals.

11. The method according to claim 10, further comprising: In an event that the determined one or more operating states of the AV are different than one or more previous operating states, the data transmission priority policy is updated.

12. The method according to any one of claims 8 to 11, wherein the data transmission priority policy describes at least a data pre-processing policy indicating how data should be processed before being transmitted from the AV.

13. The method of any one of claims 8 to 12, wherein the one or more operating states describe one or more of an environmental operating condition, a driving event, an AV operating condition, or a navigation operating condition.

14. The method according to any one of claims 8 to 13, further comprising: receiving one or more transmission type requests, the one or more transmission type requests comprising one or more requests for assistance data; as well as The data transmission priority policy is updated based at least in part on the one or more transmission type requests.

15. A computer program product comprising at least one non-transitory computer-readable storage medium having computer-executable program code instructions stored therein, the computer-executable program code instructions comprising program code instructions configured to: determining a measure of availability of an auxiliary communication network resource associated with the auxiliary data; determining a data transmission priority policy, the data transmission priority policy: a) describing one or more policies regarding the transmission of the auxiliary data, and b) determined based at least in part on one or more operating states associated with the AV; as well as An auxiliary data object of the auxiliary data is caused to be transmitted from the AV, wherein the auxiliary data object is generated based at least in part on the data transmission priority policy.

16. The computer program product of claim 15, wherein the program code instructions are further configured to: determining a measure of overall communication network resource availability available to the AV; determining a measure of mandatory communication network resource availability associated with the mandatory data; and The measure of the auxiliary communication network resource availability associated with the auxiliary data is determined based at least in part on the measure of the overall communication network resource availability and the measure of the mandatory communication network resource availability.

17. The computer program product according to claim 15 or 16, wherein the program code instructions are further configured to: receiving one or more signals from one or more sensors carried by the AV; and One or more operating states of the AV are determined based at least in part on the received one or more signals.

18. A computer program product according to any one of claims 15 to 17, wherein a data transmission priority machine learning model is used to determine the data transmission priority strategy.

19. The computer program product of any one of claims 15 to 18, wherein the data transmission priority policy describes at least one or more data types associated with the auxiliary data to be transmitted from the AV.

20. The computer program product of any one of claims 15 to 19, wherein the data transmission priority policy describes at least a frequency with which the auxiliary data is to be transmitted from the AV.

Citation Information

Cited By

  • Hybrid communication network seamless switching method for automatic driving

    CN121194267A

  • A seamless handover method for a hybrid communication network oriented to automatic driving

    CN121194267B