System and method for remote control guided autonomy of autonomous vehicles

Abstract

A system includes an autonomous vehicle and a control device. The control device detects an event trigger affecting the autonomous vehicle. In response to detecting the event trigger, the control device causes the autonomous vehicle to enter a first degraded autonomous mode. In a first degraded autonomous mode, the control device communicates sensor data to the supervisory server. A control device receives a high-level command from a supervisory server. The one or more advanced commands indicate minimum risk maneuver for the autonomous vehicle. The control device receives a maximum driving speed for the autonomous vehicle from the supervision server. The control device navigates the autonomous vehicle using adaptive cruise control in accordance with the one or more advanced commands and the maximum driving speed.

Description

[0001] This application claims priority to U.S. Provisional Application No. 63 / 364,531, filed on May 11, 2022, entitled “SYSTEM AND METHOD FOR REMOTECONTROL GUIDED AUTONOMY FOR AUTONOMOUS VEHICLES,” which is incorporated herein by reference. Technical Field

[0002] The present disclosure relates generally to autonomous vehicles. More specifically, the present disclosure relates to systems and methods for remotely controlling guided autonomy of autonomous vehicles. Background Art

[0003] In some cases, while traveling along a road, a component of the autonomous vehicle may fail. The failed component may affect the operation of the autonomous vehicle. Once there is evidence of a failure, the autonomous vehicle is suddenly forced to stop if the failure is severe, or pulled over to the side of the road if the failure is less severe. These scenarios may increase the likelihood of accidents with other vehicles. Summary of the invention

[0004] The present disclosure identifies various problems and previously unmet needs associated with navigating an autonomous vehicle in situations where hardware failures and / or software failures affect the operation of the autonomous vehicle. Certain embodiments of the present disclosure provide unique technical solutions to the technical problems of current autonomous vehicle technology, including those described above, to enable various degraded autonomy modes for an autonomous vehicle, depending on the situation.

[0005] The present disclosure contemplates systems and methods for implementing various degraded autonomy modes for an autonomous vehicle depending on the situation.

[0006] In an example scenario, assume that an autonomous vehicle is traveling along a road, and a control device associated with the autonomous vehicle detects an event trigger that affects the autonomous vehicle. The event trigger may include a hardware failure and / or a software failure with respect to the autonomous vehicle.

[0007] A control device may cause the autonomous vehicle to enter a first degraded autonomous mode if: 1) wireless communication between the control device and a supervisory server is at least partially operable (even with a delay greater than a threshold, e.g., greater than half a second, one second, two seconds, etc.); 2) the lane detection capability and position detection capability of the control device are at least partially operable; 3) the traffic sign detection capability of the control device is at least partially operable; and 4) adaptive cruise control is at least partially operable.

[0008] In a first degraded autonomous mode, the control device communicates sensor data (captured by sensors of the autonomous vehicle) to a supervisory server and, in response, receives high-level commands and a maximum travel speed from the supervisory server.

[0009] The control device detects lane markings and traffic signs based on the sensor data. The control device navigates the autonomous vehicle using adaptive cruise control based on the high-level command, the maximum travel speed, the lane markings, and the traffic signs.

[0010] The control device may cause the autonomous vehicle to enter a second, degraded autonomous mode if: 1) wireless communication between the control device and the supervisory server is at least partially operational (even if with a delay greater than a threshold, e.g., greater than half a second, one second, two seconds, etc.); 2) adaptive cruise control is at least partially operational; and 3) the control device is unable to perform lane following or detect traffic signs.

[0011] In the second degraded autonomous mode, the control device communicates sensor data (captured by sensors of the autonomous vehicle) to the supervisory server and, in response, receives high-level commands and a maximum travel speed from the supervisory server. The control device navigates the autonomous vehicle using adaptive cruise control based on the high-level commands and the maximum travel speed.

[0012] One difference between the second degraded autonomy mode and the first degraded autonomy mode is that the control device is less capable in its ability to navigate the autonomous vehicle due to the inability to perform lane following and detect traffic signs. Therefore, the control device may receive high-level commands more frequently in the second degraded autonomy mode than in the first degraded autonomy mode.

[0013] In the first degraded autonomy mode and the second degraded autonomy mode, due to the degraded operation of lane following and the low driving speed of the autonomous vehicle, up to a certain delay (e.g., up to a two-second delay, a three-second delay, etc.) in the communication between the control device and the supervisory server may be acceptable.

[0014] The control device may cause the autonomous vehicle to enter a third degraded autonomy mode under the following circumstances: 1) there is no (or very poor) network communication between the control device and the supervisory server; 2) the lane detection capability and position detection capability of the control device are at least partially operational; 3) the traffic sign detection capability of the control device is at least partially operational; and 4) adaptive cruise control is at least partially operational.

[0015] For example, because the autonomous vehicle is in an area where network coverage does not exist or is very poor (e.g., the network connection throughput speed is less than a threshold, such as 1 kilobytes per minute (kbpm), 2 kbpm), there may be no (or very poor) network communication between the control device and the supervisory server.

[0016] In a third degraded autonomous mode, the control device determines lane markings and traffic signs based on the sensor data. The control device uses adaptive cruise control to navigate the autonomous vehicle based on a predetermined maximum speed, lane markings, and traffic signs.

[0017] Thus, the disclosed system contemplates various degraded autonomy modes for various situations.

[0018] Thus, the disclosed system may be integrated into practical applications for improving navigation of autonomous vehicles and operation of autonomous vehicles.

[0019] Furthermore, the disclosed system may be integrated into additional practical applications that improve the driving experience of autonomous vehicles and other vehicles.

[0020] One potential solution in response to detecting a failure of a component of an autonomous vehicle is to abruptly stop the autonomous vehicle as quickly as possible if a severe failure is detected (e.g., loss of positioning, loss of a main computing unit, etc.), or to drive the autonomous vehicle off the road to a predetermined rescue area if the detected failure is less severe. However, this solution does not address the various situations described above and may cause potential accidents with other vehicles on the road. Therefore, by not abruptly stopping the autonomous vehicle at the first sign or indication of a failure, the autonomous vehicle may be navigated with or even without high-level commands from a supervisory server. Thus, the disclosed system may improve the driving experience of autonomous vehicles and other vehicles.

[0021] In one embodiment, a system includes an autonomous vehicle and a control device associated with the autonomous vehicle. The autonomous vehicle is configured to travel along a road, wherein the autonomous vehicle includes at least one sensor configured to capture sensor data. The control device includes a processor configured to detect an event trigger affecting the autonomous vehicle. In response to detecting the event trigger, the processor is also configured to cause the autonomous vehicle to enter a first degraded autonomous mode. In the first degraded autonomous mode, the processor is configured to communicate the sensor data to a supervisory server. The processor receives one or more high-level commands from the supervisory server, wherein the one or more high-level commands indicate a minimum risk maneuver for the autonomous vehicle. The processor receives a maximum travel speed for the autonomous vehicle from the supervisory server. The processor navigates the autonomous vehicle using adaptive cruise control based on the one or more high-level commands and the maximum travel speed.

[0022] Certain embodiments of the present disclosure may include some, all, or none of these advantages. These advantages and other features will be more clearly understood from the following detailed description taken in conjunction with the accompanying drawings and claims. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] For a more complete understanding of the present disclosure, reference is now made to the following brief description, taken in conjunction with the accompanying drawings and detailed description, wherein like reference numerals represent like parts.

[0024] Figure 1 An embodiment of a system for implementing various degraded autonomy modes for an autonomous vehicle is illustrated;

[0025] Figure 2 The diagram shows a method for implementing the first degraded autonomous mode. Figure 1 Example operation flow of the system;

[0026] Figure 3 The diagram shows a method for implementing the second degraded autonomous mode. Figure 1 Example operation flow of the system;

[0027] Figure 4 The diagram shows the method for implementing the third degraded autonomous mode. Figure 1 Example operation flow of the system;

[0028] Figure 5 An embodiment of a method for implementing a first degraded autonomy mode for an autonomous vehicle is illustrated;

[0029] Figure 6 An embodiment of a method for implementing a second degraded autonomy mode for an autonomous vehicle is illustrated;

[0030] Figure 7 illustrates an embodiment of a method for implementing a third degraded autonomy mode for an autonomous vehicle;

[0031] Figure 8 An embodiment of a method for implementing various degraded autonomy modes for an autonomous vehicle is illustrated;

[0032] Fig. 9 illustrates a block diagram of an example autonomous vehicle configured to perform autonomous driving operations;

[0033] Fig.10 The diagram shows the Fig. 9 An example system for autonomous driving operations for use with an autonomous vehicle; and

[0034] Fig.11 The diagram shows the Fig. 9 Block diagram of the onboard control computer in an autonomous vehicle. DETAILED DESCRIPTION

[0035] As described above, in the event that hardware failures and / or software failures affect the operation of an autonomous vehicle, the prior art fails to provide an effective, reliable, and safe solution to navigate an autonomous vehicle. The present disclosure provides various systems, methods, and devices to implement various degraded autonomous modes for an autonomous vehicle depending on the situation. Figures 1 to 11 The embodiments of the present disclosure and their advantages can be understood. Figures 1 to 11 Describes systems and methods for implementing various degraded autonomy modes for an autonomous vehicle depending on the situation.

[0036] System Overview

[0037] Figure 1 An embodiment of system 100 is illustrated that is configured to implement various degraded autonomy modes 140 a - 140 c to account for various hardware failures and / or software failures with autonomous vehicle 902 . Figure 1 Also illustrated is a simplified schematic diagram of a road 102 traveled by an autonomous vehicle 902, wherein the autonomous vehicle 902 may enter any of the degraded autonomous modes 140a-140c based on detected event triggers 142a-142c affecting the autonomous vehicle 902. In certain embodiments, the system 100 includes an autonomous vehicle 902 communicatively coupled to a supervisory server 160 and an application server 180 via a network 110. The network 110 enables communication between components of the system 100. The network 110 allows the autonomous vehicle 902 to communicate with other autonomous vehicles 902, systems, supervisory servers 160, application servers 180, databases, devices, etc. The autonomous vehicle 902 includes a control device 950. The control device 950 includes a processor 122 in signal communication with a memory 126. The memory 126 stores software instructions 128 that, when executed by the processor 122, cause the control device 950 to perform one or more operations described herein. The supervisory server 160 includes a processor 162 in signal communication with a memory 168. The memory 168 stores software instructions 170 that, when executed by the processor 162, cause the supervisory server 160 to perform one or more operations described herein. In other embodiments, the system 100 may not have all of the components listed and / or may have other elements instead of, or in addition to, those listed above. The system 100 may be configured as shown or in any other configuration.

[0038] In the example scenario, assume that autonomous vehicle 902 is traveling along road 102, and control device 950 detects event triggers 142 (e.g., one or more event triggers 142a-142c) that affect autonomous vehicle 902. Event triggers 142a-142c may include hardware failures and / or software failures with respect to autonomous vehicle 902. For example, a hardware module and / or software module of autonomous vehicle 902 may fail or degrade.

[0039] Faulty or degraded hardware and / or software modules of autonomous vehicle 902 may be associated with various functions of autonomous vehicle 902, such as, positioning of autonomous vehicle 902 (e.g., determining a geographic positioning system (GPS) location of autonomous vehicle 902 on map data 134), object detection (e.g., detecting objects and obstacles on road 102, such as traffic signs and lane markings), connectivity with supervisory server 160, etc.

[0040] Faulty or degraded hardware modules of autonomous vehicle 902 may include damaged sensors 946 (e.g., due to an impact), computing units (e.g., Fig. 9 ), and / or any other hardware module of autonomous vehicle 902 that is not fully operational. A faulty connector on autonomous vehicle 902 may interrupt the transmission of data and other information, thereby causing event triggers 142 (e.g., one or more event triggers 142a-event triggers 142c).

[0041] The faulty or degraded software modules of autonomous vehicle 902 may include software code associated with any component of autonomous vehicle 902 that may be damaged, for example, due to a software algorithm error or vulnerability in the code, due to a cyber-attack, or other code intrusion. For example, the faulty or degraded software modules may include software instructions 128, object detection machine algorithm module 132, positioning module 154, traffic sign detection module 156, and other software modules.

[0042] In response to detecting that the health level of at least one component of autonomous vehicle 902 has become less than a threshold percentage (e.g., less than 60%, 50%, etc.), control device 950 can determine a hardware failure and / or a software failure.

[0043] In response to detecting a hardware failure and / or a software failure, one potential solution is to abruptly stop autonomous vehicle 902 as quickly as possible if a severe failure is detected (e.g., loss of positioning, loss of a main computing unit, etc.), or to drive autonomous vehicle 902 off road 102 to a predetermined rescue area if the detected failure is less severe. In other words, the existing solution only addresses two extreme cases, where in one case autonomous vehicle 902 is forced to stop, and in another case autonomous vehicle 902 is pulled over. However, the solution does not address various scenarios between the two extreme cases and has a number of disadvantages.

[0044] For example, the omission in such a mechanism is such that if autonomous vehicle 902 is not in one of the predetermined less severe fault states, autonomous vehicle 902 is forced to stop on road 102. This approach may cause potential accidents with other vehicles on road 102. Especially on highways, it is not expected that autonomous vehicle 902 will stop on the road unless it is mechanically inoperable. In most cases where a degraded or failed hardware module and / or software module is detected, it is up to the driver to drive autonomous vehicle 902 into an emergency lane or the nearest rescue area. If there is no driver around autonomous vehicle 902 to manually operate autonomous vehicle 902, this may be impossible for autonomous vehicle 902.

[0045] Another potential solution is to stream sensor data 130 to supervisory server 160, display sensor data 130 on user interface 166 (e.g., a video feed of road 102 in front of autonomous vehicle 902), and allow remote operator 184 to remotely navigate autonomous vehicle 902. However, this potential solution is limited by the available network communication bandwidth between control device 950 and supervisory server 160, especially in certain areas where wireless network coverage is limited or even non-existent. This may result in significant delays in transmitting and streaming sensor data 130.

[0046] To provide a technical solution to these shortcomings, system 100 is configured to implement various degraded autonomy modes 140a - 140c for various scenarios and resolutions between the two extremes of autonomous vehicle 902 described above at stop and pull over.

[0047] In the first case, assume that the control device 950 is capable of: 1) communicating (e.g., streaming) sensor data 130 to the supervisory server 160 (even with a delay greater than a threshold, e.g., greater than half a second, one second, two seconds, etc.); 2) receiving data from the supervisory server 160 (e.g., maximum driving speed 172 and high-level commands 174); 3) detecting lanes and lane markings; 4) detecting traffic signs and traffic lights; and 5) navigating the autonomous vehicle 902 using adaptive cruise control 146 based on the data received from the supervisory server 160, the detected lane markings, traffic signs, and traffic lights. In this case, the control device 950 is configured to cause the autonomous vehicle 902 to enter the first degraded autonomous mode 140a. Figure 2 The operational flow 200 of the system 100 described in detail will now be described in greater detail with respect to the first degraded autonomous mode 140a.

[0048] In the second case, assume that control device 950 is capable of: 1) communicating (e.g., streaming) sensor data 130 to supervisory server 160 (even with a delay greater than a threshold, e.g., greater than half a second, one second, two seconds, etc.); 2) receiving data from supervisory server 160 (e.g., maximum driving speed 172 and high-level commands 174); and 3) navigating autonomous vehicle 902 using adaptive cruise control 146 based on the data received from supervisory server 160. In this case, control device 950 is configured to cause autonomous vehicle 902 to enter second degraded autonomous mode 140b. Figure 3 The operational flow 300 of the system 100 described in detail in the second degraded autonomous mode 140b is described.

[0049] In the third case, assume that control device 950 is capable of: 1) detecting lanes and lane markings; 2) detecting traffic signs and traffic lights; and 3) navigating autonomous vehicle 902 using adaptive cruise control 146 based on the detected lanes, lane markings, traffic signs, and traffic lights. In this case, control device 950 is configured to cause autonomous vehicle 902 to enter third degraded autonomous mode 140c. Figure 4 The operational flow 400 of the system 100 described in detail in the third degraded autonomous mode 140c is described.

[0050] System Components

[0051] The network 110 may include any interconnected system capable of transmitting audio, video, signals, data, messages, or any combination of the foregoing. The network 110 may include all or part of a local area network (LAN), a wide area network (WAN), a metropolitan area network (MAN), a personal area network (PAN), a wireless PAN (WPAN), an overlay network, a software defined network (SDN), a virtual private network (VPN), a packet data network (e.g., the Internet), a mobile phone network (e.g., a cellular network such as 4G or 5G), a plain old telephone (POT) network, a wireless data network (e.g., WiFi, WiGig, WiMax, etc.), a long term evolution (LTE) network, a universal mobile telecommunications system (UMTS) network, a peer-to-peer (P2P) network, a Bluetooth network, a near field communication (NFC) network, a Zigbee network, a Z-wave network, a WiFi network, and / or any other suitable network.

[0052] Example Autonomous Vehicle

[0053] In one embodiment, autonomous vehicle 902 may include a semi-truck tractor unit (see FIG. 1 ) that is attached to a trailer to transport cargo or goods from one location to another. Fig. 9 ). Autonomous vehicle 902 is typically configured to travel along a road in an autonomous mode. Autonomous vehicle 902 may use Figure 9-11 to navigate through the multiple components described in detail in Figure 9-11 The operation of autonomous vehicle 902 is described in more detail in . The corresponding description below includes a brief description of certain components of autonomous vehicle 902.

[0054] Control device 950 may generally be configured to control the operation of autonomous vehicle 902 and its components, and to facilitate autonomous driving of autonomous vehicle 902. Control device 950 may also be configured to determine a route ahead of autonomous vehicle 902, and to navigate autonomous vehicle 902 to travel in the route, which is safe for travel and free of objects or obstacles. Figure 9-11 Control device 950 may generally include one or more computing devices that communicate with other components of autonomous vehicle 902 (see Fig. 9 ). In the present disclosure, the control device 950 may be interchangeably referred to as the on-board control computer 950.

[0055] The control device 950 can be configured to detect objects on and around a road traveled by the autonomous vehicle 902 by analyzing the sensor data 130 and / or the map data 134. For example, the control device 950 can detect objects on and around a road by implementing an object detection machine learning module 132. The object detection machine learning module 132 can be implemented using a neural network and / or a machine learning algorithm for detecting objects based on images, videos, infrared images, point clouds, radar data, etc. The object detection machine learning module 132 is further described in more detail below. The control device 950 can receive sensor data 130 from a sensor 946 positioned on the autonomous vehicle 902 to determine a safe route for travel. The sensor data 130 may include data captured by the sensor 946.

[0056] Sensor 946 may be configured to capture any object within its detection area or field of view, such as landmarks, lane markings, lane boundaries, road boundaries, vehicles, pedestrians, road / traffic signs, etc. In some embodiments, sensor 946 may be configured to detect rain, fog, snow, and / or any other weather conditions. Sensor 946 may include a light detection and ranging (LiDAR) sensor, a radar sensor, a video camera, an infrared camera, an ultrasonic sensor system, a gust detection system, a microphone array, a thermocouple, a humidity sensor, a barometer, an inertial measurement unit, a positioning system, an infrared sensor, a motion sensor, a rain sensor, etc. In some embodiments, sensor 946 may be positioned around autonomous vehicle 902 to capture the environment around autonomous vehicle 902. For further description of sensor 946, see Fig. 9 The corresponding description.

[0057] Control Equipment

[0058] exist Fig. 9 126 and the like. The control device 950 is described in more detail in . In short, the control device 950 may include a processor 122 in signal communication with a memory 126 and a network interface 124. The processor 122 may include one or more processing units that perform the various functions described herein. The memory 126 may store any data and / or instructions used by the processor 122 to perform its functions. For example, the memory 126 may store software instructions 128 that, when executed by the processor 122, cause the control device 950 to perform one or more functions described herein.

[0059] The processor 122 may be Fig. 9126. The processor 122 includes one or more processors operably coupled to the memory 126. The processor 122 can be any electronic circuit, including a state machine, one or more central processing unit (CPU) chips, a logic unit, a core (e.g., a multi-core processor), a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), or a digital signal processor (DSP). The processor 122 can be a programmable logic device, a microcontroller, a microprocessor, or any suitable combination of the foregoing. The processor 122 can be communicatively coupled to the network interface 124 and the memory 126, and communicate with the network interface 124 and the memory 126 signals. One or more processors can be configured to process data and can be implemented in hardware or software. For example, the processor 122 can be 8-bit, 16-bit, 32-bit, 64-bit, or any other suitable architecture. The processor 122 may include an arithmetic logic unit (ALU) for performing arithmetic and logical operations, processor registers that supply operands to the ALU and store results of ALU operations, and a control unit that fetches instructions from memory and executes the instructions by directing coordinated operation of the ALU, registers, and other components. One or more processors may be configured to implement various instructions. For example, one or more processors may be configured to execute software instructions 128 to implement the functions disclosed herein, such as with respect to Figure 1-Figure 11 Some or all of those functions described. In some embodiments, a logic unit, FPGA, ASIC, DSP, or any other suitable hardware or electronic circuit is used to implement the functions described herein.

[0060] The network interface 124 may be Fig. 9 Components of the network communication subsystem 992 described in . The network interface 124 can be configured to enable wired communication and / or wireless communication. The network interface 124 can be configured to communicate data between the autonomous vehicle 902 and other devices, systems, or domains. For example, the network interface 124 may include an NFC interface, a Bluetooth interface, a Zigbee interface, a Z-wave interface, a radio frequency identification (RFID) interface, a WIFI interface, a local area network (LAN) interface, a wide area network (WAN) interface, a metropolitan area network (MAN) interface, a personal area network (PAN) interface, a wireless PAN (WPAN) interface, a modem, a switch, and / or a router. The processor 122 can be configured to use the network interface 124 to send and receive data. The network interface 124 can be configured to use any suitable type of communication protocol that can be understood by a person of ordinary skill in the art.

[0061] The memory 126 may be Fig. 9126 may include one of the data stores 990 described in . The memory 126 may be volatile or non-volatile and may include a read-only memory (ROM), a random access memory (RAM), a ternary content addressable memory (TCAM), a dynamic random access memory (DRAM), and a static random access memory (SRAM). The memory 126 may include one or more of a local database, a cloud database, a network attached storage (NAS), etc. The memory 126 may store Figure 1-Figure 11 , and any other data, instructions, logic, rules, or code that, when executed by the processor 122, is operable to implement the (multiple) functions described herein. For example, the memory 126 may store software instructions 128, sensor data 130, object detection machine learning module 132, map data 134, route plan 136, driving instructions 138, degraded autonomous mode 140a-degraded autonomous mode 140c, event trigger 142a-event trigger 142c, high-level command 174, adaptive cruise control 146, predetermined maximum speed 148, predetermined distance 150, predetermined time 152, positioning module 154, traffic sign detection module 156, and / or any other data / instructions. The software instructions 128 include code that, when executed by the processor 122, causes the control device 950 to perform the functions described herein, such as Figure 1-Figure 11 The memory 126 includes one or more disks, tape drives, or solid-state drives and can be used as an overflow data storage device to store programs when such programs are selected for execution and to store instructions and data that are read during program execution.

[0062] The object detection machine learning module 132 may be implemented by the processor 122 executing the software instructions 128, and may generally be configured to detect objects and obstacles based on the sensor data 130. The object detection machine learning module 132 may be implemented using a neural network and / or machine learning algorithm for detecting objects based on any data type, such as images, videos, infrared images, point clouds, radar data, etc.

[0063] In some embodiments, the object detection machine learning module 132 may be implemented using a machine learning algorithm such as a support vector machine (SVM), naive Bayes, logistic regression, k-nearest neighbor, decision tree, etc. In some embodiments, the object detection machine learning module 132 may utilize multiple neural network layers, convolutional neural network layers, long short-term memory (LSTM) layers, bidirectional LSTM layers, recursive neural network layers, etc., wherein the weights and biases of these layers are optimized during the training process of the object detection machine learning module 132. The object detection machine learning module 132 may be trained by a training data set, which may include samples of data types labeled with one or more objects in each sample. For example, the training data set may include sample images of objects (e.g., vehicles, lane markings, pedestrians, road signs, obstacles, etc.) labeled with (multiple) objects in each sample image. Similarly, the training data set may include samples of other data types such as videos, infrared images, point clouds, radar data, etc., labeled with (multiple) objects in each sample data. The object detection machine learning module 132 may be trained, tested, and refined by the training data set and the sensor data 130. The object detection machine learning module 132 uses the sensor data 130 (which is not labeled with objects) to increase its prediction accuracy when detecting objects. For example, supervised machine learning algorithms and / or unsupervised machine learning algorithms can be used to verify the predictions of the object detection machine learning module 132 when detecting objects in the sensor data 130.

[0064] Map data 134 may include a virtual map of a city or region including roads traveled by autonomous vehicle 902. In some examples, map data 134 may include map 1058 and map database 1036 (see FIG. 1058 for a description of map 1058 and map database 1036). Fig.10 ). The map data 134 may include drivable areas, such as roads, paths, highways, and non-drivable areas, such as terrain (determined by the occupancy grid module 1060, for a description of the occupancy grid module 1060, see Fig.10 ). The map data 134 may specify the location coordinates of road signs, lanes, lane markings, lane boundaries, road boundaries, traffic lights, obstacles, and the like.

[0065] Route plan 136 may be a plan for traveling from a starting location (e.g., a first autonomous vehicle launch pad / landing pad) to a destination (e.g., a second autonomous vehicle launch pad / landing pad). For example, route plan 136 may specify a combination of one or more streets, roads, and highways in a specific order from the starting location to the destination. Route plan 136 may specify stages, including a first stage (e.g., moving out from the starting location / launch pad), multiple intermediate stages (e.g., traveling along specific lanes of one or more specific streets / roads / highways), and a final stage (e.g., entering the destination / landing pad). Route plan 136 may include other information about the route from the starting location to the destination, such as road / traffic signs in route plan 136, etc.

[0066] The driving instructions 138 may be implemented by the planning module 1062 (see Fig.10 1062 in route plan 136). Driving instructions 138 may include instructions and rules for adapting the autonomous driving of autonomous vehicle 902 according to the driving rules for each stage of route plan 136. For example, driving instructions 138 may include instructions to maintain within a speed range for the road traveled by autonomous vehicle 902, to adapt the speed of autonomous vehicle 902 relative to changes observed by sensors 946 (such as the speed of surrounding vehicles, objects within the detection area of ​​sensors 946, etc.).

[0067] Adaptive cruise control 146 may be implemented by processor 122 executing software instructions 128, and is generally configured to navigate autonomous vehicle 902 according to given data / instructions, such as predetermined maximum travel speed 148, maximum travel speed 172, and high-level commands 174. Control device 950 may use adaptive cruise control 146 to maintain a safe distance (e.g., six feet, seven feet, or any other suitable distance) from other objects and vehicles, and to keep autonomous vehicle 902 in the current lane in which it is traveling. Figure 2-Figure 5 Example navigation of autonomous vehicle 902 using adaptive cruise control 146 is described in FIG.

[0068] The positioning module 154 may correspond to the fusion positioning module 1026 (see Fig.10 1026 in the description of the fusion positioning module 1026). Positioning module 154 can be implemented by processor 122 executing software instructions 128, and is generally configured to determine the position of autonomous vehicle 902 on road 102 and / or on map data 134. Therefore, positioning module 154 can provide position detection capabilities. Control device 950 can use positioning module 154 for lane following, for example, to maintain the current lane. Positioning module 154 can use GPS sensor 946g (see Fig. 9) to determine the location of the autonomous vehicle 902.

[0069] The traffic sign detection module 156 may be implemented by the processor 122 executing software instructions 128 and is generally configured to detect road signs, traffic signs, traffic lights, etc. In certain embodiments, the traffic sign detection module 156 may be implemented using a neural network and / or a machine learning algorithm configured to detect road signs, traffic signs, traffic lights, etc. In some embodiments, the traffic sign detection module 156 may be implemented using a machine learning algorithm such as a support vector machine (SVM), naive Bayes, logistic regression, k-nearest neighbor, decision tree, etc. In some embodiments, the traffic sign detection module 156 may utilize multiple neural network layers, convolutional neural network layers, long short-term memory (LSTM) layers, bidirectional LSTM layers, recurrent neural network layers, etc., wherein the weights and biases of these layers are optimized during the training process of the traffic sign detection module 156. The traffic sign detection module 156 may be trained by a training data set that includes multiple images of road signs, traffic signs, and traffic lights, each image being labeled with sampled data. Similarly, the training data set may include samples of other data types (such as video, infrared images, point clouds, radar data, etc.), with road signs, traffic signs, or traffic lights labeled in each sample data. The traffic sign detection module 156 may be trained, tested, and refined by the training data set and the sensor data 130. The traffic sign detection module 156 uses the sensor data 130 (which is not labeled with road signs, traffic signs, and traffic lights) to increase its prediction accuracy when detecting objects. For example, supervised machine learning algorithms and / or unsupervised machine learning algorithms may be used to verify the predictions of the traffic sign detection module 156 when detecting road signs, traffic signs, and traffic lights in the sensor data 130.

[0070] Supervision Server

[0071] Supervisory server 160 may include one or more processing devices and is generally configured to supervise the operation of autonomous vehicle 902 when it is en route and to supervise the travel of autonomous vehicle 902. Supervisory server 160 may include processor 162, network interface 164, user interface 166, and memory 168. The components of supervisory server 160 are operably coupled to each other. Processor 162 may include one or more processing units that perform various functions of supervisory server 160. Memory 168 may be used to store any data and / or instructions used by processor 162 to perform its functions. For example, memory 168 may store software instructions 170 that, when executed by processor 162, cause supervisory server 160 to perform one or more functions described herein. Supervisory server 160 may be configured as shown or in any other suitable configuration.

[0072] In one embodiment, supervisory server 160 may be implemented by a cluster of computing devices that may be used to supervise the operation of autonomous vehicle 902. For example, supervisory server 160 may be implemented by multiple computing devices using a distributed computing and / or cloud computing system. In another example, supervisory server 160 may be implemented by multiple computing devices in one or more data centers. Thus, in one embodiment, supervisory server 160 may include more processing power than control device 950. Supervisory server 160 communicates signals with autonomous vehicle 902 and its components (e.g., control device 950).

[0073] Processor 162 includes one or more processors. Processor 162 can be any electronic circuit, including a state machine, one or more CPU chips, logic units, cores (e.g., multi-core processors), FPGAs, ASICs, or DSPs. Processor 162 can be a programmable logic device, a microcontroller, a microprocessor, or any suitable combination of the foregoing. Processor 162 can be communicatively coupled to network interface 164, user interface 166, and memory 168, and communicate with network interface 164, user interface 166, and memory 168 signals. One or more processors are configured to process data and can be implemented in hardware or software. For example, processor 162 can be 8-bit, 16-bit, 32-bit, 64-bit, or any other suitable architecture. Processor 162 can include: an ALU for performing arithmetic and logical operations, a processor register for supplying operands to the ALU and storing the results of the ALU operation, and a control unit, which extracts instructions from the memory and executes the instructions by guiding the coordinated operation of the ALU, registers, and other components. One or more processors are configured to implement various instructions. For example, one or more processors are configured to execute software instructions 170 to implement the functions disclosed herein, such as with respect to Figure 1-Figure 11Some or all of those functions described.In some embodiments, a logic unit, FPGA, ASIC, DSP, or any other suitable hardware or electronic circuit can be used to implement the functions described herein.

[0074] The network interface 164 may be configured to enable wired and / or wireless communications of the supervisory server 160. The network interface 164 may be configured to communicate data between the supervisory server 160 and other devices, servers, autonomous vehicles 902, systems, or domains. For example, the network interface 164 may include an NFC interface, a Bluetooth interface, a Zigbee interface, a Z-wave interface, an RFID interface, a WIFI interface, a LAN interface, a WAN interface, a PAN interface, a modem, a switch, and / or a router. The processor 162 may be configured to send and receive data using the network interface 164. The network interface 164 may be configured to use any suitable type of communication protocol as would be appreciated by one of ordinary skill in the art.

[0075] The user interface 166 may include one or more user interfaces configured to interact with a user, such as a remote operator 184. The remote operator 184 may access the supervisory server 160 via the communication path 186. In some embodiments, the user interface 166 may include peripheral devices of the supervisory server 160, such as a monitor, keyboard, mouse, track pad, touch pad, microphone, web camera, speaker, etc. In some embodiments, the user interface 166 may include a graphical user interface, a software application, or a web application. The remote operator 184 may use the user interface 166 to access the memory 168 to examine any data stored in the memory 168. The remote operator 184 may confirm, update, and / or overwrite the route plan 136 and / or any other data stored in the memory 168.

[0076] Memory 168 may be volatile or non-volatile and may include ROM, RAM, TCAM, DRAM, and SRAM. Memory 168 may include one or more of a local database, a cloud database, NAS, etc. Memory 168 may store Figure 1-Figure 11 , and any other data, instructions, logic, rules, or code that, when executed by processor 162, is operable to implement the functionality described herein. For example, memory 168 may store software instructions 170, sensor data 130, object detection machine learning module 132, map data 134, route plan 136, maximum driving speed 172, high-level commands 174, and / or any other data / instructions. Software instructions 170 may include code that, when executed by processor 162, causes supervisory server 160 to perform the functionality described herein, such as Figure 1-Figure 11 The memory 168 includes one or more magnetic disks, tape drives, or solid-state drives and can be used as an overflow data storage device to store programs when such programs are selected for execution and to store instructions and data that are read during program execution.

[0077] Application Server

[0078] The application server 180 may be any computing device configured to communicate with other devices (such as the supervisory server 160, the autonomous vehicle 902, a database, etc.) via the network 110. The application server 180 may be configured to perform the functions described herein and interact with the remote operator 184 (e.g., using its user interface via the communication path 182). Examples of the application server 180 include, but are not limited to, a desktop computer, a laptop computer, a server, etc. In one example, the application server 180 may act as a presentation layer from which the remote operator 184 may access the supervisory server 160. In this way, the supervisory server 160 may send the route plan 136, the sensor data 130, and / or any other data / instructions to the application server 180 (e.g., via the network 110). For example, after establishing the communication path 182 with the application server 180, the remote operator 184 may review the received data and confirm, update, and / or overwrite any route plan 136.

[0079] Remote operator 184 may be an individual associated with supervisory server 160 and may have access to supervisory server 160. For example, remote operator 184 may be an administrator who may access and view information about autonomous vehicle 902, such as sensor data 130, driving instructions 138, route plan 136, and other information available on memory 168. In one example, remote operator 184 may access supervisory server 160 via network 110 from application server 180, which acts as a presentation layer.

[0080] Operational flow for implementing the first degraded autonomous mode

[0081] Figure 2 The diagram shows a method for implementing the first degraded autonomous mode 140a. Figure 1 An example operational flow 200 of the system 100 is shown. Figure 2 Also illustrated is road 102 being traveled by autonomous vehicle 902 , wherein autonomous vehicle 902 enters first degraded autonomy mode 140a .

[0082] In the first degraded autonomous mode 140a, it is assumed that: 1) the wireless communication between the control device 950 and the supervisory server 160 is at least partially operable (even with a delay greater than a threshold, for example, greater than half a second, one second, two seconds, etc.); 2) the positioning module 154 of the control device 950 (which provides lane detection capability and position detection capability) is at least partially operable; 3) the traffic sign detection module 156 of the control device 950 (which provides traffic sign detection capability) is at least partially operable; and 4) the adaptive cruise controller 146 is at least partially operable.

[0083] In the example scenario, assume that autonomous vehicle 902 is traveling along road 102. Control device 950 may detect event trigger 142a that affects autonomous vehicle 902.

[0084] In some embodiments, event trigger 142a may include one or more of a hardware failure and a software failure with respect to autonomous vehicle 902, similar to Figure 1 Described in .

[0085] In some embodiments, event trigger 142a may include one or more of hardware degradation and software degradation of autonomous vehicle 902, similar to Figure 1 Described in .

[0086] In certain embodiments, event trigger 142a may include a degradation in a hardware module of autonomous vehicle 902 (e.g., due to a malfunction of the hardware module) such that the hardware and the module are partially operational.

[0087] In certain embodiments, event trigger 142a may include a degradation in a software module of autonomous vehicle 902 (e.g., due to a vulnerability in the software module and / or the software module being corrupted) such that the software module is partially operational.

[0088] In some embodiments, event trigger 142a may include a degradation affecting a hardware component of autonomous vehicle 902, such as sensor 946 or vehicle subsystem 940 (see Fig. 9 ) is destroyed by the impact.

[0089] In certain embodiments, event trigger 142a may include a degradation affecting a software component of autonomous vehicle 902 (such as software instructions 128, object detection machine learning module 132, localization module 154, and / or traffic sign detection module 156), for example, due to an outdated software module.

[0090] In certain embodiments, event trigger 142a may include a degradation affecting network interface 124. The degradation affecting network interface 124 may be due to autonomous vehicle 902 being in an area where there is limited network coverage, may be a hardware degradation with respect to network interface 124, and / or may be a software degradation with respect to network interface 124.

[0091] Putting the autonomous vehicle into a first degraded autonomy mode

[0092] In response to detecting the event trigger 142a described above, the control device 950 causes the autonomous vehicle 902 to enter the first degraded autonomous mode 140a. In the first degraded autonomous mode 140a, the control device 950 may perform one or more of the following operations in parallel or in any suitable order.

[0093] Control device 950 may communicate sensor data 130 to supervisory server 160. Sensor data 130 may include data indicating objects on and around road 102, such as lane markings, traffic signs, traffic lights, other vehicles, and / or any other objects. For example, sensor data 130 may include data captured by one or more cameras associated with autonomous vehicle 902 (e.g., Fig. 9 In other examples, the sensor data 130 may include any other data captured by other sensors 946, such as image feeds, point cloud data feeds, etc.

[0094] Supervisory server 160 receives sensor data 130. Supervisory server 160 may display sensor data 130 on user interface 166 (see Figure 1 ). The remote operator 184 can access the supervisory server 160 directly or through the application server 180 (see Figure 1 ) to view sensor data 130, similar to Figure 1 Described in .

[0095] The remote operator 184 can provide the user interface 166 (see Figure 1) provides input, where the input includes one or more high-level commands 174 and maximum travel speed 172 for autonomous vehicle 902. One or more high-level commands 174 may indicate a minimal risk maneuver for autonomous vehicle 902, such as slowing autonomous vehicle 902. One or more high-level commands 174 may include one or more of the following instructions: 1) maintain a specific amount of time in the current lane (e.g., five minutes, six minutes, etc.); 2) change to a specific lane when traffic on the specific lane permits; 3) change to an emergency lane when traffic permits; 4) drive to a drivable safe area off main road 102; 5) select a specific exit; 6) pull over at a specific location on a specific side of road 102; and 7) drive until a specific distance and stop at a specific location.

[0096] The supervisory server 160 may receive a user interface 166 (see Figure 1 The supervisory server 160 may communicate one or more high-level commands 174 and a maximum travel speed 172 to the control device 950.

[0097] The control device 950 may receive one or more high-level commands 174 and a maximum travel speed 172 from the supervisory server 160 .

[0098] The control device 950 may implement the object detection machine learning module 132, the localization module 154, and the traffic sign detection module 156. In this process, the control device 950 may feed the sensor data 130 to the object detection machine learning module 132 to detect objects 210 (and obstacles), such as other vehicles, on the road 102. The control device 950 may feed the sensor data 130 to the localization module 154 to detect lane markings 212 on at least one or both sides of the autonomous vehicle 902 based on the sensor data 130. In this way, the control device 950 may keep the autonomous vehicle 902 within the current lane. The control device 950 may feed the sensor data 130 to the traffic sign detection module 156 to detect traffic signs 214 (and traffic lights) on the road 102 in front of the autonomous vehicle 902 based on the sensor data 130.

[0099] Control device 950 may implement adaptive cruise control 146 to navigate autonomous vehicle 902. Using adaptive cruise control 146 and the modules described above, control device 950 ensures that autonomous vehicle 902 maintains a predetermined safe distance between itself and other vehicles and objects on road 102, does not collide with other vehicles and objects, and does not veer outside of the lane in which it is currently traveling by detecting lane markings on one or both sides of autonomous vehicle 902.

[0100] Control device 950 may also ensure navigation of autonomous vehicle 902 according to traffic regulations on road 102 based on detecting and analyzing traffic signs and traffic lights (if any) on road 102 via traffic sign detection module 156 , unless overridden by remote operator 184 .

[0101] Control device 950 may use adaptive cruise control 146 to navigate autonomous vehicle 902 based on one or more high-level commands 174, maximum travel speed 172, sensor data 130, lane markings, traffic signs, and traffic lights. Maximum travel speed 172 may be equal to the posted speed limit of the road or highway on which autonomous vehicle 902 is traveling. Alternatively or additionally, maximum travel speed 172 may be based on the location of autonomous vehicle 902, environmental factors, and the nature and type of degradation of triggering event 142; a table or database of factors and appropriate maximum travel speeds may be part of autonomous vehicle 902 and may be stored in memory 126 of control device 950. Environmental factors may include visibility (e.g., reduced visibility due to fog, dust storms, etc.), weather (e.g., precipitation, extreme temperatures, gusty winds, etc.), and road conditions (e.g., icy roads, loose gravel, sheet metal, slippery or flooded roads, etc.).

[0102] During this operation, control device 950 may periodically (e.g., every two minutes, every three minutes, or any other suitable time interval) receive high-level commands 174 from supervisory server 160 and navigate autonomous vehicle 902 based on the received data.

[0103] After some time or distance depending on the teleoperator 184 , the teleoperator 184 may issue a final high-level command 174 to pull over to the side of the road 102 , change to a particular lane, or continue driving forward until reaching a particular safe area to pull over.

[0104] In certain embodiments, in first degraded autonomous mode 140a, some communication lag (such as a delay in data communication having less than a threshold delay, e.g., less than 20 seconds, 15 seconds, etc.) between control device 950 and supervisory server 160 may be acceptable. The delay may be due to one or more sensors 946 being at least partially operational. In first degraded autonomous mode 140a, control device 950 may use operational sensor(s) 946 in conjunction with adaptive cruise controller 146 to assist in navigation and lane keeping of autonomous vehicle 902 (e.g., to keep autonomous vehicle 902 in its lane) while autonomous vehicle 902 is experiencing such a delay.

[0105] Operational procedures for implementing the second degraded autonomous mode

[0106] Figure 3 The method for implementing the second degraded autonomous mode 140b is shown. Figure 1 An example operational flow 300 of the system 100 is shown. Figure 3 Also illustrated is road 102 being traveled by autonomous vehicle 902 , wherein autonomous vehicle 902 enters second degraded autonomy mode 140 b .

[0107] In the second degraded autonomous mode 140b, it is assumed that: 1) wireless communication between the control device 950 and the supervisory server 160 is at least partially operational (even if with a delay greater than a threshold, e.g., greater than half a second, one second, two seconds, etc.); 2) the adaptive cruise controller 146 is at least partially operational; and 3) the control device 950 is incapable of lane following or detecting traffic signs.

[0108] Due to a degradation or failure affecting the positioning module 154 (eg, a hardware degradation or failure and / or a software degradation or failure), the control device 950 may be unable to perform lane following.

[0109] The control device 950 may be unable to detect a traffic sign due to degradation or failure affecting the traffic sign detection module 156 , such as hardware and / or software degradation or failure.

[0110] One difference between the second degraded autonomy mode 140b and the first degraded autonomy mode 140a is that the control device 950 is less capable in its ability to navigate the autonomous vehicle 902 due to the inability to perform lane following and detect traffic signs. Therefore, in the second degraded autonomy mode 140b, the control device 950 relies more on its receipt of high-level commands 174 from the supervisory server 160 than in the first degraded autonomy mode 140a. Thus, in some embodiments, the control device 950 receives high-level commands 174 more frequently than in the first degraded autonomy mode 140a.

[0111] In the example scenario, assume that autonomous vehicle 902 is traveling along road 102. Control device 950 may detect event trigger 142b that affects autonomous vehicle 902.

[0112] In some embodiments, event trigger 142b may include one or more of a hardware failure and a software failure with respect to autonomous vehicle 902, similar to Figure 1 Described in .

[0113] In some embodiments, event trigger 142b may result in a loss of positioning capability with respect to autonomous vehicle 902. The loss of positioning capability may result in control device 950 being unable to determine the geographic location coordinates (e.g., GPS location coordinates) of autonomous vehicle 902. In other words, the loss of positioning capability may result in control device 950 being unable to determine whether autonomous vehicle 902 is located, for example, on road 102 and / or on map data 134 (see FIG. Figure 1 The loss of positioning capability may be responsive to positioning module 154 (see Figure 1 ) in the event of a failure or degradation.

[0114] In certain embodiments, event trigger 142 b may result in a loss of traffic sign detection capability with respect to autonomous vehicle 902. The loss of traffic sign detection capability may result in the inability of control device 950 to detect, for example, traffic signs and / or traffic lights on road 102. The loss of traffic sign detection capability may be in response to a failure or degradation in traffic sign detection module 156.

[0115] In certain embodiments, event trigger 142b may include a degradation in a hardware module of autonomous vehicle 902 (e.g., due to a malfunction of the hardware module) such that the hardware and modules are partially operational.

[0116] In certain embodiments, event trigger 142b event trigger 142a may include a degradation in a software module of autonomous vehicle 902 (eg, due to a vulnerability in the software module and / or the software module being corrupted) such that the software module is partially operational.

[0117] In some embodiments, event trigger 142b may include a degradation affecting a hardware component of autonomous vehicle 902, such as sensor 946 or vehicle subsystem 940 (see Fig. 9 ) is destroyed by the impact.

[0118] In certain embodiments, event trigger 142b may include a degradation affecting a software component of autonomous vehicle 902 (such as software instructions 128, object detection machine learning module 132), for example, due to an outdated software module.

[0119] In certain embodiments, event trigger 142 b may include a degradation affecting network interface 124 . The degradation affecting network interface 124 may be due to autonomous vehicle 902 being in an area where there is limited network coverage, may be a hardware degradation with respect to network interface 124 , and / or may be a software degradation with respect to network interface 124 .

[0120] Putting the autonomous vehicle into a second degraded autonomy mode

[0121] In response to detecting event trigger 142b, control device 950 causes autonomous vehicle 902 to enter second degraded autonomous mode 140b. In second degraded autonomous mode 140b, control device 950 may perform one or more of the following operations in parallel or in any suitable order.

[0122] The control device 950 may communicate the sensor data 130 to the supervisory server 160. The sensor data 130 may include data indicating objects on and around the road 102, such as lane markings, traffic signs, traffic lights, other vehicles, and / or any other objects, similar to Figure 2 For example, sensor data 130 may include data from one or more cameras associated with autonomous vehicle 902 (e.g., Fig. 9 In other examples, the sensor data 130 may include any other data captured by other sensors 946, such as image feeds, point cloud data feeds, etc.

[0123] Supervisory server 160 receives sensor data 130. Supervisory server 160 may display sensor data 130 on user interface 166 (see Figure 1 ). The remote operator 184 can access the supervisory server 160 directly or through the application server 180 (see Figure 1 ) to view sensor data 130, similar to Figure 1 Described in .

[0124] The remote operator 184 can provide the user interface 166 (see Figure 1 ) provides input, where the input includes one or more high-level commands 174 for autonomous vehicle 902 and maximum travel speed 172. One or more high-level commands 174 may indicate a minimal risk maneuver for autonomous vehicle 902, such as slowing autonomous vehicle 902. Figure 2 Examples of high-level commands 174 are described in .

[0125] The supervisory server 160 may receive a user interface 166 (see Figure 1 The supervisory server 160 may communicate one or more high-level commands 174 and a maximum travel speed 172 to the control device 950. The control device 950 may receive one or more high-level commands 174 and a maximum travel speed 172 from the supervisory server 160.

[0126] Control device 950 may implement adaptive cruise control 146 to navigate autonomous vehicle 902. Using adaptive cruise control 146, control device 950 ensures that autonomous vehicle 902 maintains a predetermined safe distance from other vehicles and objects on road 102, does not hit other vehicles and objects, and drives in a specific safe zone on road 102 (unless overridden by a command from remote operator 184, which may be provided in the case of harmless debris on road 102). Control device 950 may use adaptive cruise control 146 to navigate autonomous vehicle 902 according to one or more high-level commands 174 and maximum travel speed 172.

[0127] As described above, in the second degraded autonomous mode 140b, the control device 950 may receive high-level commands 174 more frequently than in the first degraded autonomous mode 140a, such as every thirty seconds, every minute, or any other suitable time interval.

[0128] After some time or distance depending on the teleoperator 184 , the teleoperator 184 may issue a final high-level command 174 to pull over to the side of the road 102 , change to a particular lane, or continue driving forward until reaching a particular safe area to pull over.

[0129] In some embodiments, some communication lag (such as latency in data communications with less than a threshold latency, e.g., less than 20 seconds, 15 seconds, etc.) between control device 950 and supervisory server 160 may be acceptable in second degraded autonomy mode 140b. However, because control device 950 is not capable of lane following or detecting traffic signs, remote operator 184 may need to be extra careful to handle the latency in communications and provide high-level commands 174 so that control device 950 does not inadvertently steer autonomous vehicle 902 outside of its intended path.

[0130] In some embodiments, control device 950 may cause autonomous vehicle 902 to degrade from first autonomous mode 140a (see Figure 2 ) enters the second degraded autonomous mode 140b. For example, when autonomous vehicle 902 is in the first degraded autonomous mode 140a (see Figure 2 ), if control device 950 determines that it is no longer capable of lane following and detecting traffic signs, it will cause autonomous vehicle 902 to enter a second degraded autonomous mode 140b.

[0131] In some embodiments, control device 950 may cause autonomous vehicle 902 to enter first degraded autonomous mode 140a (see Figure 2). For example, when autonomous vehicle 902 is in second degraded autonomous mode 140b, if control device 950 detects that at least one of the positioning capability and the traffic sign detection capability of control device 950 is partially restored, it may cause autonomous vehicle 902 to enter first degraded autonomous mode 140a. For example, if control device 950 determines that a software update package for a corresponding degraded or faulty software module is received (e.g., from supervisory server 160) and installs the software update package, control device 950 may determine that at least one of the positioning capability and the traffic sign detection capability of control device 950 is partially restored.

[0132] In this case, the control device 950 may perform one or more additional operations, similar to Figure 2 In the process, the control device 950 may access the sensor data 130, which includes data representing at least one of lane markings and traffic signs on the road 102. The control device 950 may send the positioning module 154 (see Figure 1 ) to feed sensor data 130 to detect lane markings on at least one side or both sides of autonomous vehicle 902 based on sensor data 130. Similarly, control device 950 may feed traffic sign detection module 156 (see Figure 1 ) feeds sensor data 130 to detect traffic signs (and traffic lights) on the road 102 in front of the autonomous vehicle 902 based on the sensor data 130.

[0133] Control device 950 may use detected lane markings and traffic signs in navigating autonomous vehicle 902 via adaptive cruise control 146 based on high-level commands 174 and maximum driving speed 172 .

[0134] Operational procedures for implementing the third degraded autonomous mode

[0135] Figure 4 The diagram shows a method for implementing the third degraded autonomous mode 140c. Figure 1 An example operational flow 400 of the system 100 is shown. Figure 4 Also illustrated is road 102 being traveled by autonomous vehicle 902 , wherein autonomous vehicle 902 enters third degraded autonomy mode 140 c .

[0136] In the third degraded autonomous mode 140c, it is assumed that: 1) there is no (or very poor) network communication between the control device 950 and the supervisory server 160; 2) the positioning module 154 of the control device 950 (which provides lane detection capability and position detection capability) is at least partially operational; 3) the traffic sign detection module 156 of the control device 950 (which provides traffic sign detection capability) is at least partially operational; and 4) the adaptive cruise controller 146 is at least partially operational.

[0137] For example, because the autonomous vehicle 902 is in an area where network coverage does not exist or is very poor (e.g., the network connection throughput speed is less than a threshold, such as 1 kilobytes per minute (kbpm), 2 kbpm), there may be no (or very poor) network communication between the control device 950 and the supervisory server 160.

[0138] In another example, due to hardware modules and / or software modules associated with a network connection at the control device 950 (such as the network interface 124 (see Figure 1 ) and / or network communication subsystem 992 (see Fig. 9 )), there may be no (or very poor) network communication between the control device 950 and the supervisory server 160.

[0139] In another example, there may be no (or very poor) network communication between the control device 950 and the supervisory server 160 due to a malfunction in a hardware module and / or software module associated with network connectivity at the supervisory server 160 , such as the network interface 164 .

[0140] In the example scenario, assume that autonomous vehicle 902 is traveling along road 102. Control device 950 may detect event trigger 142b that affects autonomous vehicle 902.

[0141] In some embodiments, event trigger 142b may include one or more of a hardware failure and a software failure with respect to autonomous vehicle 902, similar to Figure 1 Described in .

[0142] In certain embodiments, the event trigger 142b may result in a degradation in connectivity (or even a loss of connectivity) with the supervisory server 160 such that the control device 950 and the supervisory server 160 are unable to communicate with each other.

[0143] In certain embodiments, the event trigger 142 b may be a loss of connectivity between the control device 950 and the supervisory server 160 .

[0144] Because there is no network communication between control device 950 and supervisory server 160, control device 950 does not receive high-level commands 174 or maximum driving speed 172 from supervisory server 160. Therefore, control device 950 can rely solely on sensor data 130 and adaptive cruise control 146 to navigate autonomous vehicle 902.

[0145] Putting the autonomous vehicle into a third degraded autonomy mode

[0146] In response to detecting event trigger 142c, control device 950 causes autonomous vehicle 902 to enter third degraded autonomous mode 140c. In third degraded autonomous mode 140c, control device 950 may perform one or more of the following operations in parallel or in any suitable order.

[0147] The control device 950 can access the sensor data 130, which includes data representing objects 210, lane markings 212, and traffic signs 214 on and around the road 102. The control device 950 can implement the object detection machine learning module 132, the localization module 154, and the traffic sign detection module 156, similar to Figure 2 As described in , to detect object 210 , lane marking 212 , and traffic sign 214 .

[0148] Control device 950 may implement adaptive cruise control 146 to navigate autonomous vehicle 902. Using adaptive cruise control 146, control device 950 may navigate autonomous vehicle 902 so that autonomous vehicle 902 maintains within the current lane according to the detected lane markings and does not veer outside the current lane. In addition, control device 950 may ensure that autonomous vehicle 902 maintains a predetermined safe distance from other vehicles and objects on road 102 and does not collide with other vehicles and objects.

[0149] Furthermore, control device 950 may ensure that autonomous vehicle 902 is navigated according to traffic regulations on road 102 based on detecting and analyzing traffic signs and traffic lights (if any) on road 102 .

[0150] In addition, control device 950 may use predetermined maximum travel speed 148 in navigation of autonomous vehicle 902. Predetermined maximum travel speed 148 may be different from maximum travel speed 172 received by control device 950 when autonomous vehicle 902 enters first degraded autonomous mode 140a or second degraded autonomous mode 140b. For example, predetermined maximum travel speed 148 may be at the bottom end of a speed range prescribed for road 102 according to speed signs on road 102 and traffic regulations.

[0151] Determining whether connectivity to the supervisory server is at least partially restored

[0152] In some embodiments, while navigating autonomous vehicle 902, control device 950 may instruct autonomous vehicle 902 to travel a predetermined distance 150, such as one mile, two miles, or any other suitable distance. While autonomous vehicle 902 is traveling predetermined distance 150, control device 950 determines whether connectivity with supervisory server 160 is at least partially restored. For example, control device 950 may periodically (e.g., every minute, every two minutes, etc.) communicate confirmation request 410 to supervisory server 160.

[0153] If control device 950 receives a response from supervisory server 160 before a certain time (e.g., before one minute, thirty seconds, etc.), it determines that connectivity with supervisory server 160 is at least partially restored. Otherwise, control device 950 determines that connectivity with supervisory server 160 has not yet been restored. If autonomous vehicle 902 moves to an area where there is better network coverage, connectivity with supervisory server 160 may be restored.

[0154] In some embodiments, at the end of traveling the predetermined distance 150, if the control device 950 determines that connectivity with the supervisory server 160 has not been at least partially restored, the control device 950 may instruct the autonomous vehicle 902 to stop in a safe area (e.g., an obstacle-free area, an emergency lane, etc.).

[0155] In some embodiments, at the end of traveling predetermined distance 150, if control device 950 determines that connectivity with supervisory server 160 has not been at least partially restored, control device 950 may instruct autonomous vehicle 902 to pull over to a particular location on the side of road 102.

[0156] During this process, control device 950 may determine whether it is safe to pull over autonomous vehicle 902. If traffic permits, control device 950 may determine that it is safe to pull over autonomous vehicle 902. If control device 950 determines that it is safe to pull over autonomous vehicle 902, it may instruct autonomous vehicle 902 to pull over to a specific location on one side of road 102. Otherwise, control device 950 may instruct autonomous vehicle 902 to stop in a safe area (e.g., an obstacle-free area, an emergency lane, etc.).

[0157] At the end of the travel predetermined distance 150, if the control device 950 determines that the connectivity with the supervisory server 160 is at least partially restored, the control device 950 may receive one or more high-level commands 174 and a maximum travel speed 172 from the supervisory server 160. In other words, the control device 950 may cause the autonomous vehicle 902 to enter Figure 2 The first degraded autonomous mode 140a described in.

[0158] Thus, control device 950 may use adaptive cruise control 146 to navigate autonomous vehicle 902 based on one or more high-level commands 174, maximum travel speed 172, lane markings, and traffic signs, similar to Figure 2 Described in .

[0159] In some embodiments, while navigating autonomous vehicle 902, control device 950 may instruct autonomous vehicle 902 to travel until predetermined time 152, e.g., 5 minutes, 10 minutes, or any other suitable time period. While autonomous vehicle 902 is traveling until predetermined time 152, control device 950 determines whether connectivity with supervisory server 160 is at least partially restored (e.g., by sending confirmation request 410 to supervisory server 160), similar to that described in the above scenario.

[0160] In some embodiments, while driving until the end of the predetermined time 152, if the control device 950 determines that connectivity with the supervisory server 160 has not been at least partially restored, the control device 950 may instruct the autonomous vehicle 902 to stop in a safe area (e.g., an obstacle-free area, an emergency lane, etc.) (if it is determined that it is not possible to pull over the autonomous vehicle 902), similar to what is described in the above case.

[0161] In some embodiments, upon driving until the end of the predetermined time 152, if the control device 950 determines that connectivity with the supervisory server 160 has not been at least partially restored, the control device 950 may instruct the autonomous vehicle 902 to pull over to a specific location on the side of the road 102 (if it is determined that the autonomous vehicle 902 can be pulled over), similar to what is described in the above scenario.

[0162] After driving until the end of the predetermined distance 150, if the control device 950 determines that the connectivity with the supervisory server 160 is at least partially restored, the control device 950 can receive one or more high-level commands 174 and the maximum driving speed 172 from the supervisory server 160. In other words, the control device 950 can cause the autonomous vehicle 902 to enter Figure 2 Thus, similar to the first degraded autonomous mode 140a described in Figure 2 As described in , control device 950 can use adaptive cruise control 146 to navigate autonomous vehicle 902 based on one or more high-level commands 174, maximum travel speed 172, lane markings, and traffic signs.

[0163] Placing an autonomous vehicle from a degraded autonomy mode into another degraded autonomy mode

[0164] In some embodiments, when autonomous vehicle 902 is in third degraded autonomous mode 140c, control device 950 may cause autonomous vehicle 902 to enter first degraded autonomous mode 140a or second degraded autonomous mode 140b, depending on the situation and whether the positioning capability and the traffic sign detection capability are at least partially operational, similar to Figure 1-Figure 3 Described in .

[0165] For example, if the positioning capability and traffic sign detection capability of the control device 950 are not operational, the control device 950 may cause the autonomous vehicle 902 to enter the second degraded autonomous mode 140b. In another example, if the positioning capability and traffic sign detection capability of the control device 950 are at least partially operational, the control device 950 may cause the autonomous vehicle 902 to enter the first degraded autonomous mode 140a. Thus, the control device 950 may cause the autonomous vehicle 902 to enter another mode from any one of the degraded modes 140a-140c, depending on the situation and the needs of the event triggers 142a-142c.

[0166] Although the present disclosure describes three degraded autonomy modes 140a-140c, other degraded autonomy modes 140 may be implemented in view of the present disclosure. For example, in certain embodiments, in response to a degradation or failure in any hardware module and / or software module of autonomous vehicle 902, a specific degraded autonomy mode 140 may be implemented to navigate autonomous vehicle 902 without forcing autonomous vehicle 902 to a sudden stop.

[0167] Example method for implementing a first degraded autonomous mode

[0168] Figure 5 An example flow chart of a method 500 for implementing a first degraded autonomous mode 140a is illustrated. Modifications, additions, or omissions may be made to the method 500. The method 500 may include more operations, fewer operations, or other operations. For example, the operations may be performed in parallel or in any suitable order. Although sometimes discussed as the system 100, the autonomous vehicle 902, the control device 950, the supervisory server 160, or any component thereof performing operations, any suitable system or component of a system may perform one or more operations of the method 500. For example, one or more operations of the method 500 may be performed at least in part in a non-transitory, tangible, machine-readable medium (e.g., a separate computer program product from a computer program product). Figure 1 and Fig. 9 126, memory 168, and data storage 990), respectively, from Figure 1 and Fig. 9The software instructions 128, software instructions 170, and processing instructions 980 are implemented in the form of software instructions 128, software instructions 170, and processing instructions 980, which are executed by one or more processors (e.g., Figure 1 and Fig. 9 When the processor 122, the processor 162, and the processor 970) are executed, one or more processors may be caused to perform operations 502-512.

[0169] At operation 502 , the control device 950 determines whether an event trigger 142 a is detected. Figure 2 An example of event trigger 142a is described in , which may cause control device 950 to cause autonomous vehicle 902 to enter first degraded autonomous mode 140a. If control device 950 determines that event trigger 142a is detected, method 500 proceeds to operation 504. Otherwise, method 500 stays at operation 502.

[0170] At operation 504, control device 950 determines that event trigger 142a causes autonomous vehicle 902 to enter first degraded autonomous mode 140a. For example, in this case, event trigger 142a may be Figure 2 One or more events described in trigger 142a.

[0171] At operation 506, control device 950 communicates sensor data 130 to supervisory server 160. Sensor data 130 may include data representing objects and obstacles on road 102 traveled by autonomous vehicle 902, similar to Figure 1 and Figure 2 Described in .

[0172] At operation 508 , the control device 950 receives the high-level command 174 and the maximum travel speed 172 from the supervisory server 160 . Figure 2 Examples of high-level commands 174 are described in .

[0173] At operation 510, the control device 950 determines lane markings and traffic signs based on the sensor data 130. In this process, the control device 950 may implement the object detection machine learning module 132, the positioning module 154, and the traffic sign detection module 156, similar to Figure 1 and Figure 2 Described in .

[0174] At operation 512, control device 950 uses adaptive cruise control 146 to navigate autonomous vehicle 902 based on high-level commands 174, maximum travel speed 172, lane markings, and traffic signs, similar to Figure 2 Described in .

[0175] Example method for implementing a second degraded autonomous mode

[0176] Figure 6 An example flow chart of a method 600 for implementing the second degraded autonomous mode 140b is illustrated. Modifications, additions, or omissions may be made to the method 600. The method 600 may include more operations, fewer operations, or other operations. For example, the operations may be performed in parallel or in any suitable order. Although sometimes discussed as the system 100, the autonomous vehicle 902, the control device 950, the supervisory server 160, or any component thereof performing operations, any suitable system or component of a system may perform one or more operations of the method 600. For example, one or more operations of the method 600 may be performed at least in part in a non-transitory, tangible, machine-readable medium (e.g., a separate device from the system 100). Figure 1 and Fig. 9 126, memory 168, and data storage 990), respectively, from Figure 1 and Fig. 9 The software instructions 128, software instructions 170, and processing instructions 980 are implemented in the form of software instructions 128, software instructions 170, and processing instructions 980, which are executed by one or more processors (e.g., Figure 1 and Figure 8 When the processor 122, the processor 162, and the processor 970) are executed, one or more processors may be caused to perform operations 602-610.

[0177] At operation 602 , the control device 950 determines whether an event trigger 142 b is detected. Figure 3 An example of event trigger 142b is described in , which may cause control device 950 to cause autonomous vehicle 902 to enter second degraded autonomous mode 140b. If control device 950 determines that event trigger 142b is detected, method 600 proceeds to operation 604. Otherwise, method 600 stays at operation 602.

[0178] At operation 604, control device 950 determines that event trigger 142b causes autonomous vehicle 902 to enter second degraded autonomous mode 140b. For example, in this case, event trigger 142b may be Figure 3 One or more events described in trigger 142b.

[0179] At operation 606, control device 950 communicates sensor data 130 to supervisory server 160. Sensor data 130 may include data representing objects and obstacles on road 102 traveled by autonomous vehicle 902, similar to Figure 1-Figure 3 Described in .

[0180] At operation 608 , the control device 950 receives the high-level command 174 and the maximum travel speed 172 from the supervisory server 160 . Figure 2 and Figure 3 Examples of high-level commands 174 are described in .

[0181] At operation 610, control device 950 uses adaptive cruise control 146 to navigate autonomous vehicle 902 based on high-level commands 174 and maximum travel speed 172, similar to Figure 3 Described in .

[0182] Example method for implementing a third degraded autonomous mode

[0183] Figure 7 An example flow chart of a method 700 for implementing the third degraded autonomous mode 140c is illustrated. Modifications, additions, or omissions may be made to the method 700. The method 700 may include more operations, fewer operations, or other operations. For example, the operations may be performed in parallel or in any suitable order. Although sometimes discussed as the system 100, the autonomous vehicle 902, the control device 950, the supervisory server 160, or any component thereof performing operations, any suitable system or component of a system may perform one or more operations of the method 700. For example, one or more operations of the method 700 may be performed at least in part in a non-transitory, tangible, machine-readable medium (e.g., a separate computer program product from a computer program product). Figure 1 and Fig. 9 126, memory 168, and data storage 990), respectively, from Figure 1 and Fig. 9 The software instructions 128, software instructions 170, and processing instructions 980 are implemented in the form of software instructions 128, software instructions 170, and processing instructions 980, which are executed by one or more processors (e.g., Figure 1 and Fig. 9 When the processor 122, the processor 162, and the processor 970) are executed, one or more processors may be caused to perform operations 702-708.

[0184] At operation 702 , the control device 950 determines whether an event trigger 142 c is detected. Figure 3 An example of event trigger 142c is described in , which may cause control device 950 to cause autonomous vehicle 902 to enter third degraded autonomous mode 140c. If control device 950 determines that event trigger 142c is detected, method 700 proceeds to operation 704. Otherwise, method 700 stays at operation 702.

[0185] At operation 704, control device 950 determines that event trigger 142c causes autonomous vehicle 902 to enter third degraded autonomous mode 140c. For example, in this case, event trigger 142c may be Figure 4 One or more events described in trigger 142c.

[0186] At operation 706, the control device 950 determines lane markings and traffic signs based on the sensor data 130. In this process, the control device 950 may implement the object detection machine learning module 132, the positioning module 154, and the traffic sign detection module 156, similar to Figure 1 and Figure 4 Described in .

[0187] At operation 708, control device 950 uses adaptive cruise control 146 to navigate autonomous vehicle 902 according to predetermined maximum travel speed 172, lane markings, and traffic signs, similar to Figure 4 Described in .

[0188] Example methods for implementing various degraded autonomy modes

[0189] Figure 8 An example flow chart of a method 800 for implementing various degraded autonomous modes 140a-degraded autonomous modes 140c is illustrated. Modifications, additions, or omissions may be made to method 800. Method 800 may include more operations, fewer operations, or other operations. For example, the operations may be performed in parallel or in any suitable order. Although sometimes discussed as system 100, autonomous vehicle 902, control device 950, supervisory server 160, or any component thereof performing operations, any suitable system or component of a system may perform one or more operations of method 800. For example, one or more operations of method 800 may be performed at least in part in a manner stored in a non-transitory, tangible, machine-readable medium (e.g., a respective one from Figure 1 and Fig. 9 126, memory 168, and data storage 990), respectively, from Figure 1 and Fig. 9 The software instructions 128, software instructions 170, and processing instructions 980 are implemented in the form of software instructions 128, software instructions 170, and processing instructions 980, which are executed by one or more processors (e.g., Figure 1 and Fig. 9 When the processor 122, the processor 162, and the processor 970) are executed, one or more processors may be caused to perform operations 802-826.

[0190] At operation 802 , the control device 950 determines whether an event trigger 142 is detected. Figure 1-Figure 4, which may cause the control device 950 to cause the autonomous vehicle 902 to enter various degraded autonomous modes 140a-140c. If the control device 950 determines that the event trigger 142 is detected, the method 800 proceeds to operation 804. Otherwise, the method 800 stays at operation 802.

[0191] At operation 804, control device 950 determines whether event trigger 142 causes autonomous vehicle 902 to enter first degraded autonomous mode 140a. For example, control device 950 may determine whether detected event trigger 142 is Figure 2 If the control device 950 determines that the event trigger 142a causes the autonomous vehicle 902 to enter the first degraded autonomous mode 140a, the method 800 proceeds to operation 806. Otherwise, the method 800 proceeds to operation 814.

[0192] At operation 806, control device 950 communicates sensor data 130 to supervisory server 160. Sensor data 130 may include data representing objects and obstacles on road 102 traveled by autonomous vehicle 902, similar to Figure 1 and Figure 2 Described in .

[0193] At operation 808 , the control device 950 receives the high-level command 174 and the maximum travel speed 172 from the supervisory server 160 . Figure 2 Examples of high-level commands 174 are described in .

[0194] At operation 810, the control device 950 determines lane markings and traffic signs based on the sensor data 130. In this process, the control device 950 may implement the object detection machine learning module 132, the positioning module 154, and the traffic sign detection module 156, similar to Figure 1 and Figure 2 Described in .

[0195] At operation 812, control device 950 uses adaptive cruise control 146 to navigate autonomous vehicle 902 based on high-level commands 174, maximum travel speed 172, lane markings, and traffic signs, similar to Figure 2 Described in .

[0196] At operation 814, control device 950 determines whether event trigger 142 causes autonomous vehicle 902 to enter second degraded autonomous mode 140b. For example, control device 950 may determine whether detected event trigger 142 is Figure 3If the control device 950 determines that the event trigger 142a causes the autonomous vehicle 902 to enter the second degraded autonomous mode 140b, the method 800 proceeds to operation 816. Otherwise, the method 800 proceeds to operation 822.

[0197] At operation 816, control device 950 communicates sensor data 130 to supervisory server 160. Sensor data 130 may include data representing objects and obstacles on road 102 traveled by autonomous vehicle 902, similar to Figure 1-Figure 3 Described in .

[0198] At operation 818, the control device 950 receives the high-level command 174 and the maximum travel speed 172 from the supervisory server 160. Figure 2 and Figure 3 Examples of high-level commands 174 are described in .

[0199] At operation 820, control device 950 uses adaptive cruise control 146 to navigate autonomous vehicle 902 based on high-level commands 174 and maximum travel speed 172, similar to Figure 3 Described in .

[0200] At operation 822, control device 950 determines whether event trigger 142 causes autonomous vehicle 902 to enter third degraded autonomous mode 140c. For example, control device 950 may determine whether detected event trigger 142 is Figure 4 If control device 950 determines that event trigger 142c causes autonomous vehicle 902 to enter third degraded autonomous mode 140c, method 800 proceeds to operation 824. Otherwise, method 800 ends.

[0201] At operation 706, the control device 950 determines lane markings and traffic signs based on the sensor data 130. In this process, the control device 950 may implement the object detection machine learning module 132, the positioning module 154, and the traffic sign detection module 156, similar to Figure 1 and Figure 4 Described in .

[0202] At operation 708, control device 950 uses adaptive cruise control 146 to navigate autonomous vehicle 902 according to predetermined maximum travel speed 172, lane markings, and traffic signs, similar to Figure 4 Described in .

[0203] Example autonomous vehicle and its operation

[0204] Fig. 9A block diagram of an example vehicle ecosystem 900 in which autonomous driving operations may be determined is shown. Fig. 9 As shown, the autonomous vehicle 902 may be a semi-trailer truck. The vehicle ecosystem 900 may include multiple systems and components that may generate one or more sources of information / data and related services and / or deliver one or more sources of information / data and related services to an onboard control computer 950, which may be located in the autonomous vehicle 902. The onboard control computer 950 may be in data communication with multiple vehicle subsystems 940, all of which may reside in the autonomous vehicle 902. A vehicle subsystem interface 960 may be provided to facilitate data communication between the onboard control computer 950 and the multiple vehicle subsystems 940. In some embodiments, the vehicle subsystem interface 960 may include a controller area network (CAN) controller to communicate with devices in the vehicle subsystem 940.

[0205] Autonomous vehicle 902 may include various vehicle subsystems that support the operation of autonomous vehicle 902. Vehicle subsystems 940 may include vehicle driving subsystem 942, vehicle sensor subsystem 944, vehicle control subsystem 948, and / or network communication subsystems 992-992. Fig. 9 The components or devices shown of vehicle driving subsystem 942, vehicle sensor subsystem 944, and vehicle control subsystem 948 are examples. Autonomous vehicle 902 may be configured as shown or may be any other configuration.

[0206] Vehicle driving subsystem 942 may include components operable to provide powered motion to autonomous vehicle 902. In an exemplary embodiment, vehicle driving subsystem 942 may include engine / motor 942a, wheels / tires 942b, transmission 942c, electrical subsystem 942d, and power source 942e.

[0207] The vehicle sensor subsystem 944 may include a plurality of sensors 946 configured to sense information about the environment or condition of the autonomous vehicle 902. The vehicle sensor subsystem 944 may include one or more cameras 946a or image capture devices, a radar unit 946b, one or more temperature sensors 946c, a wireless communication unit 946d (e.g., a cellular communication transceiver), an inertial measurement unit (IMU) 946e, a laser rangefinder / LiDAR unit 946f, a global positioning system (GPS) transceiver 946g, and a wiper control system 946h. The vehicle sensor subsystem 944 may also include sensors configured to monitor the internal systems of the autonomous vehicle 902 (e.g., an O2 monitor, a fuel gauge, an engine oil temperature, etc.).

[0208] The IMU 946e may include any combination of sensors (e.g., accelerometers and gyroscopes) configured to sense changes in the position and orientation of the autonomous vehicle 902 based on inertial acceleration. The GPS transceiver 946g may be any sensor configured to estimate the geographic location of the autonomous vehicle 902. For this purpose, the GPS transceiver 946g may include a receiver / transmitter operable to provide information about the position of the autonomous vehicle 902 relative to the earth. The radar unit 946b may represent a system that uses radio signals to sense objects within the local environment of the autonomous vehicle 902. In some embodiments, in addition to sensing objects, the radar unit 946b may be additionally configured to sense the speed and direction of objects approaching the autonomous vehicle 902. The laser rangefinder or LiDAR unit 946f may be any sensor configured to use lasers to sense objects in the environment where the autonomous vehicle 902 is located. The camera 946a may include one or more devices configured to capture multiple images of the environment of the autonomous vehicle 902. The camera 946a may be a still image camera or a motion video camera.

[0209] The vehicle control subsystem 948 can be configured to control the operation of the autonomous vehicle 902 and its components. Therefore, the vehicle control subsystem 948 may include various elements, such as a throttle and gear selector 948a, a brake unit 948b, a navigation unit 948c, a steering system 948d, and / or an autonomous control unit 948e. The throttle and gear selector 948a may be configured to control, for example, the running speed of the engine and thereby control the speed of the autonomous vehicle 902. The throttle and gear selector 948a may be configured to control the gear selection of the transmission. The brake unit 948b may include any combination of mechanisms configured to decelerate the autonomous vehicle 902. The brake unit 948b may slow down the autonomous vehicle 902 in a standard manner, including slowing down the wheels by using friction or by engine braking. The brake unit 948b may include an anti-lock braking system (ABS), which may prevent the brakes from locking when the brakes are applied. Navigation unit 948c may be any system configured to determine a driving path or route for autonomous vehicle 902. Navigation unit 948c may additionally be configured to dynamically update the driving path as autonomous vehicle 902 operates. In some embodiments, navigation unit 948c may be configured to incorporate data from GPS transceiver 946g and one or more predetermined maps to determine a driving path for autonomous vehicle 902. Steering system 948d may represent any combination of mechanisms operable to adjust the direction of autonomous vehicle 902 in either an autonomous mode or a driver-controlled mode.

[0210] Autonomous control unit 948e may represent a control system configured to identify, evaluate, and avoid or otherwise negotiate potential obstacles or obstructions in the environment of autonomous vehicle 902. In general, autonomous control unit 948e may be configured to control autonomous vehicle 902 for operation without a driver, or to provide driver assistance in controlling autonomous vehicle 902. In some embodiments, autonomous control unit 948e may be configured to incorporate data from GPS transceiver 946g, radar unit 946b, LiDAR unit 946f, camera 946a, and / or other vehicle subsystems to determine a driving path or trajectory for autonomous vehicle 902.

[0211] The network communication subsystem 992 may include a network interface, such as a router, a switch, a modem, etc. The network communication subsystem 992 may be configured to establish communications between the autonomous vehicle 902 and other systems, servers, etc. The network communication subsystem 992 may also be configured to receive data from other systems and send data to other systems.

[0212] Many or all of the functions of autonomous vehicle 902 may be controlled by onboard control computer 950. Onboard control computer 950 may include at least one data processor 970 (which may include at least one microprocessor) that executes processing instructions 980 stored in a non-transitory computer-readable medium (such as data storage device 990 or memory). Onboard control computer 950 may also represent a plurality of computing devices that may be used to control separate components or subsystems of autonomous vehicle 902 in a distributed manner. In some embodiments, data storage device 990 may contain processing instructions 980 (e.g., program logic) that may be executed by data processor 970 to perform various methods and / or functions of autonomous vehicle 902, including with respect to Figure 1-Figure 11 Those described.

[0213] The data storage device 990 may also contain additional instructions, including instructions to transmit data to one or more of the vehicle driving subsystem 942, the vehicle sensor subsystem 944, and the vehicle control subsystem 948; instructions to receive data from one or more of the vehicle driving subsystem 942, the vehicle sensor subsystem 944, and the vehicle control subsystem 948; instructions to interact with one or more of the vehicle driving subsystem 942, the vehicle sensor subsystem 944, and the vehicle control subsystem 948; or instructions to control one or more of the vehicle driving subsystem 942, the vehicle sensor subsystem 944, and the vehicle control subsystem 948. The onboard control computer 950 may be configured to include a data processor 970 and a data storage device 990. The onboard control computer 950 may control functions of the autonomous vehicle 902 based on input received from various vehicle subsystems (e.g., the vehicle driving subsystem 942, the vehicle sensor subsystem 944, and the vehicle control subsystem 948).

[0214] Fig.10 An exemplary system 1000 for providing precise autonomous driving operation is shown. The system 1000 may include multiple modules that may be operated in the onboard control computer 950, such as Fig. 9 The vehicle control computer 950 may include Fig.10 , wherein the sensor fusion module 1002 can perform at least four image or signal processing operations. The sensor fusion module 1002 can obtain images from a camera located on the autonomous vehicle to perform image segmentation 1004 to detect the presence of moving objects (e.g., other vehicles, pedestrians, etc.) and / or static obstacles (e.g., stop signs, speed bumps, terrain, etc.) located around the autonomous vehicle. The sensor fusion module 1002 can obtain LiDAR point cloud data items from a LiDAR sensor located on the autonomous vehicle to perform LiDAR segmentation 1006 to detect the presence of objects and / or obstacles located around the autonomous vehicle.

[0215] The sensor fusion module 1002 can perform instance segmentation 1008 on the image and / or point cloud data items to identify contours (e.g., boxes) around objects and / or obstacles located around the autonomous vehicle. The sensor fusion module 1002 can perform temporal fusion 1010, in which objects and / or obstacles from one frame of one image and / or point cloud data item are correlated or associated with objects and / or obstacles from one or more images or frames received subsequently in time.

[0216] The sensor fusion module 1002 may fuse objects and / or obstacles based on images obtained from the camera and / or point cloud data items obtained from the LiDAR sensor. For example, the sensor fusion module 1002 may determine based on the positions of two cameras: an image from one of the cameras including half of the vehicle located in front of the autonomous vehicle, the same as the vehicle captured by the other camera. The sensor fusion module 1002 may send the fused object information to the intervention module 1046, and send the fused obstacle information to the occupancy grid module 1060. The onboard control computer may include an occupancy grid module 1060, which may retrieve landmarks from a map database 1058 stored in the onboard control computer. The occupancy grid module 1060 may determine the drivable area and / or obstacles based on the fused obstacles obtained from the sensor fusion module 1002 and the landmarks stored in the map database 1058. For example, the occupancy grid module 1060 may determine that the drivable area may include a speed bump obstacle.

[0217] Below the sensor fusion module 1002, the onboard control computer 950 may include a LiDAR-based object detection module 1012, which may perform object detection 1016 based on a point cloud data item obtained from a LiDAR sensor 1014 located on the autonomous vehicle. The object detection 1016 technique may provide the location of the object from the point cloud data item (e.g., in 3D world coordinates). Below the LiDAR-based object detection module 1012, the onboard control computer may include an image-based object detection module 1018, which may perform object detection 1024 based on an image obtained from a camera 1020 located on the autonomous vehicle. The object detection 1018 technique may employ deep machine learning techniques 1024 to provide the location of the object from the image provided by the camera 1020 (e.g., in 3D world coordinates).

[0218] Radar 1056 on the autonomous vehicle can scan the area in front of the autonomous vehicle, or the area towards which the autonomous vehicle is driving. The radar data can be sent to sensor fusion module 1002, which can use the radar data to associate objects and / or obstacles detected by radar 1056 with detected objects and / or obstacles from both LiDAR point cloud data items and camera images. The radar data can also be sent to intervention module 1046, which can perform data processing on the radar data to track objects through object tracking module 1048, as further described below.

[0219] The onboard control computer may include an intervention module 1046 that receives the position of the object from the point cloud and the object from the image, as well as the fused object from the sensor fusion module 1002. The intervention module 1046 also receives radar data, with which the intervention module 1046 can track the object from one point cloud data item and one image obtained at one time instance to another (or next) point cloud data item and another image obtained at another subsequent time instance via the object tracking module 1048.

[0220] The intervention module 1046 may perform object attribute estimation 1050 to estimate one or more attributes of an object detected in an image or point cloud data item. The one or more attributes of the object may include the type of the object (e.g., a pedestrian, a car, or a truck, etc.). The intervention module 1046 may perform behavior prediction 1052 to estimate or predict a motion pattern of an object detected in an image and / or point cloud. The behavior prediction 1052 may be performed to detect the position of an object in a set of images (e.g., sequential images) received at different points in time, or in a set of point cloud data items (e.g., sequential point cloud data items) received at different points in time. In some embodiments, the behavior prediction 1052 may be performed for each image received from a camera and / or each point cloud data item received from a LiDAR sensor. In some embodiments, the intervention module 1046 can be executed (e.g., run or executed) to reduce the computational load by performing behavior prediction 1052 every other, or after every predetermined number of images received from the camera or point cloud data items received from the LiDAR sensor (e.g., after every two images, or after every three point cloud data items).

[0221] The behavior prediction 1052 feature can determine the speed and direction of objects around the autonomous vehicle based on radar data, where the speed information and direction information can be used to predict or determine the object's motion pattern. The motion pattern may include predicted trajectory information of the object over a predetermined length of time in the future after receiving an image from the camera. Based on the predicted motion pattern, the intervention module 1046 can assign a motion pattern situation label to the object (e.g., "located at coordinates (x, y)", "stopped", "driving at 50 mph", "accelerating", or "decelerating"). The situation label can describe the object's motion pattern. The intervention module 1046 can send one or more object attributes (e.g., the type of object) and the motion pattern situation label to the planning module 1062. The intervention module 1046 can perform environmental analysis 1054 using any information obtained by the system 1000, and any number and combination of components of the system 1000.

[0222] The onboard control computer may include a planning module 1062 that receives object attributes and motion pattern situation labels, drivable areas and / or obstacles from the intervention module 1046, and vehicle position and posture information from the fused positioning module 1026 (described further below).

[0223] The planning module 1062 may perform navigation planning 1064 to determine a set of trajectories on which the autonomous vehicle may drive. The set of trajectories may be determined based on drivable area information, one or more object attributes of an object, a motion mode situation label of an object, the location of obstacles, and drivable area information. In some embodiments, the navigation planning 1064 may include determining an area next to a road where the autonomous vehicle can be safely parked in an emergency. The planning module 1062 may include behavioral decision making 1066 to determine driving actions (e.g., steering, braking, throttle) in response to determining a change in conditions on the road (e.g., a traffic light turns yellow, or the autonomous vehicle is in an unsafe driving condition because another vehicle is driving in front of the autonomous vehicle and in an area within a predetermined safe distance of the location of the autonomous vehicle). The planning module 1062 performs trajectory generation 1068 and selects a trajectory from the set of trajectories determined by the navigation planning operation 1064. The selected trajectory information may be sent by the planning module 1062 to the control module 1070.

[0224] The onboard control computer may include a control module 1070 that receives the proposed trajectory from the planning module 1062 and receives the autonomous vehicle position and attitude from the fused positioning module 1026. The control module 1070 may include a system identifier 1072. The control module 1070 may perform model-based trajectory refinement 1074 to refine the proposed trajectory. For example, the control module 1070 may apply filtering (e.g., Kalman filtering) to smooth the proposed trajectory data and / or minimize noise. The control module 1070 may perform robust control 1076 by determining the amount of brake pressure to be applied, the steering angle, the throttle amount to control the vehicle speed, and / or the transmission gear based on the refined proposed trajectory information and the current position and / or attitude of the autonomous vehicle. The control module 1070 may send the determined brake pressure, steering angle, throttle amount, and / or transmission gear to one or more devices in the autonomous vehicle to control and facilitate the precise driving operation of the autonomous vehicle.

[0225] The depth image-based object detection 1024 performed by the image-based object detection module 1018 can also be used to detect landmarks on the road (e.g., stop signs, speed bumps, etc.). The onboard control computer can include a fused localization module 1026 that obtains landmarks detected from the images, landmarks obtained from a map database 1036 stored on the onboard control computer, landmarks detected from point cloud data items by the LiDAR-based object detection module 1012, speed and displacement from the odometer sensor 1044, and an estimated position of the autonomous vehicle from GPS / IMU sensors 1038 (i.e., GPS sensor 1040 and IMU sensor 1042) located on or in the autonomous vehicle. Based on this information, the fused localization module 1026 can perform a localization operation 1028 to determine the position of the autonomous vehicle, which can be sent to the planning module 1062 and the control module 1070.

[0226] The fused positioning module 1026 can estimate the pose 1030 of the autonomous vehicle based on the GPS and / or IMU sensors 1038. The pose of the autonomous vehicle can be sent to the planning module 1062 and the control module 1070. The fused positioning module 1026 can also estimate the state of the trailer unit (e.g., position, possible movement angle) based on (e.g., trailer state estimation 1034), for example, information provided by the IMU sensor 1042 (e.g., angular rate and / or linear velocity). The fused positioning module 1026 can also check the map content 1032.

[0227] Fig.11 1 shows an exemplary block diagram of an onboard control computer 950 included in an autonomous vehicle 902. The onboard control computer 950 may include at least one processor 1102 and instructions stored thereon (e.g., Figure 1 and Fig. 9 The instructions, when executed by the processor 1102, configure the vehicle control computer 950 and / or various modules of the vehicle control computer 950 to perform Figure 1-Figure 11 The operations described in . Transmitter 1106 can transmit or transfer information or data to one or more devices in the autonomous vehicle. For example, transmitter 1106 can send instructions to one or more motors of a steering wheel to turn the autonomous vehicle. Receiver 1108 receives information or data transmitted or sent by one or more devices. For example, receiver 1108 receives the current speed status from an odometer sensor, or receives the current transmission gear status from a transmission. Transmitter 1106 and receiver 1108 can also be configured as described above. Fig. 9 and Fig.10The multiple vehicle subsystems 940 described in the figure communicate with the on-board control computer 950.

[0228] Although multiple embodiments are provided in the present disclosure, it should be understood that the disclosed systems and methods may be embodied in many other specific forms without departing from the spirit or scope of the present disclosure. The examples presented should be considered illustrative rather than restrictive, and are not intended to be limited to the details given herein. For example, various elements or components may be combined or integrated into another system, or certain features may be omitted or not implemented.

[0229] In addition, without departing from the scope of the present disclosure, the techniques, systems, subsystems, and methods described and illustrated as discrete or separately in various embodiments may be combined or integrated with other systems, modules, techniques, or methods. Other items shown or discussed as coupled to each other, or directly coupled or communicating, may be indirectly coupled or communicated through some interface, device, or intermediate component (whether electrical, mechanical, or other). Changes, substitutions, and alternative other examples are determinable by those skilled in the art and may be made without departing from the spirit and scope disclosed herein.

[0230] To assist the Patent Office, and any reader of any patent issuing on this application, in interpreting the claims appended hereto, Applicant notes that unless the words "means for" or "step for" are expressly used in a particular claim, Applicant does not intend to invoke 35 U.S.C. §112(f) as any of the appended claims existed on their filing date.

[0231] Implementations of the present disclosure may be described in terms of the following clauses, the features of which may be combined in any reasonable way.

[0232] Clause 1. A system comprising:

[0233] an autonomous vehicle configured to travel along a road, wherein the autonomous vehicle includes at least one sensor configured to capture sensor data;

[0234] A control device associated with an autonomous vehicle and comprising a first processor, the first processor being configured to:

[0235] Detecting event triggers that affect the autonomous vehicle;

[0236] In response to detecting the event trigger, causing the autonomous vehicle to enter a first degraded autonomous mode, wherein in the first degraded autonomous mode, the first processor is configured to:

[0237] communicating sensor data to a supervisory server;

[0238] receiving one or more high-level commands from a supervisory server, wherein the one or more high-level commands indicate a minimal risk maneuver for the autonomous vehicle;

[0239] receiving a maximum driving speed for the autonomous vehicle from a supervisory server; and

[0240] The autonomous vehicle is navigated using adaptive cruise control based on one or more high-level commands and a maximum travel speed.

[0241] Clause 2. The system of clause 1, wherein the event trigger comprises one or more of a hardware failure and a software failure with respect to the autonomous vehicle.

[0242] Clause 3. The system of clause 1, wherein:

[0243] The event triggers a loss of positioning capability with respect to the autonomous vehicle; and

[0244] The loss of positioning capability results in the control device being unable to determine the geographic location coordinates of the autonomous vehicle.

[0245] Clause 4. The system of clause 3, wherein:

[0246] The event triggering also results in a loss of traffic sign detection capability with respect to the autonomous vehicle; and

[0247] The loss of traffic sign detection capability results in the control device being unable to detect traffic signs and traffic lights.

[0248] Clause 5. The system of clause 1, wherein the first processor is further configured to communicate a message to a supervisory server indicating that the autonomous vehicle has entered the first degraded autonomy mode.

[0249] Clause 6. The system of clause 1, further comprising a supervisory server communicatively coupled to the control device and comprising a second processor configured to:

[0250] receiving sensor data from a control device;

[0251] Display sensor data on the user interface;

[0252] receiving input from a remote operator, the input including a maximum travel speed and one or more high-level commands;

[0253] communicating a maximum travel speed to a control device; and

[0254] Communicates one or more high-level commands to a controlling device.

[0255] Clause 7. The system of clause 4, wherein the first processor is further configured to:

[0256] At least one of the detection positioning capability and the traffic sign detection capability is partially restored;

[0257] In response to detecting that at least one of the positioning capability and the traffic sign detection capability is partially restored, causing the autonomous vehicle to enter a second degraded autonomy mode, wherein in the second degraded autonomy mode, in addition to operations in the first degraded autonomy mode, the first processor is further configured to:

[0258] accessing sensor data, the sensor data comprising data representing one or more of lane markings and traffic signs;

[0259] detecting lane markings on at least one side of the autonomous vehicle based on the sensor data;

[0260] detecting traffic signs on the road ahead of the autonomous vehicle based on sensor data; and

[0261] Adaptive cruise control is used to use detected lane markings and traffic signs in navigation of the autonomous vehicle based on one or more high-level commands and a maximum travel speed.

[0262] Clause 8. A method comprising:

[0263] Detection of event triggers that affect the autonomous vehicle, where:

[0264] The autonomous vehicle is configured to travel along a road; and

[0265] The autonomous vehicle includes at least one sensor configured to capture sensor data;

[0266] In response to detecting the event trigger, causing the autonomous vehicle to enter a first degraded autonomy mode, wherein in the first degraded autonomy mode, the method further comprises:

[0267] communicating sensor data to a supervisory server;

[0268] receiving one or more high-level commands from a supervisory server, wherein the one or more high-level commands indicate a minimal risk maneuver for the autonomous vehicle;

[0269] Receiving a maximum travel speed for the autonomous vehicle from a supervisory server; and navigating the autonomous vehicle using adaptive cruise control based on the one or more high-level commands and the maximum travel speed.

[0270] Clause 9. The method of clause 8, wherein the one or more high-level commands include at least one of the following instructions:

[0271] Maintaining a specific amount of time in the current lane;

[0272] changing to a particular lane when traffic in the particular lane permits; and

[0273] Pull over on a specific side of the road at a specific location.

[0274] Clause 10. The method of clause 8, further comprising maintaining a predetermined distance from other vehicles and objects on the road.

[0275] Clause 11. The method of clause 8, wherein the minimal risk maneuver includes decelerating the autonomous vehicle.

[0276] Clause 12. The method of clause 8, wherein the autonomous vehicle is a semi-truck tractor unit attached to a trailer.

[0277] Clause 13. The method of clause 8, wherein the sensor data includes data indicative of objects on the road.

[0278] Clause 14. The method of clause 8, wherein the at least one sensor comprises at least one of a camera, a light detection and ranging (LiDAR) sensor, a motion sensor, and an infrared sensor.

[0279] Clause 15. A system comprising:

[0280] an autonomous vehicle configured to travel along a road, wherein the autonomous vehicle includes at least one sensor configured to capture sensor data;

[0281] A control device associated with an autonomous vehicle and comprising a first processor, the first processor being configured to:

[0282] Detecting event triggers that affect the autonomous vehicle;

[0283] In response to detecting the event trigger, causing the autonomous vehicle to enter a first degraded autonomous mode, wherein in the first degraded autonomous mode, the first processor is configured to:

[0284] accessing sensor data, the sensor data comprising data representing one or more of lane markings and traffic signs;

[0285] detecting lane markings on at least one side of the autonomous vehicle based on the sensor data;

[0286] detecting traffic signs on the road ahead of the autonomous vehicle based on sensor data; and

[0287] The autonomous vehicle is navigated using adaptive cruise control such that the autonomous vehicle maintains within a current lane according to detected lane markings and obeys traffic regulations according to traffic signs, wherein navigating the autonomous vehicle is according to a predetermined maximum travel speed.

[0288] Clause 16. The system of clause 15, wherein the event trigger comprises one or more of a hardware failure and a software failure with respect to the autonomous vehicle.

[0289] Clause 17. The system of clause 16, wherein the event triggering causes connectivity with the supervisory server to be degraded such that the control device and the supervisory server cannot communicate with each other.

[0290] Clause 18. The system of clause 15, wherein the event trigger is a loss of connectivity between the control device and the supervisory server.

[0291] Clause 19. The system of clause 17, wherein when navigating the autonomous vehicle, the first processor is further configured to:

[0292] instructing the autonomous vehicle to travel a predetermined distance;

[0293] determining whether connectivity with the supervisory server is at least partially restored while the autonomous vehicle is traveling the predetermined distance;

[0294] In response to determining that connectivity with the supervisory server is at least partially restored:

[0295] communicating sensor data to a supervisory server;

[0296] receiving one or more high-level commands from a supervisory server, wherein the one or more high-level commands indicate a minimal risk maneuver for the autonomous vehicle;

[0297] receiving a maximum driving speed for the autonomous vehicle from a supervisory server;

[0298] Navigating the autonomous vehicle using adaptive cruise control based on the one or more high-level commands and the maximum travel speed; and

[0299] In response to determining that connectivity with the supervisory server has not even been at least partially restored, the autonomous vehicle is instructed to pull over to a particular location on a side of the road.

[0300] Clause 20. The system of clause 17, wherein when navigating the autonomous vehicle, the first processor is further configured to:

[0301] Instructing the autonomous vehicle to drive until a predetermined time;

[0302] while the autonomous vehicle is traveling until a predetermined time, determining whether connectivity with the supervisory server is at least partially restored;

[0303] In response to determining that connectivity with the supervisory server is at least partially restored:

[0304] communicating sensor data to a supervisory server;

[0305] receiving one or more high-level commands from a supervisory server, wherein the one or more high-level commands indicate a minimal risk maneuver for the autonomous vehicle;

[0306] receiving a maximum driving speed for the autonomous vehicle from a supervisory server;

[0307] navigating the autonomous vehicle using adaptive cruise control based on the one or more high-level commands and the maximum travel speed; and

[0308] In response to determining that connectivity with the supervisory server has not even been at least partially restored, the autonomous vehicle is instructed to pull over to a particular location on a side of the road.

[0309] Clause 21. The system of any of clauses 1-7, wherein the first processor is further configured to perform one or more operations of the method of any of clauses 8-14.

[0310] Clause 22. The system of any of clauses 15-20, wherein the first processor is further configured to perform one or more operations of the method of any of clauses 8-14.

[0311] Clause 23. An apparatus comprising means for performing the method according to any of clauses 8-14.

[0312] Clause 24. A non-transitory computer-readable medium storing instructions that, when executed by a processor, cause the processor to perform one or more operations according to any of clauses 1-7 and 14-20.

[0313] Clause 25. A non-transitory computer-readable medium storing instructions that, when executed by a processor, cause the processor to perform one or more operations according to any of clauses 8-14.

Claims

1. A system (100), comprising: An autonomous vehicle (902) configured to travel along a road (102), wherein the autonomous vehicle (902) includes at least one sensor (946) configured to capture sensor data (130); A control device (950) associated with the autonomous vehicle (902) and comprising a first processor (122, 970), the first processor being configured to: detecting an event trigger (142a) affecting the autonomous vehicle (902); In response to detecting the event trigger (142a), causing the autonomous vehicle (902) to enter a first degraded autonomous mode (140a), wherein in the first degraded autonomous mode (140a), the first processor (122, 970) is configured to: communicating the sensor data (130) to a supervisory server (160); receiving one or more high-level commands (174) from the supervisory server (160), wherein the one or more high-level commands (174) indicate a minimal risk maneuver for the autonomous vehicle (902); receiving a maximum travel speed (172) for the autonomous vehicle (902) from the supervisory server (160); and The autonomous vehicle (902) is navigated using adaptive cruise control (146) based on the one or more high-level commands (174) and the maximum travel speed (172).

2. The system (100) of claim 1, wherein the event trigger (142a) comprises one or more of a hardware failure and a software failure with respect to the autonomous vehicle (902).

3. The system of claim 1, wherein: The event trigger (142a) results in a loss of positioning capability with respect to the autonomous vehicle (902); and The loss of the positioning capability results in the control device (950) being unable to determine the geographic location coordinates of the autonomous vehicle (902).

4. The system (100) of claim 3, wherein: The event trigger (142a) also results in a loss of traffic sign detection capability with respect to the autonomous vehicle (902); and The loss of the traffic sign detection capability results in the control device (902) being unable to detect traffic signs and traffic lights.

5. The system (100) of claim 1, wherein the first processor (122, 970) is further configured to communicate a message to the supervisory server (160), the message indicating that the autonomous vehicle has entered the first degraded autonomy mode (140a).

6. The system (100) of claim 1, further comprising the supervisory server (160), the supervisory server (160) being communicatively coupled to the control device (950) and comprising a second processor (162), the second processor (162) being configured to: receiving the sensor data (130) from the control device (950); displaying the sensor data (130) on a user interface (166); receiving input from a remote operator (184), the input comprising the maximum travel speed (172) and the one or more high-level commands (174); communicating the maximum travel speed (172) to the control device (950); and The one or more high-level commands (174) are communicated to the control device (950).

7. The system (100) of claim 4, wherein the first processor (122, 970) is further configured to: detecting that at least one of the positioning capability and the traffic sign detection capability is partially restored; In response to detecting that at least one of the positioning capability and the traffic sign detection capability is partially restored, causing the autonomous vehicle (902) to enter a second degraded autonomous mode (142b), wherein in the second degraded autonomous mode (142b), in addition to the operations in the first degraded autonomous mode (142a), the first processor (122, 970) is further configured to: accessing the sensor data (13), the sensor data (13) comprising data representing one or more of lane markings (212) and traffic signs (214); detecting the lane markings (212) on at least one side of the autonomous vehicle (902) based on the sensor data (130); detecting the traffic sign (214) on the road (102) in front of the autonomous vehicle (902) based on the sensor data (130); as well as The detected lane markings (212) and traffic signs (214) are used in the navigation of the autonomous vehicle (902) using the adaptive cruise control (146) based on the one or more high-level commands (174) and the maximum travel speed (172).

8. A method (500), comprising: Detecting an event trigger (142a) affecting an autonomous vehicle (902), wherein: The autonomous vehicle (902) is configured to travel along a road (102); and The autonomous vehicle (902) includes at least one sensor (946) configured to capture sensor data (130); In response to detecting the event trigger (142a), causing the autonomous vehicle (902) to enter a first degraded autonomous mode (140a), wherein in the first degraded autonomous mode (140a), the method further comprises: communicating the sensor data (130) to a supervisory server (160); receiving one or more high-level commands (174) from the supervisory server (160), wherein the one or more high-level commands (174) indicate a minimal risk maneuver for the autonomous vehicle (902); receiving a maximum travel speed (172) for the autonomous vehicle (902) from the supervisory server (160); and The autonomous vehicle (902) is navigated using adaptive cruise control (146) based on the one or more high-level commands (174) and the maximum travel speed (172).

9. The method (500) of claim 8, wherein the one or more high-level commands (174) include at least one of the following instructions: Maintaining a specific amount of time in the current lane; changing to the particular lane when traffic in the particular lane permits; and Pull over on a specific side of the road at a specific location.

10. The method (500) of claim 8, further comprising maintaining a predetermined distance (150) from other vehicles and objects on the road.

11. The method (500) of claim 8, wherein the minimal risk maneuver includes decelerating the autonomous vehicle (902).

12. The method (500) of claim 8, wherein the autonomous vehicle (902) is a semi-truck tractor unit attached to a trailer.

13. The method (500) of claim 8, wherein the sensor data (130) includes data indicative of objects on the road.

14. The method (500) of claim 8, wherein the at least one sensor (946) comprises at least one of a camera, a light detection and ranging (LiDAR) sensor, a motion sensor, and an infrared sensor.

15. A system (100), comprising: An autonomous vehicle (902) configured to travel along a road, wherein the autonomous vehicle (902) includes at least one sensor (946) configured to capture sensor data (130); A control device (950) associated with the autonomous vehicle (902) and comprising a first processor (122, 970), the first processor being configured to: detecting an event trigger (142c) affecting the autonomous vehicle (902); In response to detecting the event trigger (142c), causing the autonomous vehicle (902) to enter a first degraded autonomous mode (140c), wherein in the first degraded autonomous mode (140c), the first processor (122, 970) is configured to: accessing the sensor data (130), the sensor data (130) comprising data representing one or more of a lane marking (212) and a traffic sign (214); detecting the lane markings (212) on at least one side of the autonomous vehicle (902) based on the sensor data (130); detecting the traffic sign (214) on the road (102) in front of the autonomous vehicle (902) based on the sensor data (130); as well as The autonomous vehicle (902) is navigated using adaptive cruise control (146) such that the autonomous vehicle (902) maintains within a current lane based on the detected lane markings (212) and obeys traffic regulations based on the traffic signs (214), wherein navigating the autonomous vehicle (902) is based on a predetermined maximum travel speed (148).

16. The system (100) of claim 15, wherein the event trigger (142c) comprises one or more of a hardware failure and a software failure with respect to the autonomous vehicle (902).

17. The system (100) of claim 16, wherein the event trigger (142c) causes connectivity with a supervisory server (160) to be degraded such that the control device (950) and the supervisory server (160) are unable to communicate with each other.

18. The system (100) of claim 15, wherein the event trigger (142c) is a loss of connectivity between the control device (950) and a supervisory server (160).

19. The system (100) of claim 17, wherein when navigating the autonomous vehicle (902), the first processor (122, 970) is further configured to: instructing the autonomous vehicle (902) to travel a predetermined distance (150); while the autonomous vehicle (902) is traveling the predetermined distance (150), determining whether the connectivity with the supervisory server (160) is at least partially restored; In response to determining that the connectivity with the supervisory server (160) is at least partially restored: communicating the sensor data (130) to the supervisory server (160); receiving one or more high-level commands (174) from the supervisory server (160), wherein the one or more high-level commands (174) indicate a minimal risk maneuver for the autonomous vehicle (902); receiving a maximum travel speed (172) for the autonomous vehicle (902) from the supervisory server (160); navigating the autonomous vehicle (160) using the adaptive cruise control (146) based on the one or more high-level commands (174) and the maximum travel speed (172); and In response to determining that the connectivity with the supervisory server (160) has not even been at least partially restored, the autonomous vehicle (902) is instructed to pull over to a specific location on a side of the road.

20. The system (100) of claim 17, wherein when navigating the autonomous vehicle (902), the first processor (122, 970) is further configured to: instructing the autonomous vehicle (902) to travel until a predetermined time (152); while the autonomous vehicle (902) is traveling until the predetermined time (152), determining whether the connectivity with the supervisory server (160) is at least partially restored; In response to determining that the connectivity with the supervisory server (160) is at least partially restored: communicating the sensor data (130) to the supervisory server (160); receiving one or more high-level commands (174) from the supervisory server (160), wherein the one or more high-level commands (174) indicate a minimal risk maneuver for the autonomous vehicle (902); receiving a maximum travel speed (172) for the autonomous vehicle (902) from the supervisory server (160); navigating the autonomous vehicle (902) using the adaptive cruise control (146) based on the one or more high-level commands (174) and the maximum travel speed (172); and In response to determining that the connectivity with the supervisory server (160) has not even been at least partially restored, the autonomous vehicle (902) is instructed to pull over to a specific location on a side of the road.

21. The system (100) of any one of claims 1-7, wherein the first processor (122, 970) is further configured to perform one or more operations of the method (500) of any one of claims 8-14.

22. The system (100) of any one of claims 15-20, wherein the first processor (122, 970) is further configured to perform one or more operations of the method (500) of any one of claims 8-14.

23. An apparatus comprising means for performing the method (500) according to any one of claims 8 to 14.

24. A non-transitory computer-readable medium (126) storing instructions that, when executed by a processor (122, 970), cause the processor (122, 970) to perform one or more operations according to any one of claims 1-7 and 14-20.

25. A non-transitory computer-readable medium (126) storing instructions that, when executed by a processor (122, 970), cause the processor (122, 970) to perform one or more operations according to any one of claims 8-14.

Images