Modification of Restrictions on Vehicle Dynamics for Tracks

Abstract

The present disclosure is directed to performing one or more plausibility checks on potential trajectories for a device, such as an autonomous vehicle, to navigate along. In some examples, the potential trajectory may be validated based on whether the potential trajectory matches the current trajectory the vehicle is navigating such that the potential trajectory and the current trajectory are not too different, whether the vehicle can feasibly or kinematically navigate to the potential trajectory from its current state, whether the potential trajectory is punctual in time or was received within the time of a preceding trajectory, and / or whether the potential trajectory passes a staleness check, such as the potential trajectory was created within a certain period of time. In some examples, determining whether the potential trajectory is feasible may include updating a set of feasibility constraints based on one or more operational characteristics of the status of subsystems of the vehicle.

Description

Technical Field

[0001] Cross - Reference to Related Applications This application claims priority to U.S. Patent Application No. 16 / 539,893, filed on August 13, 2019, entitled "MODIFYING LIMITS ON VEHICLE DYNAMICS FOR TRAJECTORIES", U.S. Patent Application No. 16 / 539,870, filed on August 13, 2019, entitled "SYSTEM AND METHOD FOR TRAJECTORY VALIDATION", U.S. Patent Application No. 16 / 539,878, filed on August 13, 2019, entitled "CONSISTENCY VALIDATION FOR VEHICLE TRAJECTORY SELECTION", and U.S. Patent Application No. 16 / 539,873, filed on August 13, 2019, entitled "FEASIBILITY VALIDATION FOR VEHICLE TRAJECTORY SELECTION", and incorporates by reference all of these disclosures for all purposes.

Background Art

[0002] Autonomous vehicles are programmed with safety in mind, among other goals such as passenger comfort, predictability, and responsiveness. Some systems of autonomous vehicles, such as navigation systems, are important for ensuring safety. In navigation systems, it may be necessary to consider a large number of variations that can be difficult to explain, making safe and effective navigation very complex. For example, there may be multiple subsystems of an autonomous vehicle that interact in complex ways. With additional environmental changes, the complexity of safe navigation may also increase dramatically. These interactions of multiple subsystems and the environment can introduce variability and unpredictability that make it more difficult to ensure the safety of passengers, pedestrians, property, etc.

[0003] A detailed description will be given with reference to the accompanying drawings. In the drawings, the leftmost digit(s) of a reference number identify the figure in which the reference number first appears. The same reference number in different drawings indicates a similar or identical component or feature.

Brief Description of the Drawings

[0004]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Figure 16

[0005] This disclosure describes methods, apparatuses, and systems for verifying and selecting future trajectories for devices such as autonomous vehicles. In some examples, an artificial intelligence (“AI”) unit of an autonomous vehicle considers inputs from various subsystems of the vehicle and outputs a selection of possible trajectories to a trajectory manager process (“TMP”), which can operate as a system that performs collision avoidance by selecting and verifying one of the trajectories provided using specific criteria. In this example, the TMP can then output the selected trajectory to a drive manager that controls the vehicle's propulsion system, possibly within other systems.

[0006] In some aspects, the TMP performs a criteria check on the provided trajectories independently of which trajectory the AI engine outputs, and if the AI system outputs an unsafe trajectory, the TMP sends a corresponding signal to the drive manager process or drive actuator to prevent the vehicle from navigating an invalid, e.g., unsafe, trajectory. In some examples, if none of the proposed trajectories meet the criteria of the TMP, the TMP can output a trajectory to bring the vehicle to a stop.

[0007] In some aspects, the TMP evaluates the validity of the trajectory. The validity test may include several different tests or evaluations, such as checking timeliness, consistency, staleness, and / or feasibility. For example, a given trajectory may be evaluated for timeliness, such as being received within a certain period of a previous trajectory. In other examples, a given trajectory may be evaluated for consistency with the current state of the vehicle and / or previous trajectories of the tested trajectory (e.g., whether the new trajectory and / or reference is within a threshold distance of the previous trajectory). In some aspects, the trajectory may be tested for staleness, such as verifying that the trajectory has not been generated for a long time in the past. In some cases, staleness may be considered or referenced as part of the consistency check. In yet other examples, the trajectory may be evaluated for the feasibility of the vehicle navigating the tested trajectory, such as based on the physical capabilities or limitations of the vehicle. In some aspects, two or more of these validity tests may be performed on potential trajectories to ensure that the trajectory is safe and does not impose an undesirable physical burden on the vehicle occupants.

[0008] The result of the validity evaluation may be output as a signal indicating at least whether the trajectory is valid, e.g., passed the evaluation or is invalid such as failed the evaluation. The validity signal may take one of several forms and may include simple values such as binary values indicating whether the trajectory passed the evaluation and whether it is actually valid. In some cases, it may also include an identifier of the trajectory. In yet other cases, it may also include further details of the trajectory. The validity signal or the selected trajectory itself may be sent to the drive manager process or drive actuator to cause the vehicle to execute a valid trajectory. In some cases, regardless of pass or fail, the validity signal may be communicated to the AI engine to enable selection of another trajectory or for many other reasons.

[0009] In some aspects, the TMP may also perform collision tests against potential trajectories. In these cases, an executable trajectory is a trajectory that satisfies at least a validity test and a collision test. In other cases, other evaluation or verification tests may be used alone or in combination with the examples described above. In some aspects, one or more validity checks may include a timing check related to how much time has elapsed since the trajectory was received. For example, a secondary system may expect trajectories to be received at a particular frequency (e.g., every 4 seconds). In such an example, if the difference in time between subsequently received trajectories exceeds, for example, 4 seconds, it may indicate that one or more subsystems of the vehicle are not operating correctly or as expected (e.g., due to delays, transmission issues, planning difficulties, etc.), and thus may be considered to fail the timing criterion. When determining whether the timing requirement is met, the TMP may compare the received time of the trajectory with the current time to determine whether the difference between the two is greater than a predetermined timing limit.

[0010] In some aspects, one or more validity checks may include a consistency evaluation, which is related to preventing large jumps in vehicle state or commands from one trajectory to the next based on some predetermined consistency constraints. For example, if the speed between the current trajectory and the next trajectory is too different, the next trajectory may be flagged as inconsistent. The result of the consistency or kinematic evaluation may be in the form of a kinematic validity or kinematic validity value indicating whether a given trajectory is consistent and, in some cases, to what extent it is consistent.

[0011] In some examples, one or more validity checks may include an obsolescence check where the system determines whether the age of the generated trajectory is lower than or equal to a threshold age. In such examples, the trajectories may include a timestamp when they were generated. The difference between the current time and the generated time may be compared to a threshold to determine whether the trajectory is old. As a non-limiting example, despite being periodic (e.g., received within 4 seconds from a previous trajectory as in the previous example), the timestamp of a trajectory may indicate that the time since it was generated is longer than 100 ms old (exceeding a threshold of 50 ms), in which case the trajectory is marked as invalid.

[0012] In some aspects, one or more validity checks may include a feasibility check, which is related to verifying that the trajectory being verified is within the current dynamic capabilities of the vehicle (e.g., a new trajectory commanding a speed of 180 KPH or a lateral acceleration of 6 m / s / s may not be followed).

[0013] In some cases, the system monitor may modify a set of predefined vehicle capabilities based on the operating characteristics of the vehicle. For example, the TMP may detect that one or more tires have low pressure, the battery is not operating at full capacity, the motor is operating at 75% output, etc. The TMP may correlate various numerical or qualitative operating statuses to changes in one or more feasibility limits set for the vehicle. Based on these and other operating characteristics, the system monitor can update one or more feasibility values or limits imposed on the vehicle. The TMP can access these values and verify potential trajectories based on the latest values. In some aspects, the result of the feasibility assessment may be output as a feasibility value or signal or a feasibility validity signal.

[0014] The system monitor can monitor the state of subsystems within a vehicle, send requests to the TMP that can override the determination of the TMP regarding a trajectory, and / or send an operation status update that can notify and correct the evaluation of the TMP regarding a potential trajectory. In some cases, the TMP can receive updates (e.g., of one or more operating characteristics) from the vehicle or a device's monitoring system or system monitor at various intervals, periodically, upon the occurrence of certain events such as a specific event. These updates can be of one or more subsystems that affect the operation of the vehicle. In some cases, the TMP can change the criteria for its selection / verification of a trajectory based on the update(s). In some cases, the system monitor may communicate trajectory requests and / or other constraints to the TMP, which can affect or even override the selection criteria implemented by the TMP.

[0015] In some aspects, a set of vehicle capabilities or feasibility limitations can be determined for a vehicle operating in a normal state or normal operating situation. The system monitor can detect and provide updates regarding whether various subsystems of the vehicle are operating normally or have a reduced operating state such that the operation of the vehicle is reduced in some manner. The system monitor and / or the TMP can update a set of vehicle capability limitations based on the operation update. The TMP can then modify the criteria based on the updated set of vehicle capabilities when determining whether a trajectory is valid. In a specific example, vehicle capability limitations can be used to determine whether a trajectory is feasible, such as through a feasibility check. As a non-limiting example, a reduced pressure or braking ability may be associated with a reduced maximum acceleration (lateral and / or longitudinal).

[0016] In some embodiments, the trajectory verification process may also include a collision check. The collision check criteria may include TMP-rejected trajectories that the TMP has determined will include an imminent collision with an agent or object. In some cases, the trajectory has individual validity criteria covering timeliness, consistency, staleness, and feasibility, and has a collision-free requirement. In such cases, there are two tests: validity and collision avoidance, but these two tests may be based on multiple criteria.

[0017] The collision check can be used to prevent the representation of trajectories that include a predicted collision with any agent or obstacle before it is determined that the vehicle can stop (exceeding the braking distance or related value) or before it is determined that the object can be avoided. Trajectories that are not avoidable but include any predicted collision may not need to be followed, or if all available trajectories include a predicted collision, the trajectory that minimizes the predicted collision energy or other metric may be the selected trajectory.

[0018] The collision check may include running a free space checker, evaluating a trajectory predictor based on an object list, a processor using machine learning on a heat map, or other approaches. Some trajectories may be checked for possible collisions in all frames, while other trajectories may not need to be checked. For example, if an emergency stop ("E-stop") trajectory is the least likely to occur, it may not be necessary to repeatedly check that trajectory to determine if it will result in a collision. This is because the trajectory is likely a last resort, has the goal of decelerating, and likely already represents the last option to come to a stop by locking the steering (by brakes or other means) and applying maximum deceleration, and is likely only used in cases of multiple system failures or a suspect status.

[0019] The available orbits for TMP may have a hierarchy where the first orbit is at a higher level than the second orbit, the second orbit is at a higher level than the third orbit, and so on. The first orbit is the nominal operating orbit, and the lower-level orbits can handle non-ideal situations or error situations, such as when a higher orbit is not appropriate. For example, if some errors are found in the first orbit, the first orbit may be the orbit used unless the second orbit is the contingency orbit that would be used in that case.

[0020] In at least some examples, if the newly acquired orbit is not valid and / or collision-free, one or more of the received primary and / or secondary orbits may be stored to be used in subsequent frames. Such orbits may be referred to as stored orbits. In any one or more examples, any orbit (either newly acquired or stored) may be modified if such a modification would lead to a scenario that prevents collisions (or minimizes severity, etc.). In some examples, such a modification may be limited to vertical modifications, such as applying additional braking to the orbit, but in other examples, other modifications (e.g., slight turns) may be provided. According to the hierarchy, the system can choose the first (or primary), secondary (or alternative / contingency), modified secondary, modified stored secondary, and finally, an emergency stop (also referred to herein as an "E-stop") if none of the above orbits mitigate the collision. In such an example, the E-stop can be an orbit that attempts to quickly stop the autonomous vehicle within the mechanical capabilities of the autonomous vehicle and the safety limits of the occupants.

[0021] It should be understood that the examples and descriptions in this specification referring to these specific sets of trajectories can be modified to cover examples having other sets of trajectories provided hierarchically. Also, in specific examples, a strictly ordered hierarchy is provided, although in some examples, there may be a hierarchy where multiple types of trajectories have the same level. In such an example, there may be first, second, third, and fourth trajectories, where the first trajectory is considered the highest on the trajectory, the second and third are equal and considered to be at the next lowest level, and the fourth is the lowest-level trajectory.

[0022] In the general case, there is a hierarchy of trajectories, and in the normal operating state, the TMP examines the first trajectory and uses it if it is valid and collision-free. In that normal operating state, when a new trajectory is provided, each trajectory can be checked in each frame, and the highest, valid, and collision-free trajectory can be used. In at least some examples, if either the primary or secondary is collision-free and valid, even if the lower-level trajectory is neither valid nor collision-free, the TMP may select such a trajectory. The TMP can then store a state variable representing the trajectory used. If the normal operation is such that the first trajectory is valid and collision-free, dropping to a lower trajectory may indicate an error or an abnormal situation. For example, the first trajectory may be valid and collision-free at generation, but then, because an object moving at high speed passes in front of the autonomous vehicle, when the first trajectory is checked, it is confirmed but not collision-free. As a result, the TMP uses a lower trajectory and updates the state variable to indicate which one was used. The stored value of the state variable may be used in future frames to limit the trajectories used by the TMP until at least the error situation is cleared.

[0023] Generally, a trajectory can be represented by a data structure that defines vectors, paths, vehicle states (position, orientation, velocity, and / or acceleration), commands, etc., including a series of controls (e.g., acceleration, steering, and yaw rate, etc.) that are executed over time from the start state of the trajectory to the end point and / or are associated with such waypoints. The trajectory is not necessarily reproduced to its end point and may be overwritten. The trajectory provided to the TMP can be a trajectory that involves decelerating the vehicle until it stops and a trajectory that involves the vehicle moving according to some details of the trajectory, changes in direction, changes in speed, etc. The trajectory can include data representing signals to be sent to drive actuators, such as for brakes, steering, accelerator, suspension, etc.

[0024] When followed to the end point, a trajectory that would necessarily lead to the stopping of the vehicle due to an abnormal or safety situation is called a stop trajectory in this specification, and other trajectories are nominal trajectories. Some nominal trajectories may cause the autonomous vehicle to come to a stop in non-abnormal situations, such as driving to a stop sign, stopping at a stop signal, waiting for pedestrians to cross, pulling over to the shoulder to pick up people, etc. The stop trajectory has a stationary position as the end point for safety-related reasons or due to abnormal situations.

[0025] TMP may also have a stored stop trajectory that is used by TMP when it does not have other proposed trajectories that meet the criteria of TMP. If played back to the end of the trajectory, the stored stop trajectory and / or the received stop trajectory will end with the vehicle stationary. In other words, the end point of such a trajectory is a stationary vehicle. Not all trajectories are necessarily played back to their end points. For example, a newly acquired trajectory may be received before an earlier received trajectory is executed to its end point. In at least some examples, the stop trajectory may include secondary and / or incidental trajectories as described above. In some examples, such a stop trajectory may not involve coming to a stop, but may provide an alternative route and / or control as an incidental situation if a problem occurs with the primary trajectory.

[0026] AI may provide only two trajectories, such as a primary (nominal) trajectory and a secondary (e.g., stop) trajectory, but in some examples, AI may provide three or more trajectories. As an example, AI may provide a primary trajectory with a vehicle continuing in a lane, a primary trajectory with a vehicle stopping within about 10 seconds, a secondary trajectory with a braking lane, a secondary trajectory with a vehicle changing lanes and braking, etc. In another example, AI creates and issues a new primary trajectory and a new secondary trajectory approximately every 100 ms. The modified secondary trajectory may be a trajectory created by TMP applying a vertical declaration to the secondary trajectory.

[0027] In a more general case, what is provided is a set of instructions, which may be instructions regarding a trajectory, and one, two, or more sets of instructions may be provided. Other sets of instructions may be generated from what has been provided, for example, a set of instructions derived from a previously provided and stored set of instructions, a converted or modified version of a received set of instructions, and a set of fixed instructions, etc. If the set of instructions includes a trajectory, the set of instructions may include instructions somewhat related to the trajectory, such as instructions unrelated to the trajectory, such as messages, instructions such as lighting, and / or direction instructions or other actions taken in relation to the trajectory. In this case, a state variable may be provided that represents the maximum level in a hierarchy of levels where sets of instructions having levels above that maximum level will not necessarily be executed. If it is determined that the set of instructions is invalid or abnormal, the maximum level is set to a level below the level of that invalid or abnormal set of instructions. The state variable may remain at or below that level until a release signal is received to reset the state variable to a higher level, perhaps the highest level. For example, a request indicating the selection of a sporadic trajectory rather than a nominal trajectory can set the state variable to represent a vehicle trajectory state that is a sporadic state representing the selection of a sporadic trajectory rather than a nominal trajectory, and the release signal can reset the vehicle trajectory state to a nominal request state indicating the selection of a nominal trajectory rather than a sporadic trajectory. The release signal may be a signal received from a remote system configured to transmit a signal in response to the reception of an input from a user.

[0028] In an exemplary system, the system monitor can monitor the state of the systems within the vehicle and send a request to the TMP that can overwrite the determination of the TMP regarding the "downward" direction of the trajectory (e.g., primary → secondary → modified secondary → stored secondary → modified and stored secondary → "E-stop", etc.). The request can be a request for a specific type of trajectory selected from primary, secondary, or "E-stop", which will be used to send a signal that the system monitor considers any of them as an acceptable option. The request can be an overwrite request that requests an incidental trajectory on the nominal trajectory. In some logic tables, the final result is the same for the trajectory type requests of multiple system monitors. As an example, if the TMP is trying to determine the activation of a secondary trajectory request based on some combination of states, even if the system monitor requests a primary trajectory type or a secondary trajectory type, the result is likely to be a secondary trajectory. It may be referred to herein as a system monitor that designates a primary or secondary trajectory type, but should be understood as described above.

[0029] In another example, if the AI provides trajectories to the TMP and the TMP discovers that all of them are acceptable, the TMP may select a primary trajectory. However, if the system monitor then processes an input indicating that the tire pressure of all four tires of a four-wheel vehicle is 15 PSI (i.e., running flat), the system monitor will send a signal requesting an "E-stop" trajectory.

[0030] During operation, the system monitor may send a signal requesting a secondary trajectory when an abnormal situation such as the detection of a mechanical or electrical error is detected. In response, the TMP provides a secondary trajectory to the drive manager. While the drive manager is activating the secondary trajectory to bring the autonomous vehicle to a stationary state, the system monitor detects that the problem has been sufficiently resolved and sends a signal requesting the nominal (e.g., primary) trajectory, while the TMP will provide another trajectory in some other way.

[0031] In some examples, recovery from some situations is considered to be so severe that it requires human intervention. In such cases, the system monitor may issue a secondary trajectory or "E-stop" trajectory request until it receives a fault clear signal from a system where humans intervene (and / or a remote system with additional computational power for artificially intelligent decisions), such as a system monitor release signal from a teleoperator system.

[0032] The described systems and techniques can provide many advantages and benefits, such as safer operation of autonomous vehicles or other movable devices through redundant trajectory verification and collision checking, especially when updating the feasibility limits of a vehicle based on the current operating status of the vehicle's subsystems. The described systems and techniques can also provide improvements in resource economy, such as reducing the memory and processing requirements, through the use of clear validity checks based on predefined criteria. In some aspects, the described systems and techniques can enable the use of less bandwidth in communicating information for selecting trajectories, among other benefits and advantages described throughout this disclosure, through the use of specific validity criteria.

[0033] FIG. 1 is an exemplary drawing 100 of an autonomous vehicle 102 that selects and verifies one of a plurality of trajectories 110, 112 for navigation. As shown, the autonomous vehicle 102 may include a planning system 116. The planning subsystem 116 may include a primary system 106 or an AI engine, which can generate trajectories for the autonomous vehicle 102 to follow or navigate, such as the current trajectory 104 and the trajectories 110, 112 that can be navigated by the autonomous vehicle 102 in the future. To enhance safety in the operation of the autonomous vehicle 102, to further assist in avoiding collisions, to add redundancy, and for various other reasons, a secondary system 108 is implemented to verify some different attributes of the possible trajectories generated by the primary system 106 before the autonomous vehicle is instructed to follow one of those trajectories.

[0034] The secondary system 108 may include a Trajectory Manager Process or System (TMP) 118 that receives the trajectories generated by the primary system 106 and performs several verification checks 120 on the trajectories to ensure that the trajectories are valid and / or do not result in any avoidable collisions. The TMP 118 may receive two or more trajectories from the primary system 106 for verification or evaluation. For example, the primary system 106 may generate at least one first or primary trajectory 110 for each period, which may include continuing the forward momentum of the autonomous vehicle 102 at least some distance. The primary system 106 may also generate at least one secondary trajectory 112 such that the autonomous vehicle 102 can stop at a point 114 that can be at any future time. In other examples, the TMP 118 may evaluate any one of several trajectories of the autonomous vehicle 102 for any given period.

[0035] Upon receiving potential trajectories, the TMP 118 may perform several verification checks 120 on the primary and secondary trajectories 110, 112 to ensure that the selected trajectories are safe and do not cause sudden movements or discomfort to the occupants of the autonomous vehicle 102. These verification checks may include timeliness checks, consistency checks, staleness checks, and / or feasibility checks, as described in more detail below. In some aspects, the TMP 118 may also perform collision checks on the trajectories to verify that no avoidable collisions occur on the selected trajectories.

[0036] In some cases, the secondary system 108 or the primary system 106 may monitor some of the subsystems of the autonomous vehicle 102. If one or more of the subsystems are not functioning in an appropriate state as indicated by a normal operating state, the primary or secondary system 106, 108 may modify a set of the vehicle's ability limitations. When determining whether a trajectory is achievable for the autonomous vehicle, the TMP 118 may determine whether the autonomous vehicle can safely navigate and follow a new trajectory based on a set of the vehicle's ability limitations. In such a manner, the trajectory is verified based on the current operating situation of the vehicle, and safety can be enhanced.

[0037] The autonomous vehicle 102 may include some hardware and software components implementing a propulsion system, and a control and planning system for instructing the propulsion system to move the vehicle through various terrains and obstacles and to transport passengers, among other things. The autonomous vehicle 102 may include some subsystems, as will be described in more detail below with reference to FIG. 16. Although the present disclosure is primarily focused on verifying a trajectory for an autonomous vehicle, it should be understood that the techniques described herein are equally applicable to any of several different types of devices, such as any device including some type of propulsion system or movement system.

[0038] In some aspects, the planning system 116 may be an example of the planning subsystem 1628 described below with reference to FIG. 16. The planning system 116, as well as the primary system 106 and the secondary system 108, may include one or more processors and memory including instructions for generating and verifying a trajectory for the autonomous vehicle, or more generally, for any of several different types of devices.

[0039] Figure 2 is a high-level block diagram showing an example of some elements of an autonomous vehicle management and / or control system 200, including a primary computing unit 202 and a secondary computing unit 204. The primary computing unit 202 and the secondary computing unit 204 can be separate processors having a communication channel between them, or some other structure. As shown, the primary computing unit 202 runs a system monitor AI 210, a trajectory planner AI 212, and stores sensor data 214, and the secondary computing unit 204 runs a system monitor 220, a trajectory manager process ("TMP") 222, and a safety / unconscious corridor checker 224.

[0040] The system monitor AI 210 receives data that may be included in messages sent from the system monitor 220 regarding the status of various components of the autonomous vehicle monitor. Examples may include a power system, a tire system, an environmental sensor system, a battery monitor, etc. The system monitor 220 may also receive messages from the actuator systems regarding the status and operation of these actuator systems. The system monitor AI 210 may receive data messages from other sources as well. In operation, the system monitor AI 210 can process these inputs to determine system monitor AI outputs, such as a trajectory type request message sent to the system monitor 220. As an example, the system monitor AI 210 can determine that the planned trajectory to be used is a certain type of trajectory, or an emergency stop type, or a stop type of trajectory due to some abnormal or non-nominal situation that causes the autonomous vehicle to come to a stop. The system monitor 220 can output an actuator diagnostic message.

[0041] The trajectory planner AI 212 generates one or more trajectories considering various states, situations, and / or inputs. The trajectories can be represented by trajectory data records passed in trajectory data messages. The trajectory data records may include details such as a timestamp indicating when the trajectory was created, the direction, speed, actuator inputs, etc. in the frames covered by the trajectory. For a given frame, the trajectory planner AI 212 may output one or more trajectories to the trajectory manager process (“TMP”) 222, which may include a nominal trajectory and, if the nominal trajectory is not valid or includes a collision, or when the TMP 222 determines not to execute the nominal trajectory, an emergency trajectory to be used. The trajectory planner AI 212 may receive warning messages from the TMP 222, such as a message indicating that the TMP 222 cannot process some trajectories.

[0042] The sensor data 214 may include point cloud data such as around the vehicle represented by a point cloud, image data, radar data, and other sensor data, and is included in a message. The primary computing unit 202 can provide this sensor data 214 in a message to the safety / oblivious corridor checker 224, which can use that information to provide a transformed (corrected) trajectory with a maximum deceleration to the TMP 222.

[0043] In addition to receiving messages regarding the status and operation of those actuator systems from the actuator system and receiving a trajectory type request message from the system monitor AI 210, the system monitor 220 may receive a system monitor release signal in a received message from the teleoperator system. Based on that input, the system monitor 220 may output a trajectory limit message to the TMP 222 that restricts the trajectories that the TMP 222 can select. The trajectory limit message may be an instruction that the TMP 222 should not execute the nominal (e.g., primary) trajectory and should execute a trajectory that results in the autonomous vehicle reaching a stationary position or another trajectory. The trajectory limit message may be an instruction for the TMP 222 to execute an emergency stop trajectory.

[0044] The system monitor 220 can also output a clearance message such as a "Clear to Start" message and send a signal to the TMP222 that it can reselect the highest level orbit. This can occur when the TMP222 uses a lower level orbit due to a previous request by the system monitor 220, or when a higher level orbit may be used because it is not valid and collision-free. The TMP222 maintains that level until it is released by either a "Clear to Start" message or a system monitor release signal. In some examples, a system monitor release signal is required for release. In some examples, the system monitor release signal is received by the system monitor 220 from a human operator using a teleoperator system after verifying data from the vehicle. In some examples, the system monitor release signal is received directly by the TMP222.

[0045] As described in more detail elsewhere in this specification, the TMP222 determines, from the orbits available to the TMP222, the level of the orbit in the orbit hierarchy, the level that the TMP222 caps, and the validity and collision-free status of the orbit. The TMP222 evaluates the orbits for validity, evaluates whether they will be collision-free, and selects one of those orbits, or an orbit that the TMP222 has stored, or an orbit that the TMP222 has generated by modifying another orbit (e.g., a collision avoidance orbit or an E-stop orbit). The verification process may include a timeliness test, an obsolescence test or another consistency test, and a feasibility test 234, as described in more detail below. In other examples, the TMP222 may include some or a different combination of verification and collision checks, or other checks. Next, the TMP222 outputs an orbit data message (or otherwise called an orbit record) to the drive manager. Some of the orbit processing may be performed by the TMP222 and some may be performed by the drive manager.

[0046] Collision check 236 may include execution of a free space checker, evaluation of an orbit predictor based on an object list, a processor using machine learning in a heat map, a Kalman filter, data association of sensor data and object tracking determined by an AI engine 206, or other approaches. Primary orbits, accidental orbits, and stored accidental orbits, and / or modifications thereof, may be checked periodically for possible collisions. Other orbits may not be checked. For example, an emergency stop (“E-stop”) orbit need not be repeatedly checked to determine whether the orbit will result in a collision. This is because the orbit is likely a last resort, has the goal of decelerating, and likely already represents the last option to come to a stop by locking the steering and applying maximum braking, and is likely only used in the case of multiple system failures or a suspect status.

[0047] In some cases, the orbit record may be an example of a validity signal, such as for a valid orbit. In other cases, such as when an orbit is determined to be invalid, a validity signal indicating such may be communicated as feedback to the primary calculation unit 202 or the orbit planner AI 212. In other cases, the validity signal or value may be used internally in the TMP 222. When generating a validity value or signal, the TMP 222 may output a valid orbit as an orbit record to the primary calculation unit 202, or may be able to select another orbit for evaluation if it is determined that the tested orbit is not valid. It should be understood that validity values and validity signals may be used interchangeably throughout.

[0048] In some aspects, the system monitor 220 may receive various inputs regarding the states of other systems on the autonomous vehicle. The inputs may include heartbeat signals, fault notifications, and reduced-capability notifications from components that are expected to be active and are important for the proper operation of the autonomous vehicle. In some aspects, the system monitor 220 can send status messages to the system monitor AI 210, enabling the system monitor AI 210 to process the status updates and determine any restrictions on the potential trajectories of the autonomous vehicle. In some cases, this can take the form of updating one or more of the vehicle's capability limitations, such as maximum acceleration, deceleration, yaw, and other kinematic constraints on the movement of the vehicle, based on the status updates. The system monitor AI 210 then determines whether any trajectory type should be restricted or constrained from the selection and indicates so in a trajectory type request message to the system monitor 210, which can then indicate the restrictions to the TMP222. In some cases, the request for the trajectory type can indicate the most permissive trajectory that the vehicle can safely take, enabling the TMP222 to then verify the indicated trajectory or a more restrictive trajectory (e.g., a collision avoidance trajectory or an emergency stop trajectory). In other aspects, the system monitor AI 210 can determine to what extent, e.g., how much to restrict, one or more of the vehicle's capability values based on the inputs, whereby the system monitor 220 may update the set of the vehicle's capability limitations. In this scenario, the TMP222 can request capability or feasibility limitations from the system monitor 220 when verifying the feasibility of one or more trajectories, and its determination may be based on the feasibility limitations.

[0049] Figure 3 is a block diagram 300 that more particularly shows the elements of the TMP222 from FIG. 2, including a collision checker 304, a trajectory validator 302, a trajectory status manager 320, and a trajectory selector 330. In some examples, the TMP222 may include separate processes and systems for collision avoidance.

[0050] The trajectory validator 302 may include a consistency validator 310, a timeliness validator 312, a feasibility validator 314, and an obsolescence validator 316, and potentially other validators (not shown) that can perform other tests on the trajectory. The collision checker 304 checks the trajectory to determine whether it is collision-free. The results of the validity check and the collision check may be provided to the trajectory status manager 320, and the trajectory status manager 310 may store the trajectory in the trajectory storage 322 along with those results.

[0051] In a general case, TMP222 has a plurality of trajectories available in TMP222, some of which may be received from the AI system of the primary calculation unit 102, some of which are stored from a previous frame, some of which are modified versions of the received trajectories where the received trajectories are transformed and the transformation and / or the transformed trajectories are stored in the trajectory transformation storage 326, and some of which may be relatively fixed trajectories (such as emergency stop type trajectories). The trajectories provided by the AI may be generated by the AI system based on sensor data, and the sensors may provide information regarding the road, weather, surrounding environment, etc. The provided trajectories may include trajectories that are expected to be nominal trajectories and one or more contingency trajectories in case the nominal trajectories are not available.

[0052] The tracks stored in the track storage 322 can include nominal tracks and various incidental tracks that may or may not end when the vehicle is stationary. The stored incidental tracks can be incidental tracks from previous frames received and stored by the TMP222, perhaps those that the TMP222 receives, selects not to use, and instead uses the received nominal track. Another example is an incidental type of track (i.e., a collision avoidance track) that is derived from the received incidental track but is modified to avoid a possible collision by, for example, repeatedly increasing the braking applied up to the maximum deceleration, making small adjustments in the vertical direction, and / or making other minor changes without completely abandoning the track. Yet another example is an incidental type of track that is derived from the stored incidental track but is modified in a similar manner, a converted and stored incidental type of track. Still another example is an emergency stop type of track where, instead of repeatedly increasing the braking with the steering locked, the maximum declaration is immediately applied.

[0053] The validity value and the collision value can be determined from the validity constraints and data regarding the previously or currently playing track. Using these values, the track selector 330 can select a track, which can provide a binary value and an indicator of the track selected based on these binary values.

[0054] In some examples, a system monitor (such as the system monitor 220 and / or the system monitor AI 210 described above) can monitor the health of the components of the autonomous vehicle and issue a request regarding which of the available tracks to use. For example, the system monitor may identify a situation that is not related to safety but requires an emergency stop and then indicate that an incidental track that will ultimately bring the autonomous vehicle to a stop should be used. This request can be related to the health of the component / sub-component. In at least some examples, additional information (such as, but not limited to, weather, friction, etc.) can be used by the system monitor in determining the requested track.

[0055] In some embodiments, the system monitor may obtain status updates from various subsystems of the vehicle and update a set of feasibility limits, such as acceleration and deceleration limits in both the primary direction of travel (longitudinal) and lateral directions. The set of feasibility limits may initially be set to default values, such as those specified by the component manufacturer of the vehicle's components, empirically determined values, etc. These values may then be modified based on the determined impact that one or more operating statuses have on the vehicle's feasibility limits. As a non-limiting example, if inclement weather is detected, the maximum acceleration / deceleration may be reduced by 10%.

[0056] In one example, when TMP222 uses an incidental trajectory, TMP222 will record the level of the trajectory it used in state variable storage 324, either because a higher trajectory failed a validity test or collision check, or because the system monitor indicated that an incidental trajectory should be used, and will not use a higher level trajectory until a release signal is sent. The release may have different levels of release, which may vary depending on the severity of the anomaly that led to the use of the incidental trajectory. For example, if the anomaly was a minor power drop in one of the batteries but the power returned to normal on its own, it may be an anomaly of one level, and the system monitor AI system may be given the authority to issue a release.

[0057] More generally, the system monitor sends a trajectory limit message to TMP222 indicating that it should not use a trajectory at a level that exceeds the limits specified in the trajectory limit message until a release is issued. For example, the trajectory limit message may indicate that a trajectory higher than a first incidental trajectory should not be executed. In that case, TMP222 will resume the execution of the nominal trajectory at a higher place in the trajectory hierarchy.

[0058] Once the release occurs, TMP222 can then use the highest level of trajectory that passed its test. For some exceptions, such as when all trajectories are found to be invalid when checked and TMP222 has to use an emergency stop trajectory, a higher level of release may be required, perhaps. Such a higher level may require a human to check the situation and a human to handle the exception clearance.

[0059] Trajectory storage 322 may include storage for various trajectories that can be selected by trajectory selector 330, each containing a type indicator, data describing the trajectory, a confirmation value indicating whether TMP222 has determined the trajectory to be valid, and a collision or non - collision value indicating whether TMP222 has determined the trajectory to be collision - free. In some examples, there may be no explicit storage for each of the data objects shown.

[0060] The orbit selector 330, as described in more detail herein, selects an orbit, perhaps from the orbit storage 322, based on whether the orbit is valid, collision-free, and possibly other circumstances, and then transmits a message with the proposed orbit to the drive manager. The orbit selector 330 takes into account the state variables in the state variable storage 324 that represent the "current level" of the TMP222. As described elsewhere in this document, the TMP222 evaluates the orbits and uses the highest orbit in the hierarchy unless its state indicates that it should not exceed the specified level. Examples of the hierarchy are from the nominal orbit to the emergency stop orbit. The specified level may be due to the system monitor 220 indicating the highest allowable level, or because the TMP222 processes the orbits and discovers that the highest level orbit is not valid or contains a collision. In some examples, the current level of the TMP222 remains the same from frame to frame if the orbit at the current level is valid and collision-free and no external signal to lower the level is received, and the current level of the TMP222 decreases if the orbit at the current level is not valid or collision-free or an external signal to lower the level is received. In such cases, the TMP222 looks for the "clear to proceed" signal of the system monitor and / or the release signal of the system monitor.

[0061] When the orbit selector 330 selects an orbit, it transmits a data message that matches the selected orbit to the drive manager. If the orbit is not available or for other reasons, the orbit selector 330 may transmit a failure message to the drive manager and / or may result in a default "E-stop". The orbit selector 330 may also provide its output as feedback to the safety / insensitive collision checker 224.

[0062] In some situations, the trajectory status manager 320 examines a trajectory, determines that it is valid but not collision - free, and determines that a converted trajectory would be collision - free. For example, there can be a hierarchy that goes from highest to lowest: nominal trajectory, first contingency trajectory, stored contingency trajectory, converted contingency trajectory which is a conversion from a valid first contingency trajectory with a collision to a collision - free trajectory, converted stored contingency trajectory which is a conversion from a valid stored contingency trajectory with a collision to a collision - free trajectory, and emergency stop trajectory. Examples of conversions include applying additional braking to the trajectory to make it collision - free. In such an example, the TMP222 can either continue to execute such a trajectory to the end point or be released in other ways according to the techniques described herein.

[0063] The trajectory conversion storage 326 can store details of such conversions if needed in a future frame. The converted contingency trajectories are stored as complete trajectories and can be used in future frames. In that example, if the first contingency trajectory is converted to a converted contingency trajectory and that converted contingency trajectory is used, there is a release such that the TMP222 can use a higher - level trajectory until the converted contingency trajectory is reused in a future frame until there is a release where the converted contingency trajectory is no longer valid and collision - free, in which case a lower - level trajectory will be used. In some examples, a separate trajectory conversion storage is not used and the converted trajectories are simply stored in the trajectory storage 322.

[0064] The trajectory selector 330 can update the state variable storage 324 to indicate a new maximum allowable trajectory level when it receives a system monitor "clear and go" signal and / or a system monitor release signal. In a normal process, the system monitor clears the TMP222 to use the highest - level available trajectory, but in some processes, the system monitor may partially clear the TMP222 to a level above its current level but not to the highest level.

[0065] Figure 4 is a block diagram 400 showing elements of a drive manager 402 that interfaces with an actuator or actuator system 404 of an autonomous vehicle. As shown therein, the drive manager 402 may include a trajectory tracker 406 that maintains a data structure including trajectories passed to the drive manager 402 from a trajectory selector such as the trajectory selector 330 shown in FIG. 3. The drive manager 402 may also include a control manager 408.

[0066] Examples of actuators within the actuator system 404 may include a steering system 410, a friction braking system 412, an inverter 414, a traction system 416, an electric parking brake system 418, and an active suspension controller 420. Each of these systems may affect one or more performance limitations of the vehicle such that a reduced operating capacity decreases the amount by which the vehicle can accelerate or decelerate in a primary direction of travel and / or laterally.

[0067] During operation, the drive manager 402 may output actuator command messages to the actuator system 404 and may receive feedback messages from the actuator system 404. Such feedback messages may be used by a system monitor in determining a requested trajectory, as described in more detail below.

[0068] Figure 5 is a block diagram showing an alternative view 500 of the system monitor 220 and the TMP 222. Figure 5 does not necessarily show all data structures that may exist, and some of those shown may be logical data structures and / or remote or distributed data structures. As shown therein, the system monitor 220 may have a communication system 504 that interfaces with a telematics system 508 remote from the autonomous vehicle, and may present information to a display 510 used by a telematics operator 512 to consider information and provide signals and inputs to the autonomous vehicle.

[0069] The requirements for what type of release signal is sufficient to release the TMP for higher-level orbit use may, for some levels, require that the release signal can only result from human interaction. The human interaction can be from a remote telematics operator using a system that provides telematics data to humans. In at least some examples, such a telematics operator may be equipped with a more powerful machine that can perform more advanced diagnostics to issue a clearance message.

[0070] As an example, after the system monitor 220 detects that some values or situations are outside the normal operating range and sends an inquiry to the telematics system 508 while detecting an abnormal situation of the autonomous vehicle or the environment, it may send a trajectory type request message 502 that requires an accidental trajectory leading to finally stopping and staying in a stationary position to the TMP 222. In some examples, the abnormal situation may be so severe that the telematics operator 512 should not start moving until the abnormal situation is sufficiently cleared after evaluating the situation and the autonomous vehicle reaches the stationary position.

[0071] For example, if the system monitor 220 detects a failure that should lead to the stop of the autonomous vehicle for safety reasons and issues a trajectory type request to the TMP 222, but then the telematics operator 512 determines that the failure no longer exists or is not an executable failure, the telematics operator 512 can instruct the telematics system 508 to send a system monitor release signal (e.g., a "release to start" message) to the system monitor 220. This mechanism creates a situation where once the autonomous vehicle stops, the system monitor 220 will instruct the TMP 222 to maintain the stop state until the system monitor 220 receives the "release to start" message.

[0072] FIG. 6 is a drawing 600 showing an exemplary interaction between the trajectory planner AI 212 of the primary computing unit 202, the TMP 222 of the secondary computing unit 204, and the system monitor 220, as described in more detail above with respect to FIGS. 2 and 3. As described above, the system monitor 220 operates in cooperation with the system monitor AI 210, updates one or more feasibility constraints, and notifies the updated one or more feasibility constraints to the TMP 222 to enable verification of a trajectory based on the updated feasibility constraints. In other examples, the system monitor 220 may include the functionality of the system monitor AI 210 and may be resident in either the primary or secondary computing unit 202, 204. In the illustrated interaction, the trajectory planner AI 212 may determine one or more trajectories, and the TMP 222 may select one of those trajectories based on a validity check, a collision-free check, and / or a request from the system monitor 220 that requests a particular type(s) of trajectory.

[0073] In the exemplary process flow shown, the trajectory planner AI 212 may issue several trajectories in operation 602, which may include one or more of different types of trajectories such as primary and secondary trajectories. For example, the trajectory planner AI 212 may issue, at one interval or over one period, (a) a primary trajectory (and / or a set of primary trajectories that would take the vehicle to its destination when executed by the vehicle) to continue the vehicle on its mission, and (b) a secondary (or contingency) trajectory to stop the vehicle at some point due to an abnormal situation.

[0074] In operation 604, TMP222 may receive a trajectory from the trajectory planner AI212. In some cases, TMP222 may access other available trajectories that are not necessarily received from the trajectory planner AI212 within the current time frame, such as, for example, stored secondary trajectories, or even trajectories at least partially determined by TMP222. Examples include stored secondary types of trajectories (e.g., after verification), which are secondary trajectories from received and stored previous timestamps, and perhaps trajectories that TMP222 has received and elected not to use and instead uses another trajectory. Another example is a collision avoidance trajectory or a modified or transformed incidental trajectory based on another trajectory such as a secondary trajectory or a stored secondary trajectory. The transformed incidental trajectory is derived from the received secondary trajectory but may be modified to avoid a possible collision, for example, by repeatedly increasing the braking applied up to the maximum deceleration, adjusting vertically by a small distance, or making other minor changes without completely abandoning the trajectory. Yet another example is an E-stop type of trajectory where, instead of repeatedly increasing the braking with the steering locked, the maximum declaration is immediately applied.

[0075] In some cases, simultaneous with or at a different time than the receipt of the trajectory by TMP222 in operation 604, or with other operations of TMP222, the system monitor 220 may receive, in operation 606, one or more status updates from one or more of the systems described above with reference to FIG. 4, or associated therewith, of one or more vehicle systems or subsystems. These status updates may include, for example, periodic updates or heartbeat operations of one or more subsystems and may indicate that the subsystem is fully functional. The status updates may also indicate whether the system is operating at reduced capacity, is malfunctioning, and / or other information related to the operation of the vehicle.

[0076] Based on these status updates, the system monitor 220 may determine or update a set of existing vehicle feasibility or capability limitations in operation 608 and send them to the TMP 222. Then, the TMP 222 may receive the feasibility limitations in operation 610 and use them for orbit evaluation in operation 616. This may include determining the amount to update one or more feasibility limitations based on the status of one or more subsystems. In some examples, the operating status of various subsystems of the vehicle may be determined as a matter of degree (e.g., via a percentage, etc.). Various ranges of percentages of operation of various subsystems may correlate to the vehicle's capability limitations. For example, tire pressure above 35 PSI for four tires may correlate to 100% of longitudinal acceleration, deceleration, and lateral acceleration. Tire pressure between 25 and 35 PSI may correlate to 75% or the like of one or more of their capabilities. Various subsystems of the vehicle can be monitored, and then the subsequent impact on one or more operating limitations can be determined. The impact that may be represented by a change in an operating limitation can be determined empirically, set by a component or vehicle manufacturer, or via other means.

[0077] In some aspects, the system monitor 220 may, in some aspects, check with the telematics system as needed and determine the type of orbit requested in operation 612. This may be in the form of determining the highest level of orbit that the system monitor would allow the vehicle to select. Next, the system monitor 220 may issue an orbit request to the TMP 222 in operation 614, which may be in the form of an inter-process message. It should be understood that in some cases, the TMP 222 may not anticipate or even require input from the system monitor 220 for orbit selection. For example, the TMP 222 may receive system or status updates directly or through another entity or system, etc.

[0078] In operation 616, TMP222 may evaluate the received trajectory, such as by performing one or more validity checks and / or collision checks. In some aspects, operation 606 may include performing one or more of a timeliness test, a consistency test, an obsolescence check, a feasibility test, and / or a collision test on each trajectory received in operation 604, as described in more detail below. In some cases, one or more of the tests may be performed by one or more of the trajectory validators 302, such as the consistency validator 310, the timeliness validator 314, the feasibility validator 314, and / or the obsolescence validator 316, as described above with reference to FIG. 3.

[0079] In some cases, when any trajectory request is determined or transmitted from the system monitor 220, TMP222 may further evaluate the trajectory based on the trajectory request in operation 616. In some aspects, TMP222 may select a trajectory in operation 618 based on the trajectory request, whereby the selection may be limited by the highest trajectory indicated by the trajectory request. TMP222 may execute or instruct other subsystems of the vehicle, such as the primary computing unit 202, to execute the trajectory in operation 620.

[0080] In some aspects, TMP222 may wait to receive a trajectory request from the system monitor 220 in order to evaluate the trajectory in operation 618. In other aspects, TMP222 may not have to wait for a trajectory request from the system monitor 220 before selecting a trajectory in operation 620.

[0081] In some cases, the TMP222 may evaluate the validity of the received trajectory and whether the trajectory is collision - free. In at least some examples, the result may be represented by a binary value (e.g., true or false, or 1 and 0), but herein it is contemplated that any set of values or keying system may be used for similar effects including, but not limited to, uncertainty information, error bars, etc. As explained throughout this disclosure, the validity signal or value can represent the total score of any number of different tests or operations such as timeliness checks, consistency checks, staleness checks, feasibility checks, or other tests useful for ensuring the safe operation of a vehicle or device. In some cases, for a true or positive value to be assigned, the trajectory must pass all validity tests.

[0082] In some aspects, using pre - defined trajectory selections to select and verify trajectories can provide a number of advantages and / or benefits. For example, higher consistency in trajectory selection may be achieved through using a more comprehensive set of trajectory selections when compared to a rule - based approach. Further, in some examples, by using a pre - defined set of values such as a table or other data structure, rather than a rule - based approach, the time for verifying, for example, valid and collision - free trajectories may be reduced.

[0083] Figure 7 is a state diagram that can be implemented in a TMP such as the TMP222 described above, which processes, selects, and / or modifies trajectories as described above in operation 618. Such a state diagram can illustrate how a system (such as an autonomous vehicle system) moves from one operating mode (e.g., a nominal mode) to an incident mode or a backup mode. The TMP may receive requests from one or more trajectories for a frame and / or a system monitor such as the system monitor 220 that direct a requested trajectory for the frame. Any of these may be received synchronously and / or asynchronously.

[0084] An operating mode may have a trajectory associated with that mode. For example, a nominal mode can have a trajectory that moves the system along a route, such as moving an autonomous vehicle along a route from a starting location to a destination while observing constraints related to the road, the autonomous vehicle, and / or the occupants, while an incidental mode can have a trajectory associated with a trajectory that moves the autonomous vehicle along a route that brings the autonomous vehicle to a stop while avoiding collisions.

[0085] In the state diagram of FIG. 7, the states correspond to the operating modes, and the TMP may store a state variable that indicates the state. Alternatively, the TMP may derive that state from other information. As described herein, in the TMP, several trajectories may be available in any given frame.

[0086] When in the first state (shown as "State A" in FIG. 7), the TMP checks the received nominal trajectory, which can be called a primary trajectory against one or more verification tests such as those shown in FIG. 3. From State A, if the trajectory is valid and collision-free and the system monitor permits the nominal trajectory (such as by not sending a trajectory limit message or indicating that the TMP is cleared to use any trajectory), the TMP will process that trajectory and set its state variable to indicate that state. The TMP can process the trajectory by executing the trajectory or passing it to another subsystem for execution. In the next frame, the TMP can obtain the trajectory for that next frame and return to State A.

[0087] The TMP can receive multiple nominal trajectories along with the incidental trajectories, and in some cases, can also receive one or more trajectories generated by itself, such as by modifying the received trajectories. In the examples described herein, the TMP may receive only one nominal trajectory. The TMP can check other trajectories to determine whether they are valid and collision-free in each frame, and in some examples, can check other trajectories only when the TMP determines that a trajectory higher in the trajectory hierarchy may be invalid or not collision-free, such as when the nominal trajectory is not valid or not collision-free, or when the system monitor indicates that the TMP should not use the nominal trajectory (as shown in FIG. 7 as a system monitor that sends a signal other than "Clear to Start"). The nominal trajectory may involve a stop, such as when the vehicle comes to a stop at a stop sign under normal circumstances or to pick up or drop off passengers.

[0088] From state A, if any of these situations are not met, for example, if the TMP determines that the nominal trajectory may not be valid or not collision-free, or if the system monitor indicates that the TMP should not use the nominal trajectory (as shown in FIG. 7 as a system monitor that sends a signal other than "Clear to Start"), the TMP transitions to state B where it checks the validity and collision of the received first incidental trajectory. If the first incidental trajectory is both valid and collision-free, the TMP moves to state G, executes its first incidental trajectory (here too, either by executing the trajectory or passing the trajectory to another subsystem for execution), and upon completion, moves to state H. In this example, state H is the state where the autonomous vehicle is stationary. As shown in FIG. 8, the TMP maintains this state until it receives a "Clear to Start" signal from the system monitor and / or a system monitor release request from a system monitored by a human, such as a telematics system.

[0089] In state B, if the TMP determines that the first contingency trajectory is not valid, the TMP transitions to state C and attempts the second contingency trajectory, which could be the remembered contingency trajectory from the previous frame. In state B, if the TMP determines that the first contingency trajectory is valid but not collision-free, the TMP transitions to state D and attempts to correct it by converting the first contingency trajectory to a third contingency trajectory.

[0090] In state C, the TMP checks the validity and collision of the second contingency trajectory. If the second contingency trajectory is both valid and collision-free, the TMP moves to state J, executes its second contingency trajectory (either by executing the trajectory or passing it for execution), and upon completion, moves to state K. In this example, state K is also the state where the autonomous vehicle is stationary. Since the TMP also maintains a state variable indicating the tolerance level within its hierarchy, when the TMP reaches state C, in future frames, it skips states A and B until a release is received.

[0091] In state C, if the TMP determines that the second contingency trajectory is not valid, the TMP transitions to state F and executes an emergency stop trajectory, which should be rare. This is because it requires both the nominal trajectory to be infeasible and both contingency trajectories to be invalid.

[0092] In state D, the TMP checks the validity and collision of the third contingency trajectory. If the third contingency trajectory is both valid and collision-free, the TMP moves to state M, executes its third contingency trajectory (either by executing the trajectory or passing it for execution), and upon completion, moves to state N. In this example, state N is also the state where the autonomous vehicle is stationary. Since the TMP also maintains its tolerance level within the hierarchy, when the TMP reaches state D, in future frames, it skips states A, B, and C until a release is received.

[0093] In state E, the TMP checks the validity and collision of the fourth contingency trajectory. If the fourth contingency trajectory is both valid and collision - free, the TMP moves to state P, executes its fourth contingency trajectory (either by executing the trajectory or passing the trajectory for execution), and upon completion, moves to state Q. In this example, state P is also the state where the autonomous vehicle is stationary. The TMP also maintains state variables indicating its tolerance levels within the hierarchy, so when the TMP reaches state E, in future frames it skips states A through D until a release is received.

[0094] In state D or E, if the TMP determines that it cannot convert the contingency trajectory into a collision - free transformed trajectory (or something has occurred to make it invalid or it has been rendered invalid by the conversion), the TMP moves to state F. Also, if the system monitor sends a message to the TMP to execute an emergency stop in response to perhaps detecting a system error, a mechanical error, an electrical error, etc., the TMP moves to state F.

[0095] Thus, as described with reference to FIG. 7, the TMP can identify the states it should be in and the trajectories it should execute. Since each of the contingency trajectories is likely in response to an error or an unexpected situation, the TMP will not resume normal operation until it receives a release signal.

[0096] FIG. 8 is a state diagram that can be implemented in a TMP, representing the state of the TMP when an orbit other than the nominal orbit is selected. Examples of states where the TMP is executing a certain type of accidental orbit include state G (when the TMP is executing a first accidental orbit that is a valid and collision - free orbit received from the orbit planner AI), state J (when the TMP is executing a stored accidental orbit that is a valid and collision - free orbit previously received from the orbit planner AI), state M (when the TMP is executing a converted version of the first accidental orbit that has been collision - free), state P (when the TMP is executing a converted version of the stored accidental orbit that has been collision - free), and state F (when the TMP is executing an emergency stop orbit).

[0097] In states other than state P and state F, after the autonomous vehicle has stopped (or, in some examples, has a non - zero speed that is lower than or equal to the threshold speed), if the system monitor requests a nominal type of orbit, the TMP may return to state A, but a release signal from the telematics system is required for the transition to state A. For example, in state F, when the TMP executes an emergency stop and transitions to state R where it is in a stationary state, in state S, the TMP will remain in that state until it receives a release request issued by a human operator, and the autonomous vehicle will remain stationary.

[0098] As shown in FIG. 8, while there are some states that can be brought about by the system monitor clearing an abnormal situation, which can cause the autonomous vehicle to move again or exit from a state that can lead to the stop of the autonomous vehicle, there are also other states where it is necessary to receive a release signal from the telematics system in order to cause the autonomous vehicle to move again.

[0099] In a particular example, the nominal trajectory is labeled as the primary trajectory, the first incidental trajectory is labeled as the secondary trajectory, the second incidental trajectory is the stored incidental trajectory, the third incidental trajectory is a transformation of the first incidental trajectory, and the fourth incidental trajectory is a transformation of the second incidental trajectory, although other possibilities for the nominal trajectory and incidental trajectories are conceivable. It is not necessary for the nominal trajectory to continue the movement of the vehicle, nor is it necessary for all incidental trajectories to bring the vehicle to a stop.

[0100] In a more general example, there is a nominal mode in which the nominal trajectory is executed, and a hierarchy of incidental trajectories in which the highest level of trajectory that is valid and collision - free is executed and the system remains at that level until the TMP is released. In a general example, the levels are such that higher levels can be released by an automated system, and lower levels may require human review and intervention to be released to a level higher than the level of the ultimately executed trajectory.

[0101] In some examples, the release from the system monitor may not be a complete and unconditional release (for example, allowing the TMP to return to the highest level and state A). Instead, the release from the system monitor may be to a level that is somewhat higher than the current TMP level but lower than the highest level.

[0102] FIG. 8 shows that the system monitor can clear from states H, K, and N. However, in some examples, the TMP may be programmed to enable the release of the system monitor from states G, J, and M, which are the states in which the autonomous vehicle is moving, although this may require more than the release of the system monitor from states H, K, and N. Other variations should become apparent upon reviewing the figures and descriptions herein. Further, although the system monitor is shown to clear to the primary trajectory, the system monitor may clear to any other level as reflected in the state diagram described with respect to FIG. 7.

[0103] FIG. 9 shows an exemplary process 900 for verifying alternative trajectories and selecting a trajectory based on the verification, which process can be performed by an autonomous vehicle or device, as described above. In some examples, process 900 can be performed by the planning subsystem 1628 and / or by the TMP222 and / or the system monitor 220 described above, as described below with reference to FIG. 16.

[0104] Process 900 can begin with operation 902, in which at least two alternative trajectories for the device can be determined or received. In some cases, operation 902 can be performed by the TMP222.

[0105] Next, in operation 904, one or more verification tests or evaluations can be performed on the alternative trajectories, for example, by the TMP222. The verification tests can include one or more of validity tests such as timeliness, consistency, staleness, and / or feasibility tests, and / or collision tests, as described in more detail below.

[0106] Next, in operation 906, a data set can be generated based on the verification tests. In some cases, the TMP222 may generate the data set. The data set can include the results of the verification tests performed on the alternative trajectories.

[0107] Based on the data set, a trajectory can be selected in operation 908. Operation 908 can include determining the selected trajectory based at least in part on a validity and / or collision check. In at least some examples, such a selected trajectory can include the highest level of trajectory that is both collision-free and valid, such as in accordance with the state diagrams 700 and 800 described above with reference to FIGS. 7 and 8. In various examples, an emergency stop (E-stop) may be selected if there are no valid and collision-free trajectories.

[0108] Next, in operation 910, the device may be instructed or caused to operate according to a selected trajectory. Operation 910 may include TMP222 sending the selected trajectory to the primary calculation unit 202, the drive manager 402, and / or the actuator 404, or other subsystems of the device or vehicle to cause the device to follow the selected trajectory.

[0109] FIG. 10 shows another exemplary process 1000 for verifying alternative trajectories and selecting a trajectory based on the verification, which may be executed by an autonomous vehicle or device as described above. In some examples, process 1000 may be executed by the planning subsystem 1628 and / or by the TMP222 and / or the system monitor 220 described above, as described below with reference to FIG. 16.

[0110] Process 1000 may begin at operation 1002 where first and second alternative trajectories for the device may be obtained. In some cases where the device is an autonomous vehicle, the first and second alternative trajectories may be obtained from the trajectory planner AI 212 described above, or from other trajectory generation computing devices or processes.

[0111] Next, in operations 1004 and 1006, validity values can be determined for each of the first and second alternative trajectories. In some embodiments, the determined validity values may be binary values in order to facilitate or reduce the complexity of analysis and / or comparison and increase computational efficiency. In other embodiments, the validity values may include more than three values, for example, to increase the accuracy of trajectory selection. Operations 1004 and 1006 can include performing any of several separate or interconnected tests or evaluations. For example, validity can be determined by performing a timeliness check to confirm that a trajectory was received within a certain period, a consistency check to confirm that a future trajectory matches or conforms to, or is within the defined physical parameters of, the current trajectory of the device (e.g., steering to the trajectory), a feasibility check to confirm that the vehicle can physically navigate along and on the trajectory (e.g., within speed and acceleration limits), an obsolescence check to confirm that the trajectory was generated within a certain period, and / or other validity tests. In some embodiments, operations 1004 and 1006 can be performed by the consistency validator 310, timeliness validator 312, feasibility validator 314, and / or obsolescence validator 316 of the TMP222 described above.

[0112] Next, in operations 1008 and 1010, a collision value can be determined for each of the first and second trajectories. In some cases, operations 1008 and 1010 can be performed by TMP222 and / or the collision checker 304. In some aspects, operations 1008 and 1010 can include rejecting a trajectory that TMP222 determines will include an imminent collision with an agent or object. Operations 1008 and 1010 can include execution of a free space checker, evaluation of a trajectory predictor based on an object list, a processor using machine learning in a heat map, or other approaches. In some cases, one or more of operations 1004 and 1008 can be performed in parallel with, simultaneously with, or even substantially simultaneously with (e.g., within the technical tolerance) operations 1006 and 1010.

[0113] Next, in operation 1012, based on the validity and collision values determined in previous operations, a trajectory can be selected, for example, by selecting the highest priority trajectory that is collision-free and valid according to the state diagrams 700 and 800 described above. In some aspects, operation 1012 can include a comparison of a data set that includes the validity and collision values of at least two alternative trajectories.

[0114] Next, in operation 1014, the device can be instructed or caused to operate according to the selected trajectory. Operation 1014 can include TMP222 sending the selected trajectory to the primary calculation unit 202, the drive manager 402, and / or the actuator 404, or other subsystems of the device or vehicle to cause the device to follow the selected trajectory.

[0115] FIG. 11 shows an exemplary process 1100 for verifying a trajectory, which may be executed by an autonomous vehicle or device as described above. In some examples, process 1100 may be executed by planning subsystem 1628 and / or by TMP222 and / or system monitor 220 as described below with reference to FIG. 16.

[0116] Process 1100 can begin at operation 1102 where a trajectory of the autonomous vehicle can be received. Operation 1102 can be executed by TMP222, such as from trajectory planning AI 212 or other components or subsystems of the autonomous vehicle.

[0117] At operation 1104, a feasibility value can be determined for the trajectory. In some aspects, operation 1104 can be executed by feasibility validator 314. The feasibility value can be determined based on some operating characteristics of the vehicle, as described in more detail below with reference to FIGS. 14 and 15. In some examples, the feasibility value can be related to whether it is physically possible for the vehicle to navigate a potential trajectory, such as based on the vehicle's maximum speed, maximum deceleration in one or more directions, maximum acceleration in one or more directions, maximum combined acceleration, or some other characteristics (turning radius, and / or other kinematic constraints). In some cases, the feasibility value used to evaluate a potential trajectory can be updated based on the current operating situation of the vehicle and its subsystems, or can be defined via an external or objective metric.

[0118] At operation 1106, a timeliness value for the trajectory can be determined. In some aspects, operation 1106 can be executed by timeliness validator 312. In some cases, timeliness can be defined, for example by TMP222, by how much time has elapsed since a previous trajectory was received. In some cases, operation 806 can include determining a second time difference between the time the trajectory was received and the time a previous trajectory was received.

[0119] Timeliness can be used as an indicator of whether various subsystems of a vehicle or device are functioning properly and are within expected bounds. For example, if new trajectories are normally received within a certain period or at a certain frequency, trajectories received outside that period, or not received at all, may indicate a malfunction or problem with one or more systems of the autonomous vehicle. Thus, using timeliness and other similar checks as indicators of the vehicle's operating status can be particularly useful in improving the safety of the vehicle's operation. In some cases, one or more periods or limits indicating whether the received trajectories are timely may be defined. These limits may be common to all trajectories, or may be set depending on whether the trajectory is a primary trajectory or an incidental trajectory.

[0120] In some aspects, timeliness can be defined in terms of the number of excluded trajectories. The number of excluded trajectories may follow the passage of a time interval at which trajectories are normally received, such as every tenth or hundredth of a second. In some cases, one excluded trajectory may indicate a malfunction or other situation that causes the trajectory to become invalid. In other cases, two or a different number of excluded trajectories may indicate a problem and may cause the timeliness test to fail. Further, in some cases, different periods or numbers of excluded trajectories may be set for different types of trajectories (e.g., primary, incidental, E-stop, etc.). For example, for primary trajectories, it may be determined that the trajectory is still timely or passes the timeliness test even if two or fewer are excluded. Incidental trajectories may be timely and valid only if one or fewer trajectories are excluded. In this way, the techniques described can utilize timeliness tests to verify potential trajectories, and the limits or metrics used to determine whether a trajectory passes or fails a timeliness test can be configured in many different ways.

[0121] Timing checks may be performed to verify that an orbit was recently received. For example, it may include comparing the time an orbit was received from the orbit planner AI212 by TMP222 with the current time that can be maintained by TMP. For example, there may be a possibility that a newly created orbit follows after a radio wave stop for X seconds. In this case, the orbit will fail the timing test but will pass the aging test as described in more detail below. In other words, the timing test as described herein ensures that new orbits are received regularly.

[0122] In operation 1108, the validity of consistency, represented by a value such as the binary value of an orbit, for example, can be determined. In some cases, operation 1108 can be performed by the consistency validator 310. The consistency validity or the consistency validity value can generally indicate whether a potential orbit is within a certain physical relationship with respect to the current orbit on which the vehicle is traveling. In some cases, the specific physical relationship can be defined by one or several different values, which are modified by the operating characteristics or limitations of the vehicle (e.g., the feasibility value). The values can include a distance difference, a time difference, a steering threshold, a yaw threshold, a speed deviation threshold, or a collinearity threshold. Since the comparison is based on the orbit rather than the state of the vehicle, such validity may be different from the feasibility validity. In such a comparison, during the nominal operation of the vehicle, it is expected that subsequent orbits do not deviate significantly from previously generated orbits (if generated fast enough, e.g., every 100 ms or faster). Thus, in such an example, a deviation flag can be set as an indication of an error in the planner subsystem. In some aspects, small deviations may be acceptable, and one or more threshold deviation values can be used to determine whether the difference will contradict the orbit with the previous orbit.

[0123] Determining the consistency validity value may include obtaining the current state of the vehicle, such as the current position, orientation, yaw, and / or speed of the vehicle. The potential state of the vehicle that would place the vehicle on a potential trajectory, preferably at a location on the potential trajectory that is closest to or at least proximate to the current state of the vehicle, may also be determined. Next, based on the current state and the potential state, it can be determined whether the vehicle can safely and efficiently navigate the potential trajectory. Determining the consistency validity is described in more detail below with reference to FIGS. 12-13.

[0124] In operation 1110, an orbit obsolescence value (also referred to as an obsolescence validity value) may be determined. In some embodiments, operation 1110 may be performed by an obsolescence validator 316. In other cases, obsolescence may be considered part of the consistency validity and may be performed by a consistency validator 310. In some cases, obsolescence may be defined by when the orbit was created relative to the current time or the time at which the orbit is being evaluated as an indicator of whether the device can move to and execute the orbit. For example, a first temporal difference between the time the orbit was created (such as the time shown in a timestamp associated with the orbit) and the current time may be determined. This value may, in some cases, be compared to a time limit, such as a predetermined time limit, for a realizable orbit. Depending on whether the time difference meets or exceeds the time limit, the obsolescence validity value may be set to false.

[0125] In operation 1112, the validity signal may be generated based on feasibility, timing, consistency, and / or staleness values. In some cases, a true or positive value may be generated when all four values are positive, or when the vehicle can safely navigate the track. A negative result may be generated in operation 1112 if one or more values are negative. In some cases, it should be understood that only a subset of the validity tests may be implemented (e.g., one or more of operations 1104 / 1106, 1108, 1110) to obtain a similar effect, and other tests or evaluations may be combined in the same manner as described above.

[0126] In some aspects, process 1100 may include operation 1114 that determines whether more tracks have been received and / or are available for evaluation. As used herein, the dashed lines used to indicate operations are optional so that the process may be executed without the indicated operation(s). If the determination is positive, process 1100 may loop operations 1102 - 1112 until there are no available tracks and / or until the expiration of a time threshold (e.g., a certain amount of time until an actual track must be selected and implemented). In some cases, the tracks for evaluation may be selected according to state diagrams 700 and 800 described above.

[0127] Next, in operation 1116, a track may be optionally selected based on the validity value determined above. In some aspects, selecting a track may utilize state diagrams 700 and / or 800 described above. In operation 1118, the autonomous vehicle may be controlled according to the selected track, at least partially based on the validity signal. In some examples, operation 1118 may include TMP222 sending the selected track or its instructions to the primary computing unit 202 of the device or vehicle, the drive manager 402, and / or the actuator 404, or other subsystems to cause the device to follow the selected track.

[0128] Figure 12 shows an exemplary diagram of an autonomous vehicle on a current trajectory 1214 that is in a physical relationship with a potential trajectory 1204, as an example for showing various features of the consistency and feasibility validity tests as described in this specification. In the illustrated example, the trajectory 1204 can define a right turn and can be represented by several different states of the vehicle on the trajectory, as represented by vehicles in states 1202, 1206, 1208, 1210, and 1212. The current trajectory 1214 can be represented by a vehicle in state 1216.

[0129] Each state of the vehicle, such as the current state 1216 and the potential state 1208, can be defined by or associated with one or more properties or parameters. For example, the state of the vehicle can be defined by one or more of the vehicle's position, orientation, yaw rate, speed, acceleration, jerk, and / or other physical and kinematic qualities at a given point in time and space.

[0130] To determine whether the potential trajectory matches the vehicle's current trajectory or state, the current location can first be obtained or determined. In some cases, the current location can be a future estimated location, for example, the location when the vehicle starts the potential trajectory 1204 or the location at the end of the trajectory 1214. The vehicle can be projected onto the potential trajectory 1204 into a potential state such as state 1208. In some cases, the state 1208 can be selected to be the location on the trajectory 1204 that is closest to the current state 1216 of the vehicle. The potential state 1208 can be determined relative to the current state 1216, for example, to minimize the distance between two points. This can include a geometric evaluation (such as determining the shortest Euclidean distance between the trajectory 1204 and the state 1216 or via other means).

[0131] In some cases, the current trajectory represented by the state of the vehicle at location 1216 can be compared to the state 1208 on the potential trajectory 1204. Each state 1216 and 1208 of the vehicle may be represented by several parameters such as position, orientation, yaw, and / or speed. By comparing these values, one or more deviation values can be determined, such as the distance between the two states, the commanded steering position difference between the vehicle in the first state 1216 and the vehicle in the second state 1208, the yaw deviation between the two states, and / or the speed deviation between the two states. For example, the distance to the trajectory threshold can indicate when the vehicle is too far from the potential trajectory and inconsistent. Another example includes the commanded steering position, which can indicate how much the vehicle has to change its steering angle to reach the potential trajectory. Next, the deviation value can be compared to one or more thresholds to determine whether the potential trajectory 1204 matches the current trajectory of the vehicle. In some cases, if one deviation value is greater than a set difference or tolerance, it will be known that the potential trajectory is inconsistent. In some aspects, the difference may be linear, and for other metrics such as the derived metric, other comparisons can be used to determine whether the trajectory is consistent.

[0132] In some cases, different speed deviation thresholds may be determined and used for different speeds or speed ranges of the vehicle, such as via a clamping function. For example, if the current speed of the vehicle on the current trajectory is between 0.05 and 3 meters per second, the threshold may be set to a 10% difference. At higher speeds, the threshold may be set lower so that a 5 or 7% difference would exceed the acceptable speed deviation.

[0133] In some cases, the consistency check may also include a determination as to whether the potential state 1204 is collinear with the trajectory 1214. This can be performed, for example, by taking the cross product of two vectors representing the two trajectories. This may be performed to ensure that the two trajectories 2104 and 1214 are sufficiently aligned such that they are safe for the occupant and do not stress the vehicle or the occupant or otherwise exert excessive forces.

[0134] In some aspects, consistency may include a test for staleness, such that potentially old trajectories, such as those created more than 1,000 ms ago, may not be verified for consistency. In other cases, staleness may be determined separately from consistency as its own validity check.

[0135] In some examples, the difference may be compared to one or more tolerance values or deviation values, such as some variables, tolerance values, or threshold values used to determine whether two trajectories, states and trajectories, or two states are consistent. For example, the distance to a trajectory threshold can indicate when the vehicle is too far from the potential trajectory to be consistent. Other thresholds can include the commanded steering position (e.g., how much the vehicle has to change its steering angle to reach the potential trajectory), yaw deviation, one or more speed deviations associated with different speed ranges of the vehicle, and collinearity deviation.

[0136] In some embodiments, the feasibility check may include determining whether the vehicle can proceed from the current state 1216 to the potential trajectory 1204 within one or more capabilities or feasibility limitations of the vehicle. This may include determining the kinematics, represented by line 1218, for moving the vehicle from state 1216 to state 1208. This may include determining the required acceleration or deceleration, lateral acceleration or deceleration, and / or combined acceleration in the primary direction of movement (continuing in the direction of trajectory 1214). The kinematic determination may be based on the current position of the vehicle in state 1216 (e.g., the location corresponding to the state of the vehicle), the current speed or velocity of the vehicle, the direction of travel, and other characteristics of the vehicle, as well as the corresponding resulting values in the potential state 1208.

[0137] In some embodiments, one or more of the feasibility thresholds may be modified based on one or more operating statuses of the vehicle or a subsystem of the vehicle. For example, a subsystem of the vehicle may be able to detect that one or more of the tire air pressures are low. For example, based on detecting that the tire air pressure is low, the system monitor may be able to modify a set of feasibility values for the vehicle. FIG. 13 shows an exemplary process for performing a consistency check against a potential trajectory and operating a device in accordance with the potential trajectory based on the consistency check, which process may be performed by a TMP such as TMP222 and / or by the planning subsystem 1328 as described below with reference to FIG. 16. Process 1300 may be a more detailed example of operation 1108 of process 1100 described above with reference to FIG. 11.

[0138] Process 1300 can start at operation 1302, where the current state of a device such as an autonomous vehicle can be obtained. In some aspects, one or more components of the autonomous vehicle, such as the aforementioned TMP222 and / or system monitor 220, can determine the current state. In some aspects, the current state can be defined by a set of first values including one or more of a first position, a first orientation, a first yaw rate, or a first speed.

[0139] At operation 1304, the potential state of the device to follow a potential trajectory can be obtained, for example, by the TMP222 and / or system monitor 220 if the device is an autonomous vehicle. In some cases, operation 1304 can include determining the potential state based on the current state, such as a location proximate to the location associated with the current state.

[0140] At operation 1306, the current state can be compared with the potential state, and at operation 1308, it can be determined whether the potential trajectory passes one or more consistency checks. The one or more consistency checks can include one or more of the aforementioned consistency checks. Operations 1306 and 1308 can include generating a difference between the current state and the potential state and comparing that difference with a set of deviation values such as those described above.

[0141] If the determination at operation 1308 is negative, an invalid validity signal can be generated at operation 1314 and another trajectory can be selected for evaluation. However, if the determination at operation 1308 is positive, a validity signal indicating a valid trajectory can be generated at operation 1310 and the device can be operated to follow the potential trajectory at operation 1312.

[0142] FIG. 14 shows an exemplary communication 1400 between various subsystems of an autonomous vehicle, such as those described with reference to FIGS. 2-5. Communication 1400 shows an exemplary way in which the system monitor 220 can send vehicle updates to feasibility limits or values imposed on the vehicle.

[0143] In the illustrated example, the system monitor 220 can receive system data from various sources within the autonomous vehicle, such as from the primary computing unit 202 and / or the drive manager 402. The system data can include various updates, status information, and heartbeat information from various subsystems of the vehicle. In some aspects, the system data received by the system monitor 220 can also include sensor data regarding the environment in which the vehicle is operating, such as weather, road conditions, temperature, etc.

[0144] The system monitor 220 can maintain a set of feasibility limits or a data structure for the device. These feasibility limits or capability limits can define various physical limits imposed on the movement of the vehicle. One example can include limits on various possible actions or actions of the vehicle and can include values for both the default and current capabilities. For example, possible actions can include maximum speed, maximum longitudinal deceleration, maximum longitudinal acceleration (in the direction of the current movement), maximum lateral acceleration, and maximum combined acceleration (e.g., maximum acceleration in the lateral and longitudinal directions). It should be understood that other possible actions / limits can also be imposed on the vehicle to enhance safety in the operation of the vehicle and safety to the occupants.

[0145] The stem monitor 220 may update the feasibility limits according to various inputs received by the system monitor 220, including system data and status updates from the primary computing unit 202 and / or the drive manager 402, and / or other data such as sensor data. In some embodiments, the system monitor 220 may maintain and update various status information regarding different subsystems of the vehicle, including, for example, the current state of some subsystems that may affect one or more feasibility limits. In one example, the operating states or statuses of the various subsystems of the autonomous vehicle 1400 may include tire pressure information, motor operation as a percentage, various components of the vehicle electrical system, and various other subsystems that may be found in a vehicle or an autonomous vehicle, or other movable devices. In some cases, the operating states of the various subsystems may be represented as a numerical value (e.g., as a percentage), one of three functional states, a degraded state, and a non-operational state, or via other schemes. In some embodiments, the system monitor 220 may receive status updates of the various subsystems of the vehicle periodically in response to changes in the situation, or at other regular or irregular intervals. In some embodiments, the intervals may correspond to the timing at which new trajectories are created and / or evaluated.

[0146] The system monitor 220 can also maintain or access a set of values indicating the impact on the feasibility limits or values of one or more changes to the operating states or statuses of various subsystems. In one example, the effects of different levels or PSI values of a tire pressure monitoring system (TPMS), which are examples of operating states or statuses, can correlate to different impacts or limitations on one or more of the maximum speed, maximum longitudinal acceleration, maximum longitudinal deceleration, maximum lateral acceleration, and / or maximum combined acceleration. In a similar manner, any subsystem or sensor input of the vehicle can correlate to a change in one or more of the vehicle's feasibility values. In some cases, the effects of the various operating statuses of the vehicle's subsystems are determined empirically, pre-set, or presented and adaptable to the vehicle's operating conditions, user preferences, etc. In some examples, a change in a single subsystem, such as the PSI of a tire, can affect multiple feasibility limits. In some examples, the operating statuses of two or more subsystems may contribute to a change in a single or multiple feasibility limits of the vehicle.

[0147] The system monitor 220 can update the feasibility limits or physical operating limits of the vehicle or device by using the information described above to obtain the status information of the subsystems, determine how the status information of the subsystems affects one or more of the feasibility limits, and update the affected feasibility limits. In other aspects, one or more of the described operations can be performed additionally or alternately by other aspects of the TMP222 or the primary or secondary computing units 202, 204.

[0148] In some embodiments, the set of feasibility constraints and / or adjustments thereto may be specific to one or more types of trajectories, such as a primary trajectory and an incidental trajectory. In this example, the system monitor 220 may set different capacity limits for different types of trajectories. For example, the maximum speed of an incidental trajectory may be set lower than that of a primary trajectory. Similarly, changes in the status of one or more subsystems of the vehicle may have various effects on the feasibility constraints of different types of trajectories. For example, a change in tire pressure may cause a first decrease in the maximum longitudinal deceleration of a primary trajectory and a second decrease in the maximum longitudinal deceleration of an incidental trajectory.

[0149] When updating the feasibility constraints, the system monitor 220 may transmit the updated constraints, for example, in the form of one or more values TMP222 and the primary calculation unit 202. TMP222 may use the feasibility constraints in the process of verifying potential trajectories, and the primary calculation unit 202 and / or the trajectory planner AI212 may use the feasibility constraints when generating new trajectories. When selecting a potential trajectory, TMP222 may then transmit one or more trajectories to the primary calculation unit 202, the system monitor 220, and the drive manager 402 to provide further feedback for future trajectory decisions.

[0150] FIG. 15 shows an exemplary process for verifying the feasibility of potential trajectories, which may be executed by one or more of the above-described TMP222 and system monitor 220, and / or by the planning subsystem 1628 as described below with reference to FIG. 16. Process 1500 may be a more detailed example of operation 1104 of process 1100 described above with reference to FIG. 11.

[0151] Process 1500 can start at operation 1502, where one or more status updates corresponding to at least one subsystem of the device can be obtained. The one or more status updates may be obtained directly from a subsystem, a sensor associated therewith, or a monitoring system such as system monitor 220. The status update may indicate an operating status or value of a subsystem, such as a tire air pressure value, a battery charge level, an operating level (e.g., percentage), or may indicate that a particular subsystem is malfunctioning. In this example, an operating status or status update may be derived from an indication that a subsystem is not operating at 100% capacity.

[0152] At operation 1504, the feasibility limits of the device can be reduced based on one or more status updates. In some aspects, operation 1504 may include an update of one or more capability values. In some aspects, a change in one subsystem can affect multiple feasibility limits. In this scenario, multiple feasibility limits or capability limits can be updated. In other cases, the feasibility limit can be increased or restored to a predefined limit based on one or more positive status updates of the device.

[0153] At operation 1506, a potential trajectory for the device can be obtained, for example, from trajectory planner AI 212. At operation 1508, it can be determined whether a device following the potential trajectory (or navigating to a starting point or a point on the potential trajectory) will exceed the reduced feasibility limits. Operation 1508 may include simulating or determining one or more kinematic values of the vehicle, such as acceleration, speed, and direction change, to navigate to and follow the potential trajectory.

[0154] If the determination in operation 1508 is negative, an invalidity signal that was not qualified can be generated in operation 1514, and another trajectory can be selected for evaluation. However, if the determination in operation 1508 is positive, a validity signal indicating a valid trajectory can be generated in operation 1510, and the device can be operated according to the potential trajectory in operation 1512.

[0155] FIG. 16 shows an example of an element that can be used according to an exemplary architecture of an autonomous vehicle component 1600 of an autonomous vehicle for verifying and selecting a trajectory for navigation. The autonomous vehicle may be characterized as having an autonomous vehicle operating system 1602 coupled to various controllers, which are then coupled to various components of the autonomous vehicle to handle movement, power management, etc. The elements of the autonomous vehicle operating system 1602 provide a computing system for generating trajectories and performing one or more verifications of those trajectories, as described herein. These elements may be used in other applications other than autonomous vehicles.

[0156] The architecture 1600 can specify one or more computer systems (s) including various hardware, software, firmware, etc. to implement aspects of the systems, methods, and apparatuses described herein. For example, the autonomous vehicle operating system 1602 can include a surrounding analysis system 1603 and other components that can be used in various aspects of the autonomous vehicle. The surrounding analysis system 1603 can be used to capture information that the autonomous vehicle operating system 1602 can use to operate controllers for motors, steering, object avoidance, etc.

[0157] The surrounding analysis system 1603 can be organized into multiple subsystems to simplify implementation, to enable separate teams to develop for specific subsystems, or for other reasons. In some embodiments, the subsystems are implemented independently, while in other embodiments, multiple subsystems are partially or fully integrated together. The subsystems can include a LIDAR subsystem 1604, a camera subsystem 1606, a radar subsystem 1608, a sonar subsystem 1610, a voxel space subsystem 1612, a ground determination subsystem 1614, a clustering subsystem 1616, an interpolation subsystem 1618, an object determination subsystem 1620, a dynamic object determination subsystem 1622, a ray casting subsystem 1624, a tracking subsystem 1626, a planning subsystem 1628, a sensor calibration subsystem 1630, an annotation subsystem 1632, and optionally other subsystems 1634.

[0158] A given subsystem can be implemented with program code or hardware to communicate with other subsystems, receive inputs, and provide outputs. Some of the inputs can be from sensors. In some descriptions herein, for readability, a subsystem may be described as including a sensor from which the subsystem obtains data or signals, and / or an emitter to which the subsystem outputs data or signals. For example, the sonar subsystem can be described as having an ultrasonic sensor or as receiving signals from an ultrasonic sensor. As another example, the camera subsystem can be described as having a camera and a display or as receiving signals or data from a camera and transmitting signals or data to a display.

[0159] Although not shown in FIG. 16, it should be understood that communication between subsystems may be provided as needed. A given subsystem may communicate with another subsystem by directly transmitting data to the other subsystem via some channel, and the ambient analysis system 1603 may comprise a bus subsystem or communication infrastructure through which subsystems can communicate by passing data and / or signals therebetween. The ambient analysis system 1603 may also be configured to receive external data and communicate information external to the ambient analysis system 1603.

[0160] A given subsystem may have some of its own computational processing, which may be executed by dedicated hardware for that given subsystem, or may be executed by a processor or circuit assigned to perform the calculations of that subsystem. In some cases, the subsystem may be implemented entirely in software and may be executed by one or more processors 1636 using a memory 1638 such as a program code memory and a data storage memory. The memory may be for temporary storage of variables and data such as RAM, and may also be for persistent storage (i.e., data that persists without the need for a certain lifespan, refresh, power, etc.), and should be implied at the locations shown even if not explicitly mentioned. For example, if a subsystem is described as operating on or storing data in a database, there will be some form of memory for storing data in an electronically readable form. In some cases, the data storage in the database or memory is not specific to or internal to one subsystem. In such cases, the memory is accessible from multiple subsystems. For example, one subsystem may create records based on sensor data acquired by that subsystem, write those records to a database or other data structure, and then another subsystem may read and use that data. When a subsystem is implemented in software, the subsystem may include a processor specific to that subsystem, or may include program code coupled to a more general program code memory and processor.

[0161] In some examples, the surrounding analysis system 1603 is used in an autonomous vehicle. In some examples, the surrounding analysis system 1603 can provide perception and planning functionality to the autonomous vehicle. Generally, the surrounding analysis system 1603 can interface with other controllers such as drive controllers, power controllers, environmental controllers, and communication controllers, along with LIDAR perception, radar perception, vision (camera) perception, acoustic perception, segmentation and classification, tracking and fusion, and prediction / planning.

[0162] The autonomous vehicle operation system 1602 can include a planning system 1640, a road navigation system 1642, a manifest manager 1644, and an audit / fault logger 1646. The autonomous vehicle operation system 1602 can also include or interface with various sensors 1650 and emitters 1652.

[0163] The autonomous vehicle operation system 1602 can interface with a drive controller 1670 that interacts with a motor 1680, a steering 1682, a brake 1684, and a suspension 1686, a power controller 1672 that interacts with a battery 1688 and an inverter / charger 1690, an environmental controller 1674 that interacts with heating, ventilation, and air conditioning (HVAC) components 1692 and lighting 1694, and a communication controller 1676 that processes communication between the autonomous vehicle, devices used in the autonomous vehicle, and external devices via a network, a cellular channel, or a Wi-Fi channel 1696. The combination of the autonomous vehicle operation system 1602, the controllers, and the vehicle components installed in the autonomous vehicle can provide a vehicle that can safely navigate without constant human intervention.

[0164] Referring also to the surrounding analysis system 1603 and its subsystems, the LIDAR subsystem 1604, as described herein, may include one or more LIDAR sensors for capturing LIDAR data for segmentation and may include any one or more depth sensors, as described in detail herein. In some examples, the LIDAR subsystem 1604 may include functionality for combining or synthesizing LIDAR data from multiple LIDAR sensors to generate a meta-spin of the LIDAR data, which may refer to LIDAR data based on multiple LIDAR sensors. In the case of the meta-spin of LIDAR data, the LIDAR subsystem 1604 may include functionality for determining a virtual origin of the meta-spin data (e.g., a coordinate reference frame common to all LIDAR sensors) and performing a data transformation such that the LIDAR data from each of the one or more LIDAR sensors is represented with respect to the virtual origin. As can be understood in the context of the present disclosure, the LIDAR subsystem 1604 may capture data and transmit a data set to other subsystems of the surrounding analysis system 1603 for subsequent processing.

[0165] The camera subsystem 1606 may include or interface with one or more camera sensors for capturing visual data for image segmentation and / or classification. The camera subsystem 1606 may include any number and type of camera sensors. For example, the camera subsystem 1606 may include any color camera, monochrome camera, depth camera, RGB-D camera, stereo camera, infrared (IR) camera, ultraviolet (UV) camera, etc. As can be understood in the context of the present disclosure, the camera subsystem 1606 may capture data and transmit a data set to other subsystems for subsequent processing. For example, data from the camera subsystem 1606 may be included as one or more channels of a multi-channel image that is processed as such by another subsystem.

[0166] The radar subsystem 1608 may include one or more radar sensors for capturing the range, angle, and / or velocity of objects in the environment. As can be understood in the context of the present disclosure, the radar subsystem 1608 may capture data and transmit a data set to other subsystems of the ambient analysis system 1603 for subsequent processing. For example, data from the radar subsystem 1608 may be included as one or more channels of a multi-channel image provided to another subsystem.

[0167] The sonar subsystem 1610 can include, or interface with, one or more speakers or sound emitters and one or more microphones (such as a microphone array) to capture acoustic information from objects in the environment. Additionally or alternatively, such a sonar subsystem 1610 may comprise various ultrasonic transducers. For example, the sonar subsystem 1610 can cause an ultrasonic transducer to emit a pulse of sound and listen for an echo to determine position and / or movement information related to an object in the environment. As can be understood in the context of the present disclosure, the sonar subsystem 1610 may capture data and transmit a data set to other subsystems for subsequent processing. For example, another subsystem of the ambient analysis system 1603 may fuse data obtained from the sonar subsystem 1610 with data obtained from the LIDAR subsystem 1604 to more accurately segment an object and / or to determine information about the object or for other purposes.

[0168] The autonomous vehicle operation system 1602 may include any number or type of other sensors suitable for use in autonomous vehicles other than those shown. The various sensors 1650 may include, but are not limited to, ultrasonic transducers, wheel encoders, environmental sensors, microphones, inertial measurement unit(s) (IMU), accelerometers, gyroscopes, magnetometers, temperature sensors, humidity sensors, light sensors, global positioning system (GPS) sensors, location sensors, and the like.

[0169] In some examples, the LIDAR subsystem 1604, camera subsystem 1606, radar subsystem 1608, and / or sonar subsystem 1610 may provide one or more data sets to other subsystems of the surrounding analysis system 1603 to combine and / or synthesize data for improved segmentation.

[0170] The surrounding analysis system 1603 may further include storage for simulated data generated by a computer simulation algorithm for use in part in testing. In some examples, the simulated data may include any type of simulated data, such as camera data, LIDAR data, radar data, sonar data, inertial data, GPS data, etc. In some examples, the surrounding analysis system 1603 may modify, transform, and / or execute the conversion operations described herein on the simulated data to verify operation and / or to train machine learning algorithms as described herein. For example, to test some functionality in a laboratory environment, simulated sensor data / signals can be fed to the subsystems as if they were actual sensor data to test the performance of some subsystems.

[0171] The voxel space subsystem 1612 may include functionality for converting or mapping data to a voxel map. For example, the voxel space subsystem 1612 can receive LIDAR data, camera data, radar data, sonar data, etc., and map, transform, or associate individual data points to a voxel map representing a three-dimensional space within the environment. The voxel space is a logical representation of a three-dimensional environment, such as the space surrounding an autonomous vehicle, and is represented as individual small volumes, such as voxels. The voxel map provides data or values for each voxel within the voxel space. As a three-dimensional environmental representation, the voxel map can be stored in memory and operated on by a processor.

[0172] In some examples, the voxel space subsystem 1612 can define the dimensions of the voxel space, including the length, width, and height of the voxel space. Further, the voxel space subsystem 1612 may determine the size of individual voxels. In some examples, the voxels can be of a uniform size and shape throughout the voxel space, but in some examples, the size and / or density of the voxels can vary based on their relative location within the voxel space. For example, the size of a voxel can increase or decrease proportionally to the distance of the voxel from the origin or center of the voxel space. Additionally, or alternatively, such a voxel space subsystem 1612 can include a transformation between a virtual origin and the origin of the voxel space. In some examples, the voxel space subsystem 1612 can include functionality for generating a sparse voxel space where voxels that do not contain data, or that contain some amount of data below a data threshold, need not be present in the voxel map and the values of those voxels can be assumed or ignored. In such examples, the voxel map can be organized as an octomap, or a voxel hash, etc. In some examples, the voxel space subsystem 1612 can include functionality for reducing the amount of noise in the data of the voxel map or the data used to generate the voxel map by filtering the data when it is mapped into the voxel space and stored in the voxel map. For example, the filtering can include removing data (e.g., the number of LIDAR data points associated with a voxel) that is below a threshold amount of data per voxel, or data (e.g., the number of LIDAR data points associated with some neighboring voxels) that exceeds a predetermined number of voxels. In some examples, the voxel space subsystem 1612 can update the voxel map when data is collected over time, or in response to an autonomous vehicle navigating within the corresponding real-world environment of the voxel space. For example, the voxel space subsystem 1612 can add data to, and / or discard data from, the voxel map as the autonomous vehicle navigates within the environment.

[0173] In some examples, the voxel space subsystem 1612 can initialize a voxel map and other voxel space parameters such as voxel size, orientation, and extent, treat the initial voxel map as representing free space, and the voxel space subsystem 1612 can construct a representation of objects as LIDAR data is captured over time. In other examples, the voxel space subsystem 1612 can initialize the voxel map and voxel space parameters using the global map data, whereby locally captured LIDAR data can be used to localize the autonomous vehicle within the global map space and can also be used to clean up or clear voxels of the global map.

[0174] The ground determination subsystem 1614 can include functionality for analyzing individual voxels of the voxel space to determine the ground associated with the environment within the voxel space. For example, the ground determination subsystem 1614 can determine locally flat voxels by estimating a plane representing the data associated with a particular voxel and determining the normal vector of that plane. For example, the ground determination subsystem 1614 can perform principal component analysis on the voxels of the voxel map to determine the smallest principal component associated with the data associated with the voxel. In some examples, for principal component analysis, the smallest eigenvector can correspond to the normal vector of the plane, while the eigenvalue associated with the eigenvector can correspond to the spread or level of the data associated with a particular voxel in the direction of the smallest eigenvector.

[0175] As another example, without limitation, such surface normal determination may be performed by calculating the normal of the cross product of vectors indicating the directions from a point P in a voxel to two of the nearest neighbors of P. As another example, without limitation, such surface normal determination may be performed by performing eigenvalue decomposition on the covariance matrix associated with an individual voxel. In some examples, the ground determination subsystem 1614 may determine whether a target voxel is a locally flat voxel by determining the surface associated with the target voxel based on values associated with adjacent voxels. Further, in some examples, the ground determination subsystem 1614 may utilize a marching cubes type algorithm to create a mesh based on the average point values associated with the voxels and determine triangles containing at least three points to create a surface. Further, the ground determination subsystem 1614 may receive a reference orientation that may correspond to the direction or orientation of the autonomous vehicle. The ground determination subsystem 1614 may determine that a voxel is a locally flat voxel if the normal vector associated with the voxel is within a threshold amount of the reference orientation as described above.

[0176] The clustering subsystem 1616 operates in conjunction with the ground determination subsystem 1614 and can determine the ground region, perhaps by extending the representation of the ground region in memory, starting from the surface closest to the origin of the LIDAR data or from the surface under the autonomous vehicle. That is, voxels at positions within the voxel space corresponding to real-world positions adjacent to the autonomous vehicle can be used as seed voxels by the clustering subsystem 1616, and then the voxel representation can be extended from those seed voxels. The clustering subsystem 1616 can determine that adjacent locally flat voxels belong to the same cluster and can extend the region to include the ground plane. Further, the clustering subsystem 1616 operates in conjunction with the object determination subsystem 1620, discussed below, and can determine that voxels within the cluster or elsewhere are associated with a particular object. The clustering subsystem 1616 may utilize various clustering algorithms including, but not limited to, region growing, hierarchical clustering, partitional clustering, square error clustering, graph theoretic clustering, mixture-resolving clustering, mean-seeking clustering, k-means clustering, N-cut clustering, proximity clustering, etc.

[0177] The interpolation subsystem 1618 operates in conjunction with the ground determination subsystem 1614 and / or the clustering subsystem 1616 and can combine or otherwise associate various clusters to expand the representation of the ground plane. For example, locally flat voxels may not form a single cluster when determining the ground area associated with the autonomous vehicle, in which case the interpolation subsystem 1618 can interpolate between points to determine whether the gradient exceeds or falls below a threshold gradient for expanding the ground plane cluster. Additional aspects of the ground determination subsystem 1614, the clustering subsystem 1616, and the interpolation subsystem 1618 may be provided elsewhere in this specification as needed to understand these subsystems.

[0178] The object determination subsystem 1620 may include functionality for determining objects represented in voxel space by a voxel map. For example, the object determination subsystem 1620 can receive an indication about the ground plane from the ground determination subsystem 1614 and / or receive an indication about some or all of the locally flat voxels, and can remove the voxels associated with the ground from the voxel space such that the voxel map contains only the values of other voxels. Next, the object determination subsystem 1620 can analyze the remaining voxels to determine objects based on voxel connectivity. For example, the object determination subsystem 1620 can operate in cooperation with the clustering subsystem 1616 to expand the region within the voxel space corresponding to an object by determining that adjacent voxels should be considered part of the same object. The object determination subsystem 1620 can assign an object identifier to all voxels associated with a particular object, and in some examples, the object identifier assigned or determined by the object determination subsystem 1620 can be propagated to the LIDAR data associated with the voxels that include the particular object. Additional information about objects, the ground, clusters, etc. can be stored along with the voxel map or as a separate data structure. Additional aspects of the object determination subsystem 1620 may be provided elsewhere in this specification as necessary to understand the object determination subsystem 1620.

[0179] The dynamic object determination subsystem 1622 may include functionality for distinguishing between static objects and dynamic objects that can be determined to exist in a space corresponding to the voxel space. For example, the dynamic object determination subsystem 1622 may accumulate data over time, compare the voxel values at a first time with the voxel values at a second time, and determine whether the occupancy of the voxels has changed over time to determine the movement of the object. For example, if a voxel occupied by an object at a first time is not occupied by that object at a second time, the dynamic object determination subsystem 1622 may consider that object to be a dynamic object and record that evaluation as voxel map data. Based on which voxels become occupied or unoccupied over time, the dynamic object determination subsystem 1622 can determine the movement of the dynamic object, such as the speed and direction of movement. In some examples, the dynamic object determination subsystem 1622 can provide an indication for determining movement from the dynamic object. Additional aspects of the dynamic object determination subsystem 1622 may be provided elsewhere in this specification as needed to understand the dynamic object determination subsystem 1622.

[0180] The ray casting subsystem 1624 operates in conjunction with the dynamic object determination subsystem 1622 to distinguish between static and dynamic objects. Further, the ray casting subsystem 1624 may include functionality for clearing the voxel map over time as data accumulates in the voxel map representation. For example, as an object moves through the entire voxel space over time, the representation of the voxels occupied by the dynamic object may contain increasingly more data over time. However, the ray casting subsystem 1624 can analyze the path of the rays associated with the LIDAR data to determine, for example, that some voxels through which the rays pass internally should be considered cleared and that the corresponding storage in the voxel map should be cleared. Thus, the ray casting subsystem 1624 may provide additional functionality for determining that voxels occupied at a first time are not occupied at a second time, which can be provided to various modules, for example, to determine that an object is a dynamic object. In some examples, the voxel map can be represented in a sparse manner (e.g., providing data representing occupied voxels and ignoring unoccupied voxels) or a dense manner (e.g., without voxel discard). In some examples, the ray casting subsystem 1624 may store ray casting information in a dense manner, i.e., voxels that do not exist in the sparse voxel representation (e.g., because the voxels do not have associated LIDAR data) can have ray casting information associated with such voxels. For example, voxels without associated LIDAR data can still be represented in a dense voxel map to include ray casting information associated with the voxels of the voxel space. In some examples, the dense voxel representation may associate positive information that the voxel is unoccupied with the voxel, at least in part corresponding to the ray casting operations discussed herein.Furthermore, since LIDAR data is accumulated for individual voxels, negative information may be associated with individual voxels in the voxel map, for example, to indicate that they are occupied by static objects. Since the data is accumulated over time, the information can be partially aggregated to determine, for example, whether a voxel corresponds to free space or a static object. Additionally, the raycasting subsystem 1624 can be used to clean up the global map by comparing the locally captured LIDAR data with the global map data. Additional aspects of the raycasting subsystem 1624 may be provided elsewhere in this specification as needed to understand the raycasting subsystem 1624.

[0181] The tracking subsystem 1626 may include functionality to receive indications of one or more dynamic objects and perform additional processing to track the objects. For example, the tracking subsystem 1626 can determine the speed of a dynamic object and / or determine and store the trajectory of a dynamic object over time. In some examples, the tracking subsystem 1626 can be programmed to execute a prediction algorithm that can predict the path of an object being tracked based on the object's previous movements.

[0182] The planning subsystem 1628 may include functionality to receive segmented data and / or indications of the ground plane, static objects, and / or dynamic objects to determine the trajectory of the autonomous vehicle. For example, the planning subsystem 1628 can receive segmented information that identifies the ground plane and generate a trajectory for the autonomous vehicle to follow. The planning subsystem 1628 and / or another subsystem of the operating system 1602 may include or implement a collision avoidance system as described above to perform one or several validations on the trajectory determined by the planning subsystem 1628.

[0183] The sensor calibration subsystem 1630 may include functionality for calibrating one or more sensors based at least in part on the segmentation information determined with respect to the environment. For example, sensor data from the LIDAR subsystem 1604, camera subsystem 1606, radar subsystem 1608, and / or sonar subsystem 1610 may be used (e.g., using simultaneous localization and mapping (SLAM)) to estimate location and / or orientation, although the autonomous vehicle may also include additional sensors such as an inertial measurement unit (IMU) and / or GPS unit to determine the location of the autonomous vehicle in the environment. In some examples, the IMU may indicate that the autonomous vehicle is at a first location, while the analysis of the LIDAR data described herein may indicate that the vehicle is at a second location different from the first location. The sensor calibration subsystem 1630 may determine the difference in locations and may adjust or calibrate another sensor to update the location of the autonomous vehicle, or one or more sensor specific or external characteristics.

[0184] The annotation subsystem 1632 may include functionality for receiving the segmentation information discussed herein and may annotate the ground plane, static objects, and / or dynamic objects with information associated with the objects stored as data with the voxel map or otherwise. In some examples, the annotation subsystem 1632 may provide the segmentation information in a graphical user interface for manual review and / or adjustment by, for example, a technician. In some examples, the annotation subsystem 1632 may include functionality for determining and applying classifications to the objects discussed herein. The annotation subsystem 1632 may be programmed to execute a machine learning algorithm such as a neural network process to perform the segmentation and classification operations.

[0185] An exemplary neural network can pass input data through a series of connected layers to generate an output. An example of a neural network can include a convolutional neural network (CNN). Each layer of the CNN can also include another CNN or can include several layers. As can be understood in the context of the present disclosure, a neural network can utilize machine learning, which can refer to a broad class of such algorithms where an output is generated based on learned parameters.

[0186] Although considered in the context of neural networks, any type of machine learning consistent with this disclosure may be used. For example, machine learning algorithms may include regression algorithms (e.g., ordinary least squares regression (OLSR), linear regression, logistic regression, stepwise regression, multivariate adaptive regression splines (MARS), locally estimated scatterplot smoothing (LOESS)), instance-based algorithms (e.g., ridge regression, least absolute shrinkage and selection operator (LASSO), elastic net, least angle regression (LARS)), decision tree algorithms (e.g., classification and regression tree (CART), iterative dichotomiser 3 (ID3), chi-squared automatic interaction detection (CHAID), decision stump, conditional decision tree), Bayesian algorithms (e.g., naive Bayes, Gaussian naive Bayes, multinomial naive Bayes, averaged one-dependent estimators (AODE), Bayesian belief network (BNN), Bayesian network), clustering algorithms (e.g., k-means, k-medians, expectation maximization (EM), hierarchical clustering), association rule learning algorithms (e.g., perceptron, backpropagation, Hopfield network, radial basis function network (RBFN)), deep learning algorithms (e.g., deep Boltzmann machine (DBM), deep belief network (DBN), convolutional neural network (CNN), stacked autoencoder), dimensionality reduction algorithms (e.g., principal component analysis (PCA), principal component regression (PCR), partial least squares regression (PLSR), Sammon mapping, multidimensional scaling (MDS), projection pursuit, linear discriminant analysis (LDA), mixture discriminant analysis (MDA), quadratic discriminant analysis (QDA), flexible discriminant analysis (FDA)), ensemble algorithms (e.g., boosting, bootstrap aggregating (bagging), AdaBoost, stack generalization (blending) gradient boosting machine (GBM), gradient boosting regression tree (GBRT), random forest), SVM (support vector machine), supervised learning, unsupervised learning, semi-supervised learning, etc., but are not limited thereto.

[0187] The environment shown in FIG. 16 may be implemented in one or more computer systems including storage, one or more processors, memory, and optionally, an operating system.

[0188] The systems and methods described herein may be implemented in software or hardware, or any combination thereof. The systems and methods described herein may be implemented using one or more computing devices that may or may not be physically or logically separated from each other. The methods may be executed by components arranged as any of on-premises hardware, on-premises virtual systems, or hosted private instances. Further, various aspects of the methods described herein may be combined or integrated with other functions.

[0189] Exemplary environments and computerized systems for implementing the system and method can include a processor or computer system and can be configured to specifically execute some or all of the methods described herein. In some embodiments, the method can be partially or fully automated by one or more computers or processors. The systems and methods described herein can be implemented using any combination of hardware, firmware, and / or software. The systems and methods described herein (or any part(s) or function(s) thereof) can be implemented using hardware, software, firmware, or a combination thereof and can be implemented on one or more computer systems or other processing systems. In some embodiments, the illustrated system elements can be combined into a single hardware device or separated into multiple hardware devices. When using multiple hardware devices, the hardware devices can be physically close to each other or positioned remotely from each other. The described and illustrated embodiments of the method are intended to be exemplary and not limiting. For example, some or all of the steps of the method can be combined, rearranged, and / or omitted in different embodiments.

[0190] In one exemplary embodiment, the systems and methods described herein may be directed to one or more computer systems capable of performing the functionality described herein. Exemplary computing devices can be personal computer (PC) systems that operate any operating system, such as, but not limited to, OS X®, iOS®, Linux®, Android®, and Microsoft®, Windows®, among others. However, the systems and methods described herein may not be limited to these platforms. Instead, the systems and methods described herein may be implemented on any suitable computer system that operates any suitable operating system.

[0191] The system may include one or more processors. The processor may be connected to a communication infrastructure, such as, but not limited to, a communication bus, crossover bar, or network. The process and the processor may not be located in the same physical location. In other words, the process may be executed in one or more geographically remote processors, for example, via a LAN or WAN connection. The computing device may include a display interface that can transfer graphics, text, and other data from the communication infrastructure for display on a display unit.

[0192] The computer system may also include, but is not limited to, main memory, random access memory (RAM), and secondary memory. Secondary memory may include, for example, a hard disk drive, and / or a removable storage drive such as a compact disc drive CD-ROM. The removable storage drive can read from, and / or write to, a removable storage unit. As can be understood, the removable storage unit may include a computer-usable storage medium that stores computer software and / or data internally. In some embodiments, machine-accessible media can refer to any storage device used to store data accessible by a computer. Examples of machine-accessible media can include, but are not limited to, magnetic hard disks, floppy disks, optical disks such as compact disc read only memory (CD-ROM) or digital versatile disc (DVD), magnetic tape, and / or memory chips.

[0193] The processor may also include, or be operatively coupled to communicate with, one or more data storage devices for storing data. Such data storage devices may include, by way of non-limiting example, magnetic disks (including internal hard disks and removable disks), magneto-optical disks, optical disks, read only memory, random access memory, and / or flash storage. Storage devices suitable for tangibly embodying computer program instructions and data may also include all forms of non-volatile memory, including, for example, semiconductor memory devices such as EPROM, EEPROM, and flash memory devices, magnetic disks such as internal hard disks and removable disks, magneto-optical disks, and CD-ROM and DVD-ROM disks. The processor and memory may be supplemented or incorporated in an ASIC (application specific integrated circuit).

[0194] The processing system can communicate with a computerized data storage system. The data storage system can include a non-relational data store, or a relational data store such as MySQL® or another relational database. Other physical and logical database types can be used. The data store can be a database server such as Microsoft SQL Server®, Oracle®, IBM DB2®, SQLITE®, or any other database software, relational or otherwise. The data store can store information identifying syntax tags and any information required to operate on the syntax tags. In some embodiments, the processing system can use object-oriented programming and store data within objects. In these embodiments, the processing system can use an object-relational mapper (ORM) to store data objects in a relational database. The systems and methods described herein can be implemented using any number of physical data models. In an exemplary embodiment, a relational database management system (RDBMS) can be used. In these embodiments, the tables within the RDBMS can include columns representing coordinates. In the case of an economic system, data representing companies, products, etc. can be stored in tables within the RDBMS. The tables can have predefined relationships between them. The tables can also have adjuncts associated with the coordinates.

[0195] In an exemplary alternative embodiment, the secondary memory may include other similar devices for enabling a computer program or other instructions to be loaded into the computer system. Such devices may include, for example, removable storage units and interfaces. Examples of such include program cartridges and cartridge interfaces (such as those found in video game devices, but not limited thereto), removable memory chips (such as erasable programmable read-only memory (EPROM), or programmable read-only memory (PROM), and associated sockets, but not limited thereto), and other removable storage units and interfaces that may enable software and data to be transferred from the removable storage unit to the computer system.

[0196] The computing device may also include input devices such as, but not limited to, a voice input device such as a microphone, a touch screen, a gesture recognition device such as a camera, other natural user interfaces, other pointing devices such as a mouse or a digitizer, a keyboard, or other data input devices. The computing device may also include output devices such as, but not limited to, a display and a display interface. The computing device may include input / output (I / O) devices such as, but not limited to, a communication interface, cables, and communication paths. These devices include, but are not limited to, network interface cards and modems. The communication interface(s) may enable software and data to be transferred between the computer system and one or more external devices.

[0197] In one or more embodiments, a computing device may be operably coupled to an automotive system. Such an automotive system may be manually operated, semi-autonomous, or fully autonomous. In such embodiments, the input and output devices may include one or more image capture devices, controllers, microcontrollers, and / or other processors for controlling automotive functions such as, but not limited to, acceleration, braking, and steering. Additionally, a communication infrastructure or a controller area network (CAN) bus in such embodiments may also be included.

[0198] In one or more embodiments, a computing device may be operably coupled to a vision system based on any machine. For example, such a vision system based on a machine may include, but is not limited to, manually operated, semi-autonomous, or fully autonomous industrial or agricultural robots, home robots, inspection systems, security systems, etc. That is, the embodiments described herein are not limited to one particular situation and may be applicable to any application that utilizes machine vision.

[0199] In one or more embodiments, the present embodiments can be practiced in an environment of one or more computer networks (singular or plural). The network may include a private network, a public network (e.g., the Internet as described below), or a combination of both. The network may include hardware, software, or a combination of both.

[0200] From a remote communication perspective, a network can be described as a set of hardware nodes interconnected by communication facilities, and one or more processes (hardware, software, or a combination thereof) function at each such node. The processes can communicate and exchange information with each other via communication paths between them, using inter-process communication paths. Appropriate communication protocols are used on these paths.

[0201] An exemplary computer and / or remote communication network environment according to this embodiment can include nodes that can include hardware, software, or a combination of hardware and software. The nodes can be interconnected via a communication network. Each node can include one or more processes executable by a processor incorporated in the node. For example, a single process can be executed by multiple processors, or multiple processes can be executed by a single processor. Further, each of the nodes can provide an interface point between the network and the external world and can include a set of sub-networks.

[0202] In an exemplary embodiment, the processes can communicate with each other through inter-process communication paths that support communication through any communication protocol. The paths can function sequentially or in parallel, continuously or intermittently. The paths can use any of the communication standards, protocols, or technologies described herein with respect to the communication network, in addition to standard parallel instruction sets used by many computers.

[0203] A node can include any entity capable of executing a processing function. Examples of such nodes that can be used with embodiments include a computer (such as a personal computer, workstation, server, or mainframe), a handheld wireless device and a wired device (such as a personal digital assistant (PDA), a modem cell phone with processing capabilities, a wireless email device including a BlackBerry® device, etc.), a document processing device (such as a scanner, printer, fax machine, or multifunctional document machine), or as described, a composite entity (such as a local area network or a wide area network) to which a set of processors is connected. For example, in the context of the present disclosure, the node itself can be a wide area network (WAN), a local area network (LAN), a private network (such as a virtual private network (VPN)), or a set of networks.

[0204] Communication between nodes can be enabled by a communication network. Nodes can be connected either continuously or intermittently using the communication network. As an example, in the context of the present disclosure, the communication network can be a digital communication infrastructure that provides an appropriate bandwidth and information security.

[0205] The communication network can include a wired communication capability, a wireless communication capability, or a combination of both at any frequency using any type of standard, protocol, or technology. Further, in this embodiment, the communication network can be a private network (such as a VPN) or a public network (such as the Internet).

[0206] An exemplary non-exhaustive list of wireless protocols and technologies used by a communication network can include Bluetooth®, General Packet Radio Service (GPRS), Cellular Digital Packet Data (CDPD), Mobile Solution Platform (MSP), Multimedia Messaging (MMS), Wireless Application Protocol (WAP), Code Division Multiple Access (CDMA), Short Message Service (SMS), Wireless Markup Language (WML), Handheld Device Markup Language (HDML), Binary Runtime Environment for Wireless (BREW), Radio Access Network (RAN), and Packet Switched Core Network (PS-CN). Also included are various generations of wireless technology. An exemplary non-exhaustive list of mainly wireless protocols and technologies used by a communication network includes Asynchronous Transfer Mode (ATM), Enhanced Interior Gateway Routing Protocol (EIGRP), Frame Relay (FR), High-Level Data Link Control (HDLC), Internet Control Message Protocol (ICMP), Interior Gateway Routing Protocol (IGRP), Internetwork Packet Exchange (IPX), ISDN, Point-to-Point Protocol (PPP), Transmission Control Protocol / Internet Protocol (TCP / IP), Routing Information Protocol (RIP), and User Datagram Protocol (UDP). As would be recognized by those skilled in the art, any other known or anticipated wireless or wired protocols and technologies can be used.

[0207] Embodiments of the present disclosure can include an apparatus for performing the operations herein. The apparatus may be specially constructed for the desired purpose or it may comprise a general-purpose device selectively activated or reconfigured by a program stored within the device.

[0208] In one or more embodiments, the present embodiment is embodied in machine-executable instructions. The instructions can be used to cause a processing device programmed with the instructions, such as a general-purpose or special-purpose processor, to execute the steps of the present disclosure. Alternatively, the steps of the present disclosure can be executed by specific hardware components including wiring logic for executing the steps, or by any combination of programmed computer components and custom hardware components. For example, the present disclosure can be provided as a computer program product as outlined above. In this environment, the embodiment can include a machine-readable medium having instructions stored therein. The instructions can be used to program any processor(s) (or other electronic device) to execute a process or method according to an exemplary embodiment of the present disclosure. Further, the present disclosure can also be downloaded and stored on a computer program product. Here, the program can be transferred from a remote computer (such as a server) to a requesting computer (such as a client) by a data signal embodied in a carrier wave or other propagation medium via a communication link (such as a modem or network connection), and ultimately, such a signal may be stored on a computer system for subsequent execution.

[0209] The method can be implemented in a computer program product accessible from a computer-usable or readable storage medium that provides program code for use by or in connection with a computer or any instruction execution system. The computer-usable or readable storage medium can be any device that can contain or store a program for use by or in connection with a computer, or an instruction execution system, apparatus, or device.

[0210] A data processing system suitable for storing and / or executing the corresponding program code can include at least one processor directly or indirectly coupled to a computerized data storage device such as a memory element. Input / output (I / O) devices (including, but not limited to, keyboards, displays, pointing devices, etc.) can be coupled to the system. A network adapter may also be coupled to the system to enable the data processing system to be coupled to other data processing systems, or remote printers or storage devices, through intervening private or public networks. To provide interaction with a user, these functions can be implemented on a computer comprising a display device such as an LCD (liquid crystal display), or another type of monitor for displaying information to the user, as well as a keyboard and an input device such as a mouse or trackball through which the user can provide input to the computer.

[0211] A computer program can be a set of instructions that can be used directly or indirectly in a computer. The systems and methods described herein can be implemented using programming languages, or combinations of programming languages, including compiled or interpreted languages such as CUDA, OpenCL, Flash®, JAVA®, C++, C, C#, Python, Visual Basic®, JavaScript®, PHP, XML, HTML, etc., and can be deployed in any form, as a stand-alone program or as a subsystem, component, subroutine, or other unit suitable for use in a computing environment. Software can include, but is not limited to, firmware, resident software, microcode, etc. Protocols such as SOAP / HTTP can be used when implementing interfaces between programming subsystems. The components and functionality described herein can be implemented using any programming language suitable for software development on any desktop operating system, including, but not limited to, different versions of Microsoft Windows®, Apple® Mac®, iOS®, Unix® / X-Windows®, Linux®, that are executed in a virtualized or non-virtualized environment. The system can be implemented using a web application framework such as Ruby on Rails.

[0212] A processor suitable for the execution of a program consisting of instructions can be, but is not limited to, general and special purpose microprocessors of any type of computer, as well as a single processor, or one of a plurality of processors or cores. The processor can receive and store instructions and data from a computerized data storage device, such as read-only memory, random access memory, both, or any combination of the data storage devices described herein. The processor can include any processing circuit or control circuit that operates to control the operation and execution of an electronic device.

[0213] The systems, subsystems, and methods described herein can be implemented using any combination of software elements or hardware elements. The systems, subsystems, and methods described herein can be implemented using one or more virtual machines that operate alone or in combination with each other. Any applicable virtualization solution can be used to encapsulate a physical computing machine platform into a virtual machine that is executed under the control of hardware computing platform or virtualization software operating on a host. The virtual machine can have both virtual system hardware and guest operating system software.

[0214] The systems and methods described herein can be implemented in a computer system that includes back-end components such as a data server, or middleware components such as an application server or an Internet server, or front-end components such as a client computer having a graphical user interface or an Internet browser, or any combination thereof. The components of the system can be connected by any form or medium of digital data communication, such as a communication network. Examples of communication networks include, for example, LAN, WAN, and the computers and networks that form the Internet.

[0215] One or more embodiments of the present disclosure may be practiced using other computer system configurations, including, but not limited to, handheld devices, microprocessor systems, microprocessor-based or programmable household appliances, minicomputers, mainframe computers, and the like. The systems and methods described herein may also be practiced in a distributed computing environment where tasks are performed by remote processing devices linked through a network.

[0216] The terms "computer program medium" and "computer readable medium" may be used to generally refer to media such as, but not limited to, removable storage drives, hard disks installed in hard disk drives, etc. These computer program products may provide software to a computer system. The systems and methods described herein may be directed to such computer program products.

[0217] References to "one embodiment", "an embodiment", "exemplary embodiments", "various embodiments", etc., may indicate that embodiments of the present disclosure may include a particular feature, structure, or characteristic, but not every embodiment necessarily includes the particular feature, structure, or characteristic. Further, repeated use of the phrases "in one embodiment" or "in an exemplary embodiment" does not necessarily refer to the same embodiment, but may. Similarly, references to "examples" may indicate that various examples of the present disclosure may include a particular feature, structure, or characteristic, but not every example necessarily includes the particular feature, structure, or characteristic. Further, repeated use of the phrase "in some examples" does not necessarily refer to the same example, but may.

[0218] In the description and claims, the terms "coupled" and "connected" may be used along with their derivatives. It should be understood that these terms are not necessarily intended to be synonyms of each other. Rather, in certain embodiments, "connected" may be used to indicate that two or more elements are in direct physical or electrical contact with each other. "Coupled" may sometimes mean that two or more elements are in direct physical or electrical contact with each other. However, "coupled" may also mean that two or more elements are not in direct contact with each other, but still cooperate or interact with each other.

[0219] Here, also in general, an algorithm may be regarded as a self - consistent sequence of acts or operations leading to a desired result. These include physical operations of physical quantities. Although not necessarily, usually these quantities take the form of electrical or magnetic signals that can be stored, transferred, combined, compared, and otherwise manipulated. Mainly for reasons of common usage, it has been found convenient at times to refer to these signals as bits, values, elements, symbols, characters, terms, or numbers, etc. However, it should be understood that all of these and similar terms should be associated with appropriate physical quantities and are merely convenient labels applied to these quantities.

[0220] Unless otherwise specifically noted, throughout this specification, terms such as "process", "calculate", "compute", or "determine" refer to actions and / or processes of a computer or computing system, or similar electronic computing device that operate on and / or transform data represented as physical quantities, such as electronic quantities, within the registers and / or memory of the computing system into other data similarly represented as physical quantities within the memory, registers, or other such information storage, transmission, or display devices of the computer system.

[0221] In a similar manner, the term "processor" may refer to any device, or portion of a device, that processes electronic data from registers and / or memory and converts that electronic data into other electronic data that may be stored in registers and / or memory. By way of non-limiting example, a "processor" can be a central processing unit (CPU) or a graphics processing unit (GPU). A "computing platform" may comprise one or more processors. As used herein, a "software" process may include software entities and / or hardware entities that perform operations over time, such as tasks, threads, and intelligent agents. Also, each process may refer to multiple processes for performing instructions sequentially or in parallel, continuously or intermittently. The terms "system" and "method" are used interchangeably herein insofar as a system may embody one or more methods and a method may be considered a system.

[0222] Although one or more embodiments have been described, various modifications, additions, substitutions, and equivalents are included within the scope of the disclosure.

[0223] In the description of the embodiments, reference is made to the accompanying drawings that form a part thereof, which illustrate, by way of example, specific embodiments of the claimed invention. It is to be understood that other embodiments may be utilized and that changes or modifications, such as structural changes, may be made. Such embodiments, changes or modifications are not necessarily a departure from the scope of the claimed invention. Steps may be presented herein in a particular order, but in some cases, the ordering may be changed so that a particular input is provided at different times or in a different order without changing the functionality of the described systems and methods. The disclosed procedures may also be executed in a different order. Additionally, various calculations herein need not be performed in the order disclosed, and other embodiments using alternative orderings of the calculations may be readily implemented. In addition to being reordered, the calculations may be broken down into sub-calculations having the same result.

[0224] The above considerations describe an exemplary implementation of the technology described, but other architectures may be used to implement the described functionality, which is intended to be within the scope of the present disclosure. Further, for purposes of discussion above, a particular allocation of responsibilities was defined, but various functions and responsibilities may be allocated and divided in different ways depending on the situation.

[0225] Furthermore, although the subject matter has been described in language specific to structural features and / or methodological acts, it is to be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described. Rather, the specific features and acts are disclosed as exemplary forms of implementing the claims.

[0226] Articles of the Examples

[0227] Embodiments of the present disclosure may be described with reference to the following articles.

[0228] Systems and Methods for Orbit Confirmation 1. A collision avoidance system, comprising One or more processors and a memory storing instructions, which, when executed by the one or more processors, cause the system to receive a trajectory for navigating an autonomous vehicle through an environment, determine, as staleness validity, whether a first time difference between a first time at which the trajectory was created and the current time satisfies a first time threshold, determine, as timeliness validity, whether a second time difference between a second time at which the trajectory was received and a third time at which a previous trajectory was received satisfies a second time threshold, determine, as feasibility validity, whether the trajectory can be executed by the autonomous vehicle, generate a validity signal based at least in part on one or more of the staleness validity, the timeliness validity, or the kinematic validity, and control the autonomous vehicle according to the trajectory based at least in part on the validity signal. The collision avoidance system that executes the operations.

[0229] 2. The system according to clause 1, wherein the operations further include receiving a current state of the autonomous vehicle, determining a potential state of the autonomous vehicle based at least in part on the trajectory, and determining whether the autonomous vehicle can move to the potential state based at least in part on the current state.

[0230] 3. The current state of the autonomous vehicle includes one or more of a first position, a first orientation, a first yaw rate, or a first speed, and the potential state of the autonomous vehicle includes one or more of a second position, a second orientation, a second yaw rate, or a second speed. Determining whether the autonomous vehicle can move from the current state to the potential state based at least in part on the current state includes determining at least one of an acceleration or a deceleration in at least one direction for the autonomous vehicle to progress from the current state to the potential state, and determining whether at least one of the acceleration or the deceleration in the at least one direction meets a corresponding realizable acceleration or deceleration threshold. The system according to clause 2 further includes this.

[0231] 4. The operation is receiving a second trajectory for navigating the autonomous vehicle through the environment, where the second trajectory is an alternative to the trajectory, the receiving, determining a third time difference between a fourth time when the second trajectory was created and the current time, determining a fourth time difference between a fifth time when the second trajectory was received and the third time when the previous trajectory was received, determining, as a second kinematic validity, whether the second trajectory can be executed by the autonomous vehicle, generating a second validity signal based at least in part on one or more of the third time difference, the fourth time difference, or the second kinematic validity, determining at least the trajectory and a trajectory resulting from the second trajectory based on the validity signal and the second validity signal, and controlling the autonomous vehicle according to the resulting trajectory. The system according to any one of clauses 1 to 3 further includes this.

[0232] 5. Determining the resulting trajectory includes determining the resulting trajectory from at least the trajectory, the second trajectory, and the stored trajectory, the stored trajectory being associated with a third validity value, and selecting the resulting trajectory being based on the validity signal, the second validity signal, and the third validity signal, the system according to any one of clauses 1 to 4.

[0233] 6. A method comprising receiving a trajectory for navigating a device, determining at least one of whether the trajectory is consistent with the current state of the device as consistency validity, whether the trajectory can be executed by the device as kinematic validity, or whether the trajectory was received within a time threshold as timeliness validity, generating a validity signal based at least in part on one or more of the consistency validity, the kinematic validity, or the timeliness validity, and controlling the device according to the trajectory based at least in part on the validity signal.

[0234] 7. The method of clause 6 further comprising determining a time difference between the time the trajectory was created and the current time, determining staleness validity based on whether the time difference meets a time threshold, and generating the validity signal being further based at least in part on the staleness validity.

[0235] 8. The method of clause 6 or 7 further comprising determining the consistency validity by generating a projection of the device onto the trajectory, comparing the current state of the device with the projection to generate projection data, and determining the consistency validity based on whether the projection data meets at least one deviation threshold.

[0236] 9. Determining the validity of the consistency includes generating a projection of the device on the trajectory and determining at least one physical difference between the current state of the device and the projection, wherein the at least one physical difference includes a distance, the determining, and determining the validity of the consistency based on whether the at least one physical difference meets at least one deviation threshold, and further includes the method according to any one of clauses 6 to 8.

[0237] 10. As the kinematic validity, determining whether the trajectory can be executed by the device includes receiving a potential state of the device for following the trajectory and determining whether the device can occupy the potential state at least partially based on the current state, and further includes the method according to any one of clauses 6 to 9.

[0238] 11. The current state of the device includes one or more of a first position, a first orientation, a first yaw rate, or a first velocity, and the potential state of the device includes one or more of a second position, a second orientation, a second yaw rate, or a second velocity, and the method according to any one of clauses 6 to 10.

[0239] 12. Receiving a second trajectory for the device and, as a second consistency validity, determining whether the second trajectory matches the current state of the device, as a second kinematic validity, determining whether the second trajectory can be executed by the device, or as a second timeliness validity, determining whether the second trajectory was received within a second time threshold, and determining at least one of them. Generating a second validity signal based at least in part on one or more of the second consistency validity, the second kinematic validity, or the second timing validity; selecting at least the trajectory and a trajectory resulting from the second trajectory based on the validity signal and the second validity signal; and controlling the device according to the resulting trajectory, the method according to any one of clauses 6-11.

[0240] 13. Further comprising determining a first collision value indicating whether the device navigating the trajectory is likely to cause a collision and a second collision value indicating whether the device navigating the second trajectory is likely to cause a collision, and selecting the resulting trajectory from at least the trajectory and the second trajectory is further based on the first collision value and the second collision value, the method according to any one of clauses 6-12.

[0241] 14. Further comprising receiving an operating status of at least one subsystem of the device, and selecting the resulting trajectory from at least the trajectory and the second trajectory is at least partially based on the operating status, the method according to any one of clauses 6-13.

[0242] 15. Further comprising receiving an operating status of at least one subsystem of the device and reducing one of a plurality of capacity limitations based on the operating status to result in a reduced capacity limitation, and determining whether the trajectory can be executed by the device as the kinematic validity is further based on the reduced capacity limitation, the method according to any one of clauses 6-14.

[0243] 16. A non-transitory computer-readable storage medium storing executable instructions that, when executed by one or more processors of a computer system, cause the computer system to receive a first trajectory and a second trajectory for navigating a device, a first consistency validity indicating whether the first trajectory matches the current state of the device, and a second consistency validity indicating whether the second trajectory matches the current state of the device, a first kinematic validity indicating whether the first trajectory can be executed by the device, and a second kinematic validity indicating whether the second trajectory can be executed by the device, or a first timeliness validity indicating whether the first trajectory was received within a time threshold, and a second timeliness validity indicating whether the second trajectory was received within the time threshold, determine at least one of the first consistency validity, the second consistency validity, the first kinematic validity, the second kinematic validity, the first timeliness validity, or the second timeliness validity, determine one of the first trajectory or the second trajectory based on at least one of the first consistency validity, the second consistency validity, the first kinematic validity, the second kinematic validity, the first timeliness validity, or the second timeliness validity to result in a selected trajectory, and cause the device to operate according to the selected trajectory.

[0244] 17. The non-transitory computer-readable storage medium according to clause 16, wherein the instructions, when executed by the one or more processors of the computer system, further cause the computer system to determine one of the first trajectory or the second trajectory based at least further on a hierarchy of trajectories to result in the selected trajectory, the hierarchy of trajectories including at least the first trajectory and the second trajectory.

[0245] 18. The validity of the first consistency indicates whether the first trajectory matches the current state of the device in at least one of time or space, and the validity of the second consistency indicates whether the second trajectory matches the current state of the device in at least one of time or space. The non-transitory computer-readable storage medium according to clause 16 or 17.

[0246] 19. When the instructions are executed by the one or more processors of the computer system, the computer system is further caused to obtain at least the operating status of at least one subsystem of the device, select one of the first trajectory or the second trajectory, and cause the selected trajectory, which is further at least partially based on the operating status. The non-transitory computer-readable storage medium according to any one of clauses 16 to 18.

[0247] 20. When the instructions for determining the first kinematic validity and the second kinematic validity are executed by the one or more processors of the computer system, the computer system is further caused to obtain a first set of values indicating the current state of the device, a second set of values indicating the potential state of the device for following the first trajectory, and a third set of values indicating the second potential state of the device for following the second trajectory, and determine, based at least in part on the current state, whether the device can occupy the potential state and the second potential state. The non-transitory computer-readable storage medium according to any one of clauses 16 to 19, further including instructions for causing at least one of the above to be performed.

[0248] Feasibility Check for Vehicle Trajectory Selection 21. An autonomous vehicle monitoring system, One or more processors and a memory for storing instructions, which, when executed by the one or more processors, cause the system to obtain a set of feasibility limitations for the movement of the autonomous vehicle, receive the current state of the autonomous vehicle, receive a potential trajectory for verification, determine the kinematics of the autonomous vehicle to proceed from the current state to the potential trajectory, determine a validity signal indicating whether the kinematics satisfy the set of feasibility limitations by comparing the kinematics with the feasibility limitations, and operate the autonomous vehicle according to the potential trajectory based at least in part on the validity value. The autonomous vehicle monitoring system.

[0249] 22. The system according to clause 21, wherein the set of feasibility limitations includes at least one of the maximum speed of the autonomous vehicle, the maximum deceleration of the autonomous vehicle in a first direction, the maximum acceleration of the autonomous vehicle in the first direction, the maximum acceleration of the autonomous vehicle in a second direction, and the maximum resultant acceleration.

[0250] 23. The system according to clause 21 or 22, wherein determining the kinematics of the autonomous vehicle to proceed from the current state to the trajectory further includes determining at least one of the deceleration of the autonomous vehicle in the first direction, the acceleration of the autonomous vehicle in the first direction, the acceleration of the autonomous vehicle in the second direction, and the resultant acceleration.

[0251] 24. The set of feasibility limitations includes at least one physical operation limitation imposed on the autonomous vehicle, and when the instructions are executed by the one or more processors, the system further updates at least one value of the set of feasibility limitations based on the current operation status of at least one subsystem of the autonomous vehicle. The system according to any one of clauses 21 to 23.

[0252] 25. Determining the kinematics of the autonomous vehicle to progress from the current state to the potential trajectory further includes determining a first position on the potential trajectory and comparing the first position with a second position associated with the current state of the autonomous vehicle, the system according to any one of clauses 21 to 24.

[0253] 26. A method includes obtaining at least one feasibility limit for the movement of a device, where the at least one feasibility limit includes an acceleration or deceleration limit in at least one direction, receiving the current state of the device, receiving a potential trajectory for verification, determining the kinematics of the device to progress from the current state to the trajectory, and determining a validity value indicating whether the kinematics satisfy the at least one feasibility limit.

[0254] 27. The method according to clause 26, where the at least one feasibility limit includes at least one physical operation limit imposed on the device to enhance the safety in the operation of the device.

[0255] 28. The method according to clause 26 or 27 further includes updating at least one feasibility limit based on the current operating status of at least one subsystem of the device.

[0256] 29. The method according to any one of clauses 26 to 28, where the at least one feasibility limit further includes at least one of the maximum speed of the device, the maximum acceleration of the device, the maximum deceleration of the device, or the maximum lateral acceleration of the device.

[0257] 30. The method according to any one of clauses 26 to 29, wherein the device is an autonomous vehicle, and determining the kinematics of the device to proceed from the current state to the trajectory further includes determining at least one of the deceleration of the autonomous vehicle in the first direction, the acceleration of the autonomous vehicle in the first direction, or the acceleration of the autonomous vehicle in the lateral direction.

[0258] 31. The method according to any one of clauses 26 to 30, wherein determining the kinematics of the device to proceed from the current state to the potential trajectory further includes determining a first position on the potential trajectory and comparing the first position with a second position associated with the current state of the autonomous vehicle.

[0259] 32. The method according to any one of clauses 26 to 31, wherein determining the first position on the potential trajectory further includes determining the first position so as to minimize the distance between the first position and the second position.

[0260] 33. The method according to any one of clauses 26 to 32, wherein the at least one feasibility limit is determined based on the current trajectory of the device or the potential trajectory of the device.

[0261] 34. The method according to any one of clauses 26 to 33, further including operating the device according to the potential trajectory, at least partially based on the validity value.

[0262] A non-transitory computer-readable storage medium storing executable instructions, which, when executed by one or more processors of a computer system, cause the computer system to perform at least: obtaining at least one feasibility limit for the movement of an autonomous vehicle, wherein the at least one feasibility limit includes an acceleration or deceleration limit in at least one direction; receiving a current state of the autonomous vehicle; determining a kinematics of the autonomous vehicle for proceeding to the trajectory from the current state; and determining a validity value indicating whether the kinematics is equal to or less than the at least one feasibility limit.

[0263] 36. The non-transitory computer-readable storage medium according to clause 35, wherein the at least one feasibility limit includes at least one physical operation limit of the device.

[0264] 37. The non-transitory computer-readable storage medium according to clause 35 or 36, wherein the at least one feasibility limit further includes at least one of a maximum speed of the device, a maximum acceleration of the device, a maximum deceleration of the device, or a maximum lateral acceleration of the device.

[0265] 38. The non-transitory computer-readable storage medium according to any one of clauses 35 to 37, wherein determining the kinematics of the device for proceeding to the trajectory from the current state further includes determining at least one of a deceleration of the autonomous vehicle in the first direction, an acceleration of the autonomous vehicle in the first direction, or an acceleration of the autonomous vehicle in the lateral direction.

[0266] 39. The non-transitory computer-readable storage medium according to any one of clauses 35 to 38, wherein as a result of the executable instructions being executed by one or more processors of a computer system, the computer system is further caused to update at least one feasibility limit based at least on the current operating status of at least one subsystem of the device.

[0267] 40. The non-transitory computer-readable storage medium according to any one of clauses 35 to 39, wherein the at least one feasibility limit is determined based on the current trajectory of the device or the potential trajectory of the device.

[0268] Consistency Check for Vehicle Trajectory Selection 41. A collision avoidance system, comprising one or more processors and a memory storing instructions, wherein when the instructions are executed by the one or more processors, the system is caused to receive a first set of values indicative of a current state of an autonomous vehicle, the first set of values including one or more of a first position, a first orientation, a first yaw rate, or a first speed, the receiving; receive the potential trajectory of the autonomous vehicle; determine a second set of values indicative of a potential state of the autonomous vehicle on the potential trajectory, the second set of values including one or more of a second position, a second orientation, a second yaw rate, or a second speed, the determining; determine, as a validity signal, whether the second set of values differs from the first set of values by only a corresponding set of threshold amounts; and operate the autonomous vehicle along the potential trajectory based at least in part on the validity signal.

[0269] 42. The system according to clause 41, wherein the set of threshold amounts includes at least two of a distance threshold, a time threshold, a steering threshold, a yaw threshold, a speed deviation threshold, or a collinearity threshold.

[0270] 43. Determining the validity value further includes performing a comparison between the first set of values and the second set of values to determine a set of deviation values, and determining the validity signal based at least in part on performing a second comparison between the set of deviation values and the set of threshold amounts. The system according to clause 41 or 42.

[0271] 44. Determining the second set of values includes determining the second position based on the proximity to the first position. The system according to any one of clauses 41 to 43.

[0272] 45. When the instruction is executed by the one or more processors, the system further receives a second potential trajectory of the autonomous vehicle, and determines a third set of values indicating a second potential state of the autonomous vehicle for following the second potential trajectory, where the third set of values includes one or more of a third position, a third orientation, a third yaw rate, or a third speed. Determining, as a second validity signal, determining whether the autonomous vehicle can occupy the second potential state based at least in part on the current state, selecting one of the potential trajectory or the second potential trajectory based on the validity signal and the second validity signal, resulting in the selected trajectory, and operating the autonomous vehicle according to the selected trajectory. The system according to any one of clauses 41 to 44.

[0273] 46. A method includes obtaining a current state of a device on a current trajectory, obtaining a potential state of the device for following a potential trajectory, determining a validity value indicating whether the potential state differs from the current state by at least one threshold amount, indicating whether the potential trajectory matches the current trajectory, and operating the device according to the potential trajectory based at least in part on the validity value.

[0274] 47. The method according to clause 46, wherein the current state of the device includes one or more of a first position, a first orientation, a first yaw rate, or a first speed, and the potential state of the device includes one or more of a second position, a second orientation, a second yaw rate, or a second speed.

[0275] 48. The method according to clause 46 or 47, wherein the at least one threshold includes at least one of a distance difference, a time difference, a steering threshold, a yaw threshold, a speed deviation threshold, or a collinearity threshold.

[0276] 49. Determining the validity value further includes performing a comparison between the current state and the potential state to determine a set of deviation values, and determining the validity signal based at least in part on performing a second comparison between the set of deviation values and the at least one threshold amount. The method according to any one of clauses 46 to 48.

[0277] 50. The method according to any one of clauses 46 to 49, wherein the current state includes a first position, and obtaining the potential state of the device further includes determining a second position on the potential trajectory that is the shortest distance from the first position.

[0278] 51. Determining the validity value further includes determining whether the potential trajectory is collinear with the current trajectory. The method according to any one of clauses 46 to 50.

[0279] 52. The method according to any one of clauses 46 to 51, wherein the at least one threshold is determined independently of the operating conditions of the device.

[0280] 53. Determining the validity value is further based at least in part on a time difference between a first time when the potential trajectory was created and the current time. The method according to any one of clauses 46 to 52.

[0281] Obtaining a third set of values indicative of a second potential state of the device for following a second potential trajectory; determining a second validity value indicative of whether the device can occupy the second potential state based at least in part on the current state; selecting one of the potential trajectory or the second potential trajectory based on the validity value and the second validity value to yield a selected trajectory; and operating the device in accordance with the selected trajectory, the method according to any one of clauses 46 to 53.

[0282] 55. A non-transitory computer-readable storage medium storing executable instructions, which, when executed by one or more processors of a computer system, cause the computer system to at least receive a first state of a device navigating a first trajectory, the first state including at least one of a first position and a first velocity; receive a potential state of the device following a potential trajectory, the potential state including at least one of a second position and a second velocity; determine a validity value indicative of whether the first state is within at least one threshold of the second state; and operate the device in accordance with the potential trajectory based at least in part on the validity value.

[0283] 56. The non-transitory computer-readable storage medium according to clause 55, wherein determining the validity value further includes determining a distance between the first position and the second position; comparing the distance with a distance threshold of the at least one threshold; determining a velocity difference between the first velocity and the second velocity; and comparing the velocity difference with a velocity threshold of the at least one threshold value.

[0284] 57. The non-transitory computer-readable storage medium according to clause 55 or 56, wherein the first state of the device further includes at least one of a first orientation or a first yaw rate, and the potential state of the device includes one or more of a second orientation or a second yaw rate.

[0285] 58. The non-transitory computer-readable storage medium according to any one of clauses 55 to 57, wherein determining the validity value is at least partially based on a time difference between a first time when the potential trajectory was created and the current time.

[0286] 59. The non-transitory computer-readable storage medium according to any one of clauses 55 to 58, wherein determining the at least one threshold includes at least one of a distance difference, a time difference, a steering threshold, a yaw threshold, a speed deviation threshold, or a collinearity threshold.

[0287] Correction Limitations on Vehicle Dynamics of Trajectory 60. An autonomous vehicle monitoring system comprising one or more processors and a memory storing instructions, which, when executed by the one or more processors, cause the system to receive inputs from a plurality of sensors of the autonomous vehicle, wherein one of the plurality of inputs is associated with an operating state of a subsystem of the autonomous vehicle; determine, at least in part based on the input, whether the subsystem is operating in a reduced operating state; determine reduced capabilities of a plurality of capabilities, at least in part based on determining that the subsystem is operating in the reduced operating state, the reduced capabilities being less than nominal operating capabilities; receive a potential trajectory of the autonomous vehicle; and determine a validity signal indicating whether the autonomous vehicle following the potential trajectory exceeds the reduced capabilities.

[0288] 61. The system of claim 60, wherein the reduced operating state is one of a plurality of operating states, and determining the reduced capacity limit includes determining an amount by which to reduce the nominal operating capacity limit based at least in part on a difference between the reduced operating state and a normal operating state of the subsystem.

[0289] 62. The system of claim 60 or 61, wherein the plurality of capacity limits includes at least one of a maximum speed of the autonomous vehicle, a maximum deceleration of the autonomous vehicle in a first direction, a maximum acceleration of the autonomous vehicle in the first direction, a maximum acceleration of the autonomous vehicle in a second direction, or a maximum combined acceleration.

[0290] 63. The system of any one of claims 60-62, wherein when the command is executed by the one or more processors, the system further reduces the capacity limit by an amount based at least in part on at least one characteristic of the potential trajectory.

[0291] 64. The system of any one of claims 60-63, wherein determining the validity signal further includes obtaining a first set of values indicative of a first state of the autonomous vehicle, obtaining a second set of values indicative of a potential state of the autonomous vehicle following the potential trajectory, and determining, as the validity signal, whether the device can occupy the potential state based at least in part on the first state and the reduced capacity limit.

[0292] 65. A method comprising: receiving a signal associated with an operating status of a subsystem of a device, wherein the signal indicates a reduced operating status of the subsystem; determining, based on the signal, an ability limitation imposed on the device; receiving a potential trajectory for the device to travel along; determining a validity signal indicating whether the device following the potential trajectory would exceed the ability limitation; and operating the device along the potential trajectory at least partially based on the validity signal.

[0293] 66. The method of clause 65, wherein determining the ability limitation includes determining an amount by which a default ability limitation is reduced based on the signal.

[0294] 67. The method of clause 65 or 66, wherein the amount by which the default ability limitation is reduced is determined at least partially based on a difference between the reduced operating state and a normal operating state of the subsystem.

[0295] 68. The method of any one of clauses 65 - 67, further comprising: receiving a second signal associated with a second operating status of a second subsystem of the device, wherein the second signal indicates a second reduced operating status of the second subsystem; and determining the ability limitation imposed on the device based on the signal and the second signal.

[0296] 69. The method of any one of clauses 65 - 68, wherein the ability limitation includes at least one of a maximum speed of the device, a maximum deceleration of the device in a first direction, a maximum acceleration of the device in the first direction, a maximum acceleration of the device in a second direction, a maximum resultant acceleration, a maximum yaw rate, a maximum yaw acceleration, or a maximum gradient.

[0297] 70. Determining the ability limit imposed on the device includes reducing the ability limit by an amount determined at least in part based on at least one characteristic of the trajectory, the method according to any one of clauses 65 to 69.

[0298] 71. The method according to any one of clauses 65 to 70, further comprising determining a second ability limit imposed on the device based on whether a second signal indicating a second operating status of the subsystem is received within a period.

[0299] 72. The method according to any one of clauses 65 to 71, further comprising receiving a second signal indicating an increased or restored operating status of the subsystem, increasing the ability limit based on the second signal to result in an increased ability limit, and determining a second validity signal indicating whether the device following the potential trajectory exceeds the increased ability limit.

[0300] 73. Determining the validity signal further includes obtaining a first set of values indicating a first state of the device, obtaining a second set of values indicating a potential state of the device following the trajectory, and determining, as the validity signal, whether the device can occupy the potential state based at least in part on the first state and the capacity limit, the method according to any one of clauses 65 to 72.

[0301] 74. A non-transitory computer-readable storage medium storing executable instructions that, when executed by one or more processors of a computer system, cause the computer system to at least receive an input from a sensor of a device, wherein the input is associated with a subsystem of the device, receive an operation restriction imposed on the device based on the input, receive a trajectory for the device to travel, determine a validity signal indicating whether the device following the trajectory exceeds the operation restriction, and operate the device along the trajectory based at least in part on the validity signal.

[0302] 75. The operation restriction includes a reduced operation restriction that is less than a nominal operation restriction, and the instructions, when executed by the one or more processors, further cause the computer system to determine the operation restriction from a plurality of operation restrictions based at least in part on the subsystem associated with the input. The non-transitory computer-readable storage medium according to clause 74.

[0303] 76. Determining the operation restriction includes determining an amount that restricts at least one operation characteristic of the device based on the input. The non-transitory computer-readable storage medium according to clause 74 or 75.

[0304] 77. The operation characteristic of the device includes at least one of a maximum speed of the device, a maximum deceleration of the device in a first direction, a maximum acceleration of the device in the first direction, a maximum acceleration of the device in a second direction, a maximum resultant acceleration, a maximum yaw rate, a maximum yaw acceleration, or a maximum gradient. The non-transitory computer-readable storage medium according to any one of clauses 74 to 76.

[0305] The non - transitory computer - readable storage medium according to any one of clauses 74 to 77, wherein the amount that restricts the at least one operating characteristic of the device is determined from a set of amounts corresponding to different inputs indicating different levels of operation of the subsystem.

[0306] 79. Determining the validity signal further includes obtaining a first set of values indicating a first state of the device, obtaining a second set of values indicating a potential state of the device following the trajectory, and determining, based at least in part on the operation restriction, whether the device can move from the first state of the device to the potential state for following the trajectory. The non - transitory computer - readable storage medium according to any one of clauses 74 to 78.

Claims

1. A method comprising: a computer system receiving a trajectory for an autonomous vehicle to travel, the trajectory being generated based on a capacity limit associated with a subsystem of the autonomous vehicle when the subsystem is operating in a pre-update operating status; a computer system receiving a signal associated with an operating status of the subsystem of the autonomous vehicle, the signal indicating a reduced operating status of the subsystem of the autonomous vehicle, the reduced operating status indicating a reduction in an amount by which the autonomous vehicle can accelerate or decelerate in a primary direction of travel and / or a lateral direction; the computer system determining an updated capacity limit for the autonomous vehicle based on the signal, the updated capacity limit being updated from the capacity limit associated with the subsystem operating in the pre-update operating status; the computer system determining a validity signal indicating whether the autonomous vehicle following the trajectory exceeds the updated capacity limit; the computer system determining that the autonomous vehicle operates according to the trajectory based at least in part on the validity signal; the computer system generating a future trajectory using the updated capacity limit.

2. The method of claim 1, wherein determining the updated capacity limit comprises the computer system determining an amount by which to reduce a default capacity limit based on the signal.

3. The method of claim 2, wherein the amount by which to reduce the default capacity limit is determined based at least in part on a difference between the reduced operating status of the subsystem and the pre-update operating state.

4. The computer system receiving a second signal from a second subsystem of the autonomous vehicle, the second signal indicating a second reduced operating status of the second subsystem of the autonomous vehicle, the second reduced operating status of the second subsystem indicating a reduction in a second amount by which the autonomous vehicle can accelerate or decelerate in a primary direction of travel and / or a lateral direction; The method according to any one of claims 1 to 3, further comprising: the computer system determining the updated ability limit imposed on the autonomous vehicle based on the signal and the second signal.

5. The method according to any one of claims 1 to 4, wherein the updated ability limit includes at least one of a maximum speed of the autonomous vehicle, a maximum deceleration of the autonomous vehicle in a first direction, a maximum acceleration of the autonomous vehicle in the first direction, a maximum acceleration of the autonomous vehicle in a second direction, a maximum combined acceleration, a maximum yaw rate, a maximum yaw acceleration, or a maximum gradient.

6. The method according to any one of claims 1 to 5, further comprising: the computer system determining a second ability limit imposed on the autonomous vehicle based on whether a second signal indicating a second operating status of the subsystem is received within a period.

7. the computer system receiving a second signal indicating an increased or restored operating status of the subsystem; the computer system increasing the updated ability limit based on the second signal to result in an increased ability limit; The method according to any one of claims 1 to 6, further comprising: the computer system determining a second validity signal indicating whether the autonomous vehicle following the trajectory exceeds the increased ability limit.

8. Determining the validity signal comprises: the computer system obtaining a first set of values indicating a first state of the autonomous vehicle; the computer system obtaining a second set of values indicating a potential state of the autonomous vehicle following the trajectory; The method according to any one of claims 1 to 7, further comprising: the computer system determining, as the validity signal, whether the autonomous vehicle can occupy the potential state based at least in part on the first state and the ability limit of the autonomous vehicle.

9. one or more processors; a non-transitory computer-readable storage medium storing instructions executable by the one or more processors, the system comprising: the instructions causing the system to at least Receiving a track for the autonomous vehicle to travel on, the track being generated based on the capacity limitations associated with the subsystem of the autonomous vehicle when the subsystem of the autonomous vehicle is operating in a pre-update operating status, Receiving an input from a sensor of the autonomous vehicle, the input indicating a reduced operating status of a subsystem of the autonomous vehicle, the reduced operating status of the subsystem indicating a reduction in the amount by which the autonomous vehicle can accelerate or decelerate in a primary direction of travel and / or a lateral direction, Determining updated capacity limitations to impose on the autonomous vehicle based on the input, the updated capacity limitations being updated from the capacity limitations associated with the subsystem operating in the pre-update operating status, Determining a validity signal indicating whether the autonomous vehicle following the track exceeds the updated capacity limitations, Determining, at least in part based on the validity signal, that the autonomous vehicle operates in accordance with the track, Generating a future track using the updated capacity limitations, a non-transitory computer-readable storage medium that causes the above to be performed, a system comprising.

10. Determining the updated capacity limitations includes Determining, based on the input, an amount that limits at least one operating characteristic of the autonomous vehicle, the system of claim 9.

11. The operating characteristics of the autonomous vehicle include at least one of a maximum speed of the autonomous vehicle, a maximum deceleration of the autonomous vehicle in a first direction, a maximum acceleration of the autonomous vehicle in the first direction, a maximum acceleration of the autonomous vehicle in a second direction, a maximum combined acceleration, a maximum yaw rate, a maximum yaw acceleration, or a maximum gradient, the system of claim 10.

12. The amount that limits the at least one operating characteristic of the autonomous vehicle is determined from a set of amounts corresponding to different inputs indicating different levels of operation of the subsystem, the system of claim 10 or claim 11.

13. Determining the validity signal includes Obtaining a first set of values indicating a first state of the autonomous vehicle, Obtaining a second set of values indicating a potential state of the autonomous vehicle following the track, Determining whether the autonomous vehicle can move from the first state of the autonomous vehicle to the potential state of the autonomous vehicle following the trajectory, based at least in part on the updated ability limit, further comprising the system according to any one of claims 9 to 12.

Images