Vehicle safety system

Abstract

Techniques for authorizing or determining the trajectory of a vehicle are described herein. The trajectory management component can receive status and / or error data from other safety system components and select, or in another situation determine, a safe and valid vehicle trajectory. The perception component of the safety system can authorize a trajectory that the trajectory management component can wait for to select a vehicle trajectory, can authorize a trajectory stored in a queue, and / or can utilize kinematics for trajectory authorization. The filter component of the safety system can filter and remove objects based on the trajectories stored in the queue. The collision detection component of the safety system can determine a collision state based on the trajectories stored in the queue or determine that a collision state is such that the trajectory management component can wait to select a vehicle trajectory or, in another situation, determine.

Description

Technical Field

[0001] The present invention relates to a vehicle safety system.

[0002] Cross-reference to related applications This patent application claims the benefit of U.S. Utility Patent Application Serial No. 17 / 842,469, filed Jun. 16, 2022. Application Serial No. 17 / 842,469 is hereby incorporated by reference in its entirety.

Background Art

[0003] Vehicles include various systems used to guide the vehicle through an environment that includes static and / or dynamic objects. A safety system can analyze sensor data regarding the environment. For example, sensor data such as lidar data, radar data, camera data, etc. can be used to determine the impact of detected objects on a potential operation of the vehicle. Objects encountered in the environment can include moving or potentially moving dynamic objects (e.g., vehicles, motorcycles, bicycles, pedestrians, animals, etc.) and / or static objects (e.g., buildings, road surfaces, trees, signs, fences, parked vehicles, etc.). The vehicle can generate a trajectory and use the safety system to ensure that the trajectory is safe. The trajectory must be generated in real time or near real time so that the vehicle can move accurately and effectively through the environment. The trajectory can be used to control the vehicle taking into account events around the vehicle. The decision to be used by a vehicle crossing the environment can be made to ensure the safety of the vehicle's passengers or other people or objects close to the vehicle.

[0004] The detailed description is set forth with reference to the accompanying drawings. In the drawings, the leftmost digit(s) of the reference numeral identifies the drawing in which the reference numeral first appears. The use of the same reference numeral in different drawings indicates similar or identical components or features.

Brief Description of the Drawings

[0005]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Modes for Carrying Out the Invention

[0006] Detailed description Techniques for authorizing or determining the trajectory of a vehicle are described herein. In some examples, collision avoidance of a vehicle may include a system (or "vehicle system") having a primary system and a vehicle safety system (or "safety system"). The safety system can include, but is not limited to, a trajectory management component, a perception component, a filtering component, and / or a collision detection component. The trajectory management component can receive status and / or error data from other safety system components and select a safe and effective trajectory while the vehicle is moving through the environment. The perception component can utilize sensor data to determine a corridor (e.g., a boundary region corresponding to the trajectory) associated with the trajectory and improve the prediction and avoidance of collisions by the vehicle. The perception component can authorize a trajectory that the trajectory management component can wait for to select the vehicle's trajectory, can authorize a trajectory stored in a queue, and / or can utilize kinematic data for the authorization of the trajectory. The filtering component can perform a collision check and filter out and remove an object having a trajectory that is determined not to be associated with a possible intersection of the vehicle's trajectory and the object's trajectory. In some examples, the filtering component can filter out and remove an object based on a trajectory stored in a queue. The collision detection component can determine the collision state of the vehicle and utilize the collision state to determine a safe maneuver of the vehicle that would be performed by the vehicle. The collision detection component can determine the collision state based on a trajectory stored in a queue or can determine the collision state when it is possible for the trajectory management component to wait to select the vehicle's trajectory.

[0007] The trajectory management component can select or, in another situation, determine a trajectory from the trajectories generated and received by the primary system based on the data generated and received by other components of the safety system. The trajectories can include a planned trajectory, a first safety stop trajectory, a second safety stop trajectory, and so on. In various examples, the above-mentioned planned trajectory may include a plurality of trajectories determined by the primary system, while the first and subsequent safety stop trajectories can be variations of the above-mentioned planned trajectory with different longitudinal deceleration profiles. Of course, the above-mentioned safety stop trajectories may also include, in at least some examples, a lateral deviation from the planned trajectory. The data can be received from other perception components and can be utilized by the trajectory management component. The data can include authorization results (e.g., authorization results of trajectories), status data, or error data. The data can be utilized to select a trajectory that ensures time compliance, consistency, feasibility, and collision inspection. In some examples, the trajectory management component can select the vehicle's trajectory at any time (e.g., before or after reception) regarding whether some or all of the data generated by other perception components are received. In other examples, the trajectory management component can wait to receive some or all of the data generated by other perception components before selecting the vehicle's trajectory.

[0008] The perception component can determine data (or "perception data") associated with a corridor that represents a geometric region related to the vehicle (e.g., bounding a region that will be immediately traversed by the vehicle), and / or data associated with objects in the vicinity of the vehicle. The data associated with the objects can include classification, instance and / or semantic segmentation, bounding boxes, tracks, route data, global and / or local map data, etc. The corridor can include a part of the environment (e.g., a part or the whole), such as a part of a lane (e.g., a part or the whole) and / or a part of an object (e.g., a part or the whole). The data associated with the objects and / or the corridor can be determined based on one or more of the trajectories received from the primary system, the trajectories received from the primary system and stored in the queue of the perception component, and the kinematics and / or trajectories used for authorization (e.g., authorization of a trajectory, or authorization of multiple trajectories). The trajectories can be analyzed by the perception component based on the corridor or based on a subset of the corridor.

[0009] The filter component can receive trajectories and filter out and remove objects in the environment based on the received trajectories. The filter component can be integrated with an orbit management component, and / or the trajectories can be stored in a queue of the filter component. The filter component can use the trajectories to determine objects within the vicinity of the vehicle and filter out and remove objects that are determined not to be associated with a potential intersection of the vehicle's trajectory and the object's trajectory. The objects can be filtered out and removed based on the results of an algorithm (or "model") being executed that indicates that a metric (e.g., a number metric and / or a probability metric) of a driving simulation that culminates in a collision with the object, or a collision of an abnormal approach (or other potential safety-threatening factor), is below a threshold metric, and / or can be filtered out and removed based on other factors such as, for example, vehicle speed, driving conditions, the risk tolerance configuration of the vehicle safety system. The algorithm can be used to determine possible collision points and / or intersection times between the vehicle and the object based on vehicle information (e.g., the vehicle's current trajectory, the vehicle's current speed, the vehicle's planned path, etc.) and object information (the object's current trajectory and / or speed, one or more predicted trajectories and / or speeds of the object). In at least some examples, the filtering described above can enable the vehicle safety system to direct as many resources as possible to the objects that expose it to the greatest risk of interacting with the vehicle.

[0010] The collision detection component can receive trajectories and determine a collision state based on the received trajectories. The collision state can be determined based on a predicted distance (e.g., a predicted distance determined based on the trajectory of an object and the trajectory of a vehicle) between an object and a vehicle that is less than a threshold distance. A velocity vector and / or possible collision characteristics can be predicted based on determining a possible collision with an object. The predicted velocity vector and / or possible collision characteristics can be used to determine the collision state based on the probability of a collision associated with the vehicle's trajectory, the point of impact, the relative speed of the collision, and / or the angle of the relative collision. In some examples, the collision state can be determined such that the trajectory management component can wait to select a vehicle trajectory. In other examples, the vehicle trajectory used to determine the collision state can be stored in a queue. The collision state can be used to determine a safe maneuver (e.g., a trajectory) for the vehicle. The collision detection component operates separately from other vehicle systems and / or components used in perception, prediction, trajectory planning, etc., to provide redundancy, error checking, and / or validation for other vehicle systems and / or components.

[0011] The techniques described herein can improve how a computing device functions in many additional ways. The techniques can be used to optimize available computing resources by performing operations that limit the impact on available resources (as compared to not implementing the techniques). Based on one or more outputs from a model of a filter component, the available computing resources can be directed to the most relevant objects during vehicle planning to improve the safety of the vehicle as it travels in the environment. A trajectory can be selected for the vehicle, or determined in another situation, to more effectively avoid and / or mitigate a potential collision with other moving objects in the environment. After detecting a potential collision, after executing a new perception model, and / or after generating or authorizing a new trajectory on the fly after a potential collision has been detected, the vehicle system can determine a trajectory and accurately switch to the selected trajectory.

[0012] Moreover, by operating the trajectory management component to wait for results from one or more other safety system components (e.g., a perception component, a filter component, and / or a collision detection component) that are fewer in number than all, computing resources can be conserved by not operating based on data that will later be determined to be stale or inaccurate. To avoid latency and / or lag in the operation of the trajectory management component, the trajectory management component can wait for some, but perhaps not all, of the other safety system components. Additionally or alternatively, the trajectory management component can operate without waiting for any of the other safety system components. By having the trajectory management component wait for fewer other safety system components that perform operations used to control the vehicle, the latency of the trajectory management component can be reduced.

[0013] By storing a trajectory in a queue of perception components used to determine data associated with a corridor and / or an object, computing resources that would otherwise be exhausted for an emergency in another situation can be utilized more resourcefully for a lower level of urgency. Integrating one or more safety system components (e.g., a filter component) with the trajectory management component and / or using the safety system component(s) to store the trajectory in the queue can eliminate unnecessary data processing, reduce redundant data processing, and / or reduce the urgency of data processing. By storing the trajectory(trajectories) in the corresponding queue(s) of the corresponding safety system component(s), the trajectory management component can be determined by the corresponding safety system component(s) that utilize the corresponding queue(s) and can switch back and forth from the trajectory(trajectories) (or fallback to a previously authorized trajectory) based on the information associated with each trajectory(trajectories) received from the corresponding safety system component. Computing resources that would otherwise be exhausted for an emergency in another situation can be utilized more resourcefully for a lower level of urgency based on a collision state being determined for a trajectory stored in a queue of a collision detection component. Processing data for a lower level of urgency allows for better time allocation and / or better prioritization of different types of data processing.

[0014] The techniques described in this specification can be implemented in several ways. Exemplary implementations are provided below with reference to the following drawings. Although applicable to vehicles, such as autonomous vehicles, the methods, apparatuses, and systems described in this specification can be applied to a variety of systems and are not limited to autonomous vehicles. In another example, the techniques can be utilized in an aviation or maritime context, or in any system configured to input data for determining movement associated with an object in an environment. Additionally, the techniques described in this specification can be used with real data (e.g., captured using one or more sensors), simulation data (e.g., generated by a simulator), or any two other types of data.

[0015] FIG. 1 is a block diagram of an exemplary vehicle system 100 of a vehicle proceeding through an environment in accordance with an example of the present disclosure, where the vehicle system 100 includes safety system components.

[0016] The vehicle system 100 can include a vehicle safety system 102 to ensure the safe operation of the vehicle. The vehicle safety system 102 can receive data (or "primary system data") from the primary system of the vehicle system 100. The data received from the primary system can include sensor data 104 and trajectory data 106. The sensor data 104 can include data received from any one or more sensors of one or more systems of the vehicle system 100 (e.g., sensor system(s) 606 as described below with reference to FIG. 6). The trajectory data 106 can include any data associated with one or more trajectories determined by one or more systems of the vehicle system 100 (e.g., a primary system that can be implemented as one or more of the first computing device(s) 604 as described below with reference to FIG. 6). In various examples, the above-described trajectory data 106 can include a related set of one or more desired vehicle states (e.g., orientation, position, steering angle, rotational rate, etc.) and / or commands (e.g., acceleration, torque, voltage, steering angle, etc.).

[0017] The track data 106 can include one or more tracks of the vehicle (e.g., different types of tracks (plural possible)). The track(s) (or "received track(s)") (or "candidate track(s)") can include a planned track and / or one or more other tracks (or "alternative track(s)") (e.g., a first safety stop track (or "error track"), a second safety stop track (or "high-priority error track"), a third safety stop track (or "immediate stop track"), etc.). The received planned track can be generated based on sensor data 104 that can include data (or "object data") associated with one or more objects in the environment. The planned track can include, for example, a nominal track for traversing the environment determined according to various systems such as those described in U.S. Patent No. 11,048,260 entitled "Adaptive Scaling in Trajectory Generation", the entire content of which is incorporated herein by reference. The alternative track(s) can include various types of error tracks and can include, for example, a critical error track.

[0018] The primary system data can include one or more other data (e.g., vehicle state data), some or all of which can be included in one or more of the trajectory data 106 and the sensor data 104, and / or can be separate from one or more of the trajectory data 106 and the sensor data 104. The vehicle state data can include various types of data, including performance metrics, component errors, alerts, system malfunctions, failures, and / or manual or automatic requests to change to or from an alternative trajectory. In some examples, the vehicle state data can include data that identifies system malfunctions, errors, alerts, etc. associated with any of the different types of received trajectories. Further, the vehicle state data can also include requests for an autonomous vehicle to follow a trajectory (or "requested trajectory"), which can include any of the received trajectories. The vehicle state data can be received from the primary system via internal signals according to the internal timing and processing cycles of the components of the primary system.

[0019] The vehicle safety system 102 can include an orbit management component 108 (e.g., a first component), a perception component 110 (e.g., a second component), a filter component 112 (e.g., a third component), and a collision detection (or “machine learning (ML)”) or (“neural network”) component 114 (e.g., a fourth component). In some examples, some (e.g., a portion or all) of the vehicle system data (e.g., sensor data 104 and / or orbit data 106) can be received by the orbit management component 108 from a primary system. In an example where a portion of the vehicle system data (e.g., orbit data 106) is received by the orbit management component 108, the remaining portion of the vehicle system data (e.g., sensor data 104) can be received by one or more of the remaining components of the orbit management component 108 and / or the vehicle safety system 102 (e.g., one or more of the perception component 110, the filter component 112, and the collision detection component 114). In some examples, the same vehicle system data can be received by one or more of the orbit management component 108, the perception component 110, the filter component 112, and the collision detection component 114. The orbit management component 108 can transmit some or all of the received portion of the orbit data 106 and / or the sensor data 104 to any one or more of the remaining components of the vehicle safety system 102 in some examples where the orbit management component 108 receives a portion of the orbit data 106 and / or the sensor data 104 (e.g., some or all of the portion of the orbit data 106 and / or the sensor data 104 can be received by any one or more of the remaining components of the vehicle safety system 102 from the primary system via the orbit management component 108).

[0020] Any component of the vehicle safety system 102 (the trajectory management component 108, the perception component 110, the filter component 112, and the collision detection component 114) can receive, but is not strictly limited to, a portion of the vehicle system data that is different from any other component of the vehicle safety system 102 as described above in the present disclosure. Any portion (e.g., a part or all) of the vehicle system data can be received by one or more of the components of the vehicle safety system 102 from the primary system (e.g., the trajectory data 106 and / or the sensor data 104 received by one or more of the remaining components of the vehicle safety system 102 from the primary system do not need to pass through the trajectory management component 108).

[0021] The trajectory management component 108 can process the received portion of the vehicle system data (e.g., the sensor data 104 and / or the trajectory data 106). In some examples, the trajectory management component 108 can determine the selected (or "output") trajectory that the vehicle follows. The trajectory management component 108 can determine the selected trajectory based on the trajectory approval result (e.g., the result associated with one or more stored possible trajectories), the vehicle state data (e.g., the state data from one or more vehicle components including vehicle sensors, vehicle platforms, attitude systems, vehicle monitoring systems, and / or other systems of the autonomous vehicle), and / or a manual or automated request to change to or from an alternative trajectory. The vehicle state data can include performance metrics, component errors, alerts, system malfunctions, and system failures.

[0022] In some examples, for instance, in a state where the planned trajectory and alternative trajectory(s) include the trajectory data 106, the trajectory management component 108 can determine any of the received trajectories (or "current trajectories") (e.g., the planned trajectory and alternative trajectory(s)) as the selected trajectory. In an alternative or additional example where the trajectory data 106 includes the planned trajectory but does not include alternative trajectory(s), the trajectory management component 108 can determine the planned trajectory as the selected trajectory. In these or other examples, the trajectory management component 108 can generate alternative trajectory(s) based on the planned trajectory and determine any of the received trajectories (e.g., the planned trajectory and alternative trajectory(s)) as the selected trajectory. Additional examples of generating, evaluating, and determining trajectories can be found, for example, in U.S. Patent Application No. 17 / 514,610 filed on October 29, 2021, entitled "Autonomous Vehicle Trajectory Determination Based on State Transition Model", which is hereby incorporated by reference in its entirety for all purposes.

[0023] The trajectory management component 108 can manage and store various trajectories. In some cases, the trajectories can be managed and stored via the state transition model of the trajectory management component 108. The trajectory management component 108 can store the current trajectory(s) including the selected trajectory. In some examples, the trajectory management component 108 can also hold one or more previous trajectories (e.g., any of the previous trajectories received and / or determined by the trajectory management component 108). The stored trajectories above may provide a fail-safe in the case where the vehicle safety system does not receive an updated trajectory from the primary system.

[0024] The operation of the trajectory management component 108 can be asynchronous or synchronous with respect to any of the remaining components of the vehicle safety system 102 (e.g., any of the perception component 110, the filter component 112, and the collision detection component 114). The operation of any component of the vehicle safety system 102 (e.g., any of the perception component 110, the filter component 112, and the collision detection component 114) can be asynchronous or synchronous with respect to any other component (e.g., any of the perception component 110, the filter component 112, and the collision detection component 114). In some examples, the trajectory management component 108 can select the vehicle's trajectory at any time (e.g., before or after reception) with respect to some (e.g., some or all) of the data generated by the reception of one or more of the other perception components. By not waiting, the trajectory management component 108 may ensure a minimum amount of latency for responding to a possible collision. In other examples, the trajectory management component 108 can wait to receive some (e.g., some or all) of the data generated by any of one or more of the other perception components before selecting the vehicle's trajectory. In the above examples, holster positivity may be avoided by ensuring the consistency of the data. The data that the trajectory management component 108 can wait for can include data received by signals (e.g., signals 116-120) from the components of the vehicle safety system based on the corresponding hash function of the components, as described below.

[0025] The perception component 110 is capable of determining data associated with a corridor (or “perception component data”) (or “perception data”) representing a geometric region related to the vehicle, and / or data associated with objects in the vicinity of the vehicle. In some examples, the perception component 110 determines perception data by evaluating one or more trajectories from among the trajectory(ies) received from the primary system. The trajectory(ies) can be evaluated by the perception component 110 with respect to one or more corridors associated with the vehicle.

[0026] The perception component 110 is capable of determining perception data in various ways based on various types of data. In some examples, the perception component 110 is capable of determining perception data by evaluating the corridor(s) without receiving any trajectory. In some examples, the perception component 110 is capable of determining one or more trajectories associated with a corridor based on the corridor received by the perception component 110. In other examples, the perception component 110 is capable of determining one or more trajectories associated with a subset of the corridors based on a subset of the corridors received by the perception component 110.

[0027] In some examples, the perception component 110 can evaluate a trajectory (e.g., a single trajectory received from a primary system) with respect to one or more corridors associated with the vehicle. As an example, a corridor can be a region of the environment determined at least in part based on a candidate trajectory, the extent of the vehicle (e.g., width, length, height), the current speed of the vehicle, the speed specified by the trajectory, and / or an offset based on a steering rack actuator gain, vehicle kinematics, etc. The kinematics associated with the vehicle can include the minimum turning radius of the vehicle, the maximum turning rate, tire slip, body roll, etc. In examples where kinematics are used, a corridor can be limited to a region of the environment in which the vehicle can operate over a period of time and can include a buffer. In at least some of the above examples, the period of time can correspond to the amount of time to completely stop the vehicle according to one or more deceleration profiles.

[0028] The evaluated trajectory can be stored in the perception component 110 and / or transmitted to the trajectory management component 108 as perception component data. In some examples, a trajectory (e.g., a single trajectory) can be evaluated and used to determine an intersection probability (e.g., a single intersection probability associated with one or more objects). A trajectory can be evaluated based on a corridor associated with the trajectory (e.g., a single corridor) or a subset of corridors associated with the trajectory. In some examples, the result of the trajectory evaluation by the perception component 110 can be transmitted to and received by the trajectory management component 108, which can wait for the result before making a trajectory selection.

[0029] The intersection probability determined by the perception component 110 is utilized by the trajectory management component 108 and can be analyzed and / or determined for information (or "object information") regarding one or more objects in the corridor (e.g., the trajectory management component 108 can utilize the intersection probability to determine the corresponding likelihood of a collision). By waiting to receive the object(s) identified by the perception component 110 based on the evaluated trajectory, the trajectory management component 108 is guaranteed to have relevant information from the perception component 110 regarding the identified object(s) before proceeding to the next frame (e.g., the next step) for trajectory selection.

[0030] In some examples, perception component information (e.g., information including one or more intersection probabilities (s) that can incorporate and / or be utilized to determine various characteristics associated with the corresponding object(s)) can be identified and / or determined by the perception component 110. The perception component information can include, in the corridor, information associated with the object(s) approaching the vehicle, a value indicating how close the object is to the vehicle and / or one or more predicted future locations of the vehicle, a value indicating how fast the object is approaching in each respective direction(s) of the vehicle or the predicted future location(s) of the vehicle, a value indicating how closely the direction of travel of the object aligns with the current location or the predicted future location(s), etc. The perception component information can be determined and / or utilized in a similar manner as described above for the intersection probability (e.g., the perception component information can be transmitted by the perception component 110 to the trajectory management component 108).

[0031] Additional examples of determining corridors and authorizing trajectories can be found, for example, in U.S. Patent Application No. 16 / 588,529, filed September 30, 2019, entitled "Collision Avoidance Perception System," which is hereby incorporated by reference in its entirety for all purposes. Additional examples of determining perception information utilized in trajectory selection can be found, for example, in U.S. Patent Application No. 17 / 514,542, filed October 29, 2021, entitled "Collision Avoidance and Mitigation in Autonomous Vehicles," which is hereby incorporated by reference in its entirety for all purposes.

[0032] In some examples, the perception component 110 can evaluate more than one trajectory (e.g., a trajectory received from a primary system) (e.g., a first trajectory and a second trajectory), and more than one trajectory is utilized to determine more than one intersection probability. In some examples, the first trajectory can be a previous trajectory (e.g., a trajectory generated by the primary system in a previous frame (e.g., a previous step)), and the second trajectory can be a current trajectory (e.g., a trajectory generated by the primary system in a current frame (e.g., a current step)). More than one trajectory can be stored in a queue (or "trajectory buffer", e.g., a buffer of recent trajectories) of the perception component 110. In some examples, individual ones of more than one trajectory (and / or one or more safety stop trajectories) can be evaluated based on a corridor (e.g., a single corridor) associated with the corresponding trajectory or a subset of the corridors associated with the corresponding trajectory. In other examples, more than one trajectory can be evaluated as a group for the more than one trajectory based on a corridor (e.g., a single corridor) associated with the group or a subset of the corridors associated with the group.

[0033] In some examples, for instance, in a state where the number of tracks for which the perception component 110 evaluates the intersection probability is more than one and is two, the perception component information of the two tracks (for example, the object(s) associated with the corresponding collision probability(ies) above the collision probability threshold) can be stored using two deep queues. In these examples, the trajectory management component 108 does not need to wait for the perception component information before proceeding to the next frame. In those examples, generally, sufficient information (for example, perception component information) associated with various trajectories (for example, planned trajectories and one or more of the first safety stop trajectory, the second safety stop trajectory, the nth safety stop trajectory, etc.) for various frames (for example, the current frame and previous frames (for example, stored frames)) can be identified by the perception component 110 and provided to the trajectory management component 108.

[0034] In some examples, determining the intersection probability(ies) can include making a determination (for example, continuously (or "always") publishing) about the object information of the object(s) associated with any of the trajectory(ies). What has been described above enables the trajectory management component 108 to switch back and forth from the trajectory(ies) based on the detection of any of the object(s) for the trajectory(ies). As described above, the safety and reliability of the vehicle's operation can be enhanced.

[0035] In some examples, the trajectory(ies) can be evaluated by the perception component 110 based on the kinematics and / or the trajectory(ies) associated with the vehicle. In an example where the trajectory(ies) is evaluated with respect to a subset of the corridor, the subset of the corridor can be determined based on the kinematics and / or the trajectory(ies).

[0036] Perception component information can include kinematics and / or trajectory(ies). The kinematics and / or trajectory(ies) can be analyzed to determine the intersection probability associated with an object. In some examples, the perception component 110 can determine a physics-based collision state (e.g., a state that utilizes kinematics and / or incorporates kinematics). The physics-based collision state can include information (e.g., a value) indicating a predicted collision of a vehicle with other objects (e.g., other vehicles, objects in the environment, etc.) based on kinematics. Kinematics can include any characteristics associated with the movement of a vehicle, such as, for example, the vehicle's motion data (e.g., speed, velocity, turning rate, etc.), and / or any decisions associated with the movement (e.g., the feasibility of the current speed, the rate at which the vehicle can turn (e.g., safely or permitted), etc.). The combination of kinematics and trajectory(ies) that can be included in the perception component information can be analyzed with object characteristics (e.g., characteristics of the object) and used to determine the intersection probability(ies) associated with the corresponding object and the vehicle.

[0037] By determining the kinematics and / or trajectory(ies), the perception component 110 can determine a physics-based collision state(s) (e.g., a state indicating the corresponding intersection probability(ies)). In some examples, the physics-based collision state(s) can be determined by evaluating the kinematics and trajectory(ies) (e.g., the envelope of possible trajectories). The kinematics, trajectory(ies), and / or physics-based collision state(s) can be determined based on any possible trajectory of the vehicle.

[0038] The perception component 110 can operate according to one or more modes (or "configurations" or "operating states") that evaluate the trajectory(ies). The perception component 110 can operate in a first mode that evaluates a trajectory (e.g., a single trajectory) with respect to a corridor or a subset of corridors. The perception component 110 can operate in a second mode that evaluates more than one trajectory with respect to a corridor or a subset of corridors. The perception component 110 can operate in a third mode that evaluates the trajectory(ies) based on the kinematics and / or trajectory(ies) associated with the vehicle. The third mode can be utilized to determine the physics-based collision state(s) based on the kinematics and / or trajectory(ies).

[0039] Operation in any of the mode(s) by the perception component 110 can occur at any time and in any order. The mode(s) can be determined, set, and / or changed by the perception component 110, one or more other components of the vehicle, and / or the operator. The mode(s) can be utilized based on various factors (e.g., computational resources, network resources, storage resources, etc.), preferences, etc. associated with the perception component 110 and / or one or more other components. Since the amount of storage resources for managing information associated with a single trajectory may be less than the amount of storage resources for the second mode utilized to manage more than one trajectory and / or a subset of corridors, it is possible to use the first mode to reduce the demand placed on the storage resources. In some examples, the different modes can refer to modes that switch dynamically (e.g., based on the state or context of the vehicle) or modes implemented statically.

[0040] To provide relatively higher operational safety with respect to a vehicle, a second mode that manages more than one trajectory and / or a subset of corridors can provide a previous (or “older”) trajectory if there is a need to switch back to the previous trajectory. Thus, the second mode can be used (e.g., the second mode stores information that ensures that any object information associated with the previous trajectory is available for collision checking). In other words, the second mode provides an option for the perception component 110 to fallback to the previous trajectory with the support of the perception component 110. The perception component 110 that stores information about more than one trajectory can provide the trajectory management component 108 with information that the trajectory management component 108 should determine that the current trajectory is invalid or no longer usable (e.g., invalid due to the occurrence of an obstacle).

[0041] It is possible to utilize a third mode to detect an object that may be detected by the perception component 110 in the first mode and / or the second mode, or may not be detected in another situation. In some examples, the perception component 110 operating in the third mode can approve a wider corridor than the first and / or second modes, and the wider corridor includes any possible location of the vehicle's trajectory. As an example, by determining the dynamic limitations of the vehicle, the cones in front of the vehicle can be determined and utilized to indicate any possible area where the trajectory can be located. Next, any part of the cone can be approved. The cone characteristics (e.g., size, location, etc.) of the cone are determined based on information from the primary system that indicates the possible locations of the corridors in the environment where the vehicle may safely proceed and can be used to approve all within the cone (e.g., scaled down by a desired percentage). In some examples, any feature (e.g., cone) determined in the third mode can be kinematic-based. In some examples, any feature (e.g., cone, kinematics, etc.) determined in the third mode can be utilized to determine the physics-based collision state(s).

[0042] The term "mode" is used with respect to the operation of the perception component 110 as described above in the present disclosure, but is not limited in a strict sense. In some examples, the perception component 110 can utilize some or all of the modes, as well as one or more features / functions of the corresponding modes, regardless of the presence or absence of active mode selection. In these or other examples, the perception component 110 can be configured to operate using one or more of i) a single trajectory, ii) more than one trajectory, and / or iii) kinematics and / or trajectories, at any time, by default (e.g., a predetermined configuration), by automatic and / or dynamic (e.g., real-time or near real-time) determination and / or selection, or by active selection by the perception component 110 and / or other vehicle components (e.g., other safety components).

[0043] The perception component 110 is capable of evaluating one or more objects. The evaluated trajectory(s) can be used to determine the object(s) to be evaluated. In some examples, the object(s) can include the individual ones of the object(s) determined based on the corresponding trajectory(s) (e.g., the object(s) (e.g., the first object) can be determined based on the first trajectory, and the object(s) (e.g., the second object) can be determined based on the second trajectory). The individual ones of the object(s) can be evaluated based on the corresponding trajectory(s) (e.g., the object(s) (e.g., the first object) can be evaluated based on the first trajectory, and the object(s) (e.g., the second object) can be evaluated based on the second trajectory). The object(s) can be evaluated to determine data (or "object state data") associated with the object(s) (e.g., orientation, speed, velocity, trajectory, rate of turn for a given speed, etc.). Alternatively or additionally, the object(s) can be evaluated to determine one or more intersection probabilities indicating one or more likelihoods that the vehicle and the object(s) intersect at one or more intersections along the trajectory. Additional examples for determining the intersection probability(ies) can be found, for example, in U.S. Patent Application No. 17 / 133,306, filed on December 23, 2020, entitled "Procedurally Generated Safety System Determination", which is hereby incorporated by reference in its entirety for all purposes.

[0044] In some examples, the perception component 110 can transmit a signal (e.g., a first signal) 116 based on the result of operating the perception component 110. The signal 116 can be transmitted to the trajectory management component 108 based on an evaluation of the trajectory (s). Evaluating the trajectory (s) can include authorizing the trajectory (s) and determining, based on authorizing the trajectory (s), that individual ones of the crossing probability (s) are less than a threshold crossing probability. The signal 116 can include data indicating the result of authorizing individual ones of the trajectory (s) (e.g., perception component information) (e.g., data indicating individual ones of the trajectory (s) evaluated with respect to a corridor or a subset of the corridor), and / or data including individual ones of the crossing probability (s).

[0045] The filter component 112 can receive one or more trajectories and use the trajectories to filter out and remove one or more objects in the environment. The filter component uses the trajectories to determine one or more objects within the vicinity of the vehicle and filters out and removes one or more individual ones of the determined objects that are determined not to be associated with a possible intersection between the corresponding vehicle trajectory and the trajectory of the corresponding object. As an example, an object can be filtered out and removed based on determining that the object's trajectory is not associated with a possible intersection with any of the received vehicle trajectories. The filter component 112 can be integrated with the trajectory management component 108 and / or one or more trajectories (e.g., a third trajectory and a fourth trajectory) can be stored in the queue of the filter component 112. In some examples, the third and fourth trajectories stored in the queue of the filter component 112 can be identical to the first and second trajectories stored in the perception component 110, respectively. In other examples, one or more of the third and fourth trajectories can possibly be different from one or more of the first and second trajectories.

[0046] In some examples, the filter component 112 can transmit a signal (e.g., a second signal) 118 based on the result of operating the filter component 112. The signal 118 can be transmitted to the trajectory management component 108 in filtering and removing the object(s). The signal 118 may include data indicating the result of filtering and removing the object(s) (e.g., the filtered object(s) and / or the resulting object(s) based on the filtered object(s) being excluded). In some examples, any of the objects (e.g., unfiltered objects) utilized by the filter component 112 to determine a physics-based collision state (as described below) and / or to generate the signal 118 can be the same as or different from any of the objects utilized by the perception component 110 to determine the intersection probability(ies) and / or to generate the signal 116. In some examples, any of the objects (e.g., unfiltered objects) utilized by the filter component 112 to determine a physics-based collision state (as described below) and / or to generate the signal 118 can be the same as or different from any of the objects utilized by the collision detection component 114 and / or to generate the signal 120.

[0047] In some examples, the filter component 112 can perform one or more collision checks associated with the object(s) to be excluded (e.g., filtered out) or the object(s) not to be excluded (e.g., not filtered out). The collision check(s) can be utilized to determine the corresponding collision state(s) (e.g., physics-based collision state(s)).

[0048] The collision detection component 114 can receive one or more trajectories (e.g., a planned trajectory and / or one or more of a first safety stop trajectory, a second safety stop trajectory, an nth safety stop trajectory, etc.), and based on the received trajectory(ies), can determine one or more collision states (e.g., likelihood(s) of collision). The collision state(s) can be determined based on a predicted distance between one or more objects and the vehicle that is less than a threshold distance (e.g., a predicted distance determined based on the received trajectory(ies) of the corresponding object(s) and the trajectory(ies) of the vehicle). As an example, the collision state can be determined to meet or exceed a threshold collision state based on the predicted distance between the object and the vehicle being less than the threshold distance. Additional examples of states for trajectory selection can be found, for example, in U.S. Patent Application No. 17 / 514,542, filed Oct. 29, 2021, entitled "Collision Avoidance and Mitigation in Autonomous Vehicles", which is hereby incorporated by reference in its entirety for all purposes.

[0049] The collision detection component 114 can receive the trajectory(ies) generated by the primary system, and the received trajectory(ies) includes a planned trajectory or a group of trajectories (e.g., a planned trajectory and / or one or more of a first safety stop trajectory, a second safety stop trajectory, an nth safety stop trajectory, etc.). In some examples where the collision detection component 114 receives a planned trajectory and perhaps no other trajectories, the collision detection component 114 can determine one or more alternative trajectories (e.g., a first safety stop trajectory, a second safety stop trajectory, an nth safety stop trajectory, etc.). The alternative trajectory(ies) can be determined by the collision detection component 114 based on the received planned trajectory.

[0050] The collision detection component 114 can store a planned orbit (e.g., the fifth orbit) and another planned orbit (e.g., a previous planned orbit) (e.g., the sixth orbit) and / or any number of previously determined safe stop orbits in the queue of the collision detection component 114. In some examples, the fifth and sixth orbits stored in the queue of the collision detection component 114 can be identical to the first and second orbits, respectively, and / or the third and fourth orbits, respectively. In other examples, one or more of the fifth and sixth orbits can be different from one or more of the first to fourth orbits based on the corresponding frame to which any of the orbits is associated.

[0051] In some examples, the collision detection component 114 can determine one or more collision states based on a machine learning (ML) based on the stored orbit(s) and / or one or more object orbits determined for corresponding object(s) in the environment. The collision detection component 114 can transmit a signal (e.g., the third signal) 120 based on the result of operating the collision detection component 114. The signal 120 can include ML based on the collision state(s). The signal 120 can be transmitted to the orbit management component 108 based on determining the collision state(s). The signal 120 can include data indicating the result of determining the collision state(s).

[0052] In some examples, any of the objects (e.g., unfiltered objects) utilized by the collision detection component 114 to determine an ML-based collision state and / or to generate signal 120 can be the same as, or different from, any of the objects (if any) utilized by the perception component 110 to determine an intersection probability (if any) and / or to generate signal 116. In some examples, any of the objects (e.g., unfiltered objects) utilized by the collision detection component 114 to determine an ML-based collision state and / or to generate signal 120 can be the same as, or different from, any of the objects (e.g., unfiltered objects) utilized by the filter component 112 to determine a physics-based collision state (as described below) and / or to generate signal 118.

[0053] The trajectory management component 108 is capable of outputting data based on the results from components of the vehicle safety system 102. The data output from the trajectory management component 108 can include control data 122. The control data 122 can be utilized to control the vehicle. The control data 122 can be determined based on one or more of the results received from the perception component 110 (e.g., approved trajectory(ies) and / or collision probability(ies)), the results received from the filter component 112 (e.g., filtered object(s) and / or result object(s) based on the exclusion of filtered object(s)), and the results received from the collision detection component 114 (e.g., collision state(s)). The control data 122 can include one or more of approval information (e.g., any result from any of the perception component 110, the filter component 112, and / or the collision detection component 114), a planned trajectory, a first safe stop trajectory, a second safe stop trajectory, an nth safe stop trajectory, etc. The control data 122 can include as the selected trajectory any one of a planned trajectory, a first safe stop trajectory, a second safe stop trajectory, an nth safe stop trajectory, etc. based on the selection of one of the trajectory(ies) by the trajectory management component 108. The selected trajectory can be based on the results received from the perception component 110, the filter component 112, and / or the collision detection component 114.

[0054] In some examples, the trajectory management component 108 can utilize any results from some or all of the perception component 110, the filter component 112, and / or the collision detection component 114 to avoid a collision. In other words, if any results from one or more of the perception component 110, the filter component 112, and / or the collision detection component 114 indicate a possible collision, the trajectory management component 108 can modify the vehicle's trajectory (e.g., change to a safer trajectory such as from a planned trajectory to a safe stop trajectory).

[0055] The trajectory management component 108 can utilize results associated with a relatively higher likelihood of a collision (e.g., results from the perception component 110, the filter component 112, and / or one or more other results from the collision detection component 114) over other results associated with a relatively lower likelihood of a collision (e.g., one or more other results from the perception component 110, the filter component 112, and / or the collision detection component 114). Results having a relatively higher likelihood of a collision can be utilized by the trajectory management component 108 to select another trajectory (e.g., one of the safe stop trajectories). In some examples, the trajectory management component 108 can determine to operate by changing to another trajectory (e.g., a safer trajectory) according to the relatively higher likelihood(s) of a collision from the corresponding component(s), regardless of the collision boundary or the remaining component(s) indicating a relatively lower likelihood(s) of a collision (e.g., the trajectory management component 108 can ignore the boundary or results associated with a relatively lower likelihood(s) of a collision and instead perform trajectory selection based on results associated with a relatively higher likelihood(s) of a collision). The trajectory management component 108 performs collision checks, but results from any additional collision checks performed by other components (e.g., specific collision checks performed by the collision detection component 114 to output results such as a “sum up” or “sum down” for the corresponding trajectory(s)) can also be utilized by the trajectory management component 108.

[0056] The trajectory management component 108 can use the results (e.g., tracks) received from one or more of the perception component 110 and the filter component 112, and based on each result, perform collision checks (e.g., collision checks and / or predictions including kinematic-based predictions based on one or more of signals 116, 118, and 120) at a more detailed (or "refined" or "more detailed") level (e.g., a higher level of detail, accuracy, granularity, etc.). The operation of the prediction server (e.g., the results used for the collision checks performed by the trajectory management component 108) can be used to predict the future behavior (e.g., behavior, location, position, etc.) of the corresponding object.

[0057] As an example, the prediction server can determine prediction server information (e.g., how an object will move (e.g., whether the object has performance such as turning, accelerating, decelerating, or some combination of turning, accelerating, or decelerating), how an object will move at one or more points in time, etc.). The intersection probability (s) associated with the object (s) can be determined by the prediction server and / or the trajectory management component 108 based on the prediction server information and / or the trajectory (s) stored in the trajectory management component 108 (e.g., the results of the prediction server can be compared with the corresponding stored trajectory (s)). The results of the prediction server can be collated with the corresponding stored trajectory (s) to determine the time (s) corresponding to possible collisions by using a polygon check, including comparing the trajectory (s) of the object (s) with the trajectory of the vehicle (e.g., by propagating the vehicle along the trajectory of the vehicle) by propagating (or "perturbing") the object (s) along the trajectory (s) of the object (s). In some examples, perturbing the object (s) and / or the vehicle may include perturbing the object (s) and / or the vehicle along any form of movement (e.g., predicted movement) associated with the object (s) and / or the vehicle (e.g., steering angle, speed, acceleration, etc.). Intersection probability (s). In some examples, the results of perturbing the object and / or the vehicle can be included in prediction server information that can be generated by, transmitted to, and / or utilized by the trajectory management component 108.Additional examples of conflict prediction and avoidance can be found, for example, in U.S. Patent Application No. 16 / 884,975, filed May 27, 2020, entitled "Vehicle Collision Avoidance Based on Perturbed Object Trajectories", which is hereby incorporated by reference in its entirety for all purposes.

[0058] The prediction server is described as being separate from other safety components, as described in the present disclosure at this time, but is not strictly limited. In some examples, the prediction server can be integrated with and / or implemented to interoperate with one or more other components (e.g., the trajectory management component 108, the perception component 110, the filter component 112, and / or the collision detection component 114).

[0059] By using the trajectory management component 108 to analyze the results determined by the perception component 110, the filter component 112, and / or the collision detection component 114 (e.g., the information received via signals 116, 118, and 120), the vehicle can be controlled more accurately and safely. For example, the physics-based collision state received from the perception component 110, the object received based on the filtering performed by the filter component 112, and the machine learning (ML) collision state received from the collision detection component 114 can be utilized by the trajectory management component 108 to more accurately select the vehicle trajectory(ies). Further, the robustness of the vehicle operation is also improved due to the redundancy of the operation. As an example, the collision detection component 114 determines the ML-based collision state, but the different corresponding types of analysis performed by the collision detection component 114 and the trajectory management component 108 can ensure that the likelihood of the collision determined by the trajectory management component 108 is correct. However, due to the potentially longer latency for the operation of the trajectory management component 108 because of the longer operation time required by each of the collision detection component 114, the trajectory management component 108 can perform a collision inspection operation in certain situations to reduce the latency (e.g., the trajectory management component 108 can select a different, perhaps less straight but safer alternative trajectory if the results from the collision detection component 114 related to the currently selected trajectory are delayed).

[0060] The trajectory management component 108, the perception component 110, the filter component 112, and the collision detection component 114 can be separate components, but are not strictly limited as described above in the current disclosure. In some examples, any one or more of the components of the vehicle safety system 102 (e.g., the trajectory management component 108, the perception component 110, the filter component 112, and the collision detection component 114) can be integrated into any one or more of the remaining components of the vehicle safety system 102 (e.g., the trajectory management component 108, the perception component 110, the filter component 112, and the collision detection component 114).

[0061] The signals 116, 118, 120 are received by the trajectory management component 108 as described above in the current disclosure, but are not strictly limited. In some examples, any component (e.g., the primary system) can perform the analysis and then generate and / or transmit the signals 116, 118, and 120 to the trajectory management component 108 in a similar manner to the perception component 110, the filter component 112, and the collision detection component 114, respectively. In some examples, the first primary system, the second primary system, and the third primary system, and / or any combination thereof can perform the analysis and then generate and / or transmit any one of the signals 116, 118, and 120, respectively.

[0062] In some examples, the trajectory management component 108, the perception component 110, the filter component 112, and the collision detection component 114 can exchange any information with each other regardless of whether they are implemented as one or more integrated components, one or more individual / separate components, or any combination thereof (e.g., information determined by any of the trajectory management component 108, the perception component 110, the filter component 112, and the collision detection component 114 can be exchanged with any other of the trajectory management component 108, the perception component 110, the filter component 112, and the collision detection component 114). In these or other examples, any analysis and / or determination made by any of the trajectory management component 108, the perception component 110, the filter component 112, and the collision detection component 114 can be made based on one or more information received from any other of the trajectory management component 108, the perception component 110, the filter component 112, and the collision detection component 114. By operating any of the trajectory management component 108, the perception component 110, the filter component 112, and the collision detection component 114 independently and / or redundantly, it is possible to enhance the robustness of the operation of any component and / or the entire vehicle system.

[0063] The various components of the vehicle safety system 102 can utilize, but are not strictly limited to, a subset of the corridors as described above in the present disclosure. In some examples, any of the techniques described herein can be performed using a modified corridor based on a subset of the corridors being combined together and determined as a modified corridor.

[0064] Tracks (e.g., a first track and a second track) can be stored in the respective queues of the perception component 110, the filter component 112, and the collision detection component 114 as described above in the present disclosure, but are not strictly limited thereto. Any number of tracks can be stored in the respective queues of the perception component 110, the filter component 112, and the collision detection component 114. Any one of the tracks stored in any of the queues of the perception component 110, the filter component 112, and the collision detection component 114 can be the same as or different from any of the tracks stored in any of the other queues.

[0065] FIG. 2 is a flow diagram of an exemplary process for the operation of a perception component according to an example of the present disclosure. In some examples, the exemplary process 200 can be performed by at least a perception component (e.g., the perception component 110 as described above with reference to FIG. 1).

[0066] In operation 202, the exemplary process 200 can include operating the perception component 110 to determine a mode (or “selected mode”) based on a selection from one or more modes utilized in the operation of the perception component 110. The mode(s) can include a first mode for authorizing a track, a second mode for authorizing more than one track, and a third mode for utilizing kinematic data or one or more tracks for track authorization.

[0067] In operation 204, the exemplary process 200 can include determining that the mode (e.g., the selected mode of the perception component 110) is the first mode. The perception component 110 operating in the first mode can receive a planned track generated by the primary system.

[0068] In operation 206, exemplary process 200 can include storing the received trajectory (e.g., planned trajectory). Perception component 110 can analyze the planned trajectory. Perception component 110 can analyze the objects associated with the planned trajectory. Perception component 110 can determine the intersection probability associated with the objects.

[0069] In operation 208, exemplary process 200 can include determining that the mode (e.g., the selected mode of perception component 110) is a second mode. Perception component 110 operating in the second mode can receive more than one trajectory (e.g., more than one planned trajectory) generated by the primary system. The more than one planned trajectories can include previous planned trajectories (e.g., a first trajectory (or "first planned trajectory")) and current planned trajectories (e.g., a second trajectory (or "second planned trajectory")). The more than one trajectories can be managed and stored using the queue of perception component 110.

[0070] In operation 210, exemplary process 200 can include storing the received trajectories (e.g., first and second planned trajectories). Perception component 110 can analyze the planned trajectories. By way of example, perception component 110 can analyze an object associated with the first planned trajectory (e.g., a first object) and an object associated with the second planned trajectory (e.g., a second object). In the example, perception component 110 can determine an intersection probability associated with the first object (e.g., a first intersection probability) and an intersection probability associated with the second object (e.g., a second intersection probability).

[0071] In operation 212, exemplary process 200 can include determining that the mode (e.g., the selected mode of perception component 110) is a third mode. The perception component 110 operating in the third mode can determine to utilize the kinematics and the orbit(s) received for authorization of the orbit(s).

[0072] In operation 214, exemplary process 200 can include utilizing kinematics or an orbit for authorization. The kinematics associated with the vehicle can be determined based on the received sensor data. The kinematics can be utilized for orbit authorization (authorization of the orbit(s)) together with the received orbit(s).

[0073] In operation 216, exemplary process 200 can include authorizing one or more orbits. The authorized orbit(s) can include the orbit(s) generated and transmitted by the primary system. In some examples, the authorized orbit(s) can include the orbit(s) stored in the perception component 110 based on the perception component 110 operating in the first mode. In other examples, the authorized orbit(s) can include the orbit(s) stored in the queue of the perception component 110 based on the perception component 110 operating in the second mode. In other examples, the orbit(s) can be authorized using the kinematics of the vehicle. Authorization based on the orbit(s) in any of the first to third modes can be performed based on a corridor or a subset of corridors associated with one or more of the orbit(s).

[0074] In operation 218, exemplary process 200 can include transmitting a signal based on an authorized trajectory. Perception component 110 can transmit, in a signal, the result of authorization of the trajectory(s). The signal can include data indicating individual ones of the trajectory(s) evaluated with respect to a corridor or a subset of the corridor, and / or data including individual ones of the intersection probability(ies) determined by perception component 110.

[0075] Perception component 110 can operate in any of the first to third modes, as described above in the present disclosure, but is not strictly limited thereto. In some examples, perception component 110 can operate in a hybrid mode including a combination of the first mode and the third mode, or a combination of the second mode and the third mode.

[0076] FIG. 3 is a flowchart of an exemplary process 300 for operation of a filter component according to an example of the present disclosure. In some examples, exemplary process 300 can be executed by at least a filter component (e.g., filter component 112 as described above with reference to FIG. 1) and a trajectory management component (e.g., trajectory management component 108 as described above with reference to FIG. 1).

[0077] In operation 302, exemplary process 300 can include operating filter component 112 to identify that a planned trajectory (e.g., a planned trajectory generated by a primary system) has been received. The planned trajectory can be stored in filter component 112. In some examples, the planned trajectory, which can be received as a current trajectory (e.g., a trajectory associated with the current frame of the primary system and / or trajectory management component 108), can be stored in filter component 112 together with one or more other trajectories.

[0078] In operation 304, exemplary process 300 can include adding one or more trajectories to the queue of filter component 112. In some examples, a planned trajectory (e.g., the current trajectory) can be added to a queue in which one or more other trajectories (e.g., previous planned trajectories) (e.g., trajectories associated with previous frames of the primary system and / or trajectory management component 108) are stored.

[0079] In operation 306, exemplary process 300 can include determining one or more objects closest to each trajectory. Individual ones of the trajectory (s) can be analyzed (e.g., approved) by filter component 112 to filter out and remove one or more objects. The object (s) closest to each trajectory can be determined and used to filter out and remove objects (s) farther from the trajectory than the closest object (s). As an example, the object (s) closest to the current planned trajectory can be determined and used to filter out and remove objects farther from the current planned trajectory than the closest object (s). Similarly, other object (s) closest to a previous planned trajectory can be determined and used to filter out and remove other objects farther from the previous planned trajectory than the other closest object (s).

[0080] By authorizing the orbit(s), the filter component 112 can be used to authorize each of the orbit(s) of the queue (e.g., 2 - deep queue) (e.g., compare the data associated with the object(s) closest to the vehicle) (e.g., perform a simple physics - based collision check on the object(s) closest to the vehicle) (e.g., perform a course collision check (e.g., trivalistic course collision check)). The filter component 112 can operate (e.g., authorize the orbit(s)) based on speed assumptions (e.g., physics - based speed determination, the speed remaining constant or within some bounds or thresholds, filtering out and removing orbits above or below a speed threshold). In some examples, authorizing the orbit(s) by the filter component 112 can include filtering out and removing the data associated with the corresponding orbit(s) based on detecting that there is no risk (e.g., object(s) identified by the data) associated with the data. By filtering out and removing all data (e.g., data that would otherwise be used by the orbit management component 108 in another situation), the filter component 112 can reduce the computational demand on the orbit management component 108. As described above, the orbit management component 108 can more efficiently focus on data with a higher level of safety - relatedness than the data that has been filtered out and removed.

[0081] In operation 308, exemplary process 300 may include transmitting a signal based on the determined object(s) (e.g., the closest object(s) and / or the filtered-out object(s)). The signal may include the results of the closest object(s) and / or the filtered-out object(s) for individual ones of the orbit(s) stored in the queue. The signal may be capable of identifying the closest object(s) and / or the filtered-out object(s).

[0082] In operation 310, exemplary process 300 may include determining, by orbit management component 108, whether a signal has been received from filter component 112. In some examples, orbit management component 108 may be capable of determining whether a signal generated by filter component 112 has been received. Based on determining that a signal from filter component 112 has been received, orbit management component 108 may proceed to operation 312. Based on determining that a signal from filter component 112 has not yet been received, orbit management component 108 may refrain from proceeding to operation 312 (e.g., orbit management component 108 may wait for the results from filter component 112 before moving to the next frame).

[0083] In operation 312, exemplary process 300 can include authorizing one or more trajectories. Trajectory management component 108 can determine to authorize a trajectory (or trajectories) based on determining that a signal from filter component 112 has been received. In some examples, trajectory management component 108 can authorize individual trajectories stored in a queue and determine that a trajectory (e.g., a current planned trajectory) is a selectable trajectory based on the closest object(s) and / or objects that have been filtered out. In these examples, a trajectory can be determined as a selectable trajectory (e.g., a trajectory that can be selected) based on the individual distance between the corresponding one or more trajectories of the closest object(s) and the vehicle's trajectory being greater than or equal to a threshold distance. In other examples, trajectory management component 108 can authorize a trajectory (e.g., a current planned trajectory) and determine that a trajectory is a non - selectable trajectory (e.g., a trajectory that will not be selected) based on at least one of the individual distances between the corresponding one or more trajectories of the closest object(s) and the vehicle's trajectory being less than a threshold distance. A similar authorization for a current planned trajectory can be performed for any trajectory (e.g., a previous planned trajectory).

[0084] Figure 4 is a flowchart of an exemplary process 400 for the operation of a collision detection component according to an example of the present disclosure. In some examples, exemplary process 400 can be executed by at least a collision detection component (e.g., collision detection component 114 as described above with reference to FIG. 1) and a trajectory management component (e.g., trajectory management component 108 as described above with reference to FIG. 1).

[0085] In operation 402, exemplary process 400 may include operating collision detection component 114 to determine that one or more trajectories have been received and may be utilized to determine one or more collision states. In some examples, collision detection component 114 may be capable of performing a collision check (e.g., a machine learning (ML) collision check) to approve a trajectory (or trajectories). The received and / or approved trajectory (or trajectories) may be a trajectory (or trajectories) generated by the primary system. In some examples where more than one trajectory is received by collision detection component 114, the trajectories (e.g., the current planned trajectory and previous planned trajectories) may be stored in a queue of collision detection component 114.

[0086] In operation 404, exemplary process 400 may include determining, by collision detection component 114, one or more collision states based on the trajectory (or trajectories). The collision state (or states) may be indicative of the likelihood (or likelihoods) of a collision based on the received trajectory (or trajectories). By way of example, the collision state associated with the current planned trajectory may be determined based on the individual distances between one or more object trajectories and the current planned trajectory. By way of example, i) the perception component 110 is utilized to identify a crossing probability (or probabilities), although perhaps in some cases not utilized to determine the collision state (or states), as compared to the collision detection component 114 which may be capable of determining a collision state (e.g., an ML collision state) based on the trajectory (or trajectories) and / or object (or objects).

[0087] In operation 406, exemplary process 400 may include transmitting, by collision detection component 114, a signal based on the collision state (or states). The signal may include the result of determining the collision state (or states). In some examples, the signal may indicate, for individual ones of the trajectory (or trajectories), whether the corresponding collision state meets or exceeds a threshold collision state.

[0088] In operation 408, exemplary process 400 may include determining by orbit management component 108 whether a signal is being received from collision detection component 114. In some examples, orbit management component 108 may be able to determine whether a signal generated by collision detection component 114 is being received. Orbit management component 108 may be able to determine whether a collision state(s) meets or exceeds a threshold collision state. Based on determining that a signal from collision detection component 114 is being received, orbit management component 108 may proceed to operation 410. Based on determining that a signal from collision detection component 114 has not yet been received, orbit management component 108 may refrain from proceeding to operation 410.

[0089] In operation 410, exemplary process 400 may include authorizing one or more orbits. Based on determining that a signal from collision detection component 114 is being received, orbit management component 108 may be able to determine to authorize an orbit(s). In some examples, orbit management component 108 may authorize an orbit (e.g., a current planned orbit) and, based on determining that a collision state associated with the orbit is less than a threshold collision state, may be able to determine that the orbit is a selectable orbit. In other examples, orbit management component 108 may authorize an orbit (e.g., a current planned orbit) and, based on determining that a collision state associated with the orbit meets or exceeds a threshold collision state, may be able to determine that the orbit is a non - selectable orbit. A similar authorization for a current planned orbit may be done for any orbit (e.g., a previous planned orbit).

[0090] FIG. 5 illustrates an exemplary driving environment 500 that depicts a vehicle traversing an environment and a trajectory determined for the vehicle. In some examples, a vehicle (e.g., a vehicle including the vehicle safety system 102 as described above with reference to FIG. 1) can utilize a trajectory management component 108 to receive and / or determine one or more trajectories. The trajectory(s) can include a planned trajectory 504 that can be generated by a primary system of the vehicle 502. In some examples, the planned trajectory (e.g., the current planned trajectory) 504 can be associated with the current frame.

[0091] The trajectory(s) can include a planned trajectory 506 that can be generated by a primary system of the vehicle 502. In some examples, the planned trajectory (e.g., a previous planned trajectory) 506 can be associated with a previous frame.

[0092] The trajectory(s) that can be received by one or more components (e.g., the perception component 110, the filter component 112, and / or the collision detection component 114 as described above with reference to FIG. 1) can be stored in the perception component 110, the filter component 112, and / or the collision detection component 114. The individual ones of the perception component 110, the filter component 112, and the collision detection component 114 can include queues for storing the trajectory(s) (e.g., the planned trajectory 504 and the planned trajectory 506) that can be received from the primary system (e.g., the primary system of the vehicle 502) and / or the trajectory management component 108.

[0093] The orbit(s) can include one or more safe stop orbits. The safe stop orbit(s) can include a safe stop orbit 508 (e.g., a first safe stop orbit) associated with the stop location 510 (and / or associated with a first deceleration value) and a safe stop orbit 512 (e.g., a second safe stop orbit) associated with the stop location 514 (and / or associated with a second deceleration value different from the first deceleration value). The individual ones of the safe stop orbit(s) (e.g., the safe stop orbit 508 and the safe stop orbit 512), and / or the individual ones of the stop location(s) (e.g., the stop location 510 and the stop location 514) can be determined by the orbit management component 108.

[0094] The planned orbits 504 and 506 can be determined based on various characteristics. In some examples, one or more characteristics (e.g., direction (e.g., a first direction), turning performance in the first direction (e.g., a first turning performance), speed, lateral acceleration limit, etc.) used to determine the planned orbit 504 can be the same as, or different from, one or more characteristics (e.g., direction (e.g., a second direction), turning performance in the second direction (e.g., a second turning performance), speed, lateral acceleration limit, etc.) used to determine the planned orbit 506.

[0095] In some examples, the trajectory management component 108 can determine whether to select the planned trajectory 504, the safe stop trajectory 508, or the safe stop trajectory 512 before or after receiving the results from evaluating the planned trajectory 504 by the perception component 110, the filter component 112, and / or the collision detection component 114. In other examples, the trajectory management component 108 can wait to receive the results from the perception component 110, the filter component 112, and / or the collision detection component 114 regarding the evaluation of the planned trajectory 504. In these examples, the trajectory management component 108 can determine whether to select the planned trajectory 504, the safe stop trajectory 508, or the safe stop trajectory 512 based on one or more of the received results. In some examples, in other words, the trajectory management component 108 can operate without waiting to receive results from one or more of the perception component 110, the filter component 112, and / or the collision detection component 114 to proceed with its operation (e.g., determine whether to select the planned trajectory 504). By proceeding with its operation, the trajectory management component 108 can avoid the latency that would otherwise result from waiting for results from one or more of the perception component 110, the filter component 112, and the collision detection component 114 in another situation.

[0096] In some examples, for instance, when the trajectory management component 108 receives results from one or more of the perception component 110, the filter component 112, and the collision detection component 114, the trajectory management component 108 can operate based on determining that one or more collision likelihoods associated with the corresponding result(s) are higher than other likelihood(s) associated with other result(s). As an example, the trajectory management component 108 can determine to change the trajectory (e.g., change from a planned trajectory to a safe stop trajectory) by determining that a result(s) from one or more of the perception component 110, the filter component 112, and the collision detection component 114 is associated with a higher likelihood of collision than other result(s) from other component(s).

[0097] In some examples, the trajectory management component 108 can determine whether to select one of the current trajectories (e.g., planned trajectory 504, safety stop trajectory 508, or safety stop trajectory 512), or whether to select a previous trajectory (e.g., planned trajectory 506, or one or safety stop trajectories determined in a previous frame), based on the results of evaluating one or more of the planned trajectory 504, planned trajectory 506, and / or determined safety stop trajectory(s) by the perception component 110, filter component 112, and / or collision detection component 114. In some examples, for instance, when one or more of the perception component 110, filter component 112, and / or collision detection component 114 store the corresponding queue(s), information can be received without delay via signals 116, 118, and 120, so that the trajectory management component 108 can quickly make a trajectory selection. As an example, if it is necessary to switch back to a previous trajectory, the previous trajectory is stored in the corresponding queue(s) of the perception component 110, filter component 112, and / or collision detection component 114, and it is possible to enable access to all information associated with the previous trajectory.

[0098] The safety stop orbits 508 and 512, as well as the stop locations 510 and 514, can be determined by the vehicle 502, as described above in the present disclosure, but are not strictly limited thereto. In some examples, one or more other safety stop orbits associated with the current planned orbit 504 and corresponding other stop locations can be determined and stored in the orbit management component 108. In these or other examples, one or more other safety stop orbits and corresponding other stop locations associated with the previous planned orbit 508 can be determined and stored by the orbit management component 108 (e.g., determined and / or stored in a previous frame). The orbit management component 108 can operate using other safety stop orbits or other stop locations in a similar manner as for the safety stop orbits 508 and 512 and for the stop locations 510 and 514.

[0099] The planned orbits 504 and 506 can be determined by the vehicle 502, as described above in the present disclosure, but are not strictly limited thereto. In some examples, one or more other planned orbits associated with any number of previous frames can be determined and the orbit management component 108 can be utilized by the perception component 110, the filter component 112, and / or the collision detection component 114 in a similar manner as for the planned orbits 504 and 506.

[0100]

[0101] Vehicle 602 can be, for example, a driverless vehicle, such as an autonomous vehicle configured to operate according to a Level 5 classification issued by the National Highway Traffic Safety Administration, which describes a vehicle that has the performance to perform all safety-critical functions throughout the entire journey, where the driver (or passenger) is not expected to control the vehicle at any point. - In the above example, since vehicle 602 can be configured to control all functions from the start to the end of the journey, including all parking functions, it may not include a driver, and / or controls for driving vehicle 602, such as, for example, a steering wheel, an accelerator pedal, and / or a brake pedal. What has been described above is merely an example, and the systems and methods described herein can be incorporated into any vehicle, including land transportation, air transportation, or water transportation vehicles, ranging from vehicles that always need to be manually controlled by a driver to vehicles that are partially or fully autonomously controlled.

[0102] Vehicle 602 can include one or more first computing devices 604, one or more sensor systems 606, one or more emitters 608, one or more communication connections 610 (also referred to as communication devices and / or modems), at least one direct connection 612 (for example, for physically connecting to vehicle 602 to exchange data and / or supply power), and one or more drive systems 614. As an example, the first computing device(s) 604 may be considered the primary system. In some examples, the first computing device(s) 604 can be used to implement a primary system as described above with reference to FIG. 1. One or more sensor systems 606 can be configured to capture sensor data associated with the environment.

[0103] The sensor system(s) 606 can include a time-of-flight sensor, a location sensor (e.g., GPS, compass, etc.), an inertial sensor (e.g., an inertial measurement unit (IMU), an accelerometer, a magnetometer, a gyroscope, etc.), a lidar sensor, a radar sensor, a sonar sensor, an infrared sensor, a camera (e.g., RGB, IR, intensity, depth, etc.), a microphone sensor, an environmental sensor (e.g., a temperature sensor, a humidity sensor, a light sensor, a pressure sensor, etc.), an ultrasonic transducer, a wheel encoder, etc. The sensor system(s) 606 can include multiple instances for each of the sensors described above or other types. For example, the time-of-flight sensor can include individual time-of-flight sensors located at the corners, front, rear, sides, and / or top of the vehicle 602. As another example, the camera sensor can include multiple cameras arranged at various locations both outside and / or inside the vehicle 602. The sensor system(s) 606 can provide the input to the first computing device(s) 604.

[0104] Furthermore, vehicle 602 can also include one or more emitters 608 for emitting light and / or sound. The emitter(s) 608 in the example described above include internal audio and visual emitters for communicating with the passengers of vehicle 602. By way of example and not limitation, the internal emitters can include speakers, lights, signs, display screens, touch screens, tactile emitters (e.g., vibration and / or force feedback), mechanical actuators (e.g., seatbelt tensioners, seat positioners, headrest positioners, etc.), and the like. Furthermore, the emitter(s) 608 in the example described above also include external emitters. By way of example and not limitation, the external emitters in the example described above include lights that signal the direction of travel or other indications related to the operation of the vehicle (e.g., indicator lights, signs, light arrays, etc.), and one or more audio emitters (e.g., speakers, speaker arrays, horns, etc.) that communicate audibly with pedestrians or other nearby vehicles, which may include acoustic beam steering technology.

[0105] Furthermore, vehicle 602 can also include one or more communication connections 610 that enable communication between vehicle 602 and one or more other local or remote computing device(s) (e.g., remote teleoperation computing device), or between vehicle 602 and a remote service. For example, the communication connection(s) 610 may facilitate communication between vehicle 602 and other local computing device(s) and / or drive system(s) 614. Furthermore, the communication connection(s) 610 can also enable vehicle 602 to communicate with other nearby computing device(s) (e.g., other nearby vehicles, traffic signals, etc.).

[0106] The communication connection(s) 610 can include a physical interface and / or a logical interface for connecting the first computing device(s) 604 to another computing device or to one or more external networks 616 (e.g., the Internet). For example, the communication connection(s) 610 can enable Wi-Fi-based communication via, for example, frequencies defined by the IEEE 802.11 standard, short-range wireless frequencies such as BLUETOOTH®, cellular communication (e.g., 2G, 3G, 4G, 4G LTE, 5G, etc.), satellite communication, dedicated short range communication (DSRC), or any suitable wired or wireless communication protocol that enables each computing device to interface with other computing device(s).

[0107] In at least one example, vehicle 602 can include drive system(s) 614. In some examples, vehicle 602 can have a single drive system 614. In at least one example, if vehicle 602 has multiple drive systems 614, the individual drive systems 614 can be placed at the facing ends of vehicle 602 (e.g., front and rear, etc.). In at least one example, drive system(s) 614 can include sensor system(s) 606 that detect the situation of drive system(s) 614 and / or the state around vehicle 602. By way of example and not limitation, sensor system(s) 606 can include one or more wheel encoders (e.g., rotary encoders) that sense the rotation of the wheels of the drive module, inertial sensors (e.g., inertial measurement units, accelerometers, gyroscopes, magnetometers, etc.) that measure the orientation and acceleration of the drive system, cameras or other image sensors, ultrasonic sensors that audibly detect objects around the drive system, lidar sensors, radar sensors, etc. Some sensors, such as wheel encoders, etc., can be unique to drive system(s) 614. In some cases, sensor system(s) 606 of drive system(s) 614 can overlap or supplement the corresponding system of vehicle 602 (e.g., sensor system(s) 606).

[0108] The drive system(s) 614 can include a high-voltage battery, a motor that propels the vehicle, an inverter that converts direct current from the battery to alternating current for use by other vehicle systems, a steering system including a steering motor (which can be electrically actuated) and a steering rack, a braking system including a hydraulic or electric actuator, a suspension system including hydraulic and / or pneumatic components, a stability control system for distributing braking force to reduce towing losses and maintain control, an HVAC system, lighting (e.g., lighting such as head / tail lights that illuminate the perimeter outside the vehicle), and one or more other systems (e.g., a cooling system, a safety system, an on-board charging system, such as other electrical components like a DC / DC converter, a high-voltage junction, a high-voltage cable, a charging system, a charging port). Additionally, the drive system(s) 614 can receive and preprocess data from the sensor system(s) 606 and can include a drive system controller for controlling the operation of various vehicle systems. In some examples, the drive system controller can include one or more processors and a memory communicatively connected to the one or more processors. The memory can store one or more components that perform various functionalities of the drive system(s) 614. Further, the drive system(s) 614 also includes one or more communication connections that enable communication with one or more other local or remote computing devices by each drive system.

[0109] Vehicle 602 can include one or more second computing devices 618 that provide redundancy, error checking, and / or authorization of decisions and / or commands determined by the first computing device(s) 604. As an example, the first computing device(s) 604 may be considered the primary system, while the second computing device(s) 618 may be considered the secondary system. In some examples, the second computing device(s) 618 can be utilized to implement a vehicle safety system 102 as described above with reference to FIG. 1.

[0110] The primary system may generally perform processing that controls how the vehicle maneuvers within the environment. The primary system may implement various artificial intelligence (AI) techniques, such as machine learning, to understand the environment around the vehicle and / or to instruct the vehicle to move within the environment. For example, the primary system may implement AI techniques such as localizing the vehicle, detecting objects around the vehicle, segmenting sensor data, determining object classifications, predicting object trajectories, generating vehicle trajectories, and the like. In an example, the primary system processes data from multiple types of sensors of the vehicle, such as, for example, lidar (light detection and ranging) sensors, radar sensors, image sensors, depth sensors (time of flight, structured light, etc.), and the like.

[0111] The secondary system may authorize the operation of the primary system and, if there is a problem with the primary system, may take over control of the vehicle from the primary system. In an example, the secondary system processes data from a small number of sensors, such as a subset of sensor data processed by the primary system. By way of illustration, the primary system processes lidar data, radar data, image data, depth data, etc., while the secondary system may process only lidar data and / or radar data (and / or time-of-flight data). However, in other examples, the secondary system may process sensor data from any number of sensors, such as data from each of the sensors, data from the same number of sensors as the primary system, etc.

[0112] Additional examples of vehicle architectures including a primary computing system and a secondary computing system can be found, for example, in U.S. Patent Application No. 16 / 189,726, filed November 13, 2018, entitled "Perception Collision Avoidance", which is hereby incorporated by reference in its entirety for all purposes.

[0113] The first computing device(s) 604 can include one or more processors 620 and a memory 622 communicatively connected to the one or more processors 620. In the illustrated example, the memory 622 of the first computing device(s) 604 stores a localization component 624, a perception component 626, a prediction component 628, a planning component 630, a map component 632, and one or more system controllers 634. Although depicted as residing in the memory 622 for purposes of illustration, it is contemplated that the localization component 624, the perception component 626, the prediction component 628, the planning component 630, the map component 632, and the one or more system controllers 634 can be further stored in, or alternatively, accessible to, the first computing device(s) 604 (e.g., stored in different components of the vehicle 602) and / or accessible to the vehicle 602 (e.g., stored remotely).

[0114] In the memory 622 of the first computing device 604, the localization component 624 can include functionality to receive data from the sensor system(s) 606 to determine the position of the vehicle 602. For example, the localization component 624 can include and / or request / receive a three-dimensional map of the environment and can continuously determine the location of the autonomous vehicle within the map. In some cases, the localization component 624 can use SLAM (simultaneous localization and mapping) or CLAMS (calibration, localization and mapping, simultaneously) to receive time-of-flight data, image data, lidar data, radar data, sonar data, IMU data, GPS data, wheel encoder data, or any combination and the like, and can accurately determine the location of the autonomous vehicle. In some cases, the localization component 624 can provide the data to various components of the vehicle 602 to determine the initial position of the autonomous vehicle to generate a trajectory as described herein.

[0115] The perception component 626 can include functionality to perform object detection, segmentation, and / or classification. In some examples, the perception component 626 can provide processed center data indicative of the presence of the entity closest to the vehicle 602 and / or the classification of the entity as a type of entity (e.g., car, pedestrian, cyclist, building, tree, road surface, curb, sidewalk, unknown, etc.). In additional or alternative examples, the perception component 626 can provide processed sensor data indicative of one or more characteristics associated with the detected entity and / or the environment in which the entity is located. In some examples, without limitation, the characteristics associated with the entity can include, but are not limited to, x position (global position), y position (global position), z position (global position), orientation, entity type (e.g., classification), speed of the entity, extent (size) of the entity, etc. Without limitation, the characteristics associated with the environment can include, but are not limited to, the presence of another entity in the environment, the state of another entity in the environment, time of day, day of the week, season, weather conditions, indication of darkness / light, etc.

[0116] As described above, the perception component 626 can use a perception algorithm to determine a perception-based bounding box associated with an object in the environment based on sensor data. For example, the perception component 626 can receive image data, classify the image data, and determine that an object is represented in the image data. Next, using a detection algorithm, the perception component 626 can generate a two-dimensional bounding box and / or a perception-based three-dimensional bounding box associated with the object. The perception component 626 can further generate a three-dimensional bounding box associated with the object. As described above, the three-dimensional bounding box can provide additional information such as, for example, the location, orientation, pose, and / or size (e.g., length, width, height, etc.) associated with the object.

[0117] The perception component 626 can include functionality to store perception data generated by the perception component 626. In some cases, the perception component 626 can determine a track corresponding to an object classified as an object type. For illustrative purposes only, using the sensor system(s) 606, the perception component 626 can capture one or more images of the environment. The sensor system(s) 606 can capture an image of an environment that includes an object such as, for example, a pedestrian. The pedestrian can be at a first position at time T and at a second position at time T + t (e.g., movement during the span of time t after time T). In other words, the pedestrian can move from the first position to the second position during the time span described above. The above movement can be logged, for example, as stored perception data associated with the object.

[0118] The stored perception data can, in some instances, include fused perception data captured by the vehicle 602. The fused perception data can include, for example, the fusion or other combination of sensor data from sensor system(s) 606 such as image sensors, lidar sensors, radar sensors, time-of-flight sensors, sonar sensors, global positioning system sensors, internal sensors, and / or any combination thereof. The stored perception data can further include, in addition to or instead of, classification data including the semantic classification of the objects represented by the sensor data (e.g., pedestrians, vehicles, buildings, road surfaces, etc.). The stored perception data can further include, in addition to or instead of, track data (position, orientation, sensor features, etc.) corresponding to the movement of the objects classified as dynamic objects through the environment. The track data can include, over time, multiple tracks of multiple different objects. It is possible to mine the track data described above to identify images of objects at times when a certain type of object (e.g., pedestrian, animal, etc.) is stationary (e.g., standing still) or moving (e.g., walking, running, etc.). In the example described above, the computing device determines a track corresponding to a pedestrian.

[0119] The prediction component 628 is capable of generating one or more probability maps representing the predicted probabilities of the possible locations of one or more objects in the environment. For example, the prediction component 628 is capable of generating one or more probability maps for vehicles, pedestrians, animals, and the like within a threshold distance from the vehicle 602. In some cases, the prediction component 628 is capable of measuring the tracks of the objects and generating, for the objects, a discretized prediction probability map, a heat map, a probability distribution, a discretized probability distribution, and / or a trajectory based on the observed and predicted behavior. In some cases, the one or more probability maps can represent the intent of one or more objects in the environment.

[0120] The planning component 630 can determine a path for the autonomous vehicle 602 to traverse through the environment. For example, the planning component 630 can determine various routes and paths and various levels of detail. In some cases, the planning component 630 can determine a route to proceed from a first location (e.g., the current location) to a second location (e.g., the target location). For the purposes of this discussion, a route can be a sequence of waypoints for proceeding between two locations. By way of non-limiting example, waypoints can include streets, intersections, GPS (Global Positioning System) coordinates, and the like. Further, the planning component 630 can generate instructions for guiding the autonomous vehicle along at least a portion of the route from the first location to the second location. In at least one example, the planning system 630 can determine how to guide the autonomous vehicle from a first waypoint in the sequence of waypoints to a second waypoint in the sequence of waypoints. In some examples, the instructions can be a path, or a portion of a path. In some examples, multiple paths can be generated substantially simultaneously (i.e., within the technical tolerance) according to a receding horizon technique. A single path among the multiple paths in the receding data horizon having the highest confidence level may be selected to operate the vehicle.

[0121] In other examples, the planning component 630 can alternatively or additionally use data from the perception component 626 and / or the prediction component 628 to determine a path that the vehicle 602 follows as it traverses through the environment. For example, the planning component 630 can receive data from the perception component 626 and / or the prediction component 628 regarding objects associated with the environment. Using the data described above, the planning component 630 can determine a route to proceed from a first location (e.g., the current location) to a second location (e.g., the target location) so as to avoid objects in the environment. In at least some examples, the planning component 630 can determine that there is no collision-free path and, in turn, provide a path that places the vehicle 602 in a safe stop state that avoids all collisions and / or reduces damage in another situation.

[0122] Memory 622 can further include one or more maps 632 that can be used by vehicle 602 moving in the environment. For purposes of discussion, a map can be, for example, but not limited to, a two-dimensional, three-dimensional, or N-dimensional modeled data structure capable of providing information about the environment such as, but not limited to, topology (e.g., intersections, etc.), streets, mountains, roads, terrain, and generally the environment. Optionally, in some cases, the map can include texture information (e.g., color information (e.g., RGB color information, Lab color information, HSV / HSL color information, and the like), and the like), intensity information (e.g., lidar information, radar information, and the like), spatial information (e.g., image data projected onto a mesh, individual "surfels" (e.g., polygons associated with individual colors and / or intensities)), reflectance information (e.g., specular reflection information, retroreflectivity information, BRDF information, BSSRDF information, and the like). In one example, the map can include a three-dimensional mesh of the environment. Optionally, as described herein, the map can be stored in a tile format such that, for example, individual tiles of the map represent discrete portions of the environment, and can be loaded into working memory as needed. In at least one example, one or more maps 632 can include at least one map (e.g., an image and / or a mesh). In some examples, vehicle 602 can be controlled at least in part based on map(s) 632. That is, map(s) 632 can be used in connection with localization component 624, perception component 626, prediction component 628, and / or planning component 630 to determine the location of vehicle 602, identify objects in the environment, generate prediction probability(ies) associated with the object(s) and / or vehicle 602, and / or generate a route and / or trajectory for navigating within the environment.

[0123] In some examples, one or more maps 632 can be stored on a remote computing device(s) (such as computing device(s) 648 etc.) accessible via network(s) 616. In some examples, the plurality of maps 632 can be stored, for example, based on characteristics (such as type of entity, time, day of the week, season of the year, etc.). Storing the plurality of maps 632 can have similar memory requirements, but can increase the speed at which data in the maps can be accessed.

[0124] In at least one example, the first computing device(s) 604 can include one or more system controllers(s) 634 configured to control the steering, propulsion, braking, safety, emitter, communication, and other systems of vehicle 602. The system controller(s) 634 described above can communicate with and / or control the corresponding systems of drive system(s) 614 and / or other components of vehicle 602, and may be configured to operate according to a path provided by planning component 630.

[0125] The second computing device(s) 618 can include one or more processors 636 and a memory 638, including components for verifying and / or controlling the sides of vehicle 602, as discussed herein. In at least one example, one or more processors 636 can be similar to processor(s) 620, and memory 648 can be similar to memory 622. However, in some examples, processor(s) 636 and memory 638 may include hardware different from processor(s) 620 and memory 622 for additional redundancy.

[0126] In some examples, the memory 638 can include a localization component 640, a perception / prediction component 642, a planning component 644, and one or more system controllers 646. In some examples, the perception / prediction component 642 can be used to implement any features of the perception component 110 as described above with reference to FIG. 1.

[0127] In some examples, the localization component 640 may receive sensor data from the sensor(s) 606 to determine one or more of the position and / or orientation (along with the pose) of the autonomous vehicle 602. As used herein, the position and / or orientation may be relative to one or more points and / or objects in the environment in which the autonomous vehicle 602 is located. In an example, the orientation may include indications of the yaw, roll, and / or pitch of the autonomous vehicle 602 with respect to a reference plane and / or with respect to one or more points and / or objects. In an example, the localization component 640 may perform less processing (e.g., a higher level of localization) than the localization component 624 of the first computing device(s) 604. For example, rather than determining the pose of the autonomous vehicle 602 with respect to a map, the localization component 640 may simply determine the pose (e.g., local position rather than global position) of the autonomous vehicle 602 with respect to the objects and / or surfaces detected around the autonomous vehicle 602. The position and / or orientation may be determined using probabilistic filtering techniques such as Bayesian filters (e.g., Kalman filters, extended Kalman filters, unscented Kalman filters, etc.) using some or all of the sensor data, for example.

[0128] In some examples, the perception / prediction component 642 can include functionality to detect, identify, classify, and / or track objects represented in the sensor data. For example, the perception / prediction component 642 can perform clustering operations and operations to estimate or determine the height associated with an object, as described herein.

[0129] In some examples, the perception / prediction component 642 may include M-estimation, but may lack an object classifier, such as a neural network, decision tree, and / or the like for classifying objects. In additional or alternative examples, the perception / prediction component 642 may include any type of ML model configured to clarify the classification of an object. In contrast, the perception component 626 may include a pipeline of hardware and / or software components and may include one or more machine learning models, Bayesian filters (e.g., Kalman filters), GPUs (graphics processing unit(s)), and / or the like. In some examples, the perception data determined by the perception / prediction component 642 (and / or 626) may include object detection (e.g., identification of sensor data associated with objects in the environment around an autonomous vehicle), object classification (e.g., identification of the type of object associated with a detected object), object track (e.g., the position, velocity, acceleration, and / or orientation of a historical, current, and / or predicted object), and / or the like.

[0130] Furthermore, the perception / prediction component 642 may also process the input data to determine one or more predicted trajectories of the object. For example, based on the current position of the object and the velocity of the object over a period of several seconds, the perception / prediction component 642 may predict the path that the object will move over the next several seconds. In some examples, the predicted path may involve using a linear assumption of the motion with a given position, orientation, velocity, and / or direction. In other examples, the predicted path described above may involve more complex analysis.

[0131] In some examples, the planning component 644 may be capable of receiving a trajectory from the planning component 630 and including functionality to approve that the trajectory is collision-free and / or within a safety margin. In some examples, the planning component 644 may be capable of generating a safe stop trajectory (e.g., a trajectory that stops the vehicle 602 with a "comfortable" deceleration (e.g., less than maximum deceleration)), and in some examples, the planning component 644 may be capable of generating an emergency stop trajectory (e.g., maximum deceleration regardless of the presence or absence of steering input).

[0132] In some examples, the planning component 644 may be capable of including the trajectory management component 108, the perception component 110, the filter component 112, and / or the collision detection component 114. In some examples, any of the corresponding queue(s) of the perception component 110, the filter component 112, and / or the collision detection component 114 described throughout this disclosure may be the same as, or different from, any of the other queue(s). In these examples, the perception component 110, the filter component 112, and / or the collision detection component 114 may be capable of sharing information with each other and / or with the trajectory management component 108.

[0133] In some examples, the perception component 110, the filter component 112, and / or the collision detection component 114 can be stored and / or managed by the planning component 644 as described above with reference to FIGS. 1-5. As an example, the perception component 110 can generate a signal indicating an object approaching the autonomous vehicle, the filter component 112 can filter and remove the object(s), and generate a second signal indicating a physics-based collision state associated with any object(s) that were not filtered out and removed, and the collision detection component 114 can generate a third signal indicating a collision state based on machine learning, and can determine the likelihood of a collision based at least in part on the machine learning (ML)-based collision state, and the trajectory management component 108 can determine to follow a candidate trajectory or a safe stop trajectory based on at least one of the first signal, the second signal, or the third signal.

[0134] In some examples, the system controller 646 can include functionality to control safety-critical components of the vehicle (e.g., steering, brakes, motor, etc.). In this way, the second computing device(s) 618 can provide redundancy and / or an additional hardware and software layer for vehicle safety.

[0135] Vehicle 602 can be connected to computing device(s) 648 via network 616 and can include one or more processors 650 and a memory 652 communicatively connected to the one or more processors 650. In at least one example, the one or more processors 650 can be similar to processor(s) 620, and the memory 652 can be similar to memory 622. In the illustrated example, the memory 652 of the computing device(s) 648 stores component(s) 654 that can correspond to any of the components described herein.

[0136] Processor(s) 620, 636, and / or 650 can be any suitable processor capable of processing data and executing instructions for performing operations as described herein. By way of example and not limitation, processor(s) 620, 636, and / or 650 can include one or more CPUs (Central Processing Units), GPUs (Graphics Processing Units), or any other device or portion of a device that processes electronic data and converts that electronic data into other electronic data that can be stored in registers and / or memory. In some examples, further, integrated circuits (e.g., ASICs, etc.), gate arrays (e.g., FPGAs, etc.), and other hardware devices can also be considered processors so long as they are configured to implement encoded instructions.

[0137] Memories 622, 638, and / or 652 are examples of non-transitory computer-readable media. Memories 622, 638, and / or 652 can store an operating system and one or more software applications, instructions, programs, and / or data for implementing the methods described herein and the functions attributable to various systems. In various implementations, Memories 622, 638, and / or 652 can be implemented using any suitable memory technology, e.g., SRAM (Static RAM), SDRAM (Synchronous DRAM), non-volatile / flash-type memory, or any other type of memory capable of storing information. The architectures, systems, and individual elements described herein can include many other logical, programmatic, and physical components, and those shown in the accompanying drawings are merely examples relevant to the description herein.

[0138] In some cases, some or all aspects of the components described herein can include any model, algorithm, and / or machine-learning algorithm. For example, in some cases, components in Memories 622, 638, and / or 652 can be implemented as a neural network. In some examples, components of Memories 622, 638, and / or 652 may not include machine-learning algorithms that reduce complexity and are verified and / or authenticated from a security perspective.

[0139] As described herein, a typical neural network is a biologically inspired algorithm that generates an output through input data through a series of connected layers. Further, each layer in a neural network can include another neural network or can include any number of layers (whether convolutional or not). As can be understood in the context of the present disclosure, a neural network can utilize machine learning, which can refer to a broad class of the above algorithms where the output is generated based on learning parameters.

[0140] Although described in the context of neural networks, any type of machine learning can be used without conflicting with the present disclosure. For example, but not limited to, machine learning or machine - learning algorithms can include regression algorithms (e.g., ordinary least squares regression (OLSR), linear regression, logistic regression, stepwise regression, multivariate adaptive regression splines (MARS), locally estimated scatterplot smoothing (LOESS)), instance - based algorithms (e.g., ridge regression, least absolute shrinkage and selection operator (LASSO), elastic net, least angle regression (LARS)), decision tree algorithms (e.g., classification and regression tree (CART), iterative dichotomiser 3 (ID3), chi - square automatic interaction detection (CHAID), decision stump, conditional decision tree), Bayesian algorithms (e.g., naive Bayes, Gaussian naive Bayes, polynomial naive Bayes, averaged one - dependence estimators (AODE), Bayesian belief network (BNN), Bayesian network), clustering algorithms (e.g., k - means, k - median, expectation maximization (EM), hierarchical clustering), correlation rule learning algorithms (e.g., perceptron, backpropagation, Hopfield network, radial basis function network (RBFN)), deep learning algorithms (e.g., deep Boltzmann machine (DBM), deep belief network (DBN), convolutional neural network (CNN), stacked autoencoder), dimensionality reduction algorithms (e.g., principal component analysis (PCA), principal component regression (PCR), partial least squares regression (PLSR), Sammon mapping, multidimensional scaling (MDS), projection pursuit, linear discriminant analysis (LDA), mixture discriminant analysis (MDA), quadratic discriminant analysis (QDA), flexible discriminant analysis (FDA)), ensemble algorithms (e.g., boosting, bootstrap aggregation (bagging), AdaBoost, stacked generalization (blending), gradient boosting machine (GBM), gradient boosting regression tree (GBRT), random forest), SVM (support vector machine), supervised learning, unsupervised learning, semi - supervised learning, etc.

[0141] Examples of additional architectures include neural networks such as ResNet50, ResNet101, VGG, DenseNet, PointNet, and the like.

[0142] FIG. 7 depicts an exemplary process for the operation of a vehicle safety system. In some examples, the exemplary process 700 can be performed by at least a vehicle safety system (e.g., the vehicle safety system 102 as described above with reference to FIG. 1).

[0143] In operation 702, the exemplary process 700 can include receiving a trajectory (e.g., a candidate trajectory). The candidate trajectory can be received in a first component of the vehicle (e.g., the trajectory management component 108) (e.g., an autonomous vehicle) for the vehicle to follow.

[0144] In operation 704, the exemplary process 700 can include receiving at least one of a first signal (e.g., signal 116) indicating an object approaching the vehicle, a second signal (e.g., signal 118) indicating a physics-based collision state, or a third signal (e.g., signal 120) indicating a machine learning (ML)-based collision state. Signal 116 can be received by the trajectory management component 108 from a second component (e.g., the perception component 110). Signal 118 can be received by the trajectory management component 108 from a third component (e.g., the filter component 112). Signal 120 can be received by the trajectory management component 108 from a fourth component (e.g., the collision detection component 114). In some examples, one or more of the perception component 110, the filter component 112, the collision detection component 114, and the trajectory management component 108 can be integrated together and / or share results with each other.

[0145] In operation 706, exemplary process 700 may include determining whether at least one of signal 116, signal 118, or signal 120 has been received. Exemplary process 700 may proceed to operation 702 based on determining that at least one of signal 116, signal 118, or a third signal has not been received by orbit management component 108. Exemplary process 700 may proceed to operation 708 based on determining that at least one of signal 116, signal 118, or signal 120 has been received by orbit management component 108.

[0146] In operation 708, exemplary process 700 may include determining to follow a candidate orbit or a safety stop orbit. Orbit management component 108 may be capable of determining to follow a candidate orbit or a safety stop orbit based on at least one of signal 116, signal 118, signal 120. In some examples, orbit management component 108 may be capable of determining to follow a candidate orbit or a safety stop orbit before or after receiving at least one of signal 116, signal 118, or signal 120. In other examples, orbit management component 108 may be capable of waiting for at least one of signal 116, signal 118, or signal 120 before determining to follow a candidate orbit or a safety stop orbit.

[0147] Exemplary articles A method comprising receiving an orbit candidate for an autonomous vehicle to follow, generating a first signal indicative of an object approaching the autonomous vehicle, generating a second signal indicative of a physics-based collision state and determining a first likelihood of a collision based at least in part on the physics-based collision state, generating a third signal indicative of a machine-learned collision state and determining a second likelihood of a collision based at least in part on the machine-learned (ML) collision state, and determining to follow a candidate orbit or a safety stop orbit based at least in part on at least one of the first signal, the second signal, or the third signal.

[0148] B: Determining to follow a candidate trajectory or a safety stop trajectory includes waiting until the first signal, the second signal, and the third signal are generated, the method of item A.

[0149] C: Determining to follow a candidate trajectory or a safety stop trajectory further includes determining that a first likelihood of a collision is lower than a second likelihood of the collision, and determining to follow the safety stop trajectory at least partially based on the second likelihood of the collision, the method of item A or B.

[0150] D: The component that receives at least one of the first signal, the second signal, or the third signal performs a collision check by perturbing a simulation of the second object along at least one of the acceleration or the steering angle associated with the second object, the method of any one of items A - C.

[0151] E: The object is the first object. Generating the first signal further includes identifying a second object closest to the autonomous vehicle, identifying the first object as closer to the candidate trajectory than the second object, and generating the first signal based on the first object being closer to the candidate trajectory than the second object. Generating the second signal further includes identifying a physics - based collision state at least partially based on a third object approaching the candidate trajectory, and filtering out and removing a fourth object at least partially based on the object trajectory of the fourth object being associated with a possible intersection with the candidate trajectory. Generating the third signal further includes determining that an ML - based collision state satisfies or exceeds a threshold collision state at least partially based on a predicted distance between a fifth object and the autonomous vehicle being less than a threshold distance, the method of any one of items A - D.

[0152] F: When executed, it includes receiving a candidate trajectory that the vehicle will follow, receiving a first signal indicating an object approaching the vehicle, receiving a second signal indicating a collision state based on physics, where the second signal indicates a first likelihood of a collision determined at least partially based on the physics-based collision state, or receiving a third signal indicating a collision state based on machine learning (ML), where the third signal indicates a second likelihood of a collision determined at least partially based on the ML-based collision state, and determining to follow the candidate trajectory or a safe stop trajectory based at least partially on at least one of the first signal, the second signal, or the third signal. One or more non-transitory computer-readable media storing instructions that cause one or more processors to perform the operations.

[0153] G: Determining to follow the candidate trajectory or a safe stop trajectory further includes waiting until the first signal, the second signal, and the third signal are received. One or more non-transitory computer-readable media of item F.

[0154] H: Determining to follow the candidate trajectory or a safe stop trajectory further includes determining that the first likelihood of a collision is lower than the second likelihood of a collision, and determining to follow the safe stop trajectory based at least partially on the second likelihood of a collision. One or more non-transitory computer-readable media of item F or G.

[0155] I: The first component that receives at least one of the first signal, the second signal, or the third signal performs a collision inspection with a higher level of accuracy than the second component that generates the second signal and the third component that generates the third signal. One or more non-transitory computer-readable media of any one or more of items F - H.

[0156] J: The first signal is generated by a first component, the first component being configured to store candidate trajectories and previous candidate trajectories in a queue, and determining to follow a candidate trajectory is performed by a second component based at least in part on the first signal received by the second component, one or more non-transitory computer-readable media of any one or more of items F - I.

[0157] K: The object is a first object, and generating a second signal is based at least in part on determining that a second object associated with a candidate trajectory is closer to the candidate trajectory than a third object associated with the candidate trajectory, and a physics-based collision state is determined based at least in part on the second object, one or more non-transitory computer-readable media of any one or more of items F - J.

[0158] L: The first signal is determined based at least in part on kinematic data associated with a vehicle, one or more non-transitory computer-readable media of any one or more of items F - K.

[0159] M: The system is one or more processors and one or more non-transitory computer-readable media storing instructions executable by the one or more processors, the instructions, when executed, causing the system to receive a candidate trajectory that a vehicle follows, and a first signal indicating an object approaching the vehicle, a second signal indicating a physics-based collision state, the second signal indicating a first likelihood of a collision determined based at least in part on the physics-based collision state, or a third signal indicating a second likelihood of a collision based at least in part on a machine learning (ML)-based collision state, receive at least one of the first signal, the second signal, or the third signal, and determine to follow a candidate trajectory or a safe stop trajectory based at least in part on at least one of the first signal, the second signal, or the third signal, and one or more non-transitory computer-readable media.

[0160] N: The operation further includes receiving a second signal and a third signal, and determining to follow a candidate trajectory or a safety stop trajectory includes determining that a first likelihood of a collision based at least in part on an ML-based collision state is lower than a second likelihood of a collision based at least in part on a physics-based collision state, and determining to follow a safety stop trajectory based at least in part on the second likelihood of the collision, and further includes the system of item M.

[0161] O: The operation further includes identifying a second object and a third object associated with a candidate trajectory, and identifying that the second object is the object closest to the candidate trajectory, and generating the second signal is based at least in part on the second object, and is the system of item M or N.

[0162] P: The first signal is determined based at least in part on kinematic data associated with the vehicle, and is the system of any one of items M to O.

[0163] Q: Determining to follow a candidate trajectory or a safety stop candidate trajectory includes waiting until at least a third signal is received, and is the system of any one of paragraphs M to P.

[0164] R: The first signal is generated by a first component, the first component is configured to store a candidate trajectory and a previous candidate trajectory in a queue, and determining to follow a candidate trajectory is performed by a second component based at least in part on the first signal received by the second component, and is the system of any one of items M to Q.

[0165] S: The component that receives at least one of the first signal, the second signal, or the third signal performs a collision check by perturbing a simulation of the second object along at least one of an acceleration or a steering angle associated with the second object, and is the system of any one of items M to R.

[0166] T: A system of any of clauses M - S, wherein the collision state based on physics is identified based at least in part on the object closest to the candidate trajectory.

[0167] While the exemplary clauses described above are described with respect to one particular implementation, in the context of this document, it should further be understood that the content of the exemplary clauses can also be implemented via a method, device, system, computer - readable medium, and / or another implementation. Additionally, any of Examples A - T may be implemented alone or in combination with any one or more of the other Examples A - T.

[0168] End One or more examples regarding the techniques described in this specification have been described, but various alternatives, additions, substitutions, and equivalents are included within the scope of the techniques described in this specification.

[0169] In the description of the examples, references are made to the accompanying drawings, which form a part and show specific examples of the claimed subject matter as illustrations. It is understood that other examples may be used and that changes or alternatives, such as structural changes, are possible. The above examples, changes, or alternatives are not necessarily departures from the scope with respect to the intended and claimed subject matter. The steps in this specification may be given in a particular order, but in some cases, the order may be changed so that a particular input is provided at different times or in a separate order without changing the functionality of the described system and method. Further, the disclosed procedures may be executed in different orders. Additionally, the various calculations in this specification need not be performed in the order disclosed, and other examples using alternative orders of calculations can be implemented without difficulty. In addition to being reordered, the calculations may also be broken down into sub - calculations having the same result.

Claims

1. Receiving candidate trajectories for the vehicle to follow, A first signal indicating an object approaching the vehicle, A second signal indicating a collision state based on physics, wherein the second signal indicates a first likelihood of a collision determined at least partially based on the collision state based on physics, or A third signal indicating a collision state based on machine learning (ML), wherein the third signal indicates a second likelihood of a collision determined at least partially based on the collision state based on ML. To receive at least one of the following, To determine to follow the candidate trajectory or safety stop trajectory based at least in part on at least one of the first signal, the second signal, or the third signal. A method characterized by comprising:

2. Receiving at least one of the first signal, the second signal, or the third signal is: Receiving the first signal, the second signal, and the third signal. Further including, deciding to follow the candidate trajectory or the safe stop trajectory, Waiting until the first signal, the second signal, and the third signal are received, Based at least in part on the first signal, the second signal, and the third signal, it is decided to follow the candidate trajectory or the safety stop trajectory. The method according to claim 1, further comprising:

3. Deciding to follow the candidate trajectory or the safety stop trajectory means Waiting until at least one of the first signal, the second signal, or the third signal is received. The method according to claim 1, further comprising:

4. Receiving the second signal and the third signal The further provision includes determining to follow the candidate trajectory or the safe stop trajectory, Determining that the first likelihood of the collision is lower than the second likelihood of the collision, Deciding to follow the safety stop trajectory based at least in part on the second likelihood of the collision The method according to claim 1, further comprising:

5. The method according to claim 1, characterized in that the first component that receives at least one of the first signal, the second signal, or the third signal performs collision inspection with a higher level of accuracy than the second component that generates the second signal and the third component that generates the third signal.

6. The first signal is generated by the first component, The first component is configured to store the candidate trajectory and previous candidate trajectories in a queue, The decision to follow the candidate trajectory is made by the second component at least in part based on the first signal received by the second component. The method according to claim 1, characterized by the features described above.

7. The aforementioned object is the first object, The second signal is generated at least in part on determining that the second object associated with the candidate trajectory is closer to the candidate trajectory than the third object associated with the candidate trajectory. The collision state based on the aforementioned physics is determined at least partially on the second object. The method according to claim 1, characterized by the features described above.

8. A computer program characterized by including coded instructions that, when executed on a computer, implement the method described in any one of claims 1 to 7.

9. It is a system, One or more processors, One or more non-temporary computer-readable media storing instructions that can be executed by the one or more processors, wherein when an instruction is executed, it is stored in the system. Receiving candidate trajectories for the vehicle to follow, A first signal indicating an object approaching the vehicle, A second signal indicating a collision state based on physics, wherein the second signal indicates a first likelihood of a collision determined at least partially based on the collision state based on physics, or A third signal indicating a collision state based on machine learning (ML), which at least partially indicates a second likelihood of collision based on the ML-based collision state. To receive at least one of the following, To determine to follow the candidate trajectory or safety stop trajectory based at least in part on at least one of the first signal, the second signal, or the third signal. One or more non-temporary computer-readable media that perform operations including the above A system characterized by having the following features.

10. The object is the first object, and the operation is, Identifying the second and third objects associated with the candidate trajectory, Identifying that the second object is the object closest to the candidate trajectory and It further includes, The second signal is generated at least partially based on the second object. The system according to feature 9.

11. The system according to claim 9 or 10, characterized in that the first signal is determined at least in part on kinematic data associated with the vehicle.

12. The system according to claim 9 or 10, characterized in that deciding to follow the candidate trajectory or the safety stop trajectory includes waiting until at least the third signal is received.

13. The system according to claim 9 or 10, wherein the object is a first object, and a component that receives at least one of the first signal, the second signal, or the third signal performs a crash test by perturbing a simulation of the second object along at least one of the acceleration or steering angle associated with the second object.

14. The system according to claim 9 or 10, characterized in that the collision state based on the aforementioned physics is identified at least partially based on the object closest to the candidate trajectory.

15. The object is the first object, and the first signal is, Identifying the second object closest to the aforementioned vehicle, Identifying the first object as being closer to the candidate trajectory than the second object, The first signal is generated based on the fact that the first object is closer to the candidate trajectory than the second object. The second signal is generated by, Identifying a collision state based on physics, at least partially, on a third object approaching the candidate trajectory, Filtering out the fourth object based at least in part on the fact that the object trajectory of the fourth object is associated with a possible intersection with a candidate trajectory. The third signal is generated by, The collision condition based on the ML is determined to satisfy or exceed a threshold collision condition, at least in part, on the fact that the predicted distance between the fifth object and the vehicle is less than the threshold distance. The system according to claim 9 or 10, characterized in that it is generated by

Images