Vehicle safety system

Abstract

Techniques for verifying or determining a vehicle trajectory are discussed herein. A trajectory management component may receive status and / or error data from other safety system components and select or otherwise determine a safe and effective vehicle trajectory. A perceiving component of the security system may verify the trajectory, a trajectory management component may wait for selecting a vehicle trajectory based on the trajectory, verify the trajectory stored in the queue, and / or verify the trajectory using kinematics. A filter component of the security system may filter out objects based on tracks stored in a queue. A collision detection component of the security system may determine a collision state based on the trajectories stored in the queue, or determine a collision state based on which a trajectory management component may wait for selection or otherwise determine the vehicle trajectory.

Description

[0001] Cross-references to related applications

[0002] This patent application claims priority to U.S. utility patent application serial number 17 / 842,469 filed on June 16, 2022. Application serial number 17 / 842,469 is incorporated herein by reference in its entirety. Background Art

[0003] The vehicle includes various systems for guiding the vehicle through an environment that includes static and / or dynamic objects. The safety system can analyze sensor data about the environment. Sensor data, such as lidar data, radar data, camera data, etc., can be used to determine the impact of detected objects on potential vehicle actions. Objects encountered in the environment can include dynamic objects that are moving or capable of moving (e.g., vehicles, motorcycles, bicycles, pedestrians, animals, etc.) and / or static objects (e.g., buildings, road surfaces, trees, signs, obstacles, parked vehicles, etc.). The vehicle can generate a trajectory and use the safety system to ensure that the trajectory is safe. The trajectory must be generated in real time or near real time so that the vehicle can navigate the environment accurately and effectively. The trajectory can be used to control the vehicle while taking into account events around the vehicle. The decisions taken by the vehicle when traversing the environment can ensure the safety of passengers in the vehicle and other people or objects near the vehicle. BRIEF DESCRIPTION OF THE DRAWINGS

[0004] The detailed description is described with reference to the accompanying drawings. In the drawings, the leftmost digit(s) of a reference number identifies the drawing in which the reference number first appears. The use of the same reference number in different drawings indicates similar or identical components or features.

[0005] Figure 1 is a block diagram of an example vehicle system included in a vehicle and including safety system components according to an example of the present disclosure.

[0006] Figure 2 is a flow chart of an example process for operation of a sensing component according to examples of the present disclosure.

[0007] Figure 3 is a flow chart of an example process for operation of a filter component according to examples of the present disclosure.

[0008] Figure 4 is a flow chart of an example process for operation of a collision detection component according to examples of the present disclosure.

[0009] Figure 5 An example driving environment is shown, illustrating a vehicle traversing the environment, and several candidate trajectories determined for the vehicle.

[0010] Figure 6is a block diagram of an example system for implementing the techniques described herein.

[0011] Figure 7 An example method for operation of a vehicle safety system is depicted. DETAILED DESCRIPTION

[0012] This article discusses techniques for verifying or determining vehicle trajectories. In some examples, a collision avoidance system for a vehicle may include a system (or "vehicle system") having a main system and a vehicle safety system (or "safety system"). The safety system may include, but is not limited to, a trajectory management component, a perception component, a filter component, and / or a collision detection component. The trajectory management component may receive state and / or error data from other safety system components and select a safe and effective trajectory when the vehicle navigates the environment. The perception component may use sensor data to determine a corridor associated with the trajectory (e.g., a bounded area corresponding to the trajectory) to improve collision prediction and avoidance of the vehicle. The perception component may verify the trajectory, the trajectory management component may wait for the vehicle trajectory to be selected, verify the trajectory stored in the queue, and / or verify the trajectory using kinematic data. The filter component may perform a collision check to filter out objects having a trajectory that is determined to be unrelated to a potential intersection between the vehicle trajectory and the object trajectory. In some examples, the filter component may filter out objects based on the trajectory stored in the queue. The collision detection component may determine the collision state of the vehicle and use the collision state to determine the vehicle safety operation to be performed by the vehicle. The collision detection component may determine a collision state based on the trajectories stored in the queue, or determine a collision state based on which the trajectory management component may wait to select a vehicle trajectory.

[0013] The trajectory management component can select or otherwise determine a trajectory from the trajectory generated and received by the main system based on data generated and received by other components of the safety system. The trajectory can include a planned trajectory, a first safe stop trajectory, a second safe stop trajectory, etc. In various examples, such a planned trajectory can include multiple trajectories determined by the main system, and the first and subsequent safe stop trajectories can be variations of such planned trajectories with different longitudinal deceleration curves. Of course, in at least some examples, such a safe stop trajectory can include a lateral deviation from the planned trajectory. Data can be received from other perception components and utilized by the trajectory management component. The data can include verification results (e.g., verification results of the trajectory), state data, or error data. These data can be used to select a trajectory to ensure punctuality, consistency, feasibility, and collision checks. In some examples, the trajectory management component can select a trajectory for the vehicle at any time (e.g., before or after) after receiving some or all of the data generated by other perception components. In other examples, the trajectory management component can wait to receive some or all of the data generated by other perception components before selecting a trajectory for the vehicle.

[0014] The perception component can determine data (or "perception data") associated with a corridor representing a geometric area relative to the vehicle (e.g., defining an area that the vehicle is about to traverse), and / or data associated with objects near the vehicle. The data associated with the objects can include classification, instance and / or semantic segmentation, bounding boxes, trajectories, route data, global and / or local map data, etc. The corridor can include a portion (e.g., part or the entire portion) of an environment, such as a portion (e.g., part or the entire portion) of a road and / or a portion (e.g., part or the entire portion) of an object. Data associated with the object and / or corridor can be determined based on one or more of a trajectory received from a host system, a trajectory received from a host system and stored in a perception component queue, and a kinematic and / or trajectory used for verification (e.g., verification of one or more trajectories). The perception component can analyze the trajectory based on a corridor or a subset of corridors.

[0015] The filter component can receive the trajectory and filter out objects in the environment based on the received trajectory. The filter component can be integrated with the trajectory management component, and / or the trajectory can be stored in the queue of the filter component. The filter component can use the trajectory to determine the objects near the vehicle and filter out the objects determined to be irrelevant to the potential intersection between the vehicle trajectory and the object trajectory. The object can be filtered out based on the results of the executed algorithm (or "model"), which indicates that the metric (e.g., quantity metric and / or probability metric) of the driving simulation that causes a collision or near collision (or other potential safety hazards) with the object is lower than the threshold metric, and / or can be based on other factors such as vehicle speed, driving conditions, risk tolerance configuration of the vehicle safety system, etc. The algorithm can be used to determine the possible collision point and / or intersection time between the vehicle and the object based on vehicle information (e.g., the current trajectory of the vehicle, the current speed of the vehicle, the planned path of the vehicle, etc.) and object information (the current trajectory and / or speed of the object, one or more predicted trajectories and / or speeds of the object). In at least some examples, this filtering can enable the vehicle safety system to devote as many resources as possible to the object with the greatest risk of interaction with the vehicle.

[0016] The collision detection component can receive the trajectory and determine the collision state based on the received trajectory. The collision state can be determined based on the predicted distance between the object and the vehicle being less than a threshold distance (e.g., a predicted distance determined based on the trajectory of the object and the trajectory of the vehicle). The velocity vector and / or potential collision characteristics can be predicted based on determining a potential collision with the object. The predicted velocity vector and / or potential collision characteristics can be used to determine the collision state based on the collision probability, impact point, relative collision speed, and / or relative collision angle associated with the vehicle trajectory. In some examples, the collision state can be determined, and the trajectory management component can wait for the selection of the vehicle trajectory based on this collision state. In other examples, the vehicle trajectory for determining the collision state can be stored in a queue. The collision state can be used to determine the safe operation of the vehicle (e.g., trajectory). The collision detection component can be operated separately from other vehicle systems and / or components for perception, prediction, trajectory planning, etc. to provide redundancy, error checking, and / or verification of other vehicle systems and / or components.

[0017] The techniques discussed herein can improve the functionality of a computing device in a variety of additional ways. These techniques can be used to optimize available computing resources (compared to not implementing the techniques) by performing operations that limit the impact on available resources. Based on one or more outputs from the model of the filter component, available computing resources can be directed to the most relevant objects during vehicle planning, thereby improving the safety of the vehicle as it navigates through an environment. A trajectory can be selected or otherwise determined for a vehicle to more effectively avoid and / or mitigate potential collisions with other moving objects in the environment. The vehicle system can determine a trajectory and accurately switch to a selected trajectory after detecting a potential collision, executing a new perception model, and / or dynamically generating or validating a new trajectory after detecting a potential collision.

[0018] In addition, by operating the trajectory management component to wait for results from some of the components in one or more other safety system components (e.g., a perception component, a filter component, and / or a collision detection component), computing resources can be saved by not performing operations based on data that is later determined to be outdated or inaccurate. To avoid operational delays and / or lags in the trajectory management component, the trajectory management component can wait for some, but perhaps not all, other safety system components. Additionally or alternatively, the trajectory management component can operate without waiting for any other safety system components. By having the trajectory management component wait for fewer than all other safety system components to perform operations for controlling the vehicle, delays in the trajectory management component can be reduced.

[0019] By storing the trajectories in a queue of a perception component for determining data associated with corridors and / or objects, computing resources that may be exhausted due to an emergency can be utilized more resourcefully and with less urgency. One or more safety system components (e.g., filter components) integrated with the trajectory management component and / or used to store the trajectories in a queue can eliminate unnecessary data processing, reduce redundant data processing, and / or reduce any urgency of data processing. By storing the trajectories in a queue of a corresponding safety system component, the trajectory management component can switch back and forth between trajectories (or fall back to a previously verified trajectory) based on information associated with the corresponding trajectory determined by and received from the corresponding safety system component utilizing the corresponding queue. Computing resources that may be exhausted due to an emergency can be utilized more resourcefully with less urgency and based on the collision state determined for the trajectories stored in the collision detection component queue. Processing data with less urgency can better allocate time and / or prioritize different types of data processing.

[0020] The technology described herein can be implemented in a variety of ways. Example embodiments are provided below with reference to the following figures. Although applicable to vehicles, such as autonomous vehicles, the methods, devices and systems described herein can be applied to various systems and are not limited to autonomous vehicles. In another example, these technologies can be used in aviation or navigation scenarios, or in any system configured to input data to determine the motion associated with an object in an environment. In addition, the technology described herein can be used with real data (e.g., captured using a sensor), simulated data (e.g., generated by a simulator), or any combination of the two.

[0021] Figure 1 is a block diagram of an example vehicle system 100 of a vehicle traveling in an environment including safety system components according to an example of the present disclosure.

[0022] The vehicle system 100 may include a vehicle safety system 102 for ensuring safe operation of the vehicle. The vehicle safety system 102 may receive data (or "host system data") from a host system of the vehicle system 100. The data received from the host system may include sensor data 104 and trajectory data 106. The sensor data 104 may include data received from one or more systems in the vehicle system 100 (e.g., sensor system 606, as described below with reference to FIG. 1 ). Figure 6 The trajectory data 106 may include data received by one or more sensors in the vehicle system 100 (e.g., a host system, which may be implemented as one or more first computing devices 604, as described below). Figure 6In various examples, such trajectory data 106 may include one or more desired vehicle states (e.g., heading, position, steering angle, rotation rate, etc.) and / or related command sets (e.g., acceleration, torque, voltage, steering angle, etc.).

[0023] The trajectory data 106 may include one or more trajectories of the vehicle (e.g., different types of trajectories). The trajectory (or "received trajectory") (or "candidate trajectory") may include a planned trajectory and / or one or more other trajectories (or "alternative trajectories") (e.g., a first safe stop trajectory (or "error trajectory"), a second safe stop trajectory (or "high priority error trajectory"), a third safe stop trajectory (or "immediate stop trajectory"), etc.). The received planned trajectory may be generated based on the sensor data 104, which may include data associated with one or more objects in the environment (or "object data"). Such a planned trajectory may include a nominal trajectory determined according to various systems for traversing the environment, for example, as described in U.S. Patent No. 11,048,260, entitled "Adaptive Scaling in Trajectory Generation", the entire contents of which are hereby incorporated by reference herein. Alternative trajectories may include various types of error trajectories, which may include critical error trajectories, etc.

[0024] The master system data may include one or more other data (e.g., vehicle state data), some or all of which may be included in and / or separate from one or more of the trajectory data 106 and the sensor data 104. The vehicle state data may include various types of data, including performance metrics, component errors, alarms, system failures, malfunctions, and / or manual or automatic requests to change to or from an alternative trajectory. In some examples, the vehicle state data may include data identifying system failures, errors, alarms, etc., which are associated with any type of received trajectory. The vehicle state data may also include a request for the autonomous vehicle to follow a trajectory (or a "requested trajectory"), which may include any received trajectory. The vehicle state data may be received via internal signals from the master system according to the internal timing and processing cycles of the master system components.

[0025] The vehicle safety system 102 may include a trajectory management component 108 (e.g., a first component), a perception component 110 (e.g., a second component), a filter component 112 (e.g., a third component), and a collision detection (or "machine learning (ML)") or ("neural network") component 114 (e.g., a fourth component). In some examples, a portion (e.g., part or all) of the vehicle system data (e.g., sensor data 104 and / or trajectory data 106) may be received from the host system and received by the trajectory management component 108. In those examples where a portion of the vehicle system data (e.g., trajectory data 106) is received by the trajectory management component 108, the remaining portion of the vehicle system data (e.g., sensor data 104) may be received by the trajectory management component 108 and / or one or more of the remaining components of the vehicle safety system 102 (e.g., one or more of the perception component 110, the filter component 112, and the collision detection component 114). In some examples, the same vehicle system data may be received by one or more of the trajectory management component 108, the perception component 110, the filter component 112, and the collision detection component 114. In some examples where the trajectory management component 108 receives a portion of the trajectory data 106 and / or the sensor data 104, the trajectory management component 108 may transmit (e.g., route) some or all of the received trajectory data 106 and / or the sensor data 104 to one or more of any remaining components of the vehicle safety system 102 (e.g., some or all of the trajectory data 106 and / or the sensor data 104 may be received from the host system via the trajectory management component 108 and through any one or more of the remaining components of the vehicle safety system 102).

[0026] While any component of the vehicle safety system 102 (the trajectory management component 108, the perception component 110, the filter component 112, and the collision detection component 114) may receive a different portion of the vehicle system data than any other component of the vehicle safety system 102, as described in the present disclosure, it is not limited thereto. Any portion (e.g., some or all) of the vehicle system data may be received from the host system and received by one or more of any of the components of the vehicle safety system 102 (e.g., trajectory data 106 and / or sensor data 104 received from the host system and received by one or more of any of the remaining components of the vehicle safety system 102, without necessarily passing through the trajectory management component 108).

[0027] The trajectory management component 108 can process the received vehicle system data portions (e.g., sensor data 104 and / or trajectory data 106). In some examples, the trajectory management component 108 can determine a selected (or "output") trajectory for the vehicle to follow. The trajectory management component 108 can determine the selected trajectory based on trajectory validation results (e.g., results associated with one or more stored possible trajectories), vehicle state data (e.g., state data from one or more vehicle components, including vehicle sensors, vehicle platforms, posture systems, vehicle monitoring systems, and / or any other system of the autonomous vehicle), and / or manual or automatic requests to change to or from an alternative trajectory. The vehicle state data can include performance metrics, component errors, alarms, system failures, and system failures.

[0028] In some examples, for example, when the trajectory data 106 includes a planned trajectory and an alternative trajectory, the trajectory management component 108 can determine any received trajectory (or "current trajectory") (e.g., the planned trajectory and the alternative trajectory) as a selected trajectory. In alternative or additional examples where the trajectory data 106 includes a planned trajectory but does not include an alternative trajectory, the trajectory management component 108 can determine the planned trajectory as the selected trajectory. In those or other examples, the trajectory management component 108 can generate an alternative trajectory based on the planned trajectory, determining any received trajectory (e.g., the planned trajectory and the alternative trajectory) as the selected trajectory. Other examples of generating, evaluating, and determining trajectories can be found, for example, in U.S. patent application Ser. No. 17 / 514,610, entitled “Autonomous Vehicle Trajectory Determination Based on State Transition Model,” filed on October 29, 2021, the entire contents of which are incorporated herein by reference for all purposes.

[0029] The trajectory management component 108 can manage and store various trajectories. In some cases, the trajectories can be managed and stored by the state transition model of the trajectory management component 108. The trajectory management component 108 can store the current trajectory, including the selected trajectory. In some examples, the trajectory management component 108 can also retain one or more previous trajectories (e.g., any previous trajectory received and / or determined by the trajectory management component 108). Such stored trajectories can provide failsafe protection in the event that the vehicle safety system does not receive an updated trajectory from the main system.

[0030] The operation of the trajectory management component 108 can be asynchronous or synchronous with respect to the remaining components of the vehicle safety system 102 (e.g., any one of the sensing component 110, the filter component 112, and the collision detection component 114). The operation of any component of the vehicle safety system 102 (e.g., any one of the sensing component 110, the filter component 112, and the collision detection component 114) can be asynchronous or synchronous with respect to any other component (e.g., any one of the sensing component 110, the filter component 112, and the collision detection component 114). In some examples, the trajectory management component 108 can select the trajectory of the vehicle at any time (e.g., before or after) for a portion (e.g., part or all) of the data generated by any one of the one or more other sensing components. By not waiting, the trajectory management component 108 can ensure that the amount of delay in responding to a potential collision is minimized. In other examples, the trajectory management component 108 can wait to receive a portion (e.g., part or all) of the data generated by any one of the one or more other sensing components, and then select a trajectory for the vehicle. In such an example, false positives can be avoided by ensuring that there are no inconsistencies in the data. The data that the trajectory management component 108 may be waiting for may include data in signals (eg, signals 116 - 120 ) received from components of the vehicle safety system based on the components' respective hash functions, as described below.

[0031] The perception component 110 can determine data (or "perception component data") related to one or more corridors representing a geometric area relative to the vehicle (or "perception data"), and / or data related to objects near the vehicle. In some examples, the perception component 110 determines the perception data by evaluating one or more of the trajectories received from the host system. The perception component 110 can evaluate the trajectories relative to the one or more corridors relative to the vehicle.

[0032] The perception component 110 can determine the perception data in various ways based on various types of data. In some examples, the perception component 110 can determine the perception data by evaluating the corridor without receiving any tracks. In some examples, the perception component 110 can determine one or more tracks associated with the corridor based on the corridor received by the perception component 110. In other examples, the perception component 110 can determine one or more tracks associated with a subset of corridors based on the subset of corridors received by the perception component 110.

[0033] In some examples, the perception component 110 can evaluate the trajectory relative to one or more corridors associated with the vehicle (e.g., a single trajectory received from a host system). For example, the corridor can be an environmental area determined at least in part based on the candidate trajectory, the vehicle range (e.g., width, length, height), the vehicle's current speed, the speed specified by the trajectory, and / or offsets based on steering rack actuator gains, vehicle kinematics, etc. The kinematics associated with the vehicle can include the vehicle's minimum turning radius, maximum turning rate, tire slip, body roll, etc. In those examples using kinematics, the corridor can be limited to an environmental area in which the vehicle can operate for a period of time and can include a buffer zone. In at least some such examples, the time period can correspond to the amount of time to bring the vehicle to a complete stop according to any one or more deceleration curves.

[0034] The evaluated trajectories may be stored in the perception component 110 and / or transmitted to the trajectory management component 108 as perception component data. In some examples, a trajectory (e.g., a single trajectory) may be evaluated and used to determine an intersection probability (e.g., a single intersection probability associated with one or more objects). Trajectories may be evaluated based on corridors associated with the trajectory (e.g., a single corridor) or a subset of corridors associated with the trajectory. In some examples, the results of the evaluation of the trajectory by the perception component 110 may be transmitted to and received by the trajectory management component 108, which may wait for the results before performing trajectory selection.

[0035] The trajectory management component 108 can use the intersection probability determined by the perception component 110 to analyze and / or determine information (or "object information") about one or more objects in the corridor (e.g., the trajectory management component 108 can use the intersection probability to determine the corresponding collision probability). By waiting to receive the object identified by the perception component 110 based on the evaluated trajectory, the trajectory management component 108 ensures that any relevant information about the identified object is obtained from the perception component 110 before advancing to the next frame (e.g., the next step) for trajectory selection.

[0036] In some examples, the perception component 110 can identify and / or determine perception component information (e.g., information including one or more intersection probabilities, which can be combined and / or used to determine various characteristics associated with the corresponding object). The perception component information can include information associated with an object close to the vehicle, a value indicating the distance of the object from the vehicle and / or one or more predicted future positions of the vehicle, a value indicating the speed at which the object is approaching the vehicle or the corresponding direction of the predicted future vehicle position of the vehicle, a value indicating how close the object's direction of travel is to the current position or predicted future vehicle position in the corridor, etc.). The perception component information can be determined and / or utilized in a similar manner as discussed above with respect to the intersection probability (e.g., the perception component information can be transmitted by the perception component 110 to the trajectory management component 108).

[0037] Other examples of determining corridors and validating trajectories can be found, for example, in U.S. Patent Application No. 16 / 588,529, filed on September 30, 2019, entitled “Collision Avoidance Perception System,” which is incorporated herein by reference in its entirety for all purposes. Further examples of determining perception information for trajectory selection can be found, for example, in U.S. Patent Application No. 17 / 514,542, filed on October 29, 2021, entitled “Collision Avoidance and Mitigation in Autonomous Vehicles,” which is incorporated herein by reference in its entirety for all purposes.

[0038] In some examples, the perception component 110 may evaluate more than one trajectory (e.g., a trajectory received from a host system) (e.g., a first trajectory and a second trajectory), and determine more than one intersection probability using the more than one trajectory. In some examples, the first trajectory may be a previous trajectory (e.g., a trajectory generated by the host system in a previous frame (e.g., a previous step)), and the second trajectory may be a current trajectory (e.g., a trajectory generated by the host system in a current frame (e.g., a current step)). The more than one trajectory may be stored in a queue (or "trajectory buffer", e.g., a recent trajectory buffer) of the perception component 110. In some examples, the trajectories (and / or one or more safety stop trajectories) in the more than one trajectory may be evaluated based on a corridor (e.g., a single corridor) associated with the corresponding trajectory or a subset of corridors associated with the corresponding trajectory. In other examples, the more than one trajectory may be evaluated as a group of more than one trajectory based on a corridor (e.g., a single corridor) associated with the group or a subset of corridors associated with the group.

[0039] In some examples, for example, if the number of more than one trajectories for which the perception component 110 evaluates the probability of intersection is two, a two-deep queue can be used to store the perception component information of the two trajectories (e.g., objects associated with respective collision probabilities above a collision probability threshold). In these examples, the trajectory management component 108 does not have to wait for the perception component information to proceed to the next frame. In these examples, generally, the perception component 110 can identify sufficient information (e.g., perception component information) associated with various trajectories (e.g., planned trajectories, and one or more of the first safety stop trajectory, the second safety stop trajectory, the nth safety stop trajectory, etc.) of various frames (e.g., the current frame and the previous frame (e.g., the stored frame)) and provide it to the trajectory management component 108.

[0040] In some examples, determining the intersection probability can include continuously determining (e.g., continuously (or "always") publishing) object information for objects associated with any track. This can enable the track management component 108 to switch back and forth between tracks based on any object detection for any track. Therefore, the operational safety and reliability of the vehicle can be improved.

[0041] In some examples, the perception component 110 can evaluate the trajectory based on kinematics and / or trajectory associated with the vehicle. In examples where the trajectory is evaluated relative to a subset of corridors, the subset of corridors can be determined based on the kinematics and / or trajectory.

[0042] The perception component information may include kinematics and / or trajectories. The kinematics and / or trajectories may be analyzed to determine the intersection probability associated with the object. In some examples, the perception component 110 may determine a physics-based collision state (e.g., a state utilizing and / or combining kinematics). The physics-based collision state may include information (e.g., a value) indicating a predicted collision between the vehicle and another object (e.g., another vehicle, object in the environment, etc.) based on kinematics. Kinematics may include any characteristics related to vehicle motion, such as vehicle motion data (e.g., speed, rate, turning rate, etc.), and / or any determination related to motion (e.g., the feasibility of the current speed, the rate at which the vehicle can (e.g., safely or allowed) turn, etc.). The combination of kinematics and trajectories may be analyzed together with object characteristics (e.g., characteristics of the object), which may be included in the perception component information, and the combination may be used to determine the intersection probability associated with the corresponding object and vehicle.

[0043] By determining the kinematics and / or trajectory, the perception component 110 can determine a physics-based collision state (e.g., a state indicating a corresponding intersection probability). In some examples, the physics-based collision state can be determined by evaluating the kinematics and trajectory (e.g., an envelope of possible trajectories). The kinematics, trajectory, and / or physics-based collision state can be determined based on any possible trajectory of the vehicle.

[0044] The perception component 110 may operate according to one or more modes (or "configurations" or "operating states") to evaluate trajectories. The perception component 110 may operate in a first mode to evaluate trajectories (e.g., a single trajectory) relative to a corridor or a subset of corridors. The perception component 110 may operate in a second mode to evaluate more than one trajectory relative to a corridor or a subset of corridors. The perception component 110 may operate in a third mode to evaluate trajectories based on kinematics and / or trajectories associated with the vehicle. The third mode may be used to determine a physics-based collision state based on the kinematics and / or trajectories.

[0045] Operation of the perception component 110 in any mode may occur in any order and at any time. The mode may be determined, set, and / or modified by the perception component 110, one or more other components of the vehicle, and / or an operator. The mode may be used based on various factors (e.g., computing resources, network resources, storage resources, etc.), preferences, etc. associated with the perception component 110 and / or one or more other components. In order to reduce the demand for storage resources, a first mode may be used because the amount of storage resources used to manage information associated with a single trajectory may be less than the amount of storage resources of a second mode used to manage more than one trajectory and / or corridor subset. In some examples, different modes may refer to dynamically switched modes (e.g., based on vehicle state or context) or statically implemented modes.

[0046] To provide relatively higher operational safety for the vehicle, the second mode can be used because the second mode that manages more than one trajectory and / or corridor subset can provide a previous (or "older") trajectory in the event that a switch back to a previous trajectory is needed (e.g., the second mode stores information to ensure that any object information associated with the previous trajectory is available for collision checking). In other words, the second mode provides the perception component 110 with the option to support falling back to a previous trajectory. If the trajectory management component 108 determines that the current trajectory is invalid or no longer available (e.g., invalid due to the presence of an obstacle), the perception component 110 that stores more than one trajectory information can provide the information to the trajectory management component 108.

[0047] In order to detect objects that may or may not be detected by the perception component 110 in the first mode and / or the second mode, the third mode can be used. In some examples, the perception component 110 operating in the third mode can verify a wider corridor than in the first mode and / or the second mode, and the wider corridor includes any possible location of the vehicle trajectory. For example, by determining the dynamic limits of the vehicle, a cone in front of the vehicle can be determined and the cone can be used to display any possible area where the trajectory may be located. Then, any part of the cone can be verified. The cone characteristics (e.g., size, position, etc.) of the cone can be determined based on information from the main system (e.g., narrowing the range by a desired percentage), which indicates the possible location of the environmental corridor where the vehicle can safely travel, and can be used to verify all objects within the cone. In some examples, any feature determined in the third mode (e.g., cone) can be based on kinematics. In some examples, any feature determined in the third mode (e.g., cone, kinematics, etc.) can be used to determine a physics-based collision state.

[0048] Although the term "mode" is used with respect to the operation of the perception component 110, as described above in the present disclosure, it is not limited thereto. In some examples, the perception component 110 can utilize some or all of the modes, and one or more features / functionality of the corresponding modes, with or without active mode selection. In these or other examples, the perception component 110 can be configured to operate at any time utilizing one or more of the following: i) a single trajectory, ii) more than one trajectory, and / or iii) kinematics and / or trajectory, by default (e.g., a predetermined configuration), by automatic and / or dynamic (e.g., real-time or near real-time) determination and / or selection, by active selection of the perception component 110 and / or other vehicle components (e.g., other safety components), etc.

[0049] The perception component 110 can evaluate one or more objects. The evaluated trajectory can be used to determine the object to be evaluated. In some examples, the object may include a separate object in the object determined based on the corresponding trajectory (for example, the object (for example, the first object) can be determined based on the first trajectory and the object (for example, the second object) can be determined based on the second trajectory). A separate object in the object can be evaluated based on the corresponding trajectory (for example, the object (for example, the first object) can be evaluated based on the first trajectory and the object (for example, the second object) can be evaluated based on the second trajectory). The object can be evaluated to determine data related to the object (or "object state data") (for example, heading, speed, rate, trajectory, turning rate at a given speed, etc.). Additionally or alternatively, the object can be evaluated to determine one or more intersection probabilities, which indicate one or more possibilities that the vehicle and the object intersect along the trajectory at one or more intersection points. Other examples of determining intersection probabilities can be found, for example, in U.S. patent application No. 17 / 133,306, entitled "Procedurally Generated Safety System Determination," filed on December 23, 2020, the entire contents of which are incorporated herein by reference for all purposes.

[0050] In some examples, the perception component 110 can transmit a signal (e.g., a first signal) 116 based on the results of operating the perception component 110. The signal 116 can be transmitted to the trajectory management component 108 based on the evaluation of the trajectory. The evaluation of the trajectory can include validating the trajectory and determining that the respective intersection probabilities are less than the threshold intersection probability based on the validation of the trajectory. The signal 116 can include data (e.g., perception component information) indicating the validation results of the individual trajectories (e.g., data indicating the individual trajectories evaluated relative to the corridor or corridor subset) and / or data including the individual intersection probabilities.

[0051] The filter component 112 may receive the trajectory and use the trajectory to filter out one or more objects in the environment. The filter component may use the trajectory to determine one or more objects near the vehicle, and filter out individual objects that are determined to be irrelevant to the potential intersection between the corresponding vehicle trajectory and the corresponding object trajectory in the one or more determined objects. For example, the object may be filtered out based on determining that the trajectory of the object is irrelevant to the potential intersection of any received vehicle trajectory. The filter component 112 may be integrated with the trajectory management component 108, and / or the trajectories (e.g., the third trajectory and the fourth trajectory) may be stored in a queue of the filter component 112. In some examples, the third and fourth trajectories stored in the queue of the filter component 112 may be the same as the first and second trajectories stored in the perception component 110, respectively. In other examples, one or more of the third and fourth trajectories may be different from one or more of the first and second trajectories.

[0052] In some examples, the filter component 112 can transmit a signal (e.g., a second signal) 118 based on the result of operating the filter component 112. The signal 118 can be transmitted to the trajectory management component 108 based on filtering out the object. The signal 118 may include data indicating the result of filtering out the object (e.g., the filtered out object and / or the generated object that was excluded based on the filtered out object). In some examples, any object (e.g., an unfiltered object) used by the filter component 112 to determine a physics-based collision state (described below) and / or generate the signal 118 can be the same or different from any object used by the perception component 110 to determine an intersection probability and / or to generate the signal 116. In some examples, any object (e.g., an unfiltered object) used by the filter component 112 to determine a physics-based collision state (described below) and / or generate the signal 118 can be the same or different from any object used by the collision detection component 114 to generate the signal 120.

[0053] In some examples, filter component 112 can perform one or more collision checks associated with excluded (e.g., filtered) objects or non-excluded (e.g., non-filtered) objects. The collision checks can be used to determine corresponding collision states (e.g., physics-based collision states).

[0054] The collision detection component 114 can receive a trajectory (e.g., a planned trajectory, and / or one or more of a first safety stop trajectory, a second safety stop trajectory, an nth safety stop trajectory, etc.) and determine one or more collision states (e.g., collision likelihood) based on the received trajectory. The collision state can be determined based on a predicted distance between one or more objects and the vehicle being less than a threshold distance (e.g., a predicted distance determined based on the received trajectories of the corresponding objects and the trajectory of the vehicle). For example, a collision state can be determined to reach or exceed a threshold collision state based on a predicted distance between the object and the vehicle being less than a threshold distance. For example, additional examples of collision states can be found in U.S. patent application Ser. No. 17 / 514,542, filed on October 29, 2021, entitled “Collision Avoidance and Mitigation in Autonomous Vehicles,” the entire contents of which are incorporated herein by reference for all purposes.

[0055] The collision detection component 114 may receive a trajectory generated by the host system, the received trajectory including a planned trajectory or a set of trajectories (e.g., a planned trajectory, and one or more of a first safety stop trajectory, a second safety stop trajectory, an nth safety stop trajectory, etc.). In some examples, the collision detection component 114 receives the planned trajectory and may not receive other trajectories, and the collision detection component 114 may determine one or more alternative trajectories (e.g., a first safety stop trajectory, a second safety stop trajectory, an nth safety stop trajectory, etc.). The collision detection component 114 may determine the alternative trajectory based on the received planned trajectory.

[0056] The collision detection component 114 can store the planned trajectory (e.g., the fifth trajectory) and another planned trajectory (e.g., a previously planned trajectory) (e.g., the sixth trajectory) and / or any number of previously determined safe stop trajectories in a queue of the collision detection component 114. In some examples, the fifth and sixth trajectories stored in the queue of the collision detection component 114 can be the same as the first and second trajectories, respectively, and / or the same as the third and fourth trajectories, respectively. In other examples, one or more of the fifth and sixth trajectories can be different from one or more of the first-fourth trajectories based on corresponding frames associated with any of the trajectories.

[0057] In some examples, the collision detection component 114 can determine one or more collision states based on machine learning (ML) based on stored trajectories and / or based on one or more object trajectories determined for corresponding objects in the environment. The collision detection component 114 can transmit a signal (e.g., a third signal) 120 based on the result of operating the collision detection component 114. The signal 120 may include a collision state based on ML. The signal 120 may be transmitted to the trajectory management component 108 based on determining the collision state. The signal 120 may include data indicating the result of determining the collision state.

[0058] In some examples, any object (e.g., an unfiltered object) used by collision detection component 114 to determine an ML-based collision state and / or generate signal 120 can be the same as or different from any object (e.g., an unfiltered object) used by perception component 110 to determine intersection probability and / or to generate signal 116. In some examples, any object (e.g., an unfiltered object) used by collision detection component 114 to determine an ML-based collision state and / or to generate signal 120 can be the same as or different from any object (e.g., an unfiltered object) used by filter component 112 to determine a physics-based collision state (described below) and / or to generate signal 118.

[0059] The trajectory management component 108 may output data based on the results from the components of the vehicle safety system 102. The data output from the trajectory management component 108 may include control data 122. The control data 122 may be used to control the vehicle. The control data 122 may be determined based on one or more of the results received from the perception component 110 (e.g., a verified trajectory and / or a collision probability), the results received from the filter component 112 (e.g., filtered objects and / or objects obtained based on the exclusion of filtered objects), and the results received from the collision detection component 114 (e.g., collision status). The control data 122 may include one or more of the verification information (e.g., any results from any of the perception component 110, the filter component 112, and / or the collision detection component 114), the planned trajectory, the first safety stop trajectory, the second safety stop trajectory, the nth safety stop trajectory, etc. Based on the selection of one or more trajectories by the trajectory management component 108, the control data 122 may include any of the planned trajectory, the first safety stop trajectory, the second safety stop trajectory, the nth safety stop trajectory, etc. as the selected trajectory. The selected trajectory may be based on results received from the perception component 110 , the filter component 112 , and / or the collision detection component 114 .

[0060] In some examples, trajectory management component 108 can utilize any results from some or all of perception component 110, filter component 112, and / or collision detection component 114 to avoid a collision. In other words, if any results from one or more of perception component 110, filter component 112, and / or collision detection component 114 indicate a potential collision, trajectory management component 108 can modify the trajectory of the vehicle (e.g., to a safer trajectory, such as from a planned trajectory to a safe stop trajectory).

[0061] The trajectory management component 108 can utilize results associated with a higher likelihood of collision (e.g., results from the perception component 110, the filter component 112, and / or the collision detection component 114) instead of other results associated with a relatively lower likelihood of collision (e.g., one or more other results from the perception component 110, the filter component 112, and / or the collision detection component 114). The trajectory management component 108 can utilize the results with a relatively higher likelihood of collision to select a different trajectory (e.g., one of the safe stop trajectories). In some examples, the trajectory management component 108 can determine to operate by changing to another trajectory (e.g., a safer trajectory) based on the higher likelihood of collision of the corresponding component, regardless of whether the remaining components indicate a borderline or lower likelihood of collision (e.g., the trajectory management component 108 can ignore the results associated with a borderline or lower likelihood of collision and perform trajectory selection based on the results associated with a higher likelihood of collision). Although the trajectory management component 108 performs collision checks, the results of any additional collision checks performed by other components (e.g., specific collision checks performed by the collision detection component 114 to output results, such as "thumbs up" or "thumbs down" for corresponding trajectories) can also be utilized by the trajectory management component 108.

[0062] The trajectory management component 108 can utilize the results (e.g., trajectories) received from one or more of the perception component 110 and the filter component 112 to perform collision checks (e.g., utilizing collision checks and / or predictions performed by a prediction server, which may include kinematic-based predictions based on one or more of the signals 116, 118, and 120) at a finer (or "refined" or "more detailed") level (e.g., a higher level of detail, precision, granularity, etc.) based on the corresponding results, respectively. The operations of the prediction server (e.g., the results used by the collision checks performed by the trajectory management component 108) can be used to predict future outcomes (e.g., behavior, position, location, etc.) of the corresponding objects.

[0063] For example, the prediction server may determine prediction server information (e.g., information indicating how an object may travel (e.g., whether the object is capable of turning, accelerating, decelerating, or some combination of turning, accelerating, or decelerating, etc.), how the object may travel at one or more points in time, etc.). The intersection probability associated with the object may be determined by the prediction server and / or the trajectory management component 108 based on the prediction server information and / or the trajectory stored in the trajectory management component 108 (e.g., the prediction server's results may be checked against the corresponding stored trajectory). The prediction server's results may be checked against the corresponding stored trajectory using a polygon check, including comparing the trajectory of the object (e.g., by propagating (or "perturbing") the object along the object's trajectory) with the trajectory of the vehicle (e.g., by propagating the vehicle along the vehicle's trajectory) to determine the corresponding time at which a collision may occur. In some examples, perturbing the object and / or vehicle may include perturbing the object and / or vehicle along any form of motion (e.g., predicted motion) associated with the object and / or vehicle (e.g., steering angle, velocity, acceleration, etc.). Intersection probability. In some examples, the results of disturbing objects and / or vehicles may be included in prediction server information that may be generated, transmitted to, and / or utilized by trajectory management component 108. Additional examples of collision prediction and avoidance may be found, for example, in U.S. Patent Application No. 16 / 884,975, filed on May 27, 2020, entitled “Vehicle CollisionAvoidance Based on Perturbed Object Trajectories,” the entire contents of which are incorporated herein by reference for all purposes.

[0064] Although the prediction server is discussed as being separate from other safety components, as described in the current disclosure, it is not limited thereto. In some examples, the prediction server can be integrated with and / or implemented to interoperate with one or more other components (e.g., trajectory management component 108, perception component 110, filter component 112, and / or collision detection component 114).

[0065] By analyzing the results determined by the perception component 110, the filter component 112, and / or the collision detection component 114 (e.g., information received via signals 116, 118, and 120) using the trajectory management component 108, the vehicle can be controlled more accurately and safely. For example, the trajectory management component 108 can use the physics-based collision state received from the perception component 110, the objects received based on the filtering performed by the filter component 112, and the machine learning (ML) collision state received from the collision detection component 114 to more accurately select the vehicle trajectory. Due to operational redundancy, the robustness of vehicle operation is also improved. For example, although the collision detection component 114 determines an ML-based collision state, the different corresponding types of analysis performed by the collision detection component 114 and the trajectory management component 108 can ensure that the collision probability determined by the trajectory management component 108 is correct. However, since the collision detection component 114 requires a longer operation time respectively, there may be a longer delay in the operation of the trajectory management component 108. In some cases, the trajectory management component 108 can perform a collision check operation before receiving the result from the collision detection component 114 to reduce the delay (for example, if the result of the collision detection component 114 related to the currently selected trajectory is delayed, the trajectory management component 108 can select a different, possibly less direct but safer alternative trajectory).

[0066] Although the trajectory management component 108, the perception component 110, the filter component 112, and the collision detection component 114 can be separate components, as described in the current disclosure, they are not limited thereto. In some examples, one or more of any components of the vehicle safety system 102 (e.g., the trajectory management component 108, the perception component 110, the filter component 112, and the collision detection component 114) can be integrated with one or more of the remaining components of the vehicle safety system 102 (e.g., the trajectory management component 108, the perception component 110, the filter component 112, and the collision detection component 114).

[0067] Although signals 116, 118, and 120 are received by trajectory management component 108, as described in the present disclosure, they are not limited thereto. In some examples, any component (e.g., a host system) can perform analysis and then generate and / or transmit signals 116, 118, and 120, respectively, to trajectory management component 108 in a manner similar to perception component 110, filter component 112, and collision detection component 114. In some examples, the first host system, the second host system, and the third host system, and / or any combination thereof, can perform analysis and then generate and / or transmit any one of signals 116, 118, and 120, respectively.

[0068] In some examples, the trajectory management component 108, the perception component 110, the filter component 112, and the collision detection component 114, whether implemented as one or more integrated components, one or more individual / separate components, or any combination thereof, can exchange any information between each other (e.g., any information determined by any of the trajectory management component 108, the perception component 110, the filter component 112, and the collision detection component 114 can be exchanged with any other of the trajectory management component 108, the perception component 110, the filter component 112, and the collision detection component 114). In these or other examples, any analysis and / or determination made by any of the trajectory management component 108, the perception component 110, the filter component 112, and the collision detection component 114 can be made through information received from any one or more of the trajectory management component 108, the perception component 110, the filter component 112, and the collision detection component 114. By operating any of the trajectory management component 108 , the perception component 110 , the filter component 112 , and the collision detection component 114 independently and / or redundantly, the operational robustness of any component and / or the entire vehicle system may be improved.

[0069] Although various components of the vehicle safety system 102 can utilize corridor subsets, as described in the current disclosure, they are not limited thereto. In some examples, any of the techniques discussed herein can be performed utilizing modified corridors based on combining corridor subsets together and determining them as modified corridors.

[0070] Although the trajectories (e.g., the first trajectory and the second trajectory) can be stored in queues of separate items in the sensing component 110, the filter component 112, and the collision detection component 114, as described in the present disclosure, it is not limited thereto. Any number of trajectories can be stored in queues of separate items in the sensing component 110, the filter component 112, and the collision detection component 114. Any trajectory stored in any queue of the sensing component 110, the filter component 112, and the collision detection component 114 can be the same as or different from a trajectory stored in any other queue.

[0071] Figure 2 is a flow chart of an example process of sensing component operation according to an example of the present disclosure. In some examples, the example process 200 may be performed by at least one sensing component (e.g., sensing component 110, as described above with reference to Figure 1 as described) is executed.

[0072] At operation 202, the example process 200 may include operating the perception component 110 to determine a mode (or "selected mode") based on a selection of one or more modes for operating the perception component 110. The modes may include a first mode for validating a trajectory, a second mode for validating more than one trajectory, and a third mode for utilizing kinematic data or one or more trajectories for trajectory validation.

[0073] At operation 204, the example process 200 may include determining that the mode (eg, the selected mode of the perception component 110) is the first mode. The perception component 110 operating in the first mode may receive a planned trajectory generated by the host system.

[0074] At operation 206, the example process 200 may include storing the received trajectory (e.g., the planned trajectory). The perception component 110 may analyze the planned trajectory. The perception component 110 may analyze the objects associated with the planned trajectory. The perception component 110 may determine the intersection probability associated with the object.

[0075] At operation 208, the example process 200 may include determining that the mode (e.g., the selected mode of the perception component 110) is the second mode. The perception component 110 operating in the second mode may receive more than one trajectory (e.g., more than one planned trajectory) generated by the host system. The more than one planned trajectory may include a previously planned trajectory (e.g., a first trajectory (or "first planned trajectory")) and a current planned trajectory (e.g., a second trajectory (or "second planned trajectory")). The more than one trajectory may be managed and stored using a queue in the perception component 110.

[0076] At operation 210, the example process 200 may include storing the received trajectories (e.g., the first and second planned trajectories). The perception component 110 may analyze the planned trajectories. For example, the perception component 110 may analyze an object associated with the first planned trajectory (e.g., the first object) and an object associated with the second planned trajectory (e.g., the second object). In such an example, the perception component 110 may determine an intersection probability associated with the first object (e.g., the first intersection probability) and an intersection probability associated with the second object (e.g., the second intersection probability).

[0077] At operation 212, the example process 200 may include determining that the mode (eg, the selected mode of the perception component 110) is the third mode. The perception component 110 operating in the third mode may determine to verify the trajectory using the received kinematics and trajectory.

[0078] At operation 214, the example process 200 may include using kinematics or trajectory for validation. Kinematics associated with the vehicle may be determined based on the received sensor data. The kinematics may be used with the received trajectory for trajectory validation (validation of the trajectory).

[0079] At operation 216, the example process 200 may include validating one or more trajectories. The validated trajectories may include trajectories generated and transmitted by the host system. In some examples, the validated trajectories may include trajectories stored in the perception component 110 based on the perception component 110 operating in the first mode. In other examples, the validated trajectories may include trajectories in a queue stored in the perception component 110 based on the perception component 110 operating in the second mode. In other examples, the trajectories may be validated using the kinematics of the vehicle. Validation based on trajectories in any of the first to third modes may be performed based on corridors or corridor subsets associated with one or more trajectories.

[0080] At operation 218, the example process 200 may include transmitting a signal based on the validated trajectory. The perception component 110 may transmit the validation result of the trajectory in the signal. The signal may include data indicating individual trajectories evaluated relative to the corridor or corridor subset, and / or data including respective probabilities of intersection probabilities determined by the perception component 110.

[0081] Although the sensing component 110 can operate in any one of the first to third modes, as described above in the present disclosure, it is not limited thereto. In some examples, the sensing component 110 can operate in a mixed mode, which includes a combination of the first mode and the third mode, or a combination of the second mode and the third mode.

[0082] Figure 3 300 is a flow chart of an example process 300 for operation of a filter component according to an example of the present disclosure. In some examples, the example process 300 may be performed by at least one filter component (e.g., filter component 112, as described above with reference to FIG. 300 ). Figure 1 ) and a track management component (e.g., track management component 108, as described above with reference to Figure 1 as described) is executed.

[0083] At operation 302, the example process 300 may include operating the filter component 112 to identify a received planning trajectory (e.g., a planning trajectory generated by a host system). The planning trajectory may be stored in the filter component 112. In some examples, the planning trajectory that may be received as a current trajectory (e.g., a trajectory associated with a current frame of the host system and / or trajectory management component 108) may be stored in the filter component 112 along with one or more other trajectories.

[0084] At operation 304, the example process 300 may include adding one or more trajectories to a queue in the filter component 112. In some examples, the planned trajectory (e.g., the current trajectory) may be added to a queue that stores one or more other trajectories (e.g., previously planned trajectories) (e.g., trajectories associated with a previous frame of the host system and / or the trajectory management component 108).

[0085] At operation 306, the example process 300 may include determining one or more objects closest to each trajectory. Individual trajectories in the trajectory may be analyzed (e.g., verified) by the filter component 112 to filter out one or more objects. The objects closest to each trajectory may be determined and used to filter out objects that are farther from the trajectory than the nearest object. For example, the objects closest to the current planned trajectory may be determined and used to filter out objects that are farther from the current planned trajectory than the nearest object. Similarly, other nearest objects from the previous planned trajectory may be determined and used to filter out objects that are farther from the previous planned trajectory than the other nearest objects.

[0086] By validating the trajectory, the filter component 112 can be used to validate each trajectory in the queue (e.g., a two-deep queue) (e.g., compare data associated with the object closest to the vehicle) (e.g., perform a simple physics-based collision check on the object closest to the vehicle) (e.g., perform a heading collision check (e.g., a three-point heading collision check)). The filter component 112 can operate (e.g., validate the trajectory) based on a speed assumption (e.g., a physics-based speed determination, the speed remains constant or within certain limits or thresholds, and filters out trajectories above or below a speed threshold). In some examples, the validation of the trajectory by the filter component 112 can include filtering out data associated with the corresponding trajectory based on the filter component 112 not detecting a potential hazard associated with the data (e.g., an object identified by the data). By filtering out any data (e.g., data that would otherwise be utilized by the trajectory management component 108), the filter component 112 can reduce the computational requirements of the trajectory management component 108. In this way, the trajectory management component 108 can more effectively focus on higher-level safety issue data than the filtered data.

[0087] At operation 308, the example process 300 may include transmitting a signal based on the determined objects (e.g., the nearest objects and / or the filtered objects). For individual tracks in the tracks stored in the queue, the signal may include the results of the nearest objects and / or the filtered objects. The signal may identify the nearest objects and / or the filtered objects.

[0088] At operation 310, the example process 300 may include determining, by the trajectory management component 108, whether a signal is received from the filter component 112. In some examples, the trajectory management component 108 may determine whether a signal generated by the filter component 112 is received. The trajectory management component 108 may proceed to operation 312 based on determining that a signal from the filter component 112 is received. The trajectory management component 108 may refrain from proceeding to operation 312 based on determining that a signal from the filter component 112 has not been received (e.g., the trajectory management component 108 may wait for a result from the filter component 112 before moving on to the next frame).

[0089] At operation 312, the example process 300 may include verifying one or more tracks. The track management component 108 may determine to verify the track based on determining that a signal from the filter component 112 is received. In some examples, for a separate track in a track stored in a queue, the track management component 108 may verify the track (e.g., the current planning track) and determine that the track is an optional track based on the nearest object and / or the object filtered out. In these examples, the track may be determined to be an optional track (e.g., a selectable track) based on the individual distance between the corresponding one or more tracks of the nearest object and the vehicle track reaching or exceeding a threshold distance. In other examples, the track management component 108 may verify the track (e.g., the current planning track) and determine that the track is an unselectable track (e.g., an unselectable track) based on at least one of the individual distances between the one or more corresponding tracks of the nearest object and the vehicle track being less than a threshold distance. Any track (e.g., a previously planned track) may be subjected to a similar verification as the current planning track.

[0090] Figure 4 4 is a flow chart of an example process 400 of the operation of a collision detection component according to an example of the present disclosure. In some examples, the example process 400 can be performed by at least one collision detection component (e.g., collision detection component 114, as described above with reference to Figure 1 ) and a track management component (e.g., track management component 108, as described above with reference to Figure 1 as described) is executed.

[0091] At operation 402, the example process 400 may include operating the collision detection component 114 to determine that one or more trajectories are received, which can be used to determine one or more collision states. In some examples, the collision detection component 114 can perform a collision check (e.g., a machine learning (ML) collision check) to verify the trajectory. The received and / or verified trajectory can be a trajectory generated by a host system. In some examples where the collision detection component 114 receives more than one trajectory, the trajectory (e.g., the current planned trajectory and the previously planned trajectory) can be stored in a queue of the collision detection component 114.

[0092] At operation 404, the example process 400 may include determining, by the collision detection component 114, one or more collision states based on the trajectory. The collision state may indicate a likelihood of collision based on the received trajectory. For example, a collision state associated with the current planned trajectory may be determined based on a separate distance between one or more object trajectories and the current planned trajectory. For example, in contrast to i) the perception component 110 being used to identify intersection probabilities, but in some cases may not be used to determine a collision state, the collision detection component 114 may determine a collision state (e.g., an ML collision state) based on the trajectory and / or the object.

[0093] At operation 406, the example process 400 may include transmitting, by the collision detection component 114, a signal based on the collision state. The signal may include a result of determining the collision state. In some examples, for individual tracks in the track, the signal may indicate whether the corresponding collision state reaches or exceeds a threshold collision state.

[0094] At operation 408, the example process 400 may include determining, by the trajectory management component 108, whether a signal is received from the collision detection component 114. In some examples, the trajectory management component 108 may determine whether a signal generated by the collision detection component 114 is received. The trajectory management component 108 may determine whether the collision state reaches or exceeds a threshold collision state. The trajectory management component 108 may proceed to operation 410 based on determining that a signal from the collision detection component 114 is received. The trajectory management component 108 may avoid proceeding to operation 410 based on determining that a signal from the collision detection component 114 has not been received.

[0095] At operation 410, the example process 400 may include validating one or more tracks. The track management component 108 may determine to validate the track based on determining that a signal from the collision detection component 114 is received. In some examples, the track management component 108 may validate the track (e.g., the current planned track) and determine that the track is an optional track based on determining that the collision state associated with the track is less than a threshold collision state. In other examples, the track management component 108 may validate the track (e.g., the current planned track) and determine that the track is a non-optional track based on determining that the collision state associated with the track reaches or exceeds a threshold collision state. A similar validation as the current planned track can be performed on any track (e.g., a previously planned track).

[0096] Figure 5 An example driving environment 500 is shown, which shows a vehicle traversing the environment and a trajectory determined for the vehicle. In some examples, a vehicle (e.g., a vehicle including a vehicle safety system 102, as described above with reference to Figure 1 The ) can utilize the trajectory management component 108 to receive and / or determine one or more trajectories. The trajectory can include a planned trajectory 504, which can be generated by a host system of the vehicle 502. In some examples, the planned trajectory (e.g., the current planned trajectory) 504 can be associated with the current frame.

[0097] The trajectory may include a planned trajectory 506, which may be generated by a host system of the vehicle 502. In some examples, the planned trajectory (eg, a previous planned trajectory) 506 may be associated with a previous frame.

[0098] The trajectory may be generated by one or more components (e.g., a sensing component 110, a filter component 112, and / or a collision detection component 114, as described above with reference to Figure 1 The trajectories may be received from a host system (e.g., a host system of the vehicle 502) and / or the trajectory management component 108. The trajectories may be stored in the perception component 110, the filter component 112, and / or the collision detection component 114. Individual components in the perception component 110, the filter component 112, and the collision detection component 114 may include queues to store trajectories (e.g., the planned trajectory 504 and the planned trajectory 506), which may be received from a host system (e.g., a host system of the vehicle 502) and / or the trajectory management component 108.

[0099] The trajectory may include one or more safety stop trajectories. The safety stop trajectory may include a safety stop trajectory 508 (e.g., a first safety stop trajectory) associated with a stop position 510 (and / or associated with a first deceleration value), and a safety stop trajectory 512 (e.g., a second safety stop trajectory) associated with a stop position 514 (and / or associated with a second deceleration value different from the first deceleration value). The trajectory management component 108 may determine separate trajectories in the safety stop trajectory (e.g., safety stop trajectory 508 and safety stop trajectory 512) and / or separate stop positions in the stop positions (e.g., stop position 510 and stop position 514).

[0100] Planned trajectories 504 and 506 may be determined based on various features. In some examples, one or more features used to determine planned trajectory 504 (e.g., direction (e.g., first direction), turning capability in the first direction (e.g., first turning capability), speed, lateral acceleration limit, etc.) may be the same as or different from one or more features used to determine planned trajectory 506 (e.g., direction (e.g., second direction), turning capability in the second direction (e.g., second turning capability), speed, lateral acceleration limit, etc.).

[0101] In some examples, the trajectory management component 108 may determine whether to select the planned trajectory 504, the safe stop trajectory 508, or the safe stop trajectory 512 before or after receiving the evaluation results of the planned trajectory 504 from the perception component 110, the filter component 112, and / or the collision detection component 114. In other examples, the trajectory management component 108 may wait to receive the evaluation results of the planned trajectory 504 by the perception component 110, the filter component 112, and / or the collision detection component 114. In these examples, the trajectory management component 108 may determine whether to select the planned trajectory 504, the safe stop trajectory 508, or the safe stop trajectory 512 based on one or more of the received results. In some examples, in other words, the trajectory management component 108 may continue its operation (e.g., determine whether to select the planned trajectory 504) without waiting to receive the results from one or more of the perception component 110, the filter component 112, and / or the collision detection component 114. By continuing its operations, trajectory management component 108 can avoid any delays caused by waiting for results from one or more of perception component 110 , filter component 112 , and collision detection component 114 .

[0102] In some examples, for example, when trajectory management component 108 receives results from one or more of perception component 110, filter component 112, and collision detection component 114, trajectory management component 108 can operate based on determining that one or more collision probabilities associated with the corresponding results are higher than other probabilities associated with other results. For example, trajectory management component 108 can determine to change the trajectory (e.g., from the planned trajectory to the safe stop trajectory) based on the results from one or more of perception component 110, filter component 112, and collision detection component 114 being associated with a higher collision probability than other results from other components.

[0103] In some examples, the trajectory management component 108 can determine whether to select one of the current trajectories (e.g., the planned trajectory 504, the safe stop trajectory 508, or the safe stop trajectory 512) or whether to select a previous trajectory (e.g., the planned trajectory 506 or one or more safe stop trajectories determined in a previous frame) based on the evaluation results of the perception component 110, the filter component 112, and / or the collision detection component 114 on one or more of the planned trajectory 504, the planned trajectory 506, and / or any safe stop trajectories determined therefor. In some examples, for example, when one or more of the perception component 110, the filter component 112, and / or the collision detection component 114 store corresponding queues, the trajectory management component 108 can quickly perform trajectory selection because information can be received via the signals 116, 118, and 120 without delay. For example, if it is necessary to switch back to a previous trajectory, the previous trajectory can be stored in the corresponding queues of the perception component 110, the filter component 112, and / or the collision detection component 114 so that any information related to the previous trajectory can be accessed.

[0104] Although the safety stop trajectory 508 and the safety stop trajectory 512 and the stop position 510 and the stop position 514 can be determined by the vehicle 502, as discussed above in the current disclosure, they are not limited to this. In some examples, one or more other safety stop trajectories and corresponding other stop positions associated with the current planned trajectory 504 can be determined and stored in the trajectory management component 108. In these or other examples, one or more other safety stop trajectories and corresponding other stop positions associated with the previous planned trajectory 508 can be determined and stored in the trajectory management component 108 (e.g., determined and / or stored in the previous frame). The trajectory management component 108 can operate with other safety stop trajectories or other stop positions in a manner similar to the safety stop trajectory 508 and the safety stop trajectory 512 and the stop position 510 and the stop position 514.

[0105] Although planned trajectory 504 and planned trajectory 506 can be determined by vehicle 502, as described above in the current disclosure, they are not limited thereto. In some examples, one or more other planned trajectories associated with any number of previous frames can be determined, and trajectory management component 108, perception component 110, filter component 112, and / or collision detection component 114 can utilize the one or more other planned trajectories in a manner similar to planned trajectory 504 and planned trajectory 506.

[0106] Figure 6 600 for implementing the techniques described herein. In at least one example, the system 600 can include a vehicle 602. In the illustrated example system 600, the vehicle 602 can be an autonomous vehicle; however, the vehicle 602 can be any other type of vehicle.

[0107] Vehicle 602 may be implemented as a driverless vehicle, such as an autonomous vehicle configured to operate in accordance with a Level 5 classification issued by the National Highway Traffic Safety Administration, which describes a vehicle capable of performing all safety-critical functions throughout a journey in which a driver (or occupant) is not expected to be in control of the vehicle at any time. In such an example, because vehicle 602 may be configured to control all functions from the start of a journey to completion, including all parking functions, it may not include a driver and / or controls for driving vehicle 602, such as a steering wheel, accelerator pedal, and / or brake pedal. This is merely an example, and the systems and methods described herein may be incorporated into any ground, air, or water vehicle, including those ranging from vehicles that require manual control by a driver at all times to those that are partially or fully autonomously controlled.

[0108] The vehicle 602 may include one or more computing devices 604, one or more sensor systems 606, one or more transmitters 608, one or more communication connections 610 (also referred to as communication devices and / or modems), at least one direct connection 612 (e.g., for physically coupling with the vehicle 602 to exchange data and / or provide power), and one or more drive systems 614. By way of example, the first computing device 604 may be considered a primary system. In some examples, the first computing device 604 may be used to implement the primary system, as described above with reference to Figure 1 The one or more sensor systems 606 can be configured to capture sensor data associated with the environment.

[0109] The sensor system 606 may include a time-of-flight sensor, a position sensor (e.g., GPS, compass, etc.), an inertial sensor (e.g., an inertial measurement unit (IMU), an accelerometer, a magnetometer, a gyroscope, etc.), a lidar sensor, a radar sensor, a sonar sensor, an infrared sensor, a camera (e.g., RGB, IR, intensity, depth, etc.), a microphone sensor, an environmental sensor (e.g., a temperature sensor, a humidity sensor, a light sensor, a pressure sensor, etc.), an ultrasonic transducer, a wheel encoder, etc. The sensor system 606 may include multiple instances of each of these or other types of sensors. For example, the time-of-flight sensor may include separate time-of-flight sensors located at the corners, front, rear, sides, and / or top of the vehicle 602. As another example, the camera sensor may include multiple cameras disposed at various locations around the exterior and / or interior of the vehicle 602. The sensor system 606 may provide input to the first computing device 604.

[0110] The vehicle 602 may also include a transmitter 608 for emitting light and / or sound. The transmitter 608 in this example includes internal audio and visual transmitters for communicating with passengers of the vehicle 602. By way of example and not limitation, the internal transmitters may include speakers, lights, signs, display screens, touch screens, tactile transmitters (e.g., vibration and / or force feedback), mechanical actuators (e.g., seat belt tensioners, seat positioners, headrest positioners, etc.), and the like. The transmitter 608 in this example may also include external transmitters. By way of example and not limitation, the external transmitters in this example include lights indicating the direction of travel or other indicators of vehicle motion (e.g., indicator lights, signs, light arrays, etc.), and one or more audio transmitters (e.g., speakers, speaker arrays, horns, etc.) to communicate with pedestrians or other nearby vehicles using sound, one or more of which may include acoustic beam steering technology.

[0111] The vehicle 602 may also include a communication connection 610 that enables communication between the vehicle 602 and one or more other local or remote computing devices (e.g., a teleoperation computing device) or remote services. For example, the communication connection 610 may facilitate communication with other local computing devices and / or drive system 614 on the vehicle 602. Moreover, the communication connection 610 may allow the vehicle 602 to communicate with other nearby computing devices (e.g., other nearby vehicles, traffic lights, etc.).

[0112] The communication connection 610 may include a physical interface and / or a logical interface for connecting the computing device 604 to another computing device or one or more external networks 616 (e.g., the Internet). For example, the communication connection 610 may enable Wi-Fi-based communications, such as frequencies defined by the IEEE 802.11 standard, short-range wireless frequencies such as Bluetooth, cellular communications (e.g., 2G, 3G, 4G, 4G LTE, 5G, etc.), satellite communications, dedicated short-range communications (DSRC), or any suitable wired or wireless communication protocol that enables respective computing devices to interact with other computing devices.

[0113] In at least one example, the vehicle 602 may include a drive system 614. In some examples, the vehicle 602 may have a single drive system 614. In at least one example, if the vehicle 602 has multiple drive systems 614, the individual drive systems 614 may be positioned on opposite ends (e.g., front and rear, etc.) of the vehicle 602. In at least one example, the drive system 614 may include a sensor system 606 to detect conditions of the drive system 614 and / or the surrounding environment of the vehicle 602. By way of example and not limitation, the sensor system 606 may include: one or more wheel encoders (e.g., rotary encoders) to sense the rotation of the wheels of the drive system, inertial sensors (e.g., inertial measurement units, accelerometers, gyroscopes, magnetometers, etc.) to measure the orientation and acceleration of the drive system, cameras or other image sensors, ultrasonic sensors to acoustically detect objects in the surrounding environment of the drive system, lidar sensors, radar sensors, etc. Some sensors, such as wheel encoders, may be unique to the drive system 614. In some cases, sensor system 606 on drive system 614 may overlap or supplement a corresponding system of vehicle 602 (eg, sensor system 606 ).

[0114] The drive system 614 may include many of the systems in the vehicle system, including: a high-voltage battery; a motor for propelling the vehicle; an inverter that converts direct current from the battery into alternating current for use by other vehicle systems; a steering system, including a steering motor and a steering rack (which may be electric); a braking system, including a hydraulic actuator or an electric actuator; a suspension system, including hydraulic components and / or pneumatic components; a stability control system for distributing braking force to mitigate loss of traction and maintain control; an HVAC system; lighting (e.g., lighting such as head / tail lights for illuminating the external surroundings of the vehicle); and one or more other systems (e.g., cooling systems, safety systems, on-board charging systems, other electrical components, such as DC / DC converters, high-voltage connectors, high-voltage cables, charging systems, charging ports, etc.). In addition, the drive system 614 may include a drive system controller that may receive and pre-process data from the sensor system 606 and control the operation of various vehicle systems. In some examples, the drive system controller may include one or more processors and a memory coupled to the one or more processors in communication. The memory may store one or more components to perform various functions of the drive system 614. Additionally, the drive systems 614 may also include one or more communication connections that enable the respective drive systems to communicate with one or more other local or remote computing devices.

[0115] The vehicle 602 may include one or more second computing devices 618 to provide redundancy, error checking, and / or verification of decisions and / or commands determined by the first computing device 604. For example, while the first computing device 604 may be considered a primary system, the second computing device 618 may be considered a secondary system. In some examples, the second computing device 618 may be used to implement the vehicle safety system 102, as described above with reference to Figure 1 described.

[0116] The host system may generally perform processing to control how the vehicle maneuvers within the environment. The host system may implement various artificial intelligence (AI) techniques, such as machine learning, to understand the environment around the vehicle and / or direct the vehicle to move within the environment. For example, the host system may implement AI techniques to locate the vehicle, detect objects around the vehicle, segment sensor data, determine the classification of objects, predict object trajectories, generate vehicle trajectories, etc. In an example, the host system processes data from multiple types of sensors on the vehicle, such as light detection and ranging (lidar) sensors, radar sensors, image sensors, depth sensors (time of flight, structured light, etc.), etc.

[0117] The auxiliary system can verify the operation of the primary system and can take over control of the vehicle from the primary system when there is a problem with the primary system. In an example, the auxiliary system processes data from several sensors, such as a subset of the sensor data processed by the primary system. For illustration, the primary system may process lidar data, radar data, image data, depth data, etc., while the auxiliary system may only process lidar data and / or radar data (and / or time of flight data). However, in other examples, the auxiliary system may process sensor data from any number of sensors, such as data from each sensor, data from the same number of sensors as the primary system, etc.

[0118] Other examples of vehicle architectures including primary and auxiliary computing systems can be found, for example, in U.S. patent application Ser. No. 16 / 189,726, filed on Nov. 13, 2018, and entitled “Perception Collision Avoidance,” which is incorporated herein by reference in its entirety for all purposes.

[0119] The first computing device 604 may include one or more processors 620 and a memory 622 communicatively coupled to the one or more processors 620. In the example shown, the memory 622 of the first computing device 604 stores a positioning component 624, a perception component 626, a prediction component 628, a planning component 630, a map component 632, and one or more system controllers 634. Although described as being present in the memory 622 for purposes of illustration, it is contemplated that the positioning component 624, the perception component 626, the prediction component 628, the planning component 630, the map component 632, and the one or more system controllers 634 may additionally or alternatively be accessible to the computing device 604 (e.g., stored in a different component of the vehicle 602 and / or accessible to the vehicle 602 (e.g., stored remotely)).

[0120] In the memory 622 of the first computing device 604, the positioning component 624 may include functionality to receive data from the sensor system 606 to determine the location of the vehicle 602. For example, the positioning component 624 may include and / or request / receive a three-dimensional map of the environment, and may continuously determine the location of the autonomous vehicle within the map. In some cases, the positioning component 624 may use SLAM (simultaneous localization and mapping) or CLAMS (simultaneous calibration, localization and mapping) to receive time-of-flight data, image data, lidar data, radar data, sonar data, IMU data, GPS data, wheel encoder data, or any combination thereof, etc., to accurately determine the location of the autonomous vehicle. In some cases, the positioning component 624 may provide data to various components of the vehicle 602 to determine the initial location of the autonomous vehicle for generating a trajectory, as discussed herein.

[0121] The perception component 626 may include functionality for performing object detection, segmentation, and / or classification. In some examples, the perception component 626 may provide processed sensor data representing the presence of an entity near the vehicle 602 and / or an entity classified as an entity type (e.g., a car, a pedestrian, a cyclist, a building, a tree, a pavement, a curb, a sidewalk, an unknown, etc.). In additional and / or optional examples, the perception component 626 may provide processed sensor data representing one or more characteristics associated with the detected entity and / or the environment in which the entity is located. In some examples, the characteristics associated with the entity may include, but are not limited to, x-position (global position), y-position (global position), z-position (global position), orientation, entity type (e.g., classification), speed of the entity, range (size) of the entity, etc. The characteristics associated with the environment may include, but are not limited to, the presence of another entity in the environment, the state of another entity in the environment, the time of day, the day of the week, the season, the weather condition, the representation of darkness / light, etc.

[0122] As described above, perception component 626 can use perception algorithms to determine a perception-based bounding box associated with an object in the environment based on sensor data. For example, perception component 626 can receive image data and classify the image data to determine that an object is represented in the image data. Then, using the detection algorithm, perception component 626 can generate a two-dimensional bounding box associated with the object and / or a perception-based three-dimensional bounding box. Perception component 626 can also generate a three-dimensional bounding box associated with the object. As described above, the three-dimensional bounding box can provide additional information associated with the object, such as position, orientation, posture, and / or size (e.g., length, width, height, etc.).

[0123] Perception component 626 may include functionality to store perception data generated by perception component 626. In some cases, perception component 626 may determine a track corresponding to an object that has been classified as an object type. For purposes of illustration only, perception component 626 may capture images of one or more environments using sensor system 606. Sensor system 606 may capture images of an environment including an object such as a pedestrian. A pedestrian may be at a first position at time T, and at a second position at time T+t (e.g., movement during a time span t after time T). In other words, a pedestrian may move from a first position to a second position during this time span. Such movement may, for example, be recorded as stored perception data associated with an object.

[0124] In some examples, the stored perception data may include fused perception data captured by the vehicle 602. The fused perception data may include a fusion or other combination of sensor data from the sensor system 606, such as an image sensor, a lidar sensor, a radar sensor, a time-of-flight sensor, a sonar sensor, a global positioning system sensor, an internal sensor, and / or any combination of these sensors. The stored perception data may additionally or alternatively include classification data, which includes semantic classifications of objects (e.g., pedestrians, vehicles, buildings, road surfaces, etc.) represented in the sensor data. The stored perception data may additionally or alternatively include trace data (position, orientation, sensor features, etc.) corresponding to the movement of objects classified as dynamic objects in the environment. The trace data may include multiple traces of multiple different objects over time. The trajectory data may be mined to identify images of certain types of objects (e.g., pedestrians, animals, etc.) when the objects are stationary (e.g., standing still) or moving (e.g., walking, running, etc.). In this example, the computing device determines a trace corresponding to a pedestrian.

[0125] Prediction component 628 can generate one or more probability maps that represent the predicted probabilities of the possible locations of one or more objects in the environment. For example, prediction component 628 can generate one or more probability maps for vehicles, pedestrians, animals, etc. within a threshold distance from vehicle 602. In some cases, prediction component 628 can measure the traces of objects based on observed behavior and predicted behavior and generate discrete predicted probability maps, heat maps, probability distributions, discrete probability distributions, and / or trajectories of objects. In some cases, one or more probability maps can represent the intentions of one or more objects in the environment.

[0126] Planning component 630 can determine the path that vehicle 602 will follow to cross the environment. For example, planning component 630 can determine various routes and paths and various levels of details. In some cases, planning component 630 can determine the route from a first position (e.g., current position) to a second position (e.g., target position). For the purpose of this discussion, a route can be a sequence of waypoints for traveling between two positions. As a non-limiting example, waypoints include streets, intersections, global positioning system (GPS) coordinates, etc. In addition, planning component 630 can generate instructions for guiding the autonomous vehicle along at least one portion of the route from the first position to the second position. In at least one example, planning component 630 can determine how to guide the autonomous vehicle from the first waypoint in the waypoint sequence to the second waypoint in the waypoint sequence. In some examples, the instruction can be a path or a part of a path. In some examples, multiple paths can be generated substantially simultaneously (i.e., within the technical tolerance) according to the back-up level technology. A path with the highest confidence level in multiple paths in the back-up data level can be selected to operate the vehicle.

[0127] In other examples, planning component 630 can alternatively or additionally use data from perception component 626 and / or prediction component 628 to determine a path for vehicle 602 to follow through the environment. For example, planning component 630 can receive data about objects associated with the environment from perception component 626 and / or prediction component 628. Using this data, planning component 630 can determine a route to travel from a first location (e.g., current location) to a second location (e.g., target location) to avoid objects in the environment. In at least some examples, such planning component 630 can determine that there is no such conflict-free path and, in turn, provide a path to bring vehicle 602 to a safe stop, avoiding all collisions and / or otherwise mitigating damage.

[0128] The memory 622 may further include one or more maps 632, which can be used by the vehicle 602 to navigate within the environment. For the purposes of this discussion, a map can be any number of data structures modeled in two, three, or N dimensions that can provide information about the environment, such as, but not limited to, topology (e.g., intersections), streets, mountains, roads, terrain, and the overall environment. In some cases, a map may include, but is not limited to: texture information (e.g., color information (e.g., RGB color information, Lab color information, HSV / HSL color information), etc.), intensity information (e.g., lidar information, radar information, etc.); spatial information (e.g., image data projected onto a grid, individual "surfaces" (e.g., polygons associated with individual colors and / or intensities), reflectivity information (e.g., specular reflection information, retroreflection information, BRDF information, BSSRDF information, etc.). In one example, a map may include a three-dimensional grid of the environment. In some instances, a map may be stored in a tiled format so that each individual portion of the map is a tiled grid. Tiles represent discrete portions of an environment and can be loaded into working memory as needed, as discussed herein. In at least one example, one or more maps 632 can include at least one map (e.g., an image and / or a grid). In some examples, vehicle 602 can be controlled based at least in part on map 632. That is, map 632 can be used in conjunction with positioning component 624, perception component 626, prediction component 628, and / or planning component 630 to determine the location of vehicle 602, identify objects in the environment, generate predicted probabilities associated with objects and / or vehicle 602, and / or generate routes and / or trajectories to navigate within the environment.

[0129] In some examples, one or more maps 632 can be stored on a remote computing device (e.g., computing device 648) that can be accessed via network 616. In some examples, multiple maps 632 can be stored based on, for example, characteristics (e.g., entity type, time of day, day of week, season of year, etc.). Storing multiple maps 632 can have similar memory requirements, but can increase the speed at which data in the maps can be accessed.

[0130] In at least one example, the first computing device 604 can include one or more system controllers 634, which can be configured to control steering, propulsion, braking, safety, transmitter, communication, and other systems of the vehicle 602. These system controllers 634 can communicate with and / or control corresponding systems of the drive system 614 and / or other components of the vehicle 602, which can be configured to operate in accordance with the path provided from the planning component 630.

[0131] The second computing device 618 can include one or more processors 636 and memory 638, which include components to verify and / or control aspects of the vehicle 602, as discussed herein. In at least one example, the one or more processors 636 can be similar to the processor 620 and the memory 638 can be similar to the memory 622. However, in some examples, the processor 636 and the memory 638 can include different hardware than the processor 620 and the memory 622 for additional redundancy.

[0132] In some examples, memory 638 may include a positioning component 640, a sensing / prediction component 642, a planning component 644, and one or more system controllers 646. In some examples, sensing / prediction component 642 may be used to implement any of the features of sensing component 110, as described above with reference to Figure 1 described.

[0133] In some examples, the positioning component 640 may receive sensor data from the sensor 606 to determine one or more positions and / or orientations (collectively referred to as attitudes) of the autonomous vehicle 602. Here, the position and / or orientation may be relative to a point and / or object in the environment in which the autonomous vehicle 602 is located. In an example, the orientation may include a representation of the yaw, roll, and / or pitch of the autonomous vehicle 602 relative to a reference plane and / or relative to a point and / or object. In an example, the positioning component 640 may perform less processing (e.g., a higher level of positioning) than the positioning component 624 of the first computing device 604. For example, the positioning component 640 may not determine the attitude of the autonomous vehicle 602 relative to the map, but only determine the autonomous vehicle 602 relative to the detected objects and / or surfaces around the autonomous vehicle 602 (e.g., local position rather than global position). Such a position and / or orientation may be determined, for example, using some or all of the sensor data using probabilistic filtering techniques, such as Bayesian filters (Kalman filters, extended Kalman filters, unscented Kalman filters, etc.).

[0134] In some examples, perception / prediction component 642 can include functionality to detect, identify, classify, and / or track objects represented in sensor data. For example, perception / prediction component 642 can perform clustering operations and operations to estimate or determine a height associated with an object, as discussed herein.

[0135] In some examples, the perception / prediction component 642 may include an M-estimator, but may lack an object classifier, such as a neural network, a decision tree, etc., for classifying objects. In additional or alternative examples, the perception / prediction component 642 may include any type of ML model that is configured to disambiguate object classification. In contrast, the perception component 626 may include a pipeline of hardware and / or software components that may include one or more machine learning models, Bayesian filters (e.g., Kalman filters), graphics processing units (GPUs), and / or the like. In some examples, the perception data determined by the perception / prediction component 642 (and / or 626) may include object detection (e.g., identification of sensor data associated with objects in the environment surrounding the autonomous vehicle), object classification (e.g., identification of an object type associated with a detected object), object trajectory (e.g., historical, current, and / or predicted object position, velocity, acceleration, and / or heading), and / or the like.

[0136] The perception / prediction component 642 may also process the input data to determine one or more predicted trajectories of the object. For example, based on the current position of the object and the speed of the object over a period of several seconds, the perception / prediction component 642 may predict the path that the object will move over the next few seconds. In some examples, such predicted paths may include using linear assumptions of motion given a given position, orientation, speed, and / or orientation. In other examples, such predicted paths may include more complex analysis.

[0137] In some examples, planning component 644 may include functionality to receive a trajectory from planning component 630 to verify that the trajectory is collision-free and / or within a safety margin. In some examples, planning component 644 may generate a safe stop trajectory (e.g., a trajectory that stops vehicle 602 at a "comfortable" deceleration (e.g., less than a maximum deceleration)) and in some examples, planning component 644 may generate an emergency stop trajectory (e.g., a maximum deceleration with or without steering input).

[0138] In some examples, planning component 644 can include trajectory management component 108, perception component 110, filter component 112, and / or collision detection component 114. In some examples, any corresponding queue of perception component 110, filter component 112, and / or collision detection component 114 discussed in the present disclosure can be the same as or different from any other queue. In these examples, perception component 110, filter component 112, and / or collision detection component 114 can share information with each other and / or with trajectory management component 108.

[0139] In some examples, the perception component 110, the filtering component 112, and / or the collision detection component 114 can be stored and / or managed by the planning component 644, as described above with reference to Figure 1-Figure 5 For example, the perception component 110 may generate a signal indicating an object in proximity to the autonomous vehicle; the filtering component 112 may filter out the objects and generate a second signal indicating a physics-based collision state associated with any object that is not filtered out; the collision detection component 114 may generate a third signal indicating a machine learning-based collision state and determine the likelihood of a collision based at least in part on the machine learning (ML)-based collision state; and the trajectory management component 108 may determine to follow a candidate trajectory or a safe stop trajectory based on at least one of the first signal, the second signal, or the third signal.

[0140] In some examples, system controller 646 may include functionality to control safety-critical components of the vehicle (e.g., steering, brakes, motors, etc.) In this manner, second computing device 618 may provide redundancy and / or additional hardware and software layers for vehicle safety.

[0141] The vehicle 602 can be connected to a computing device 648 via a network 616 and can include one or more processors 650 and a memory 652 communicatively coupled to the one or more processors 650. In at least one example, the one or more processors 650 can be similar to the processor 620 and the memory 652 can be similar to the memory 622. In the example shown, the memory 652 of the computing device 648 stores components 654, which can correspond to any of the components discussed herein.

[0142] Processors 620, 636, and / or 650 may be any suitable processor capable of executing instructions to process data and perform operations as described herein. By way of example and not limitation, processors 620, 636, and / or 650 may include one or more central processing units (CPUs), graphics processing units (GPUs), or any other device or portion of a device that processes electronic data to convert the electronic data into other electronic data that may be stored in registers and / or memory. In some examples, integrated circuits (e.g., ASICs, etc.), gate arrays (e.g., FPGAs, etc.), and other hardware devices may also be considered processors configured to implement encoded instructions.

[0143] Memory 622, 638, and / or 652 is an example of a non-temporary computer-readable medium. Memory 622, 638, and / or 652 can store an operating system and one or more software applications, instructions, programs, and / or data to implement the methods described herein and the functions attributed to various systems. In various embodiments, memory 622, 638, and / or 652 can be implemented using any suitable memory technology, such as static random access memory (SRAM), synchronous dynamic RAM (SDRAM), non-volatile / flash memory, or any other type of memory capable of storing information. The architecture, system, and separate elements described herein may include many other logical components, programmed components, and physical components, of which those components shown in the drawings are merely examples relevant to the discussion herein.

[0144] In some cases, aspects of some or all of the components discussed herein may include any model, algorithm, and / or machine learning algorithm. For example, in some cases, the components in memory 622, 638, and / or 652 may be implemented as a neural network. In some examples, the components in memory 622, 638, and / or 652 may not include a machine learning algorithm to reduce complexity and perform verification and / or authentication from a security perspective.

[0145] As described herein, an exemplary neural network is a biologically inspired algorithm that passes input data through a series of connected layers to produce an output. Each layer in the neural network may also include another neural network, or may include any number of layers (whether convolutional or not). As can be understood in the context of the present disclosure, a neural network may utilize machine learning, which may refer to a broad classification of such algorithms in which an output is generated based on learned parameters.

[0146] Although discussed in the context of neural networks, any type of machine learning consistent with the present disclosure may be used. For example, machine learning or machine learning algorithms may include, but are not limited to, regression algorithms (e.g., ordinary least squares regression (OLSR), linear regression, logistic regression, stepwise regression, multivariate adaptive regression splines (MARS), local estimated scatter plot smoothing (LOESS)), instance-based algorithms (e.g., ridge regression, least absolute shrinkage and selection operator (LASSO), elastic net, least angle regression (LARS)), decision tree algorithms (e.g., classification and regression trees (CART), iterative dichotomy 3 (ID3), chi-squared automatic interaction detection (CHAID), decision stumps, conditional decision trees), Bayesian algorithms (e.g., naive Bayes, Gaussian naive Bayes, multinomial naive Bayes, averaged single dependency estimator (AODE), Bayesian belief network (BNN), Bayesian network), clustering algorithms (e.g., k-means, k-median, expectation maximization (EM), hierarchical clustering), association rule learning algorithms (e.g., perceptron, backpropagation, jump field network, radial basis function network (RBFN)), deep learning algorithms (e.g., deep Boltzmann machine (DBM), deep belief network (DBN), convolutional neural network (CNN), stacked autoencoder), dimensionality reduction algorithms (e.g., principal component analysis (PCA), principal component regression (PCR), partial least squares regression (PLSR), Sammon mapping, multidimensional scaling (MDS), projection pursuit, linear discriminant analysis (LDA), hybrid discriminant analysis (MDA), quadratic discriminant analysis (QDA), flexible discriminant analysis (FDA)), ensemble algorithms (e.g., boosting methods, bootstrap aggregation (Bagging), AdaBoost, stacked generalization (hybrid), gradient boosting machine (GBM), gradient boosting regression tree (GBRT), random forest), SVM (support vector machine), supervised learning, unsupervised learning, semi-supervised learning, etc.

[0147] Other examples of architectures include neural networks such as ResNet50, ResNet101, VGG, DenseNet, PointNet, etc.

[0148] Figure 7 An example process for operation of a vehicle safety system is depicted. In some examples, the example process 700 may be performed by at least a vehicle safety system (e.g., as described above with reference to Figure 1 The vehicle safety system 102) is executed.

[0149] At operation 702, the example process 700 may include receiving a trajectory (eg, a candidate trajectory). The candidate trajectory may be received at a first component (eg, trajectory management component 108) of a vehicle (eg, an autonomous vehicle) for the vehicle to follow.

[0150] At operation 704, the example process 700 may include receiving at least one of a first signal (e.g., signal 116) indicating an object in proximity to the vehicle, a second signal (e.g., signal 118) indicating a physics-based collision state, or a third signal (e.g., signal 120) indicating a machine learning (ML)-based collision state. Signal 116 may be received by the trajectory management component 108 and may come from a second component (e.g., perception component 110). Signal 118 may be received by the trajectory management component 108 and may come from a third component (e.g., filter component 112). Signal 120 may be received by the trajectory management component 108 and may come from a fourth component (e.g., collision detection component 114). In some examples, one or more of the perception component 110, the filtering component 112, the collision detection component 114, and the trajectory management component 108 may be integrated together, and / or may share results with each other.

[0151] At operation 706, the example process 700 may include determining whether at least one of the signal 116, the signal 118, or the signal 120 is received. The example process 700 may proceed to operation 702 based on determining that the trajectory management component 108 did not receive at least one of the signal 116, the signal 118, or the third signal. The example process 700 may proceed to operation 708 based on determining that the trajectory management component 108 received at least one of the signal 116, the signal 118, or the signal 120.

[0152] At operation 708, the example process 700 may include determining to follow a candidate trajectory or a safe-stop trajectory. The trajectory management component 108 may determine to follow a candidate trajectory or a safe-stop trajectory based on at least one of the signal 116, the signal 118, or the signal 120. In some examples, the trajectory management component 108 may determine to follow a candidate trajectory or a safe-stop trajectory before receiving at least one of the signal 116, the signal 118, or the signal 120. In other examples, the trajectory management component 108 may wait for at least one of the signal 116, the signal 118, or the signal 120 before determining to follow a candidate trajectory or a safe-stop trajectory.

[0153] Sample Clauses

[0154] A: A method comprising: receiving a candidate trajectory for an autonomous vehicle to follow; generating a first signal indicating an object proximate to the autonomous vehicle; generating a second signal indicating a physics-based collision state and determining a first collision likelihood based at least in part on the physics-based collision state; generating a third signal indicating a machine learning-based collision state and determining a second collision likelihood based at least in part on the machine learning (ML)-based collision state; and determining to follow the candidate trajectory or a safe stop trajectory based at least in part on at least one of the first signal, the second signal, or the third signal.

[0155] B: The method of clause A, wherein determining to follow the candidate trajectory or the safe stop trajectory comprises waiting until the first signal, the second signal, and the third signal are generated.

[0156] C: A method as described in clause A or B, wherein determining to follow the candidate trajectory or the safe stop trajectory also includes: determining that the first collision probability is lower than the second collision probability; and determining to follow the safe stop trajectory based at least in part on the second collision probability.

[0157] D: A method as described in any of clauses AC, wherein the component receiving at least one of the first signal, the second signal, or the third signal performs a collision check by perturbing a simulation of the second object along at least one of an acceleration or a steering angle associated with the second object.

[0158] E: A method as described in any of clauses AD, wherein the object is a first object, and generating a first signal further includes: identifying a second object proximate to the autonomous vehicle; identifying that the first object is closer to the candidate trajectory than the second object; and generating the first signal based at least in part on the first object being closer to the candidate trajectory than the second object, wherein generating a second signal further includes: identifying the physics-based collision state based at least in part on a third object proximate to the candidate trajectory; and filtering out a fourth object based at least in part on the object trajectory of a fourth object not being associated with a potential intersection of the candidate trajectory, and wherein generating a third signal further includes: determining that the ML-based collision state reaches or exceeds a threshold collision state based at least in part on the predicted distance between a fifth object and the autonomous vehicle being less than a threshold distance.

[0159] F: One or more non-transitory computer-readable media storing instructions that, when executed, cause one or more processors to perform operations including: receiving a candidate trajectory for a vehicle to follow; receiving a first signal indicating an object proximate to the vehicle; receiving a second signal indicating a physics-based collision state, the second signal indicating a first collision likelihood determined at least in part based on the physics-based collision state; or receiving a third signal indicating a machine learning (ML)-based collision state, the third signal indicating a second collision likelihood determined at least in part based on the ML-based collision state; and determining to follow the candidate trajectory or a safe stop trajectory based at least in part on at least one of the first signal, the second signal, or the third signal.

[0160] G: One or more non-transitory computer-readable media as described in clause F, wherein determining to follow the candidate trajectory or the safe stop trajectory further includes: waiting until the first signal, the second signal, and the third signal are received.

[0161] H: One or more non-transitory computer-readable media as described in clauses F or G, wherein determining to follow the candidate trajectory or the safe stop trajectory also includes: determining that the first collision probability is lower than the second collision probability; and determining to follow the safe stop trajectory based at least in part on the second collision probability.

[0162] I: One or more non-transitory computer-readable media as described in clauses FH, wherein a first component that receives at least one of the first signal, the second signal, or the third signal performs collision checking with higher accuracy than a second component that generates the second signal and a third component that generates the third signal.

[0163] J: One or more non-transitory computer-readable media as described in clause FI, wherein: the first signal is generated by a first component; the first component is configured to store the candidate trajectory and previous candidate trajectories in a queue; and determining to follow the candidate trajectory is performed by a second component at least in part based on the second component receiving the first signal.

[0164] K: One or more non-transitory computer-readable media as described in clauses FJ, wherein: the object is a first object; generating the second signal is based at least in part on determining that a second object associated with the candidate trajectory is closer to the candidate trajectory than a third object associated with the candidate trajectory; and the physics-based collision state is determined at least in part based on the second object.

[0165] L: One or more non-transitory computer-readable media as described in clause FK, wherein the first signal is determined at least in part based on kinematic data associated with the vehicle.

[0166] M: A system comprising: one or more processors; and one or more non-transitory computer-readable media storing instructions executable by the one or more processors, wherein when the instructions are executed, the system performs the following operations, including: receiving a candidate trajectory for a vehicle to follow; receiving at least one of the following: a first signal indicating an object close to the vehicle; a second signal indicating a physics-based collision state, the second signal indicating a first collision possibility determined at least in part based on the physics-based collision state; or a third signal indicating a machine learning (ML)-based collision state, the third signal indicating a second collision possibility based at least in part on the ML-based collision state; and determining to follow the candidate trajectory or a safe stop trajectory based at least in part on at least one of the first signal, the second signal, or the third signal.

[0167] N: In the system as described in clause M, the operation further includes: receiving a second signal and a third signal, wherein determining to follow the candidate trajectory or the safe stop trajectory further includes: determining that a first collision probability based at least in part on the ML-based collision state is lower than a second collision probability based at least in part on the physics-based collision state; and determining to follow the safe stop trajectory based at least in part on the second collision probability.

[0168] O: A system as described in clause M or N, the operation further includes: identifying a second object and a third object associated with the candidate trajectory; and identifying that the second object is the object closest to the candidate trajectory, wherein generating the second signal is at least partially based on the second object.

[0169] P: A system as described in any of clauses MO, wherein the first signal is determined at least in part based on kinematic data associated with the vehicle.

[0170] Q: A system as described in any of clauses MP, wherein determining to follow the candidate trajectory or the safe stop candidate trajectory includes waiting until at least the third signal is received.

[0171] R: A system as described in any of clauses MQ, wherein: the first signal is generated by a first component; the first component is configured to store the candidate trajectory and previous candidate trajectories in a queue; and determining to follow the candidate trajectory is performed by a second component at least in part based on the second component receiving the first signal.

[0172] S: A system as described in any of clauses MR, wherein the component receiving at least one of the first signal, the second signal, or the third signal performs a collision check by perturbing a simulation of the second object along at least one of an acceleration or a steering angle associated with the second object.

[0173] T: A system as described in any of clauses MS, wherein the physics-based collision state is identified based at least in part on an object closest to the candidate trajectory.

[0174] Although the example clauses described above are described with respect to a specific implementation, it should be understood that in the context of this article, the content of the example clauses may also be implemented via a method, an apparatus, a system, a computer-readable medium, and / or another implementation. In addition, any one of the example ATs may be implemented alone or in combination with any other one or more of the example ATs.

[0175] in conclusion

[0176]

[00136] While one or more examples of the techniques described herein have been described, various alterations, additions, permutations, and equivalents thereof are included within the scope of the techniques described herein.

[0177] In the description of the examples, reference is made to the accompanying drawings forming a part thereof, which illustrate specific examples of the claimed subject matter by way of illustration. It should be understood that other examples may be used, and changes or modifications such as structural changes may be made. Such examples, changes or modifications do not necessarily deviate from the scope of the subject matter claimed with respect to the intent. Although the steps herein may be presented in a certain order, in some cases, the ordering may be changed so that certain inputs are provided at different times or in different orders without changing the functionality of the described systems and methods. The disclosed programs may also be executed in different orders. In addition, it is not necessary to perform the various calculations herein in the order disclosed, and other examples of alternative orders using calculations may be easily implemented. In addition to being reordered, calculations may also be decomposed into sub-calculations with the same results.

Claims

1. A method comprising: receiving candidate trajectories for the vehicle to follow; Receive at least one of the following: a first signal indicative of an object in proximity to the vehicle; a second signal indicative of a physics-based collision state, the second signal indicative of a first collision likelihood determined based at least in part on the physics-based collision state; or a third signal indicative of a machine learning (ML) based collision state, the third signal indicative of a second collision likelihood determined based at least in part on the ML based collision state; as well as A determination to follow the candidate trajectory or a safe-stop trajectory is made based at least in part on at least one of the first signal, the second signal, or the third signal.

2. The method of claim 1, wherein: Receiving the at least one of the first signal, the second signal, or the third signal further includes: receiving the first signal, the second signal and the third signal, Wherein, determining to follow the candidate trajectory or the safe stop trajectory further includes: Wait until the first signal, the second signal and the third signal are received; and A determination is made to follow the candidate trajectory or a safe-stop trajectory based at least in part on the first signal, the second signal, and the third signal.

3. The method according to claim 1 or 2, wherein: Determining to follow the candidate trajectory or the safe stop trajectory further includes: Wait until at least one of the first signal, the second signal, or the third signal is received.

4. The method according to any one of claims 1 to 3, further comprising: receiving the second signal and the third signal, Wherein, determining to follow the candidate trajectory or the safe stop trajectory further includes: determining that the first collision likelihood is lower than the second collision likelihood; and A determination to follow the safe-stop trajectory is based at least in part on the second collision likelihood.

5. The method according to any one of claims 1 to 4, wherein: The first component receiving the at least one of the first signal, the second signal, or the third signal performs collision checking with higher accuracy than the second component generating the second signal and the third component generating the third signal.

6. The method of any one of claims 1 to 3, wherein: The first signal is generated by a first component; The first component is configured to store the candidate trajectory and previous candidate trajectories in a queue; as well as Determining to follow the candidate trajectory is performed by a second component based at least in part on receipt of the first signal by the second component.

7. The method of any one of claims 1 to 6, wherein: The object is a first object; The second signal is generated based at least in part on determining that a second object associated with the candidate trajectory is closer to the candidate trajectory than a third object associated with the candidate trajectory; as well as The physics-based collision state is determined based at least in part on the second object.

8. A computer program product comprising coded instructions, which, when run on a computer, implements the method according to any one of claims 1 to 7.

9. A system comprising: one or more processors; as well as One or more non-transitory computer-readable media storing instructions executable by the one or more processors, wherein when the instructions are executed, the system performs the following operations, including: receiving candidate trajectories for the vehicle to follow; Receive at least one of the following: a first signal indicative of an object in proximity to the vehicle; a second signal indicative of a physics-based collision state, the second signal indicative of a first collision likelihood determined based at least in part on the physics-based collision state; or a third signal indicative of a machine learning (ML) based collision state, the third signal indicative of a second collision likelihood based at least in part on the ML based collision state; and A determination to follow the candidate trajectory or a safe-stop trajectory is made based at least in part on at least one of the first signal, the second signal, or the third signal.

10. The system of claim 9, wherein: the object is a first object, The operations also include: identifying a second object and a third object associated with the candidate track; and identifying that the second object is the object closest to the candidate trajectory, Wherein the second signal is generated at least in part based on the second object.

11. The system of any one of claims 9 or 10, wherein: The first signal is determined based at least in part on kinematic data associated with the vehicle.

12. The system of any one of claims 9 to 11, wherein: Determining to follow the candidate trajectory or the safe stop candidate trajectory includes waiting until at least the third signal is received.

13. The system of any one of claims 9 or 12, wherein: The object is a first object, and the component receiving the at least one of the first signal, the second signal, or the third signal performs a collision check by perturbing a simulation of the second object along at least one of an acceleration or a steering angle associated with the second object.

14. The system of any one of claims 9 or 13, wherein: The physics-based collision state is identified based at least in part on an object that is closest to the candidate trajectory.

15. The system of any one of claims 9 or 14, wherein: The object is a first object, and the first signal is generated by: identifying a second object proximate to the autonomous vehicle; identifying that the first object is closer to the candidate trajectory than the second object; and generating the first signal based on the first object being closer to the candidate trajectory than the second object, Wherein, the second signal is generated by: identifying the physics-based collision state based at least in part on a third object proximate to the candidate trajectory; and filtering out a fourth object based at least in part on the object trajectory of the fourth object not being associated with a potential intersection of the candidate trajectory, and Wherein, the third signal is generated by: The ML-based collision state is determined to have reached or exceeded a threshold collision state based at least in part on a predicted distance between a fifth object and the autonomous vehicle being less than a threshold distance.

Images