Method and system for assessing and mitigating risks that may be encountered by an autonomous vehicle

Abstract

A method 100 for assessing and mitigating risks that an autonomous vehicle may encounter includes collecting information related to the environment of the ego-vehicle and identifying and assessing a set of risks that the ego-vehicle may encounter. The system for assessing and mitigating risks may include and / or interface with an ego-vehicle (also referred to herein as an autonomous vehicle, an autonomous agent, an ego-agent, an agent, etc.) and a set of computing subsystems (also referred to herein as a set of computers) and / or processing subsystems (also referred to herein as a set of processors) that function to perform some or all of the processes of the method.

Description

[Technical Field]

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS This application claims the benefit of U.S. Provisional Application No. 63 / 432,137, filed December 13, 2022, and U.S. Provisional Application No. 63 / 442,636, filed February 1, 2023, each of which is incorporated by this reference in its entirety.

[0002] The present invention relates generally to the autonomy, transportation, and vehicle fields, and more particularly to novel and useful systems and methods for assessing and mitigating risks that may be encountered by autonomous vehicles in the autonomy, transportation, and vehicle fields. [Brief explanation of the drawings]

[0003] [Figure 1] FIG. 1 is a schematic diagram of a method for assessing and mitigating risks that a vehicle may encounter. [Figure 2] FIG. 2 is a schematic diagram of a system for assessing and mitigating risks that a vehicle may encounter. [Figure 3] 3A-3F illustrate an example method for assessing and mitigating risks that a vehicle may encounter. [Figure 4] FIG. 4 is a demonstrative example of a series of risk profiles that are generated and evaluated during a method for assessing and mitigating risks that a vehicle may encounter. [Figure 5] Figure 5 shows an example of the future time points simulated in the first subset of simulations. [Figure 6] FIG. 6 shows an example of the future time points simulated in the first subset of simulations. [Figure 7] FIG. 7 shows a variant of the method for assessing and mitigating risks that a vehicle may encounter. DETAILED DESCRIPTION OF THE INVENTION

[0004] The following description of preferred embodiments of the invention is not intended to limit the invention to those preferred embodiments, but rather to enable any person skilled in the art to make and use the invention.

[0005] 1. Overview 1, a method 100 for assessing and mitigating risks that an autonomous vehicle may encounter includes steps S100 of collecting information related to the environment of the ego-vehicle and S200 of determining and assessing a set of risks that the ego-vehicle may encounter. Additionally or alternatively, method 200 may include some or all of steps S300 of operating the ego-vehicle based on the assessed risks, S210 of running a set of simulations, S220 of analyzing the simulation results, and / or any other processing. Additionally or alternatively, method 100 may be implemented using any of the methods described in U.S. Application No. 16 / 514,624, filed July 17, 2019; U.S. Application No. 16 / 505,372, filed July 8, 2019; U.S. Application No. 16 / 540,836, filed August 14, 2019; U.S. Application No. 16 / 792,780, filed February 17, 2020; U.S. Application No. 17 / 365,538, filed July 1, 2021; U.S. Application No. 17 / 550,461, filed December 14, 2021; and U.S. The method may include and / or interface with some or all of the processes described in some or all of U.S. Application No. 17 / 554,619, filed December 17, 2021, U.S. Application No. 17 / 712,757, filed April 4, 2022, U.S. Application No. 17 / 826,655, filed May 27, 2022, U.S. Application No. 18 / 072,939, filed December 1, 2022, and U.S. Application No. 18 / 109,689, filed February 14, 2023, or any other suitable processes, performed in any suitable order.

[0006] Method 100 may be performed using system 200 described below and / or any other suitable system.

[0007] 2, system 200 for assessing and mitigating risk may include and / or interface with an ego-vehicle (also referred to herein as an autonomous vehicle, an autonomous agent, an ego-agent, an agent, etc.) and a set of computing subsystems (also referred to herein as a set of computers) and / or processing subsystems (also referred to herein as a set of processors) that function to perform some or all of the processes of method 100. Additionally or alternatively, system 200 may include and / or interface with a set of one or more sensors (e.g., onboard the ego-agent, onboard a set of infrastructure devices, etc.), a simulation subsystem (e.g., executable on one or more computing subsystems) that includes a set of simulators, a set of infrastructure devices, a remotely operated platform, a tracker, a positioning system, a guidance system, a communication interface, and / or any other components.

[0008] Additionally or alternatively, the system may be any of the systems described in U.S. Application No. 16 / 514,624, filed July 17, 2019, U.S. Application No. 16 / 505,372, filed July 8, 2019, U.S. Application No. 16 / 540,836, filed August 14, 2019, U.S. Application No. 16 / 792,780, filed February 17, 2020, U.S. Application No. 17 / 365,538, filed July 1, 2021, and U.S. Application No. 17 / 550, each of which is incorporated herein by reference in its entirety. 461, filed December 14, 2021, U.S. Application No. 17 / 554,619, filed December 17, 2021, U.S. Application No. 17 / 712,757, filed April 4, 2022, U.S. Application No. 17 / 826,655, filed May 27, 2022, U.S. Application No. 18 / 072,939, filed December 1, 2022, and U.S. Application No. 18 / 109,689, filed February 14, 2023.

[0009] 2.Effects Systems and methods for assessing and mitigating risks that autonomous vehicles may encounter can provide several advantages over current systems and methods.

[0010] In a first variation, the technology provides the advantage of enabling an autonomous vehicle to be operated (e.g., driven) in a manner similar to how a human driver operates a manually driven vehicle. For example, the technology enables the autonomous vehicle to assess and triage risks, effectively buying time (and optionally adjusting actions to gain this time) before making a decision for risks that do not pose immediate harm and / or may not pose harm in the future (e.g., due to changes in the behavior of nearby vehicles). This is similar, for example, to how humans modify their behavior (e.g., taking their foot off the accelerator if they are uncertain about whether another vehicle will turn in front of them, or beginning to brake if there is a potential risk ahead) and / or do not execute sudden or overly conservative responses in advance to risks that may not actually exist.

[0011] In one set of examples, a temporal risk profile (e.g., as shown in FIG. 4) is generated that quantifies the level of risk and indicates when that predicted risk will occur, allowing the autonomous vehicle to tolerate risk far enough into the future (e.g., past a predetermined threshold number of seconds, past a predetermined reaction time, etc., with enough time for the ego-agent or other agents to brake safely and / or smoothly) without taking overly cautious or unnecessarily conservative action, such as stopping (e.g., braking for a vehicle that will never actually turn in front of the ego-agent).

[0012] In a second variation, in addition to or instead of the first variation, the present technology provides the advantage of overcoming some or all of the limitations encountered when using traditional cost and / or loss functions for autonomous vehicle decision-making. For example, in some examples of traditional cost functions, the cost function calculation often involves aggregating many different factors with different units in a weighted manner, with relative weights determined for a particular scenario (e.g., tuned, hypertuned, etc.), which can result in undesirable trade-offs and unexpected vehicle behavior in other scenarios. In the example systems and methods, some or all of the calculations used for vehicle decision-making are performed using factors that are measurable and meaningful (e.g., generated according to principles of physics) and use the same unit type, resulting in a robust analysis that does not require special use cases to be treated separately.

[0013] In a third variation, additionally or alternatively to the above, the technique provides the advantage of utilizing forward simulation (simulation that simulates a step ahead in time) to accurately consider risks associated with future scenarios (also equivalently referred to herein as "the future") that the autonomous vehicle may encounter. By way of example, the system and / or method maximizes the value and effectiveness of simulations and simulation types to accurately determine risks that the vehicle needs to respond to immediately and risks that can be further analyzed over time.

[0014] In a fourth variation, additionally or alternatively to the above, the technique provides the advantage of leveraging multiple types of simulations, where the types of simulations differ in how the vehicle's environment is represented (e.g., what objects the environment contains), and the differences in the types of simulations can determine a set of relative metrics that can be used to evaluate some or all of the risks to prioritize, the risks to consider, the vehicle's contribution to risk, the contribution of other agents to risk, and / or other information.

[0015] In a fifth variation, additionally or alternatively to the above, the technique provides the advantage of allowing a spectrum and / or multitude of risk types and risk severities to be considered and evaluated, rather than using only the most severe risk (e.g., collision) in vehicle decision-making. Using only the most severe risk (e.g., collision) in vehicle decision-making may traditionally result in the ego vehicle exhibiting overly conservative, unpredictable, and impeding behavior. In a variation, the method includes considering multiple types of risk, such as, but not limited to, collision, conflict zone (e.g., the likelihood that the ego vehicle is in a high-risk area with another agent, the likelihood that the ego vehicle will intersect with another object, etc.), reduced following distance, and / or other risks.

[0016] In a sixth variation, additionally or alternatively to the above, the technique provides the advantage of predicting not only future risk to the ego-vehicle, but (alternatively) risk associated with objects in the ego-vehicle's environment and (1) the ability to mitigate risk from the perspective of those objects (e.g., braking to avoid the ego-vehicle, braking to avoid another object, etc.), and (2) the ability to mitigate risk resulting from environmental objects interacting with each other (e.g., when the ego-vehicle is not involved at all). This may be possible, for example, through some or all of different types of simulations (e.g., simulations in which the ego-vehicle is not present), analyzing the likelihood of different predictive scenarios, incorporating prediction uncertainty in the simulation when calculating the risk metric, and / or other features of the system and / or method.

[0017] Additionally or alternatively, the systems and methods may provide other benefits.

[0018] 3. Method 100 1, a method 100 for assessing and mitigating risks that an autonomous vehicle may encounter includes steps S100 of collecting information related to the environment of the ego-vehicle and S200 of identifying and assessing a set of risks that the ego-vehicle may encounter (e.g., may encounter in the future). Additionally or alternatively, method 200 may include steps S300 of operating the ego-vehicle based on the assessed risks, S210 of running a set of simulations, S220 of analyzing the simulation results, and / or any other processing. Additionally or alternatively, method 100 may be implemented using any of the methods described in U.S. Application No. 16 / 514,624, filed July 17, 2019; U.S. Application No. 16 / 505,372, filed July 8, 2019; U.S. Application No. 16 / 540,836, filed August 14, 2019; U.S. Application No. 16 / 792,780, filed February 17, 2020; U.S. Application No. 17 / 365,538, filed July 1, 2021; U.S. Application No. 17 / 550,461, filed December 14, 2021; and U.S. The method may include and / or interface with some or all of the processes described in some or all of U.S. Application No. 17 / 554,619, filed December 17, 2021, U.S. Application No. 17 / 712,757, filed April 4, 2022, U.S. Application No. 17 / 826,655, filed May 27, 2022, U.S. Application No. 18 / 072,939, filed December 1, 2022, and U.S. Application No. 18 / 109,689, filed February 14, 2023, or any other suitable processes, performed in any suitable order.

[0019] Method 100 may be performed using system 200 described below and / or any other suitable system.

[0020] Method 100 preferably functions to accurately assess and optimally respond to potential risks that the ego-vehicle may encounter in the future. In a preferred embodiment, for example, method 100 enables the vehicle to accurately consider the risks of potential future situations without responding in an overly conservative manner (e.g., by overestimating the risk and / or its likelihood) or an overly aggressive manner (e.g., by underestimating the risk and / or its likelihood). In a specific example, method 100 functions to handle uncertainty about what the future will actually be like (e.g., whether a pedestrian will cross the road, whether another vehicle will run a stop sign, whether another vehicle will merge in front of the ego-vehicle, etc.), where handling uncertainty may include allowing time to pass (and / or taking certain actions) to gather more information about the environment and which scenarios will materialize, thereby allowing the vehicle to delay and / or mitigate its response to risks that do not pose an immediate danger to the ego-vehicle (or other vehicles / objects). This prevents the vehicle from taking overly conservative actions (e.g., sudden stops or pulling over) when not necessary, overly aggressive actions (e.g., speeding up) when it is unsafe, and / or not acting in the most intelligent way based on its current understanding of the environment and the likelihood that an identified risk becomes a real hazard. This contrasts with traditional approaches in which the vehicle finds the worst possible outcome (e.g., now, in the future, etc.) and acts accordingly (which can lead to overly conservative and / or unexpected actions compared to, for example, human driving behavior).

[0021] Additionally or alternatively, method 100 may function to prevent special use cases from having to be handled separately by the autonomous vehicle. Further, additionally or alternatively, method 100 may perform any other function.

[0022] Risks that an autonomous vehicle may encounter (equivalently referred to herein as hazards and / or hazardous events) preferably refer to potentially hazardous scenarios that the autonomous vehicle can detect, such as potentially hazardous events (e.g., collisions, potential collisions, etc.) detected based on data collected by the vehicle's perception subsystem and processed by the prediction subsystem and / or planning subsystem.

[0023] Method 100 can optionally be configured to interface with an own-agent's multi-policy decision-making process (e.g., a multi-policy decision-making task block of a computer-readable medium) and any associated components (e.g., a computer, processor, software module, etc.), although it may additionally or alternatively interface with any other decision-making process. In a preferred set of variations, for example, the multi-policy decision-making module of a computing subsystem (e.g., an on-board computing system) includes a simulator module (or similar machine or system) (e.g., a simulator task block of a computer-readable medium) that functions to predict (e.g., estimate) the effects of future (i.e., future in time) action policies (maneuvers or actions) to be implemented on the own-agent, and optionally, the effects of action policies to be implemented on each of a set of environmental agents (e.g., other vehicles in the own-agent's environment) and / or objects (e.g., pedestrians) identified in the own-agent's operating environment. The simulation can be based on each agent's current state (e.g., current hypotheses) and / or each agent's past actions or historical behavior derived from a history data buffer (preferably containing data up to the present time). The simulation can provide data related to the interaction (e.g., relative position, relative velocity, relative acceleration) between each environmental agent's predicted action policy and one or more potential action policies that may be executed by the autonomous agent.Data from the simulation can be used to determine (e.g., calculate) any number of metrics, which may individually and / or collectively function to assess some or all of the potential impact of the own agent on some or all of the environmental agents when executing a particular policy, the risk of executing a particular policy (e.g., collision risk), how much progress the own agent will make toward a particular goal by executing a particular policy, and / or determine any other metrics involved in selecting a policy for the own agent to execute.

[0024] The multi-policy decision-making process may additionally or alternatively include and / or interface with any other processes, such as, but not limited to, some or all of the processes described in U.S. Application No. 16 / 514,624, filed July 17, 2019, and U.S. Application No. 17 / 365,538, filed July 1, 2021, each of which is incorporated by this reference in its entirety, or any other suitable processes performed in any suitable order.

[0025] In a preferred set of variations, for example, method 100 is performed for each of a set of policies being considered by the autonomous vehicle, a risk metric and / or risk profile is generated for each policy (e.g., as shown in FIG. 4 ), and a particular policy is selected based on the metric and / or profile. This policy selection may be based, for example, on some or all of the following: the future time (e.g., how far into the future) at which the maximum risk for the policy is predicted to occur; the total level of risk associated with the policy (e.g., the integral of the risk curve); the magnitude of the risk (e.g., the magnitude of the maximum point in the risk profile); the average risk; the median risk; and / or any other metric. In some examples, a policy is selected based, for example, on both the value of the risk magnitude and the future time at which the risk is predicted to occur. In a particular example, a policy may be selected that has a higher overall risk compared to other policy options, but whose risk lies further into the future (e.g., slower than a predetermined reaction time and within the ego-vehicle's braking limits, etc.). Additionally or alternatively, any other characteristics may be considered and evaluated (e.g., comparison to one or more thresholds, combination in a weighted manner, evaluation of algorithms and / or models, etc.).

[0026] Additionally or alternatively, the risk metric may dynamically inform which policy options are considered in future selection cycles, such as the next time point. For example, if the risk to the ego vehicle is predicted far into the future (e.g., 4 or more seconds, 5 or more seconds, 6 or more seconds, 7 or more seconds, or more than a predetermined threshold number of seconds into which the ego vehicle or other object can be stopped with braking force below a predetermined braking threshold, etc.), a "slow down" or "wait" or "increase distance from the vehicle ahead" policy may be incorporated into the next set of policy actions considered, which may act to mitigate the detected risk and / or postpone the risk further into the future.

[0027] In some instances, for example, the types of policies considered in the next selection cycle are determined at least in part based on the risk profile generated in the current selection cycle.

[0028] Additionally or alternatively, the risk metric may inform parameters (e.g., speed, position, etc.) involved in generating the trajectory of the vehicle (e.g., as part of the route planning process), such as a trajectory generated according to a policy or a trajectory generated in the absence of a policy.

[0029] Additionally or alternatively, the method 100 may include and / or interface with other decision-making processes.

[0030] 3.1 Method—Step S100 of Collecting Information About the Vehicle's Environment Method 100 includes step S100 of collecting information related to the environment of the host vehicle, which may function to evaluate the environment of the host vehicle and receive information to inform the performance of some or all of the remaining processes of method 100.

[0031] S100 is preferably performed continuously (e.g., at a predetermined frequency, at irregular intervals, etc.) throughout the operation of the own agent, but may additionally or alternatively be performed according to a cycle associated with the own agent, such as a selection cycle associated with the own agent (e.g., a 10 Hz cycle, a 5-20 Hz cycle, etc.) (e.g., the own agent selects a new policy), a perception cycle associated with the own agent, a planner cycle associated with the own agent (e.g., a 30 Hz, a 20-40 Hz cycle, occurring more frequently than the selection cycle, etc.), in response to a trigger (e.g., a request, the start of a new cycle, etc.), and / or at any other time during method 100.

[0032] The inputs preferably include sensor inputs received from a sensor suite onboard the own agent (e.g., cameras, lidar, radar, motion sensors (e.g., accelerometers, gyroscopes, etc.), OBD port outputs, etc., position sensors (e.g., GPS sensors), etc.), but may additionally or alternatively include historical information related to the own agent (e.g., historical state estimates of the own agent) and / or environmental agents (e.g., historical state estimates of the environmental agents), sensor inputs from sensor systems not onboard the own agent (e.g., onboard other own agents or environmental agents, onboard a set of infrastructure devices and / or roadside units, etc.), information and / or any other input.

[0033] The inputs preferably include information characterizing the environment of the ego-agent, which may include other objects (e.g., vehicles (equivalently referred to herein as environmental agents), pedestrians, stationary objects, etc.) that are near the ego-agent (e.g., within the field of view of its sensors, within a predetermined distance, etc.), environmental features around the ego-agent (e.g., environmental features around the ego-agent (e.g., as referenced on a map, locating the ego-agent, etc.), and / or other information. In some variations, for example, the set of inputs may include the location, type / class (e.g., vehicle vs. pedestrian, etc.), and / or movement of environmental objects being tracked by system 200, where the environmental objects may be static objects (e.g., parked or other non-moving vehicles, stationary objects, etc.). The set of inputs may include information that characterizes some or all of the following: a moving object (e.g., a pedestrian moving in the distance), a dynamic object (e.g., a moving vehicle, a walking or running pedestrian, a bicyclist, etc.), or any other object or combination of objects (e.g., information from sensors mounted on the own agent, information from sensors in the own agent's environment, information from sensors mounted on objects, etc.). Additionally or alternatively, the set of inputs may include information that characterizes (e.g., locates, identifies, etc.) road and / or other landmark / infrastructure features (e.g., where the lanes are, where the edges of the road are, where the traffic lights are, what type they are, where the agent is relative to these landmarks, etc.), and / or other information that allows the own agent to locate itself within its environment (e.g., for map reference).

[0034] The inputs preferably also include information related to the "ego" agent, which herein refers to a vehicle being operated during execution of method 100. This may include information characterizing the "ego" agent's position (e.g., position relative to the world, position relative to one or more maps, position relative to other objects, etc.), the "ego" agent's motion (e.g., speed, acceleration, etc.), the "ego" agent's orientation (e.g., heading angle), the performance and / or health of the "ego" agent and its subsystems (e.g., sensor health, computing system health, etc.), and / or any other information.

[0035] Some or all of this information may additionally or alternatively be specified for the environmental object.

[0036] Additionally or alternatively, S100 may include any other processes and / or the collection of any other suitable information.

[0037] In a preferred set of variations, S100 includes collecting sensor data that is used to run the simulation described in S210.

[0038] 3.2 Method—Identifying and Evaluating a Set of Risks Possibly Encountered by the Ego Vehicle Step S200 Method 100 includes step S200 of identifying and evaluating a set of risks that the ego-vehicle may encounter, which functions to perform some or all of the following: detecting potential hazards that the ego-vehicle may encounter in the future; identifying whether (and / or when) these potential hazards pose a current or future risk; and identifying whether the potential hazards can be avoided (e.g., by the ego-vehicle, by other vehicles or entities in the ego-vehicle's environment, etc.). S200 also preferably functions to notify how the vehicle is being operated in S300. Additionally or alternatively, S200 may perform other suitable functions.

[0039] S200 is preferably performed in response to and based on information received in S100, but may additionally or alternatively be performed at any other suitable time and / or based on any other suitable information.

[0040] S200 preferably includes step S210 (e.g., as shown in FIG. 7 ) of running a set of simulations, which function to enable (e.g., in S220) to detect and further characterize risks that may occur in the future. The set of simulations preferably includes a forward simulation (based on a time forward) that examines future scenarios of the ego-vehicle and objects in its environment (e.g., other vehicles, pedestrians, etc.), such as if the ego-vehicle (and / or other vehicles) were to execute a particular policy (e.g., behavior, action, etc.). Additionally or alternatively, the simulations may be run in other suitable manners.

[0041] In a preferred set of variations, some or all of the simulations are performed within (e.g., as part of, during) a multi-policy decision-making process (e.g., performed during the own agent's planning cycle, as described above), some or all of which are described in U.S. Application No. 16 / 514,624 (filed July 17, 2019) and / or U.S. Application No. 17 / 365,538 (filed July 1, 2021), each of which is incorporated herein by reference in its entirety. Alternatively, the simulations may be performed according to a different decision-making process, may be performed independently of and / or asynchronously with the multi-policy decision-making process (e.g., during a trajectory generation and / or trajectory correction phase, during a path planning process, etc.), and / or may be performed at any other time.

[0042] In one set of embodiments, for example, for each potential policy that the ego vehicle considers executing (eg, as a next policy), a set of simulations (eg, including each simulation type) is run.

[0043] The set of simulations preferably includes multiple types of simulations (e.g., as described below) that collectively function to provide an accurate and robust assessment of potential future risks and / or risk characteristics (e.g., likelihood, characteristics, etc.). Multiple simulation types can be simulated in parallel, serially, and / or in any combination, etc. Also, S210 may perform a single type of simulation or other types of simulations.

[0044] 3.21 First subset of simulations The set of simulations performed in method 100 preferably includes a first subset of simulations, where the first subset of simulations functions to represent the future environment of the own agent. Additionally or alternatively, the first subset of simulations may function to identify potential risks, identify objects associated with the potential risks, utilize the potential risks and / or associated objects to inform future processing in method 100, and / or perform other functions.

[0045] The first subset of simulations preferably includes a first type of simulation, in which the ego-vehicle is simulated with environmental objects (e.g., other vehicles, pedestrians, other objects, etc.) in its environment (e.g., as shown in FIG. 3A ), thereby representing the ego-vehicle's actual, predicted environment. In this manner, the first subset of simulations preferably functions to detect potential hazardous events (also referred to as risks) that may result from interactions (e.g., collisions, near-collisions, etc.) between the ego-vehicle and objects in its environment. Furthermore, the first type of simulation may detect hazardous events (e.g., collisions between two other agents) that may occur between other agents in the ego-vehicle's environment depending on actions performed by the ego-vehicle. For example, if the ego-vehicle gets in front of a first environmental agent, the simulation may indicate that the first environmental agent will brake suddenly to avoid colliding with the ego-vehicle, resulting in a second environmental agent behind the first environmental agent colliding with (e.g., rear-ending) the first environmental agent.

[0046] Thus, in a first type of simulation, the forward movement of the ego-vehicle and environmental object (e.g., as specified in the intention estimation process, based on a proposed policy for the environmental object, based on a proposed path for the environmental object, based on the current and / or past detected position and speed of the environmental object, etc.) is simulated and evaluated to determine, for example, some or all of the following: whether the ego-vehicle will collide (intersect, contact, etc.) with the environmental object; whether another potentially dangerous or undesirable event will occur (e.g., the ego-vehicle will nearly collide with the environmental object, the ego-vehicle will collide with another environmental object, etc.); metrics associated with such event (e.g., as described below); and / or whether other types of outcomes will potentially occur at a future time.

[0047] The first type of simulation preferably checks (can analyze) multiple risk types and / or risk severity, or the first type of simulation may check a single type of risk, check risks in different types of simulation, and / or analyze risks as appropriate in one or more types of simulation.

[0048] The risk type preferably includes a predicted collision (e.g., the ego-vehicle is co-located with another object at the same time, two objects are co-located at the same time, etc.), which represents the probability of actual contact between the objects and indicates, for example, the highest severity risk. The predicted collision may be further analyzed (e.g., based on data from the first collision type) to determine some or all of the following: whether the ego-vehicle and / or other object can prevent the collision (e.g., based on the time the collision is predicted, based on the braking capability of the ego-vehicle, based on the predicted braking capability of the other object, based on the predicted reaction time and / or braking time of the ego-vehicle or other object, etc.), how alert the other object is to the possibility of a collision (e.g., based on historical data related to the other object, based on the past driving behavior of the other object, based on the estimated intentions of the other object, etc.), the probability that the collision will occur (e.g., based on uncertainty associated with the position and / or speed of the simulated object), and / or other characteristics.

[0049] The risk types preferably further include the ego vehicle being in proximity to a conflict zone (e.g., within a predetermined distance, indicated by entering the conflict zone in a forward simulation, etc.) and / or approaching and / or entering the conflict zone, where the conflict zone represents an area where a conflict may occur. A conflict may reflect any number of scenarios or be associated with various characteristics, such as a near-miss collision, a specific type of collision, a distance between objects below a predetermined threshold, the ego vehicle or other object failing to maintain a minimum separation distance relative to the other object, a specific angle range between the headings of the ego vehicle and / or other object, a path intersection between objects (e.g., the ego vehicle and another vehicle as shown in FIG. 5), and / or any other event.

[0050] Conflict zones may be partially or entirely predetermined (e.g., map-defined / obtained from a map), dynamically determined (e.g., based on the predicted behavior / path of the ego-vehicle and / or other objects in the simulation), or any combination of predetermined and dynamically determined.

[0051] In some variations, for example, the conflict zone may be at least partially predetermined (e.g., encoded on a map) and may include areas where the ego-vehicle intersects with and / or approaches the same location as other objects, and may optionally further include areas where the ego-vehicle may be traveling in a different orientation than the orientation of the other objects (e.g., by an angle of 90 degrees, between 0 and 360 degrees, between 90 and 360 degrees, etc.), areas containing pedestrians or areas of potential pedestrians, and / or any other areas.

[0052] In a specific example, a conflict zone in the shape of a crosswalk functions to create a model that encodes the risk associated with an interaction between the self and a VRU (Vulnerable Road Users, e.g., pedestrians) on the crosswalk. For example, the conflict zone and its associated response method can specify how to safely and effectively handle the associated risk. Depending on the position and speed of the ego-vehicle and the object, a risk profile can be created that prevents the ego-vehicle from entering the conflict zone if a pedestrian is present in the conflict zone. The generated risk profile enables the ego-vehicle (e.g., in S300) to maintain a safe distance from other agents by penalizing their approach to the conflict zone (e.g., if another object is predicted to be in the conflict zone with the ego-vehicle in a first subset of simulations). As a result, a set of risk metrics associated with each interacting VRU is generated, with the most constrained agent being the riskiest (e.g., scaled based on the probability and severity of the interaction). The calculation of the probability of interaction may be based on some or all of the following: where the VRU is located on the crosswalk, how far the ego vehicle is from the crosswalk, how fast the ego vehicle is approaching the crosswalk, and / or other characteristics. The severity may be determined based on the ego vehicle's distance from the conflict zone, the object's speed, the object's direction, and / or other characteristics.

[0053] The conflict zone may optionally be dynamically adjusted (e.g., considered, removed from consideration) based on the results of the first subset of simulations, based on whether other objects may be located in the conflict zone at the same time as the ego vehicle, based on whether other objects may be within a predetermined distance of the ego vehicle in the conflict zone, and / or based on any other information.

[0054] In some variations, for example, checking for a conflict zone may include checking whether the ego-vehicle is approaching and / or within an intersection (e.g., within a particular forward simulation time, the time when another object is predicted to be in the conflict zone, etc., as shown in FIG. 3A ). Additionally or alternatively, checking for a conflict zone may include checking the relative predicted time difference between when the ego-vehicle is in the conflict zone and / or at a particular location and when another object is in that zone and / or at that location (e.g., serving as a proxy for how hard the object must brake to avoid the ego-vehicle, providing an indication of whether the object will be able to avoid hitting the ego-vehicle, etc.), checking whether the ego-vehicle (or other object) is entering the other object's lane during a lane-changing or lane-crossing maneuver, checking whether the ego-vehicle is approaching a crosswalk or other area with pedestrians, checking whether the ego-vehicle is about to cross and / or enter a bike lane, and / or checking for other scenarios.

[0055] The risk type preferably further includes an assessment of whether (and / or to what extent) the ego-vehicle is expected to maintain a predetermined interval and / or following distance relative to other objects. For example, when following another vehicle, the ego-vehicle may aim to maintain at least a predetermined interval (e.g., a 2-second interval, a 2-4 second interval, at least a 1 second interval, a reliable following distance, etc.) relative to the leading vehicle. Additionally, it may be desirable to maintain a certain interval (e.g., distance) or minimum distance when passing an object (e.g., to prevent a car door from opening toward the ego-vehicle, to prevent the ego-vehicle from obstructing and / or side-spanning a cyclist, etc.). For example, if the first type of simulation determines that the ego-vehicle may not be able to achieve these intervals (e.g., as shown in FIG. 6), this risk can be assessed and used to reflect that the ego-vehicle (or other object) may not be able to react in time if something unexpected occurs (e.g., a scenario not reflected in the simulation).

[0056] In a variant, for example, some or all of the risk metrics calculated below may reflect that the overall risk to the ego-vehicle increases as the gap (e.g., time gap, distance gap, etc.) between the ego-vehicle and other objects decreases. This may prompt, for example, (e.g., after taking other risks / factors into account in the calculation) to select a policy that slows the ego-vehicle down to increase this gap.

[0057] As described below, method 100 may further include quantifying, scaling, discounting, and / or excluding from further consideration, aggregating, and / or otherwise processing these risks.

[0058] Additionally or alternatively, other risk types may be checked, assessed, and / or used in calculating risk metrics (eg, as described below).

[0059] Some or all of the above risks may optionally also be calculated with respect to non-self-vehicle interactions, such as between environmental objects.

[0060] The presence or absence of these risks preferably serves to identify which objects are involved in the risk and therefore which calculations should be performed in the risk calculation (e.g., as described below), but may additionally or alternatively perform any other function.

[0061] For example, a first subset of simulations may ultimately allow for the identification of some or all of: which objects pose risks to the environment; which of these risks can be mitigated (e.g., by slowing down, speeding up, changing lanes, etc.); how the risks are changing over time (e.g., increasing, decreasing, staying the same, etc.); and / or any other characteristics of the risks.

[0062] Additionally or alternatively, the first subset of simulations may include other simulation types and / or may be suitably performed in other ways.

[0063] 3.22 Second subset of simulations The set of simulations performed in method 100 preferably includes a second subset of simulations, where the second subset of simulations functions to represent one or more alternative versions of the future ego-vehicle's environment. The second subset of simulations further preferably functions to identify which data to obtain for comparison from the first subset of simulations. Additionally or alternatively, the second subset of simulations may perform functions to identify which objects risk is attributable to, identify uncertainty associated with scenarios in the first subset of simulations, identify the magnitude of worst-case scenarios, and / or perform other functions.

[0064] The second subset of simulations preferably includes multiple simulation types that differ in how the environment is modified. The environment is preferably modified by selectively including specific objects, but may additionally or alternatively be modified in other ways (e.g., by changing the velocity of objects, changing the position of objects, etc.). In a preferred variation, for example, the second subset of simulations includes a first type of simulation in which the ego-vehicle is removed from the environmental representation, and a second type of simulation in which all objects other than the ego-vehicle are removed from the environmental representation. Additionally or alternatively, the second subset of simulations may include any other simulation.

[0065] For example, in the first type of the second subset of simulations (e.g., as shown in FIG. 3B ), the ego-vehicle is not present, and only the position, movement, and actions of environmental objects are simulated. Preferably, this modification functions to determine the relative influence (e.g., impact) of the ego-vehicle's movement / action / behavior on environmental objects, quantifiable by a risk metric (e.g., as described below). In some variations, for example, the second subset of simulations allows further insight into whether (and / or to what extent) potentially dangerous events detected in the first subset of simulations were caused by the presence and / or behavior of the ego-vehicle. This may inform (e.g., in S300) the extent to which risk can be mitigated by the ego-vehicle and / or which policy is best suited for the ego-vehicle to implement. Additionally, this simulation type may function to identify the amount of energy and / or work required by other agents to mitigate risk caused by the ego-vehicle's actions (e.g., braking hard to avoid a collision with the ego-vehicle or another agent, steering onto the shoulder, etc.).

[0066] The set of simulations can optionally additionally or alternatively include a second type of second subset of simulations (e.g., as shown in FIG. 3C ), in which only the ego-vehicle is present and no environmental objects are simulated. This type of simulation preferably serves to provide information about uncertainties naturally present in the forward simulation. For example, simulating only the ego-vehicle can capture worst-case uncertainties present in the ego-vehicle's environment. In instances where increased energy and / or motion is observed relative to the environment caused by the ego-vehicle, the second type of simulation of the second subset can serve to minimize the energy and / or motion that the ego-vehicle would incur if there were no other objects in the environment.

[0067] In an embodiment, for example, a "potential" energy metric is calculated later in method 100, which functions to estimate the maximum energy (e.g., worst-case energy) that an action (policy) could potentially introduce into the environment when there is uncertainty about what the action taken by the ego-vehicle will actually be. For example, if a simulation indicates that the vehicle must stop for an obstacle, but actually performing that action results in the vehicle not stopping due to prediction uncertainty (e.g., a small error), the potential energy may indicate that the vehicle may be moving at a high speed that was not accounted for in the simulation. Other types of uncertainty include, for example, uncertainty in the position of some or all objects (e.g., based on imperfect sensors, a limited field of view of the sensors, etc.), uncertainty in the motion of some or all objects (e.g., based on imperfect sensors, a limited field of view of the sensors, etc.), uncertainty in the classification of objects (e.g., bicycles vs. cars, cars vs. trucks, scooters vs. pedestrians, etc.), uncertainty in the predicted future behavior of other objects, and / or other uncertainties.

[0068] These uncertainties can be particularly high in crowded environments, for example, where multiple environmental agents are present.

[0069] Additionally or alternatively, the first type of the second subset of simulations may function to account for one or more uncertainties in the risk analysis.

[0070] Additionally or alternatively, the set of simulations may include other simulations and / or types of simulations.

[0071] The second subset of simulations preferably further functions to determine which data from the first subset to retrieve for relevant comparison with data from the second subset. For example, (e.g., as shown in FIG. 3D, as described below) predicted positions of the ego-vehicle or other objects from the second subset of simulations may be used to find the same (or similar) positions in the first subset, and velocities corresponding to the same positions from different simulations may be retrieved and used in risk calculations (e.g., as described below). Additionally or alternatively, time may be referenced or other metrics may be retrieved and compared.

[0072] Additionally or alternatively, the second subset of simulations may include other simulations and / or be suitably performed in other ways.

[0073] 3.23 Method: Analyzing Simulation Results Step S220 S200 preferably further includes a step S220 of analyzing the simulation results, which functions to detect, characterize, and / or quantify some or all of the risks that the vehicle and / or other agents in the vehicle's environment (e.g., other vehicles, pedestrians, cyclists, etc.) may encounter. The output of this analysis preferably includes a set of risk metrics that can be used for ego-vehicle decision-making (e.g., policy selection) (e.g., in S300), but may additionally or alternatively be used to trigger further analysis (e.g., reward analysis) and / or be used in any other manner. In a preferred variation, for example, S220 functions to inform vehicle behavior in S300 (e.g., determining whether the vehicle needs to immediately respond to a risk, determining whether the risk can be avoided, etc.).

[0074] Analyzing the simulation results preferably includes determining whether a potential risk (e.g., a potentially hazardous event) may be encountered by the host vehicle (and / or environmental objects) in the future. The potential risk is preferably identified based on the results of at least a first subset of simulations, but may additionally or alternatively be identified based on other simulation types (e.g., a second subset of simulations). S220 may further include characterizing the type of potential risk (e.g., collision, near collision, collision involving death, injury, or property damage, property damage, traffic violation, etc.) and / or the expected severity of the potential risk, which objects are involved in the potential risk, which objects contribute to the potential risk (e.g., whether at fault), and / or any other characteristics.

[0075] Determining whether a potential risk has been detected and / or characterizing the potential risk and / or its severity is preferably performed based on the calculation of a set of risk metrics. In a preferred set of variations, the risk is detected and / or characterized through the calculation of a set of energy metrics, which represent the impact (e.g., total amount of energy, total amount of modified energy, total amount of work done on some environmental objects, etc.) of the ego-vehicle performing a particular action (also known as a policy). The impact may, for example, indicate the amount of energy introduced into the environment by the ego-vehicle performing a certain behavior and / or the amount of work required by the ego-vehicle and / or other objects to avoid a collision (or near-collision) and / or to respond to that behavior (e.g., slowing down to avoid a collision, stopping because a forward collision has blocked their path, etc.). Additionally or alternatively, the impact may reflect other scenarios or impacts.

[0076] The risk metric is preferably determined based on one or more energy metrics, such as kinetic energy and / or a modified (e.g., weighted, scaled) version of kinetic energy (e.g., derivative of velocity, velocity squared, velocity squared multiplied by a scaling factor, mass and / or velocity squared multiplied by a scaling factor, kinetic energy without mass, etc.). Additionally or alternatively, other energy metrics and / or work metrics and / or aggregated work and energy metrics can be calculated.

[0077] The energy metric may reflect actual energy values ​​(e.g., kinetic energy, potential energy, energy difference, etc.), modified energy values ​​(e.g., scaled kinetic energy, scaled potential energy, modified energy difference, etc.), relative energy values, work and / or modified work metrics (e.g., work performed by environmental agents in response to the ego-vehicle (e.g., slowing down, stopping, turning, etc.)), relative work metrics, and / or any other metrics.

[0078] For example, in a preferred variation, the energy metric includes a derivative of the simulated velocity values ​​and / or a derivative of the difference in simulated velocity values ​​(e.g., between simulation types). The derivative is preferably a second derivative (e.g., velocity squared, (velocity difference) squared, etc.), but may additionally be a first derivative of velocity, a derivative greater than the second derivative, a derivative of position, and / or any other parameter.

[0079] Preferably, one or more energy values ​​are calculated for each type of risk applied in the first subset of simulations (crash, conflict zone, interval, etc.) The calculation of the energy metric preferably differs for at least some of the risk types, but may alternatively be calculated uniformly.

[0080] For collision risk types that detect a collision in the first subset of simulations, the energy metric preferably reflects the energy of the collision (e.g., kinetic energy, scaled kinetic energy, etc.). This energy metric is preferably determined based on aggregating (e.g., adding, multiplying, etc.) the squared velocities of each object involved in the predicted collision at the time and / or location of the collision. This effectively represents, for example, a modification of the collision's kinetic energy. Optionally, it may be scaled (as shown in FIG. 3E by a factor "a") based, for example, on some or all of the angle between the orientation values ​​of the objects involved (e.g., a frontal collision will have a higher energy value than a rear-end collision, etc., as shown in FIG. 3A), the type of objects involved (e.g., a host vehicle colliding with a pedestrian will have a higher energy than other vehicles), the likelihood of an actual collision occurring, and / or any other factors (e.g., as described below). The velocity values ​​used in the calculations (e.g., as shown in Dataset 1_1 of FIG. 3D ) are preferably arbitrarily selected based on position information generated in a first subset of simulations and calculated from a second subset of simulations, but may alternatively be selected based on information from the first subset (e.g., the location of the collision, the velocity of the objects involved just before they begin to decrease and / or just before they drop to zero), and / or may be suitably selected in any other manner.

[0081] As an example, as shown in FIGS. 3A to 3F, a collision between a host vehicle (A_1) and another vehicle (EO_2) occurs at time t=t3 and position L=L in the first simulation. 17 The energy of the collision is determined by the velocity of these agents at the time and / or location of the collision (V 22 and V9). In a specific example, E in FIG. collision is (V 22 ) 2 +(V9) 2and then optionally scaled by a scaling factor "a" (or the scaling factor for each velocity squared term).

[0082] Non-collision energy metrics that may be calculated in determining the risk metric preferably indicate the likelihood that the ego-vehicle and / or other objects introduce energy into the environment, and are further preferably calculated if one or more other risk types are detected in the first type of simulation. These energy metrics are preferably associated with individual objects, but may also be associated with combinations of objects (e.g., all objects predicted to be present in the conflict zone simultaneously). The non-collision energy metrics may further function to account for uncertainties associated with the simulation, preventing the vehicle from frequently stopping or stalling while ensuring worst-case scenarios are considered.

[0083] These non-collision energy metrics (equivalently referred to herein as potential energy metrics and / or relative energy metrics) are preferably calculated based on differences in parameter values ​​(e.g., speed) between different types of simulations, and more preferably, energy metrics for objects other than the vehicle are calculated based on data from a first type (in which the vehicle is not present) of a first subset and a second subset of simulations, thereby representing the influence of the vehicle, and energy metrics for the vehicle are calculated based on data from a second type (in which only the vehicle is present) of a first subset and a second subset of simulations.

[0084] Analyzing differences between simulation types can reflect the relative risk of an object not responding appropriately to the actions of other objects. For example, comparing the speed (or other parameters) of a simulation in which all objects are present (first subset) with a simulation in which the ego-vehicle is not present can quantify the influence of objects other than the ego-vehicle reacting to the ego-vehicle (i.e., the impact of the ego-vehicle on other objects) (because the ego-vehicle is the only factor that can explain these differences). Similarly, comparing the speed (or other parameters) of a simulation in which all objects are present (first subset) with a simulation in which only the ego-vehicle is present can quantify the influence of the ego-vehicle reacting to objects other than the ego-vehicle (i.e., the impact of objects other than the ego-vehicle on the ego-vehicle) (because the other objects are the only difference in the outcome). This can further function to account for uncertainty about how objects will react (e.g., whether other objects will slow down appropriately, whether the drivers of other objects will pay attention, etc.), uncertainty in the simulation (e.g., what if there is an error in the simulation and an object thought to be next to the ego-vehicle actually cuts in front of the ego-vehicle), and / or other characteristics.

[0085] In some variations, for example, the potential energy associated with multiple objects being present in the conflict zone (e.g., simultaneously present, overlapping paths, etc.) is calculated by calculating the difference in squared velocity for each object in the conflict zone at its corresponding location (e.g., between a first subset of simulations and a second subset of simulations that includes that object) and optionally scaling this number. These potential energies can be aggregated in calculating an overall risk metric.

[0086] Alternatively, the velocity difference can be calculated first, then squared and optionally scaled.

[0087] Additionally or alternatively, in the conflict zone, the potential energy can be calculated using the velocity value from the first subset of simulations for each object (e.g., squared and multiplied by a scaling factor) rather than the squared velocity difference, which can be arbitrarily different from the collision metric by scaling this value (e.g., by the expected time difference for the objects to arrive at the same location, or by the distance between the objects once in the conflict zone, etc.).

[0088] Some variations use both the speed values ​​of a single simulation and the speed differences between simulation types to assess the risk of a conflict zone.

[0089] In the case of distance-related risks, the associated energy metric is preferably calculated based on the difference in the ego-vehicle's speed between the simulations of the first subset and the simulation of the second subset in which only the ego-vehicle is present. This reflects the amount of "energy" with which the ego-vehicle is moving, and in effect, represents how quickly it is moving towards objects with a small distance from the ego-vehicle (which may change in the future). This helps motivate the ego-vehicle (e.g., in S300) to select a policy that increases this distance.

[0090] In a preferred variation, this is calculated by taking the difference in the squared velocity of the ego vehicle between simulations and optionally scaling this number. Alternatively, the velocity difference can be scaled or some other parameter can be calculated. For objects that do not pose a risk, preferably no energy metric is calculated, although alternatively, the energy may be calculated and scaled down (e.g., to zero). Additionally or alternatively, for risks to objects that are not at fault other than the vehicle (e.g., objects following the vehicle, objects that do not have the right of way in certain situations, etc.), an energy metric for that object may be calculated and optionally scaled down (e.g., reduced, scaled to zero, etc.).

[0091] In a preferred variation, the risk metric (eg, at any time, any simulation / selection cycle, etc.) comprises a sum of energy values ​​(eg, scaled velocity derivatives) as shown in FIG. 3E.

[0092] Additionally or alternatively, the risk metric may include a maximum risk value and / or other metrics.

[0093] In some examples, risk metrics for each time point are calculated and collectively form a risk profile (e.g., as shown in Figure 3F), and in S300, information about how far into the future the risk is expected can be utilized (allowing risk mitigation rather than immediate response to the risk).

[0094] In one variation (e.g., as shown in FIG. 3E), risk metrics (e.g., for each time point, summed over all future time periods represented by the forward simulation, etc.) and / or risk profiles (e.g., as shown in FIG. 3F) are calculated based on a set of energy values, where the energy values ​​are calculated based on data from some or all of the set of simulations. These energy values ​​preferably include energy metrics resulting from some or all of the risk types (e.g., as described above), but may additionally or alternatively reflect other risk types and / or be calculated in other suitable ways.

[0095] In additional or alternative variations, the risk metric includes a set of effort values ​​and a set of aggregated energy values. For example, in one example, the risk metric includes any effort metrics occurring in the scenario (e.g., the effort required by the object to prevent the risk) and any aggregated calculated energy metrics (e.g., kinetic energy, scaled kinetic energy, etc.). In some examples, for example, a simulation may show that if the ego vehicle immediately takes an overly conservative action (e.g., immediately stopping due to a potential collision), it may actually increase the total energy of the entire system (e.g., the amount the ego vehicle must brake, the amount of effort other vehicles must expend to stop the ego vehicle, etc.) more than a potential hazardous event (e.g., the ego vehicle hitting a curb).

[0096] The risk metrics and / or components of the risk metrics (e.g., individual energy metrics) may optionally be scaled and / or weighted by any number of scaling factors (e.g., as shown by "a," "b," "c," "d," and "p" in FIG. 3E) to accurately reflect the most significant risks, enable comparisons between types of risk, and / or serve to guide the vehicle in S300 to most optimally respond to risks.

[0097] The scaling factor may optionally be determined based on the level (e.g., magnitude) of potential harm resulting from the risk. For example, the scaling factor may increase the magnitude of a risk metric for a collision of the ego-vehicle depending on the classification of the object type. For example, the scaling factor may increase the risk (e.g., energy metric “a”) of a collision between the ego-vehicle and a pedestrian, which is more likely to cause harm, compared to a collision between the ego-vehicle and a rigid object (e.g., another vehicle, a static object, etc.). For example, the softness of the object (e.g., generated through object classification by the ego-vehicle's perception subsystem) and / or the human-likeness of the object (e.g., pedestrian, vehicle with pedestrian, biker, number of humans expected to be involved, etc.) may be used to scale the risk metric (e.g., modified kinetic energy metric). Additionally or alternatively, any other features or information may be used to scale the energy metric, such as, without limitation, the predicted type of impact, the predicted location of the impact (e.g., rear-end, head-on, etc.), and / or other features. Additionally or alternatively, the predicted or identified mass of the object and / or other features of the object may be utilized.

[0098] The scaling factor may additionally or alternatively be determined based on driving rules (e.g., determined in the first subset of simulations) associated with the object's location and movement. For example, if the object is behind the ego-vehicle (e.g., within a spacing guideline, outside a spacing guideline (e.g., close following distance), etc.), the driving rules governing the following agent may scale down (or not calculate) the potential energy of the following object because the following agent would be at fault if it rear-ended the leading agent (and / or the following agent is likely to respond appropriately to the leading agent's behavior because the leading agent is within its direct line of sight). Additionally or alternatively, conventions associated with a four-way stop, common courtesy, or other scenarios may be used to measure the risk metric. Additionally or alternatively, objects may be excluded from further consideration based on these driving rules.

[0099] The scaling factor can additionally or alternatively be determined based on the collision angle or predicted collision between the objects, with more severe collision angles (e.g., T-shaped collisions, head-on collisions, etc.) increasing the associated risk value. In some variations, this applies to risks from collisions or conflict zones, but can additionally or alternatively apply to other risk types, all risk types, and / or any other type of risk. The collision angle is preferably determined based on (e.g., calculated correspondingly, etc.) the difference in heading angles between the objects (e.g., as shown in FIG. 3A ), but can additionally or alternatively be suitably calculated in other ways. In some variations, for example, an angle of 180 degrees (head-on collision) is scaled to maximize the associated term, and an angle of 0 degrees (or 360 degrees) is scaled to minimize the associated term, but any angle can scale the metric in any suitable manner.

[0100] The scaling factor may additionally or alternatively be determined based on the probability of occurrence of a risk (such as a collision risk, a non-collision risk, or a collective risk), which is preferably calculated based on data (e.g., position, speed, etc.) from a first subset of simulations, but may additionally or alternatively be determined based on a second subset of simulations, past simulations, past recorded data, current recorded data, the ego vehicle's current policy (e.g., following a leading vehicle), and / or any other data.

[0101] This probability can be scaled (or an additional metric can be calculated) to account for uncertainty about where other objects are located. This can represent, for example, the likelihood that an object is actually in the path of another object (e.g., the ego-vehicle). If the probability is high (a reliable prediction), the simulated reaction is likely to occur. If this probability is low (e.g., the other object is at the edge of a lane, or entering or exiting a lane), the associated energy metric (how the ego-vehicle or other objects will actually react) may be less reliable. For example, in such a scenario, the other object may not actually be in the ego-vehicle's way, allowing the ego-vehicle to eventually pass safely (effectively reducing the risk).

[0102] In some examples, for example, the potential energy of the ego vehicle may be scaled to reflect the likelihood that the ego vehicle actually has that potential energy, taking into account uncertainty in its environment (e.g., how many other objects are present, where the other objects are located, how fast the other objects are traveling, etc.) The potential energies of other objects may also be calculated additionally or alternatively.

[0103] In a specific example, if the host vehicle is following an object ahead, the potential energy may be more reliable than if the object is snaking in and out of lanes.

[0104] Collision risk can additionally or alternatively be used to identify the likelihood of a collision and scale the energy of the collision.

[0105] The scaling factor may additionally or alternatively be scaled by one or more distance values, such as the simulated distance between objects (inside the conflict zone, outside the conflict zone, during a chase, etc.), with smaller distance values ​​indicating a smaller risk posed by the potential risk.

[0106] The scaling factor may additionally or alternatively be scaled by one or more temporal factors, such as the time difference between objects (e.g., the time difference between when objects are predicted to be in the same location in the conflict zone, the time difference between a leading object and a trailing object, etc.), the time when a risk is predicted to occur, and / or any other temporal parameter; the temporal factors may be determined based on a simulation, based on a risk profile, and / or in any other suitable manner.

[0107] Additionally or alternatively, the scalar coefficients may be determined based on other features or information.

[0108] S220 may optionally include a step of determining whether the detected potentially dangerous event can be prevented and / or mitigated (e.g., reasonably prevented), such as through one or more actions of other environmental objects and / or the host vehicle. This is a function that determines whether the host vehicle needs to respond immediately to a potential risk or whether there is time to wait and see how the environment changes (how environmental objects move). This may be integrated into a scaling factor, taken into account in S300 through a risk profile, or utilized in any other suitable manner.

[0109] The preventability of a risk is preferably determined at least in part based on the time associated with the risk (e.g., comparison to a threshold, scaling the risk, etc.), but may additionally or alternatively be determined based on calculating the predictive ability of other objects to stop or otherwise react (e.g., stopping time, braking limits, etc.).

[0110] For example, in some variations, the preventability of risk from a conflict zone is calculated based on the time difference between the positions of objects within the conflict zone.

[0111] Additionally or alternatively, determining whether a risk can be prevented and / or mitigated may include calculating values ​​of work metrics (e.g., work, force multiplied by displacement, modified work, scaled and / or weighted work, relative work, etc.) that the environmental objects and / or the ego-vehicle must exert / perform individually and / or collectively (e.g., by all environmental agents, by all environmental agents and the ego-vehicle, etc.) to prevent the identified event from occurring. The work metrics preferably have the same units as the energy metrics (e.g., as described above), thereby allowing the work metrics and energy metrics to be partially or completely aggregated, compared, and / or otherwise used. Based on these work calculations, other parameters (e.g., vehicle control commands and temporal parameters) may be calculated and compared to thresholds (e.g., predetermined thresholds, class label-specific thresholds, etc.) to determine the extent to which the identified event can be avoided.

[0112] In one example, for example, when a potential hazardous event is detected, the amount of work required by each involved object (e.g., environmental agent, ego-vehicle, etc.) to avoid the hazardous event is calculated (e.g., based on a first subset of simulations, based on both the first and second subsets of simulations, etc.). Based on these work metrics, the level of braking required to perform this work can be calculated for each involved vehicle (e.g., car, motorcycle, etc.) and compared to one or more braking thresholds (e.g., braking force, braking magnitude, etc.) to determine whether and / or to what extent the vehicle can stop and avoid the event (e.g., if the level of braking is below a predetermined threshold, the vehicle is determined to be able to stop; if the level of braking is above a predetermined threshold, the vehicle is determined to be unable to stop, etc.). In addition to or instead of braking level, metrics such as, but not limited to, braking rate, braking distance, stopping distance, acceleration and / or deceleration rate, reaction time (e.g., compared to average human reaction time), turning effectiveness (e.g., whether turning can be performed to avoid the event), and / or other metrics can be calculated.

[0113] The amount of work can optionally be calculated in a relative manner, such as the difference between the work in a first subset of simulations and the work in a second subset of simulations, and / or can be calculated in any other manner.

[0114] Additionally or alternatively, the risk metric may be calculated in other suitable ways.

[0115] S220 may optionally include aggregating the energy metrics and work metrics to calculate an overall energy associated with the potential hazardous event (e.g., an aggregation of the kinetic energy of the collision and the work required for vehicles not involved in the collision to stop or slow down due to the collision). Additionally or alternatively, some or all of the metrics may be aggregated and / or compared with metrics from a third subset of simulations (e.g., an aggregation of kinetic energy with work energy and / or potential energy, etc.).

[0116] Additionally or alternatively, determining whether a potentially hazardous event can be prevented may include comparing energy metrics and / or work metrics (e.g., corrected kinetic energy, relative corrected kinetic energy, work, aggregated work and energy, etc.) to one or more predetermined thresholds and / or any other processing.

[0117] S220 may optionally include discounting some or all metrics (e.g., energy metrics, work metrics, overall cost metrics, etc.) if it is determined that a potentially dangerous event can be avoided (e.g., by action of the ego vehicle, by action of another agent, by action of the ego vehicle and another agent, etc.). This functions to discount risk in the ego vehicle's decision-making (e.g., not responding to risk immediately, waiting for additional information, etc.). Additionally or alternatively, S220 may include triggering a consequence that is used in the ego vehicle's decision-making.

[0118] For example, in one set of embodiments, if a risk is determined to be close enough in the future (e.g., according to a set of criteria and / or thresholds) that it cannot be avoided and / or cannot be avoided in an acceptably manner, then the risk may be persisted in the vehicle's decision-making (e.g., with a large negative score such that in S300, the relevant risk policy is not selected from among all risk policies unless it is a best-case risk (e.g., with the lowest corrected kinetic energy)), and if the risk is determined to be far enough in the future that the vehicle and / or environmental objects can react (e.g., in an acceptably manner), then the risk may be ignored and / or downweighted in the vehicle's decision-making (e.g., thereby preventing the vehicle from behaving in an overly conservative and / or premature manner).

[0119] In a first variation of S200 (e.g., as shown in FIGS. 3A-3F ), ​​a collision is detected in a first subset of simulations (between A_1 and EO_2), and a corrected kinetic energy of the collision (e.g., relative energy transfer between objects with conservation of momentum) is calculated to contribute to the overall risk. This corrected kinetic energy is preferably calculated based on velocity values ​​(e.g., the sum of the squared velocities of the involved objects) from the first subset of simulations of the objects involved in the collision, and may further be determined based on (e.g., scaled based on) the angle and / or orientation of the objects (e.g., the orientation of the objects at / before the collision), the type of the involved objects, and / or any other characteristics. For an object (EO_1) behind the ego-vehicle, considering driving rules / practices, it may be determined that since EO_1 is behind the ego-vehicle (and not within the conflict zone), its potential energy does not need to be calculated (or can be calculated and scaled down). Alternatively, since EO_1 is about to enter the conflict zone, its potential energy may be calculated. Since EO_n is within the conflict zone, we can calculate its potential energy (optionally scaling it to reflect that EO_n is a pedestrian). In the example in Figures 3A-3B, the pedestrian has the same velocity between simulations at each location, so the potential energy is zero.

[0120] Additionally, potential energies can be calculated to reflect that A_1 and / or EO_2 are in the same conflict zone (e.g., no collision but a near-miss collision occurs).

[0121] In calculating the potential energy, the object's position is preferably determined using the second subset of simulations that generated datasets 2_1 and 2_2, and this position is used to find the corresponding position in the data of the first subset of simulations (dataset 1_1) (highlighted in grey), allowing the velocity difference corresponding to the same position to be calculated.

[0122] These energies can then be summed together and / or over time to generate a parameter or set of parameters (e.g., as shown in Figure 3E), a risk profile (e.g., as shown in Figure 3F), or any combination of information. Additionally or alternatively, S200 may include other processes and / or be performed in other ways as appropriate.

[0123] 3.3 Method - Step S300 of Operating the Ego Vehicle Based on the Assessed Risk Method 200 may optionally include a step S300 of operating the ego-vehicle based on the assessed risk, which S300 functions to optimally respond to the risk based on the characteristics of the risk characterized in S200. Additionally or alternatively, S300 may perform other functions.

[0124] S300 is preferably executed during the ego vehicle's multi-policy decision-making process (e.g., as described above), but may additionally or alternatively be executed according to any other decision-making process of the ego vehicle. In a preferred variation, for example, S300 includes selecting a policy from a set of policy options for the ego vehicle based on an assessed risk obtained from a set of simulations performed for each of the policy options.

[0125] S300 optionally includes a step of calculating a reward metric, which functions to evaluate how close the vehicle is to achieving a goal (e.g., reducing the distance to a destination, following traffic rules, etc.), and which can optionally be incorporated into the vehicle's decision-making (e.g., policy selection).

[0126] In a first set of variations, the decision-making in S300 is performed hierarchically (e.g., a decision tree), and the reward metric is calculated only if no risk is detected in S200 and / or if a risk detected in S200 can be avoided. In a set of examples, if no potentially dangerous event is detected, other lower-level events (e.g., legal risk events, comfort risk events, delay risk events, etc.) may be considered (e.g., before the reward metric).

[0127] In a second set of variations, the reward metric can be aggregated with one or more risk metrics (e.g., total energy, modified kinetic energy, modified kinetic energy aggregated with work, etc.) to determine an overall score for each policy. S300 preferably includes the steps of selecting a policy (eg, action, behavior, etc.) for the ego-vehicle based on a risk and / or reward metric, and steering the vehicle in accordance with the policy.

[0128] This may include, for example, assessing risk throughout the simulation / policy deployment, such as aggregating risk across all time steps, discounting risk in future time steps if the object can sufficiently brake or otherwise avoid the risk, deferring risk to the future (e.g., by slowing down), and / or otherwise analyzing risk.

[0129] For example, the S300 can leverage various simulation rollouts to estimate whether the likelihood of a collision is increasing or decreasing based on what the vehicle is doing or about to do (e.g., by looking at past rollouts and analyzing whether the risk is increasing or decreasing), and then select a policy based on that (e.g., changing to a new policy if the risk continues to increase with the same policy, or maintaining the same policy if the risk continues to decrease).

[0130] Additionally or alternatively, a selection can be made by appropriately comparing the risks among various policy options (e.g., as shown in Figure 4).

[0131] Additionally or alternatively, S300 may include any other suitable process.

[0132] Method 200 may additionally or alternatively include any other processes, including, but not limited to, repeating some or all of the processes described above (e.g., to determine whether an avoidable risk develops into an unavoidable risk, to determine whether an avoidable risk disappears, etc.), and / or any other suitable processes.

[0133] Although omitted for simplicity, preferred embodiments include all combinations and permutations of the various system components and various method processes, which may be performed in any suitable order, sequentially or simultaneously.

[0134] Embodiments of the systems and / or methods may include any combination and permutation of the various system components and various method processes, and one or more instances of the methods and / or processes described herein may be performed asynchronously (e.g., serially), simultaneously (e.g., simultaneously, in parallel, etc.), or in any other suitable order by and / or using one or more instances of the systems, elements, and / or entities described herein. The following system and / or method components and / or processes may additionally, alternatively, or otherwise be integrated with all or a portion of the systems and / or methods disclosed in the above-referenced applications, each of which is incorporated by this reference in its entirety.

[0135] In additional or alternative embodiments, the methods and / or processing modules described above are implemented in a non-transitory computer-readable medium storing computer-readable instructions. The instructions may be executed by computer-executable components integrated with the computer-readable medium and / or processing system. The computer-readable medium may include any suitable computer-readable medium, such as RAM, ROM, flash memory, EEPROM, optical devices (CD or DVD), hard drives, floppy drives, non-transitory computer-readable media, or any suitable device. The computer-executable components may include a computing system and / or processing system (e.g., including one or more co-located or distributed, remote or local processors) connected to the non-transitory computer-readable medium, such as a CPU, GPU, TPUS, microprocessor, or ASIC, although the instructions may alternatively or additionally be executed by any suitable dedicated hardware device.

[0136] Those skilled in the art will recognize from the foregoing detailed description and drawings and the appended claims that modifications and variations can be made to the preferred embodiments of the invention without departing from the scope of the invention as defined in the appended claims.

Claims

1. 1. A method for assessing risk in a vehicle, comprising: During each set of times while the vehicle is in operation, collecting a dataset of environmental information; identifying a set of environmental objects in an environment of the vehicle based on the dataset; simulating a set of future scenarios based on the data set, the set of future scenarios comprising: a first scenario representing the environment; and a second scenario representing a first change in the environment; and a third scenario representing a second change in the environment; and identifying a set of risks based on the first scenario; calculating a set of metrics based on the set of risks and a set of velocity metrics associated with the set of future scenarios, calculating a first set of metrics based on the first scenario; calculating a second set of metrics based on the first scenario and at least one of the second or third scenarios; and controlling the vehicle according to the set of metrics.

2. 2. The method of claim 1, wherein the set of risks includes multiple categories of risk, the multiple categories of risk including a first category of risk, the first category of risk including multiple objects of a set of objects, the set of objects including the vehicle and the set of environmental objects.

3. The method of claim 2 , wherein the first category of risk includes a crash simulated in the first scenario.

4. 3. The method of claim 2, wherein the plurality of categories of risk further includes a second category of risk, and wherein a metric in the second set of metrics calculated based on the second category is determined based only on behavior of a single object in the set of objects.

5. The method of claim 4 , wherein the second category of risk is not associated with a simulated crash.

6. The second set of metrics comprises: a first subset of metrics, the first subset of metrics being determined based on speed metrics from the first scenario and the second scenario; and a second subset of metrics, the second subset of metrics being determined based on speed metrics from the first scenario and the third scenario.

7. 7. The method of claim 6, wherein the second scenario and the third scenario differ in which objects of a set of objects are represented in each scenario, the set of objects including the vehicle and the set of environmental objects.

8. The method of claim 7 , wherein in the second scenario, the vehicle is removed from the environment, and in the third scenario, the set of environmental objects is removed from the environment.

9. The method of claim 1 , wherein the method is performed according to a single policy option of a set of policy options for the vehicle.

10. 10. The method of claim 9, further comprising repeating the method for each remaining policy option in the set of the plurality of policy options, thereby generating a plurality of sets of aggregated risks, wherein the plurality of sets of aggregated risks comprises aggregated risks.

11. 11. The method of claim 10, further comprising the steps of selecting a primary policy from the set of policy options based on the set of aggregated risks, and controlling the vehicle in accordance with the primary policy.

12. The method of claim 1 , wherein at least one of the plurality of categories of risk is identified based on a predetermined set of zones of a labeled map.

13. 13. The method of claim 12, wherein the category of risk is further identified based on a simulated co-location of the vehicle and one of the set of environmental objects in one of the predetermined set of zones during the first scenario.

14. Further comprising applying a scaling factor to at least one of the set of metrics, the scaling factor being: a set of uncertainty values ​​generated in a simulation of the first scenario; a set of classifications associated with the set of environmental objects; a predetermined set of driving rules.

15. A system for assessing risk to a vehicle, comprising: a set of sensors mounted on the vehicle; a set of processors in communication with the set of sensors, the set of processors comprising: During each of a set of times while the vehicle is in operation, collecting a dataset of environmental information from the set of sensors; Identifying a set of environmental objects within an environment of the vehicle based on the dataset; a simulator implemented by the set of processors simulating a set of future scenarios based on the dataset, each of the set of future scenarios including a set of time points after the time, the set of future scenarios including: a first set of scenarios representing the environment; and a second set of scenarios representing one or more changes in the environment; Identifying a set of risks based on the first scenario; calculating a set of metrics based on the set of risks, wherein the calculation of the set of metrics comprises: calculating the first set of metrics based on the first set of scenarios; calculating a second set of metrics based on at least one of the first set of scenarios and the second set of scenarios; aggregating the first and second sets of metrics to generate an aggregated risk; A system that performs vehicle operation in accordance with the aggregated risks.

16. The system of claim 15 , wherein the vehicle is an autonomous vehicle.

17. 16. The system of claim 15, wherein the set of metrics is calculated based on a set of velocity metrics associated with the set of future scenarios, each of the set of velocity metrics comprising a second derivative of one of the set of velocity values.

18. The system of claim 17 , wherein a portion of the set of metrics is further calculated based on a difference in a speed metric between two different scenarios of the set of future scenarios.

19. The system of claim 15 , wherein the aggregated risk comprises a potential risk profile.

20. the second set of scenarios includes a first scenario and a second scenario; In the first scenario, the movement of only the set of environmental objects is simulated; The system of claim 15 , wherein in the second scenario, movement of only the vehicle is simulated.

21. While driving the vehicle, identifying a set of environmental objects within an environment of the vehicle based on a dataset of environmental information; simulating a plurality of future scenarios based on the dataset, the plurality of future scenarios comprising: a first set of scenarios representing the environment; and a second set of scenarios, each representing a respective change in the environment; identifying a set of risks based on the first set of scenarios; calculating a set of metrics based on the set of risks, the second set of scenarios, and a set of velocity metrics associated with the plurality of future scenarios; and controlling the vehicle based on the set of metrics.

22. 22. The method of claim 21, wherein the second set of scenarios includes a first scenario and a second scenario, and in the first scenario, movement of only the set of environmental objects is simulated.

23. 23. The method of claim 22, wherein in the second scenario, movement of only the vehicle is simulated.

24. 22. The method of claim 21, wherein the set of risks includes multiple categories of risk, the multiple categories of risk including a first category of risk, the first category of risk including multiple objects of a set of objects, the set of objects including the vehicle and the set of environmental objects.

25. The method of claim 24 , wherein the first category of risk includes a crash simulated in the first set of scenarios.

26. 25. The method of claim 24, wherein the plurality of categories of risk further includes a second category of risk, and wherein a metric in the set of metrics is calculated based on the second category of risk, and wherein the metric is determined based on only the movement of a single object in the set of objects.

27. The method of claim 6 , wherein the second category of risk is not associated with a simulated crash.

28. The method of claim 21 , wherein the method is performed according to a single policy option of a set of policy options for the vehicle.

29. The method of claim 21 , wherein the vehicle is an autonomous vehicle.

30. 22. The method of claim 21, wherein the set of metrics comprises a potential risk profile.

31. 1. A method for a vehicle, comprising: identifying a set of environmental objects within the vehicle's environment based on a dataset of environmental information; simulating a plurality of future scenarios based on the dataset, the future scenarios including a set of change scenarios representing respective changes in the environment; identifying a set of risks based on one of the plurality of future scenarios; calculating a set of metrics based on the set of risks, the set of change scenarios, and a set of velocity metrics associated with the plurality of future scenarios; and controlling the vehicle based on the set of metrics.

32. 32. The method of claim 31 , wherein at least one of the set of risks is identified based on a labeled map.

33. 32. The method of claim 31 , wherein at least one of the risks is further identified based on the vehicle and an environmental object from the set of environmental objects being simulated in the same location during the scenario.

34. 32. The method of claim 31 , wherein the set of risks includes multiple categories of risk, the multiple categories of risk including a first category of risk, the first category of risk including multiple objects of a set of objects, the set of objects including the vehicle and the set of environmental objects.

35. 35. The method of claim 34, wherein the first category of risk includes a crash simulated in the first set of scenarios.

36. 35. The method of claim 34, wherein the plurality of categories of risk further includes a second category of risk, and wherein a metric in the set of metrics is calculated based on the second category of risk, and wherein the metric is determined based on only movement of a single object in the set of objects.

37. The method of claim 16 , wherein the second category of risk is not associated with a simulated crash.

38. 32. The method of claim 31 , further comprising applying a scaling factor to at least one of the set of metrics, the scaling factor being determined based on a set of uncertainty values ​​generated in simulating the scenario.

39. 32. The method of claim 31, further comprising applying a scaling factor to at least one of the set of metrics, the scaling factor being determined based on a set of classifications associated with the set of environmental objects.

40. The method of claim 11 , further comprising applying a scaling factor to at least one of the set of metrics, the scaling factor being determined based on a set of predetermined driving rules.

Images