Methods and systems for expanding the operational design domain of autonomous agents

Abstract

A system for extending an operational design domain (ODD) of an autonomous agent includes a decision-making platform (equivalently referred to herein as a decision-making architecture). A method for extending an operational design domain (ODD) includes determining a decision-making architecture for a first domain and adapting the decision-making architecture to a second domain. Additionally or alternatively, the method 200 can include implementing the decision-making architecture S300 and / or any other process.

Description

Technical Field

[0001] Cross - Reference to Related Applications This application claims the benefit of U.S. Provisional Application No. 63 / 290,531, filed Dec. 16, 2021, and U.S. Provisional Application No. 63 / 316,108, filed Mar. 3, 2022, each of which is hereby incorporated by reference in its entirety.

[0002] The present invention generally relates to the field of autonomous vehicles, and more particularly, to new and useful systems and methods for expanding the operational design domain of autonomous agents in the field of autonomous vehicles.

Background Art

[0003] Making safe and effective decisions in autonomous vehicles is a difficult task. This type of decision - making requires an understanding of the current environment around the vehicle, the evolution of this environment into the future, and a safe and continuous progression towards a predefined driving goal. All decisions must be continuously constrained by both the driving rules of the road and human driving habits, which, combined with a vast number of possible interactions, make decision - making a very complex and difficult problem for autonomous systems. Even when a decision - making system can be appropriately provided, when scaling the area of deployment, the decision - making systems in conventional systems perform insufficient generalization, which can lead to insufficient vehicle performance, reliability issues, and safety risks. Conventional approaches to alleviating this problem are to recollect large amounts of labeled or partially labeled data having the same distribution as the test data and then train a machine - learning model with respect to the new data. However, many factors prevent easy access to such data, which among other limitations, leads to high annotation costs, increased time to productization, and limited area scaling.

[0004] Therefore, in the field of autonomous vehicles, there is a need to create improved and useful systems and methods to expand the operational design domain of autonomous agents in the field of autonomous vehicles. [Brief explanation of the drawing]

[0005] [Figure 1] Figure 1 is a schematic diagram of a system for expanding the operational design domain of autonomous agents. [Figure 2] Figure 2 is a schematic diagram of a method for expanding the operational design domain of autonomous agents. [Figure 3] Figure 3 shows a schematic variation of a method for extending the operational design domain of autonomous agents. [Figure 4] Figure 4 shows a modified form of the decision-making platform. [Figure 5] Figure 5 shows the deformation forms of a set of models in the decision-making platform. [Figure 6] Figure 6 shows a schematic variation of a method for expanding the operational design domain of autonomous agents. [Figure 7] Figure 7 shows a schematic example of a part of a method for extending the operational design domain of autonomous agents. [Figure 8] Figure 8 shows a modified form of a method for expanding the operational design domain of autonomous agents. [Modes for carrying out the invention]

[0006] The following description of preferred embodiments of the present invention is not intended to limit the invention to these preferred embodiments, but rather to enable those skilled in the art to manufacture and use the invention.

[0007] 1. Overview As shown in Figure 1, System 100 for extending the operational design domain (ODD) of an autonomous agent includes a decision-making platform (hereinafter equivalently referred to as a decision-making architecture). Additionally or alternatively, System 100 may include and / or interface with any or all of the following components: a set of computing subsystems (e.g., a set of computers, a set of software modules and / or task blocks implemented in the set of computers) and / or processing subsystems (e.g., a set of processors, a set of processing modules and / or task blocks implemented in the set of processors), a set of sensors, a control subsystem, an actuation subsystem (e.g., a drive-by-wire subsystem), a communication subsystem (e.g., for communicating with other agents, remote computing subsystems, remote operator platforms, etc.), an autonomous agent, and / or any other suitable components. In addition or alternatively, system 100 may include and / or interface with any or all of the components described in any or all of U.S. applications 17 / 116,810 filed on 9 December 2020, 17 December 2020, 17 December 2020, 17 December 2020, 18 December 2020, and 17 / 962,459 filed on 7 October 2022, each of which is incorporated in its entirety by this reference.

[0008] As shown in Figure 2, Method 200 for extending the Operational Design Domain (ODD) includes S100 determining a decision-making architecture for a first domain and S200 adapting that decision-making architecture to a second domain. Additionally or alternatively, Method 200 may include S300 implementing the decision-making architecture and / or any other processes. In addition or alternatively, Method 200 may include and / or interface with any or all of the processes described in any or all of U.S. applications 17 / 116,810 filed on 9 December 2020, 17 / 125,668 filed on 17 December 2020, 17 / 127,599 filed on 18 December 2020, and 17 / 962,459 filed on 7 October 2022, each of which is incorporated in its entirety by this reference, or any or all of any other preferred processes carried out in any preferred order. Method 200 may be carried out by the system 100 described above and / or by any other preferred system.

[0009] 2.Profit Systems and methods for extending the ODD of autonomous agents can offer several advantages over current systems and methods.

[0010] In a first variant, the technique benefits from reducing the time and / or amount of data required to train one or more models in the decision-making architecture of an autonomous agent when extending the operational design domain (ODD) associated with the autonomous agent. In a preferred variant, the method and / or system leverages data and training from previous training of the decision-making architecture to adapt it for use in other scenarios (e.g., routes, contexts, actions, etc.) through transfer learning and / or domain adaptation, which enables the autonomous agent to constantly accelerate the learning of new tasks based on knowledge learned from previously demonstrated tasks, thereby reducing the amount of demonstration and and therefore the time required to generate a decision-making architecture that is adequately prepared to handle (one or more) new tasks.

[0011] In a second variant, an addition to the first variant, the technique provides the benefit of preventing and / or minimizing the effects of negative transfers (e.g., leading to decision-making errors made by the vehicle) or other performance degradations when adapting the decision-making architecture to a new domain. In a particular set of examples, implementing a compact latent space representation that extracts environmental features of a vehicle, where the compact latent space representation acts as input to a modular set of models in the decision-making architecture, enables an efficient and accurate task mapping process to be implemented from source domain tasks to target domain tasks, and the inventors have found that this prevents or minimizes the effects of negative transfers.

[0012] In a third variant, an addition or alternative to the variants described above, the technique benefits by enabling the optimal selection of a particular model to be used as a starting point for determining a new model for a new operational design domain, which can increase the effectiveness of transferring learning to the new model, improve the performance of the new model, reduce the amount of data and / or time required to generate the new model, and / or provide any other desirable benefits. In the set of examples, the optimal selection of a particular model is carried out by a manual process (e.g., a rule-based process), which allows the use of a particular model that is most suitable (e.g., most closely related) to determining the new model. As an addition or alternative, the models of the vehicle decision-making architecture are arranged in a modular, sequential arrangement (e.g., a first set of models is evaluated before the selection of a second set of models and informs the selection of the second set of models), which allows only the most relevant models to be identified and used when determining the new set of models (e.g., those most closely related to a new context based on shared features, those most closely related to a new action based on shared features, etc.). This may be in contrast to conventional systems and methods that utilize end-to-end deep learning models, which often result in negative transfers due to the lack of concrete and / or measurable similarities between the old model use case (source model) and the new model use case (target model).

[0013] In a fourth variant, which is an addition to or alternative to the variants described above, the technology extends the set of routes (e.g., fixed routes) that a vehicle is configured to travel through (e.g., autonomously travel through) (e.g., is capable of doing so, is qualified to do so, is permitted to do so, etc.), extends the types of routes that a vehicle is configured to travel through (e.g., fixed routes, dynamically determined routes, etc.) (e.g., route length, route locations, starting points of (one or more) routes, destinations of (one or more) routes, etc.), and extends the number and / or types of contexts (e.g., scenarios) to which a vehicle can respond (e.g., one or more) (e.g., road type, number of lanes on a road, intersection type, traffic sign type, zone type [e.g., school zone, residential zone, highway, roadway, etc.], etc.). The benefits of efficiently and robustly expanding the operational design domain of a set of autonomous vehicles are provided by any or all of the following: expanding the number and / or types of actions (e.g., merging, slowing down, parking, loading, unloading, lane changes, etc.) that a vehicle can perform; expanding the number and / or types of trajectories that a vehicle can perform; expanding the number and / or types of driving conditions (e.g., weather conditions, lighting conditions, traffic conditions, pedestrian congestion, cyclist congestion, etc.) that a vehicle can operate under; expanding the performance conditions (e.g., sensor performance, number and / or type of sensors, sensor degradation level, etc.) that a vehicle can reliably operate under; expanding the use cases in which a vehicle can be used (e.g., goods delivery, passenger transport, dynamic route use cases, etc.); and / or separately expanding the operational design domain of a vehicle.

[0014] In addition or as an alternative, this system and method may provide any other benefits.

[0015] 3. System 100 As shown in Figure 1, System 100 for extending the operational design domain (ODD) of an autonomous agent (hereinafter equivalently referred to herein as autonomous vehicle, self-agent, self-vehicle, or vehicle) preferably includes a decision-making platform (hereinafter equivalently referred to herein as a decision-making architecture). Additionally or alternatively, System 100 may include and / or interface with any or all of the following components: a set of computing subsystems (e.g., a set of computers, a set of software modules and / or task blocks implemented in the set of computers) and / or processing subsystems (e.g., a set of processors, a set of processing modules and / or task blocks implemented in the set of processors), a set of sensors, a control subsystem, an actuation subsystem (e.g., a drive-by-wire subsystem), a communication subsystem (e.g., for communicating with other agents, remote computing subsystems, remote operator platforms, etc.), an autonomous agent, and / or any other suitable components. In addition or alternatively, system 100 may include and / or interface with any or all of the components described in any or all of U.S. applications 17 / 116,810 filed on 9 December 2020, 17 December 2020, 17 December 2020, 17 December 2020, 18 December 2020, and 17 / 962,459 filed on 7 October 2022, each of which is incorporated in its entirety by this reference.

[0016] System 100 includes and / or implements a decision-making architecture that functions to perform decision-making for an autonomous vehicle through any or all of the high-level actions and / or behavioral decisions, route planning, operation planning, trajectory planning, and / or any other decision-making processes for operating an autonomous agent.

[0017] System 100 may, in addition or alternatively, function to control autonomous agents, perform cognition for autonomous agents, perform prediction for autonomous agents, and / or otherwise enable autonomous agents to operate (e.g., drive safely, drive optimally, etc.). Further additional or alternatively, System 100 may function to enable updates to the decision-making architecture and / or any other processes.

[0018] At least a portion of the decision-making architecture used by the autonomous agent is preferably trained (hereinafter equivalently referred to as learned). In preferred variants, for example, the decision-making architecture includes one or more sets of trained models, including, but not limited to, machine learning models, deep learning models (e.g., neural networks, deep neural networks, convolutional neural networks, etc.), and / or any combination. In preferred variants (for example, described below), the decision-making architecture includes one or more sets of trained micromodels (hereinafter equivalently referred to as learning modules, learning models, etc.) that are selected and evaluated in a modular manner in response to a particular scenario (e.g., context) encountered by the autonomous agent and optionally, the output from previously evaluated micromodels. Variations of this decision-making architecture are described in any or all of U.S. applications 17 / 116,810 filed on 9 December 2020, 17 / 125,668 filed on 17 December 2020, 17 / 127,599 filed on 18 December 2020, and 17 / 962,459 filed on 7 October 2022, each of which is incorporated in its entirety by this reference. In addition or alternatively, the decision-making architecture may include and / or implement any other trained processes.

[0019] The modular and / or continuous nature of the trained models can provide numerous benefits in efficiently and optimally extending the operational design domain of vehicles. In a preferred variant (for example, further described below), each of the first set of models is associated with a specific context (and optionally a specific context of a particular fixed route), and each of the second set of models is associated with a specific action (determined, for example, based on evaluating one or more of the first set of models), and the models selected for evaluation in the first and second sets are made by a rule-based process (e.g., predetermined mappings, lookup tables, etc.). In contrast to an end-to-end decision-making architecture, for example, models from the first and second sets can be selected in a logical and explainable way for determining new models (for example, in the task mapping process described below). For example, if a new route is added and that new route includes a new context (e.g., a scenario), a model from the first set most relevant to that new context may be selected and used (e.g., exclusively) as a starting point for the new context model (e.g., providing an initial set of weights). This can also prevent the negative transfer effects (e.g., poor performance and / or unexpected output from the new model) that can occur if the source model (the model used to train the new model) is not explainably relevant to the use case of the new model.

[0020] Each of the models is preferably in the form of, and / or includes, a machine learning model, more preferably one or more neural networks and / or network models (e.g., deep Q-learning network, convolutional neural network [CNN], inverse reinforcement learning [IRL] model, reinforcement learning [RL] model, imitation learning [IL] model, etc.), but can additionally or alternatively include any other suitable model, algorithm, decision tree, lookup table, and / or other tool.

[0021] Each of the models is more preferably trained with inverse reinforcement learning, which functions to determine a reward function and / or an optimal operation policy for each of the context-aware learning modules. The output of this training is more preferably a compact fully-connected network model representing the reward function and the optimal policy for each learning module. Additionally or alternatively, the learning modules can be appropriately trained and / or implemented separately (e.g., with reinforcement learning, etc.).

[0022] In a first set of variant forms, the decision-making architecture includes and / or defines a first subset of models (e.g., the deep decision network shown in FIGS. 4 and 5) that function to implement a first part of the decision-making of the autonomous agent, and a second subset of models (e.g., the deep trajectory network shown in FIG. 4) that function to implement a second part of the decision-making. Additionally or alternatively, the decision-making architecture can include any other subset of models, a single subset of models, and / or any other combination of models.

[0023] Each of these subsets of models is preferably, equivalently herein, referred to as a modular set of models, comprising a plurality of models (e.g., a plurality of deep decision networks in a first subset, a plurality of deep trajectory networks in a second subset), and a single model from each subset is preferably selected at each point in time during vehicle decision-making (e.g., at a predetermined frequency during the operation of an autonomous vehicle, continuously during the operation of an autonomous vehicle, etc.). Additionally or alternatively, any or all of the subsets can include a single model, the set of models can include any other subset of models, a plurality of models can be selected from any or all of the subsets, and / or the decision-making architecture can be otherwise configured and / or orchestrated.

[0024] The subsets of models are more preferably related to the order of evaluation and / or other interfacing (e.g., communication) between subsets, and the output from one or more subsets is used in the selection of the model from the next subset to be evaluated. In a set of variant forms, for example, the output(s) of the model selected in a first subset of models (e.g., the selected deep decision network) is used to select which model from a second subset of models is to be evaluated next (e.g., the selected deep trajectory network). Additionally or alternatively, the subsets of models can be otherwise related, unrelated, and / or otherwise appropriately evaluated.

[0025] In a particular set of examples, a first subset of the model includes a set of deep decision networks (DDNs), one or more of which (e.g., single, multiple, etc.) are selected based on the vehicle's current context, which are responsible for making decisions about the actions an autonomous agent should take during the current planning cycle (for example, as described in U.S. Patent Application No. 17 / 125,668 filed December 17, 2020, which is incorporated herein by this reference in its entirety). A second subset of the model includes a set of deep trajectory networks (DTNs) which are selected, optimized, and / or safely constrained based on a particular action. Based on the action determined by the selected DDN, the corresponding DTN is selected and used (e.g., along with a localized view around the vehicle) to plan a safe, effective, and natural trajectory that the vehicle should follow.

[0026] The input to any or all of the set of models preferably includes environmental information related to the autonomous agent, thereby enabling the agent to utilize its awareness of its environment in its decision-making. This awareness preferably includes information from the current time step as well as from previous time steps, but can alternatively include any other information. In a preferred variant, the environmental information is refined into a smaller latent space representation. This can simplify the learning process of any or all of the set of models, and enable efficient and logical mapping to be determined in S220 in accordance with updating the decision-making architecture through transfer learning.

[0027] The latent space representation is more preferably used to train any or all of the set of models and / or to refine that training. Additionally or alternatively, the latent space representation (e.g., features of the latent space representation) may be used to determine which model should be used as the source model when determining a new target model, the models may be trained on other data, the method may be implemented without the latent space representation, and / or the method may be adequately implemented otherwise.

[0028] The latent space representation (hereinafter equivalently referred to as the abstract space representation) is preferably configured to represent the environment (or a targeted subset of the environment) of an autonomous agent with information and / or features that are comparable between different use cases (e.g., context, action, route, etc.), thereby enabling learning to be easily and efficiently transferred between models. This is in contrast to, and / or may be advantageous in comparison to, models and / or implementations that use raw data (e.g., raw sensor data as input), since raw data is not easily consumable or transferable between models. In a preferred variant, for example, the latent space representation determines a set of features that define the environment, and these features are preferably abstract and / or extracted (e.g., determined using algorithms, models, trained algorithms and / or models, etc.), and more preferably order-independent (e.g., enabling comparison between latent space representations for different contexts, actions, routes, etc.). For example, the latent space representation generates an order-independent representation of how many objects are in the autonomous agent's environment, what features (e.g., extracted features, cost values, or other metrics) are associated with the objects and / or environment, and / or any other information. In certain specific examples where features are extracted from the latent space representation (e.g., not qualitatively meaningful), a curb may be relevant to a vehicle's motion in some contexts and / or actions, but in other contexts and / or actions, similar obstacles (e.g., barricades) may exist that are related to the vehicle's motion, and since the latent space representation extracts these objects, learning from navigating a curb can be effectively utilized and used when learning how to navigate a barricade.

[0029] As an addition or alternative, the latent spatial representation may include any or all of the following information, namely, vehicle location information (e.g., latitude, longitude, altitude, etc.), vehicle movement information (e.g., speed, acceleration, etc.), chassis information (e.g., fuel level, tire pressure, etc.), external input (e.g., map), cognitive input (e.g., location of (one or more) other vehicles or other objects, movement information related to other vehicles or other objects, etc.), predicted information (e.g., predicted trajectory of other vehicles, predicted future location or future movement of other vehicles, etc.), decisions from previous decision-making processes (e.g., selected context for the vehicle based on the vehicle's location relative to the map), and / or any other information, wherein the information is preferably standardized into a latent spatial representation format (e.g., normalized, processed to non-unit values, extracted, processed according to machine learning algorithms, etc.).

[0030] The potential benefit of adding and / or substituting latent space representations is that it allows the same basic architecture (e.g., structure) to be implemented in any or all of the models (e.g., all of the first set of models, all of the second set of models, all models, etc.), which allows learning to be transferred more effectively and efficiently between models. For example, all of the first set of models (e.g., deep decision networks, context-based models, etc.) and / or any or all of the second set of models can have the same model architecture (e.g., number of neural network layers, type of neural network, size of neural network, model input type and / or model format, etc.), which allows the method to transfer learning between models more quickly. As an addition or substitution, any or all of the models can have different architectures and / or other differences.

[0031] In a particular set of examples, each of the first set of models has the same architecture (e.g., number of neural network layers, neural network type, etc.) despite being related to different contexts, and the models differ in some or all of their weight values. As an addition or alternative, each of the second set of models has the same architecture (e.g., number of neural network layers, neural network type, etc.) despite being related to different actions, and the models differ in some or all of their weight values.

[0032] As an addition or alternative, the model may be otherwise appropriately designed and / or related.

[0033] The latent spatial representation preferably differs between a first set of models and a second set of models, and the latent spatial representation utilized by the second set of models, configured to be implemented after the corresponding models of the first set, is improved based on the output of the corresponding models of the first set (e.g., localized, targeted, etc.). In a set of variant forms, for example, each of the first set of models (a set of deep decision networks) is associated with a particular context type (e.g., a scenario, road geometry, road features, etc.) and optionally a particular context type within a particular fixed route, and each of these models selects an action for the vehicle using the current context of the vehicle (e.g., determined based on the vehicle's location, taken from a given set of labels in a map) and a latent spatial representation of its entire environment (e.g., all of the environment within the field of view of its sensor stack), and this action signals a selection of one of the second set of models (a set of deep orbital networks). The second set of selected models then uses a latent spatial representation that is informed (e.g., constrained) based on the selected action, so that the representation of the vehicle's environment for this model is localized to only the areas that are important to the vehicle to know in order to perform the action (e.g., only the view in front of the vehicle if the vehicle is moving forward, only the view behind the vehicle if the vehicle is moving backward, etc.).

[0034] As an addition or alternative, the model may use the same latent spatial representation, different latent spatial representations, and / or any combination of latent spatial representations.

[0035] In a preferred set of variants (for example, shown in Figure 5), either or all of the first subset of the model and / or the second subset of the model take as input any or all of the following information: information relating to a set of detected dynamic objects (e.g., their representation) including any or all of their current position, size, previous path, and predicted path toward the future (alternatively, the system can perform self-prediction of dynamic object motion); information relating to all of the static objects and their current state; maps (e.g., defining a context for a vehicle based on the vehicle's position); routing information; and the current state of the autonomous agent (e.g., position, orientation, attitude, etc.). Additional or alternative, any other information may be used to determine the latent spatial representation. The latent spatial representation is preferably determined using a model, more preferably a trained model (e.g., separate from the first and second subsets of the model), which functions to output an effective latent spatial representation (e.g., giving order invariance for objects working as input). Additional or alternative, the latent spatial representation may be determined otherwise as appropriate.

[0036] In a specific set of examples, static and dynamic objects returned by the system's cognitive module (along with their predicted future paths) serve as input to a neural network, which generates a latent spatial representation that gives order invariance to the objects. This data is then combined with maps, routing information, and vehicle status and used as input to a second network that represents the entire input space as the most effective latent spatial representation.

[0037] The decision-making architecture more preferably implements one or more programmed processes (e.g., rule-based, coded, hardcoded, etc.) such as programmed selection of trained models to be evaluated during decision-making (e.g., as shown in Figure 4, as shown in Figure 5, etc.). In a preferred variant, for example, the decision-making architecture (e.g., for motion planning, for trajectory generation, etc.) implements programmed selection of its trained components (e.g., trained micromodels, trained micromodel task blocks, trained algorithms and / or models, etc.).

[0038] A combination of both trained and programmed features in an autonomous agent's decision-making architecture (e.g., as shown in Figure 4, as shown in Figure 5) can have numerous advantages, such as minimizing and / or eliminating the drawbacks of exclusively implementing either trained or programmed methods. These drawbacks and / or cumbersome issues may include, for example, unnatural decisions and / or movements performed by the autonomous agent (e.g., when using only a programmed motion planner), an exhaustive and specific list of scenarios to be programmed (e.g., hardcoded) (e.g., when using only a programmed motion planner), a lack of safety assurance and / or safety integration (e.g., resulting from implementing motion planning in an end-to-end trained manner), the requirement to incorporate all conceivable scenarios the agent may encounter, and / or any other drawbacks.

[0039] In a preferred variant, for example, the decision-making architecture incorporates the flexibility of machine learning techniques while ensuring safety across the ODD associated with the autonomous agent and a practical and efficient learning framework for scalability. For example, each of the set of models is a trained neural network, but model selection from the associated modular set is performed by a rule-based (e.g., programmed) process.

[0040] Alternatively, decision-making architectures can be distributed between being fully trained (e.g., including end-to-end deep learning models), fully programmed, and / or separately trained and programmed.

[0041] A decision-making architecture can optionally be configured for operational design domains (ODDs) related to one or more specific use cases for autonomous agents. In some variations, for example, a decision-making architecture may be configured for limited ODDs in the form of a fixed-route architecture (e.g., for use when performing deliveries using autonomous agents, for use cases without passengers). As an addition or alternative, autonomous agents and associated decision-making architectures may be configured for any other use cases.

[0042] In the set of examples, ODDs are further related to the transportation of goods (e.g., deliveries, business-to-business [B2B] deliveries, commercial goods transport, transport of non-human objects, etc.) between any or all of these locations, such as distribution centers, retail centers (e.g., stores), warehouses or goods storage locations, manufacturers, and / or other locations. In certain examples, for example, any or all of fixed routes may involve loading and / or unloading locations (e.g., loading docks) associated with any or all of these locations.

[0043] As an addition or alternative, the ODD of (one or more) vehicles may be otherwise appropriately configured.

[0044] As an addition or alternative, the decision-making architecture may include any other components and / or features, and / or be configured otherwise appropriately.

[0045] In a first variant of system 100, the system includes a decision-making architecture developed for a fixed-root ODD using a context-aware data-driven modular learning system (for example, as described in U.S. Patent Application No. 17 / 125,668 filed December 17, 2020, which is incorporated herein by this reference in its entirety).

[0046] In a particular example, the system allows the use of deep learning micromodels optimized and trained for a specific route in a specific context for a specific action, which can then be reused for use on different routes in similar contexts for similar sets of actions (e.g., reused through transfer learning, through retraining with smaller datasets, through retraining with more targeted datasets). The proposed system is more preferably designed to ensure that such transfers allow for transfers that reduce target training time without resulting in negative transfers (NT), although this may be configured separately as an addition or alternative. This may be done by designing a decision-making architecture system with multiple subsets of models whose intended functionality is limited to specific explainable tasks, and the inputs for such models are transformed into a latent feature space to allow for a common space where variance mismatches between different source and target domain data can be minimized. A first subset of the models uses the vehicle's current context and, optionally, a complete representation of the environment around the vehicle to select the action the vehicle should undertake. Vehicle actions may include, in particular, "stopping behind a vehicle," "yielding to a vehicle," or "merging onto a road." A second subset of the model is selected, optimized, and safely constrained based on specific actions. This subset of the model may optionally use localized views (e.g., relative to the environment representation) around the vehicle to plan a safe, effective, and natural trajectory that the vehicle should follow.

[0047] As an addition or alternative, system 100 may include any other components.

[0048] 4. Method 200 As shown in Figure 2, Method 200 for extending the Operational Design Domain (ODD) includes S100 determining a decision-making architecture for a first domain and S200 adapting that decision-making architecture to a second domain. Additionally or alternatively, Method 200 may include S300 implementing the decision-making architecture and / or any other processes. Furthermore, or alternatively, Method 200 may include and / or interface with any or all of the processes described in any or all of U.S. applications 17 / 116,810 filed on 9 December 2020, 17 / 125,668 filed on 17 December 2020, 17 / 127,599 filed on 18 December 2020, and 17 / 962,459 filed on 7 October 2022, each of which is incorporated in its entirety by this reference, or any or all of any other preferred processes carried out in any preferred order.

[0049] Method 200 works to enable the decision-making architecture of an autonomous agent to efficiently adapt to new scenarios (e.g., new contexts, actions, environments, etc.) when extending the number of routes, the types of routes, the tasks and / or contexts and / or actions within the routes, the conditions associated with the routes (e.g., daytime vs. nighttime, bad weather, etc.), non-fixed route scenarios, and / or other features related to extending the operational design domain (ODD) of the autonomous agent. This can enable minimizing and / or reducing the time required to adapt the decision-making architecture, minimizing and / or reducing the amount of data required to adapt the decision-making architecture, and / or any other or all of these results. Additionally or alternatively, Method 200 can work to optimize the decision-making architecture for various domains, enabling the decision-making architecture to adapt reliably and / or safely to different domains (e.g., without the effects of negative transfers, etc.), and / or separately, to provide a robust but adaptable decision-making architecture for use by autonomous agents.

[0050] For example, the method can be adapted for use in any or all of the following uses: different contexts, different actions, different context-action pairs, different routes (e.g., the same context and / or actions on different routes, different contexts and / or actions on different routes), different weather conditions, different traffic conditions, different hardware and / or vehicle types (e.g., different sensor types and / or number of sensors and / or sensor placement), and the occurrence of anomalies in the vehicle's hardware and / or software (e.g., sensor degradation) (e.g., indicated by uncertainty values ​​associated with vehicle input exceeding a given threshold), as well as / or any other use.

[0051] In a preferred variant, Method 200 functions to enable scaling of the agent's ODD through transfer learning while minimizing and / or preventing the negative effects of domain scaling. For example, learning from demonstrations (examples and / or interactions) could be implemented to operate a vehicle that mimics a human driver. However, even for a single task (e.g., making a protected right turn), such techniques generally require a large number of demonstrations. For scalability, such a system must learn many tasks and scenarios through demonstrations, and this process would be quite burdensome for the system developer if each task were learned separately. Method 200 can optionally function to leverage transfer learning from previously learned scenarios and / or tasks, which allows the agent to constantly accelerate the learning of new tasks based on knowledge learned from previously demonstrated tasks, thereby reducing the amount of demonstrations and therefore the time required for learning. A further troubling issue in implementing transfer learning is that its effectiveness is not guaranteed, and the following can add complexity: the learning tasks in the two domains are not related or similar, the source domain data and target domain data variances have different (e.g., significant differences), and a suitable model may not be applicable to both domains. In these cases, negative transfer (NT) can occur, where introducing source domain data / knowledge undesirably reduces learning performance in the target domain. The inventors discovered that a solution to the above problem is transfer learning (TL), or domain adaptation (DA), which attempts to enable learning in a new domain (called the target domain) by utilizing data or knowledge from related domains (called the source domain). In machine learning applications, TL can function to improve the ability of a model to generalize in a target domain, which typically has zero or very few labeled data points.

[0052] As an addition or alternative, method 200 may perform any other function and / or provide any other benefit.

[0053] Method 200 is preferably carried out and / or utilized by the system 100 described above, but may be carried out according to any other suitable system (one or more) in addition or alternative.

[0054] Method 200 is preferably implemented at least partially in a computing and / or processing system, in a computing and / or processing system installed on the agent, in a computing and / or processing system located remotely from the agent (e.g., in a cloud-based computing system and / or server), in another location, and / or in any combination of locations. In a preferred set of variations, for example, the decision-making architecture is determined and / or updated in the remote computing system, then transmitted to the agent, stored, and installed and utilized by the agent. As an addition or alternative, Method 200 may be implemented in and / or by any other preferred component.

[0055] 4.1 Method - Determining a decision-making architecture for the first domain S100 Method 200 may include determining a decision-making architecture for a first domain, which functions to determine an initial decision-making architecture for an autonomous agent. Additionally or alternatively, S100 may function to determine a decision-making architecture that can be efficiently and / or rapidly adapted to extend to a new domain for an autonomous agent, as well as / or perform any other preferred function.

[0056] S100 is preferably performed first in Method 200 (e.g., before the deployment of autonomous agents, before the execution of any or all remaining processes of Method, before S200, etc.), but additionally or alternatively, it may be performed multiple times during Method 200 (e.g., when new inputs are received, when the decision-making architecture is constantly / continuously revised and / or updated, etc.), and / or at any other time. Alternatively, Method 200 may be performed in the absence of S100.

[0057] Preferably, in this specification, the domain refers to an operational design domain (ODD) related to an autonomous agent, which specifies an operational domain in which the decision-making architecture is configured and / or designed to operate (e.g., operate safely, operate reliably, etc.). In a variant that includes a fixed route use case, for example, the domain may include a fixed route in which the autonomous agent is trained and / or verified to travel.

[0058] As an addition or alternative, a region may include and / or specify any or all of the following: a specific set of contexts (e.g., single-lane roads, multi-lane roads, parking lots, residential zones, school zones, highways, one-way roads, two-way roads, etc.), a specific set of actions and / or behaviors (e.g., within a specific context, or unrelated to a specific context), environmental conditions (e.g., weather conditions, lighting conditions, timing conditions, traffic conditions, etc.), infrastructure conditions (e.g., road quality [e.g., smoothness, potholes, asphalt, dirt, etc.], whether the road has an adjacent sidewalk, etc.), and / or any other features or conditions (e.g., together with a set of fixed routes, or unrelated to a set of fixed routes, etc.). As an addition or alternative, a region may include and / or point to any other information.

[0059] In some examples, expanding the domain of autonomous vehicles could include adding new routes, adding new contexts, adding new actions for the vehicle to perform, changing the type of data used to train and / or update one or all of the models (e.g., training on simulation data in addition to real-world sensor data), changing the vehicle's driving habits (e.g., adapting the vehicle to drive on the left side of the road instead of the right), adding new driving conditions under which the vehicle can operate (e.g., weather conditions, traffic conditions, etc.), adding new use cases under which the vehicle can operate (e.g., making deliveries at a residence now, instead of going to a loading dock to make deliveries), and / or changing or expanding the use of the vehicle in any other way.

[0060] In a preferred variant of Method 200, the first region refers to an initial region, where the second region (for example, as described below) extends over the first region (e.g., including additional routes and / or contexts and / or actions, etc.) (e.g., including the first region with additional scenarios for autonomous agents). As an addition or alternative, the second region may include a portion of the first region, be separate from and / or unrelated to the first region (e.g., not overlapping with it), partially overlap with the first region, have an equal or smaller extent to the first region, and / or be configured otherwise.

[0061] In the example set, the first domain includes a given set of fixed routes from which a decision-making architecture model (e.g., DDN, DTN, etc.) has been trained (e.g., comprehensively trained). This could include a given set of fixed routes from which data has been collected and used to build (e.g., train, determine its basic architectural features, etc.) an initial set of models for autonomous vehicle decision-making (e.g., an initial first subset of the model, an initial second subset of the model, etc.).

[0062] As an addition or alternative, the first domain may include, and / or be defined based on, any other information, including, but not limited to, a given geographical area, use cases related to autonomous vehicles (e.g., delivery of goods, transport of passengers, etc.), a set of road features (e.g., number of lanes, road surface, road type, etc.), zone / zoning types related to the road area in the first domain (e.g., residential zone, commercial zone, etc.), and / or any other information.

[0063] As an addition or alternative, S100 may include any other preferred process (for example, an addition or alternative to the one described below).

[0064] 4.11 Method - Collecting the first set of inputs S110 S100 may optionally include S110 collecting a first set of inputs (for example, as shown in Figure 3), which serves to receive data for determining (e.g., training) any or all of the decision-making architectures. Additional or alternative, the first set of inputs may be used for any or all of evaluating and / or validating any or all of the decision-making architectures, testing any or all of the decision-making architectures, performing task mapping between domains in S220, updating the decision-making architectures in S230, implementing the decision-making architectures in S300, and / or otherwise as appropriate.

[0065] S110 is performed preferably first, and more preferably multiple times (e.g., repeatedly, continuously, at a predetermined frequency, in a random set of intervals, etc.) while the vehicle is operating within a first area (for example, having a first set of fixed routes), but may be performed at any preferred time in Method 200 as an addition or alternative.

[0066] The first set of inputs is preferably related to a first region (e.g., as described above) related to an autonomous agent, which is equivalently referred to herein as the source region, but may be related to other regions (e.g., a second region), a combination of regions, (e.g., in particular) no region, and / or any other region, as an addition or alternative.

[0067] In a preferred set of variant forms, for example, the first set of inputs includes information (e.g., sensor data) collected while the autonomous vehicle is traveling through one or more of a predetermined set of fixed routes that define a first region.

[0068] The first set of inputs preferably includes data collected in a set of sensors (e.g., sensors mounted on an autonomous agent, sensors in the environment of an autonomous agent, sensors mounted on an aggregated set of autonomous agents, sensors mounted on an aggregated set of non-autonomous agents, etc.), such as data collected from any or all of the following: cameras, radar sensors, lidar sensors, audio sensors, location sensors (e.g., GPS sensors), motion and / or orientation sensors (e.g., accelerometers, speedometers, gyroscopes, audio sensors (e.g., microphones), and / or any other sensors). Additionally or alternatively, the first set of inputs may include historical information (e.g., previously collected in one or more agents), information from one or more databases (e.g., maps, labeled maps, etc.), simulated data, and / or any other data.

[0069] A first set of inputs (and / or a portion of a first set of inputs) is preferably used as training data for the decision-making architecture, but may also be used as additional or alternative test data (equivalently referred herein to test data), evaluation data (e.g., in performing hyperparameter tuning on any or all of the set of models for the decision-making architecture), validation data, and / or any other data.

[0070] The first set of inputs may, as an addition or alternative, be used to define and / or characterize domains related to the autonomous agent (e.g., used in cognition, prediction, localization, etc.) when implementing the decision-making architecture, and / or may be used and / or determined otherwise as appropriate.

[0071] As an addition or alternative, S110 may include any other suitable process.

[0072] 4.12 Method - Training the Decision-Making Architecture S120 Preferably, S100 includes training a decision-making architecture (for example, as shown in Figure 3) S120, which functions to create an initial iteration of the decision-making architecture for the operation of an autonomous agent (for example, following a first domain). Additional or alternative, S120 may function to generate a decision-making architecture that can be efficiently adapted and improved in subsequent processes of method 200, as well as / or perform any other functions.

[0073] S120 is preferably performed in response to and based on S110, but may be performed multiple times during method 200 and / or at any other time (one or more) prior to S110, based on a portion of S110 (for example, subsequent iterations of S110 are used to improve and / or update and / or retrain the decision-making architecture).

[0074] The decision-making architecture trained in S120 is preferably determined for a first domain and based on a first set of inputs (e.g., a portion of the first set of inputs, all of the first set of inputs, etc.). Additionally or alternatively, the decision-making architecture may be trained for any other domain.

[0075] S120 preferably includes training one or all of the set of models for the decision-making architecture (e.g., determining the architecture of each of the set of models, determining the learned model parameters for each of the set of models), but may also include, in addition or alternatively, testing, evaluating (e.g., hyperparameter tuning), and / or validating any or all of the models for the decision-making architecture.

[0076] Training preferably involves determining the values ​​of one or all parameters related to the decision-making architecture, such as parameters (e.g., weights) that define one or all of the sets of micromodels (e.g., deep decision networks, deep orbit networks, etc.) that are evaluated during the decision-making of the autonomous agent. This preferably involves determining at least the learned model parameters (e.g., node weights, parameters that optimize the loss function related to (one or more) models and / or the overall architecture) and optionally, one or all of the architecture of each model (e.g., number of layers, layer organization, etc.). Additionally or alternatively, training may include determining one or all of the parameters (e.g., constraints, weights, learning rate, etc.) (e.g., unlearned parameters) that define the model architecture, which are equivalently referred to herein as hyperparameters, through one or more hyperparameter tuning processes (e.g., grid search tuning, random search tuning, parameter sweep tuning, Bayesian optimization process, gradient-based optimization process, evolutionary optimization process, etc.). Additionally or alternatively, training may include testing, evaluating, optimizing, and / or differently deciding on any or all of the decision-making architecture models.

[0077] In some variations, any or all of the models within a decision-making architecture are optimized for one or more of a specific action related to a domain, a specific context, and / or a specific route. In hyper-optimization use cases, for example, any or all of the micromodels (e.g., a first subset of models) in a decision-making architecture have parameters (e.g., weights) that are optimized for a specific action in a specific context for a specific route (e.g., a fixed route). As an addition or alternative, any or all of the models may be optimized, not optimized, and / or any combination of any of this information.

[0078] At least a portion of the decision-making architecture is preferably trained by inverse reinforcement learning. Additionally or alternatively, any or all of the models may be trained by reinforcement learning, imitation learning, and / or any other type of training / learning. Further additional or alternatively, training any or all of the models may involve and / or interface with any or all of the processes described in any or all of U.S. applications 17 / 116,810 filed 9 December 2020, 17 December 2020, 17 December 2020, 17 / 125,668 filed 18 December 2020, and 17 / 127,599 filed 7 October 2022, each of which is incorporated by this reference in its entirety.

[0079] In a preferred set of variant forms, S120 includes training each of a first subset of the model (e.g., a deep decision network) and each of a second subset of the model (e.g., a deep orbit network), such as those described in any or all of U.S. applications 17 / 116,810 filed December 9, 2020, 17 / 125,668 filed December 17, 2020, and 17 / 127,599 filed December 18, 2020, each of which is incorporated in its entirety by this reference.

[0080] In a particular example, S120 includes training each of the first subsets of the model based on the context associated with each of the first subsets of the model, and training each of the second subsets of the model based on one and / or more actions associated with each of the second subsets of the model.

[0081] S120 may, as an addition or alternative, include determining any or all of the programmed processes and / or features of the decision-making architecture (for example, those described above).

[0082] As an addition or alternative, S100 may include any other suitable process.

[0083] 4.2 Method - Determining a decision-making architecture for the second domain S200 Method 200 includes determining a decision-making architecture for a second domain, S200, which functions to adapt the decision-making architecture to the second domain, such as an extended domain. Additionally or alternatively, S200 may function to enable the decision-making architecture to be adapted efficiently and / or with a minimal set of data, as well as / or to perform any other preferred functions.

[0084] S200 is preferably performed in response to and / or based on S100, but may be performed multiple times, in addition or alternatively, in response to other processes, in response to triggers, and / or at any other time.

[0085] 4.21 Method - Collecting a second set of inputs S210 S200 preferably includes S210 collecting a second set of inputs (for example, as shown in Figure 3), which functions to receive information for performing S200 and / or any or all of the remaining processes of method 200, such as performing task mapping in S220, updating the decision-making architecture in S230, implementing the decision-making architecture in S300, and / or any or all of any other processes. Additional or alternative, the second set of inputs may be used in any of the processes described above, and / or S210 may perform any other preferred function.

[0086] S210 may be performed before S220, in response to S220, before and / or in response to any other process of Method 200, during the operation of one or more autonomous agents, independently of any or all of the processes of Method 200, at the same time as S110 (e.g., overlapping with it, partially overlapping with it, simultaneously with it, etc.), multiple times (e.g., continuously, at a predetermined frequency, in a set of random intervals, etc.), and / or at any other time.

[0087] A second set of inputs (hereinafter equivalently referred to as auxiliary data) is preferably related to a second region (hereinafter equivalently referred to as the target region), which is more preferably an extension of and / or distinct from the first region (e.g., not overlapping with it, partially overlapping with it, overlapping with it, etc.) (e.g., including additional context, including additional actions, including additional routes, including additional environmental conditions, any combination, etc.). Alternatively, the second set of inputs may be related to the first region, multiple regions, and / or any other region.

[0088] The second set of inputs preferably includes the same types of data as in the first set of inputs (as described above), such as sensor data from the same or similar sensors, simulation data from the same or similar simulation subsystem, and / or any other data, but may include, as an addition or substitution, different types of data, any other data types, and / or any combination of data.

[0089] In a preferred set of variant forms, the second set of inputs includes data relating to a set of contexts and / or actions and / or routes, as well as / or environmental conditions, that are not included in the first domain. Additionally or alternatively, the second set of inputs may include data relating to the first domain (e.g., categorized within the first domain), data from combinations of domains, and / or any other data.

[0090] A second set of inputs more preferably includes data specific to (and collected from) a particular use case associated with the new model being developed in S200. For example, each new model is preferably refined (e.g., retrained, updated, etc.) in S230 based on data specific to a particular use for which the new model is specifically configured (e.g., a particular new context, a particular new action type, a particular new weather condition, a particular new fixed route, etc.). This could include, for example, for a new model of a first set of models (e.g., a deep decision network), collecting data from that new fixed route in a new context along a new fixed route, and this data is part of the second set of inputs and is used to refine the weights of this particular new model. For a new model of a second set of models (e.g., a deep orbit network), the data used to refine the model could be taken from the vehicle's sensors while the vehicle is performing its new action (e.g., within a new fixed route, in a particular context, etc.) or in temporal proximity to it. This serves to allow the collection of a minimum, targeted set of data for refining the new model for its particular use.

[0091] As an addition or alternative, S210 may include any other suitable process and / or be otherwise appropriately implemented.

[0092] 4.22 Method - Perform mapping between the first and second regions S220 S200 preferably includes S220 performing a mapping (equivalently referred to herein as task mapping) between the first and second domains (as shown, for example, in Figure 3), which functions to enable the decision-making architecture determined in S110 to be adapted to the second domain (e.g., efficiently adapted) without requiring comprehensive and new decision-making architecture training. S220 may, as an addition or alternative, leverage the decision-making architecture determined in S100 to prevent negative transfers and / or other undesirable consequences that may arise from inappropriate updates of the set of models, and / or perform any other preferred functions.

[0093] S220 may be performed at any (one or more) preferred time in Method 200, such as in response to S210, before S210 (for example, before a second set of inputs is collected in response to determining a second region and / or performing task mapping), before and / or in response to any other process in Method, independently of other processes in Method, multiple times in Method (for example, continuously, at a predetermined frequency, in a random set of intervals, for each new model being developed, etc.), during the development of a new model, and / or at any other time.

[0094] The tasks herein refer to any features and / or aggregations of features of an ODD related to an autonomous agent, including, but not limited to, context, action, behavior, route, environmental conditions, and / or any other features. In a preferred set of variant forms (for example, shown in Figure 7), a task may include context-action pairings (e.g., lane change action in a multi-lane residential context), and a second area may reflect the addition of any or all of a new context, a new action (e.g., overall), and / or new actions within that context. The mappings herein refer to mappings between these tasks (e.g., defined associations between those tasks, relationships between those tasks, etc.).

[0095] S220 preferably includes comparing tasks in a second domain with tasks in a first domain to determine mappings (e.g., relationships, overlaps, similarities, etc.) between tasks and / or domains (e.g., between a source domain and a target domain as shown in Figure 6), the mappings preferably function to identify which tasks in the source domain are relevant to the target domain (e.g., which can be utilized by the target domain, which can be used in a modular manner to form a target task in the target domain, etc.). This allows the decision-making architecture to be updated in an efficient and reliable manner (e.g., through transfer learning) (e.g., without negative transfers, without large amounts of data in the second domain, etc.), so that the most relevant tasks in the source domain can be leveraged to efficiently update the decision-making architecture.

[0096] In a decision-making architecture that includes a first subset of models each trained for a specific context, and a second subset of models each trained for a specific action (for example, arising from a model in the first subset of models), assuming a systematic categorical classification of the models with respect to context and action, a clearly defined mapping between tasks (e.g., context-action pair tasks) can be used to efficiently update the decision-making architecture (e.g., in the shortest time, with the shortest data).

[0097] The system and / or method is preferably designed and / or configured to use multiple source tasks (e.g., multiple source models, multiple source models of the same set, etc.) which any or all of which may be used when determining the mapping to the target task. In some variations, for example, all relevant experienced source tasks (determined, for example, based on shared features) are leveraged when learning a new target task. Alternatively, a subset of previously experienced tasks may be used. Which of these variations is implemented preferably depends on knowledge and / or assumptions about task allocation, and it may not be necessary to select a subset if the tasks are expected to be similar enough that all past experiences are determined and / or estimated to be useful. On the other hand, if task allocation is multimode, it may be suboptimal to transfer from all tasks (e.g., inefficient, costly in terms of time and / or computation and / or data), and a subset may be determined and utilized.

[0098] In a preferred implementation, S220 is performed for each new model being generated (equivalently referred to herein as the updated model), such as for each new deep decision network being generated, for each new deep orbit network being generated, and / or for any other model being generated. Additionally or alternatively, S220 may be performed for each new domain expansion (e.g., each new route being added, each new action requirement for a vehicle, each new context occurrence in one or more routes, each new location in a deployment [e.g., with a new route, new weather conditions, new traffic conditions, etc.]), for multiple sets of new models at once (e.g., simultaneously, sequentially, etc.), and / or at any other time (one or more).

[0099] In a set of variant forms, for example, for each new model being generated / developed, a mapping is determined that indicates which of the previous sets of models should be used to generate the new model. The mapping is preferably determined based on the similarity between the features of the previous set of models and the features related to the (one or more) purpose and / or (one or more) use of the new model. The previous set of models (equivalently referred to herein as the source model) can include any or all of multiple models, a single model, and / or any number of models. The previous set of models is preferably of the same set of models (e.g., all of the first set, all of the second set, etc.), and as an addition or alternative, the source model may arise from multiple sets and / or types (e.g., a combination of the first set and the second set).

[0100] The mapping is preferably determined manually and / or otherwise, using any combination of the following methods: using a programmed (e.g., manually determined) process, using a rule-based process, using a lookup table, using a decision tree, using a predetermined set of mapping assignments, using a set of similarity scores between models, using a set of heuristics, etc. (e.g., according to a rule-based and / or programmed process, without a trained model, using human input / feedback, etc.). This can prevent and / or minimize the occurrence of negative transfers, for example, because manual determination of mappings can allow for the selection of the most relevant (e.g., directly comparable, explainably comparable, modularly related, etc.) models and relevant parts of the domain to use when determining a new model for a new domain.

[0101] As an addition or alternative, any or all of the decision-making platform architectures can function to allow for precise, relevant mapping between models, including, but not limited to, the modular nature of models, the sequential evaluation of models, and / or the volume of models. In some variant forms, the modular nature and numerous models (e.g., hundreds for one domain, thousands for another, etc.), each with high specificity (e.g., targeting a specific context within a particular route, or targeting a specific action within a particular route), allow the most relevant models to be identified and used (e.g., exclusively) when generating new models.

[0102] While not limited to, the first set of new models may include contextual features (e.g., number of lanes, road type, specific objects in the context [e.g., specific traffic signs, traffic lights, intersections, shoulders, curbs, barricades, bike lanes (bike Any number of features between the model (and / or related regions or regions) may be determined and / or used in determining the mapping, such as the presence of lane, crosswalks, loading docks, etc., shape features, zone type and / or number, type of traffic in the context and / or pedestrians and / or cyclists, etc., action features (e.g., the action involves a specific direction of travel [e.g., going left, going right, forward, backward, etc.], the action involves a specific type of operation [e.g., decelerating, accelerating, stopping, waiting, merging, crossing oncoming traffic, parking, etc.], the action involves monitoring other vehicles [e.g., for right-of-way determination, etc.], the action involves interaction with other types of objects [e.g., other vehicles, pedestrians, cyclists, etc.], scores associated with any or all of the features (e.g., cost score, risk score, similarity score, proximity score, etc.), and / or any other features or information.

[0103] Additionally or alternatively, some or all of the mappings may be determined using trained models and / or algorithms, and / or otherwise appropriately determined.

[0104] Determining a mapping can, as an addition or alternative, leverage latent space representations, such as those described above, which can enable efficient, logical, and / or simple mappings to be determined between source and target domain tasks. This can be made possible, for example, by lower-dimensional, meaningful, and comparable metrics that can be refined (e.g., from a large amount of detailed data received in a cognitive subsystem). For example, when considering transfer learning via dimensionality reduction, a low-dimensional latent feature space (e.g., for inputs received / determined in a cognitive module) where the variances between source and target domain data are the same or close to each other can yield numerous benefits. On this latent feature space, for example, data in the relevant domains can be projected, and a training process can be applied to train a model (e.g., for a first subset of the model, a second subset of the model, etc.). Thus, the latent feature space can effectively function as a bridge for transferring knowledge from the source domain to the target domain (e.g., and used to determine efficient mappings). A dimensionality reduction process is more preferably implemented when determining the latent space, which minimizes the distance between the variances of data in different domains within the latent space. This framework can be further adapted to take advantage of the fact that some aspects of the reward function, such as the negative reward an autonomously driven vehicle might receive for stopping too close to an obstacle, are often shared across different (but related) tasks. Thus, a variation of the method can optionally assume that the reward function "r(t)" for different tasks is related via a latent basis "L" of reward components. These components can be used to reconstruct the true reward function through loose coupling of such components with task-specific coefficients s(t), using L as a transfer mechanism.

[0105] In a specific example, a latent spatial representation of data associated with a particular model (e.g., data used to train the source model, data collected for a new domain / new model) may be used to compute a proximity metric and / or to perform proximity checks to determine which source model is sufficiently similar to be used when generating a new model (e.g., sufficiently similar to the purpose / use case / context / action / etc. of the new model).

[0106] In a first variant of S220, S220 includes determining a first set of new context-based models (e.g., a deep decision network that determines actions for a particular context), and generating the new context-based models includes determining in S230 which of the existing first set of models should be used based on a shared set of features between the existing models and the features of the new context. Additional or alternative features of action options considered and / or selected by the new model and / or the first set of models may be further considered and / or compared between the models. In a particular example, a model configured to select actions that perform a right turn (and / or any other action) in a residential zone could be used to develop a model configured to select actions that perform a right turn (and / or any other action) in a highway context. In another specific example (for example, shown in Figure 7), one or more models are selected (e.g., mapped) for a first context of single-lane residential roads, one or more models for a second context of one-way residential roads, and one or more models for a third context of parking lots, and a new model for a fourth context of multi-lane residential zones is developed based on the similarities between the features of these contexts (e.g., residential designation, relevant speed limits [e.g., low speeds in both residential zones and parking lots], number of lanes, lane direction, etc.). Additionally or alternatively, similarities between available actions related to those contexts may be used when determining the mapping.

[0107] In a second variant of S220, S220 includes determining a second set of new action-based models (e.g., a deep trajectory network that generates a trajectory for a particular action), and generating the new action-based model includes determining in S230 which of the existing second sets of models should be used based on a shared set of features between the actions of the existing models and the features of the new action. In a particular example, for example, an existing model for a right-turn action may be used (e.g., together with other models, or alone) to generate a model for a new action of changing lanes into the rightmost lane, due to the similarity of the features between the actions and / or the objectives of the actions (e.g., preventing driving onto a curb adjacent to the right lane, preventing crossing a yellow line on the lane, preventing crossing a set of solid lines on the lane, performing a smooth trajectory, etc.). In another specific example, for a lane-keeping action in a new context and / or a new route, other models for lane-keeping (e.g., in a different context, in a different route, etc.) may be used to generate the new model, models for other actions may be used based on similarities in action features (e.g., left nudge, right nudge, etc.), and / or any other model or combination of models may be used to generate the new model. In another specific example, an existing model for implementing a slow-down action (e.g., a vehicle temporarily slows down / stalls to collect more data about what is potentially approaching the vehicle's path before proceeding) may be used to determine a model for a new action of merging (e.g., onto a highway, in a highway context, etc.), where there are some differences between these actions and / or contexts (e.g., speeds may differ, environments may differ, etc.), but the resulting motion (e.g., trajectory) may have shared features and / or goals (e.g., including a deceleration motion, including a gently angling drive when the path becomes clear, a lane change, etc.).

[0108] As an addition or alternative, S220 may include any other process and / or be appropriately implemented otherwise.

[0109] 4.23 Method - Updating the Decision-Making Architecture S230 S200 includes S230, which updates the decision-making architecture (as shown, for example, in Figure 3), and functions to extend the ODD associated with the autonomous agent. As an addition or alternative, S230 can function to enable the vehicle to navigate new contexts (e.g., a completely new context, a previous context in a new route, etc.), perform new actions (e.g., a completely new action, a previous action in a new context, a previous action in the same context in a new route, etc.), execute a new set of routes, as well as / or otherwise extend the vehicle's behavior and / or usability.

[0110] S230 is preferably carried out by a transfer learning process (and / or domain adaptation), and based on any or all of the mappings determined in S220, and optionally by any or all of the training processes described in S120, but may be carried out in response to any other process of Method 200 and / or at any other time (one or more).

[0111] S230 is more preferably carried out based on the decision-making architecture determined in S120, and therefore the decision-making architecture determined in S230 starts with any or all of the following: a general model architecture, model architecture and parameters (e.g., hyperparameters, learned parameters, etc.), parameters, and / or any other information of the previous decision-making architecture.

[0112] The transfer learning process preferably includes determining and / or refining (e.g., retraining, updating, tuning, etc.) a set of weights associated with (one or more) target models, but may also include, additionally or alternatively, determining and / or tuning the model architecture (e.g., the number of neural network layers, the organization of the neural network layers, the number and / or arrangement of neural network nodes, the type of neural network, the number of neural networks, etc.) and / or determining any other information.

[0113] In a preferred set of modified forms (for example, shown in Figure 8), S230 starts by determining a new model using a common model architecture (e.g., a basic model framework shared among all of the first set of models, a basic model framework shared among all of the second set of models, a modified framework, etc.), extracting a set of source models based on the mapping performed in S220, and finding a weighted mean using manual adjustments (e.g., algorithms, models, formulas, mathematical operations [e.g., addition, averaging, calculating the median, calculating the minimum, calculating the maximum, etc.]) through gradient descent-based optimization. This includes either or all of the following: aggregating (using methods such as finding a weighted mean, isotropic Gaussian approximation, and / or approximating the precision matrix using Fisher information, etc., according to approximate sampling from the posterior of the model weights for any individual context action pair); and refining the initial set of weights for the new model based on training the new model with data specific to the new model (e.g., data collected within a specific route and context related to the model [e.g., from vehicle sensors], data collected while the vehicle is performing a specific action, data collected from sensors with the same level of degradation as when the model was constructed, etc.). Additionally or alternatively, S230 may include any other suitable processes.

[0114] Refining the initial set of weights (for example, during training, during retraining, etc.) preferably involves a target model evaluation and tuning process (e.g., providing the model with new data represented in latent space representations and tuning the model while evaluating its performance), but may include any other processes as additions or alternatives.

[0115] In addition or alternatively, S230 may include a hyperparameter tuning process (for example, based on a second set of inputs as described above), and any other processes as additional or alternative.

[0116] S230 is preferably performed in less time and / or with less data compared to S120, but alternatively, it may be performed in any other time and / or with any other data.

[0117] As an addition or alternative, S230 may include any other process.

[0118] 4.3 Method - Implementing a Decision-Making Architecture in S300 Method 200 may optionally include implementing a decision-making architecture 300, which functions to make an autonomous agent operate (e.g., operate) according to the decision-making architecture. S300 may, as an addition or alternative, function to enable the vehicle to operate within new and / or expanded areas (e.g., along a new set of fixed routes, within a new context, while performing new actions, etc.).

[0119] The decision-making architecture implemented in S300 may include the one determined in S100, the one determined in S200 and / or any subsequent iterations, a combination of architectures, and / or any other decision-making architectures.

[0120] 5. Transformation Form In one variation of the method, such as for use in developing a new deep decision network (as described above), the method is used to enable a decision-making process learned through an inverse reinforcement learning (IRL) process in a first domain to be adapted for use in a second domain, where the decision-making is configured to generate an optimal policy (e.g., output) that outputs a specific action given a particular vehicle state of the vehicle. The policy preferably refers to a suggested action that the vehicle should take for any possible vehicle state (e.g., including).

[0121] In a particular example using inverse reinforcement learning, where the goal is to obtain (e.g., generate) a reward function based on state-action pairs (e.g., from human driving data, simulated data, data collected from an autonomous vehicle, a combination of data sources / types, etc.), data is first collected from a set of source domains containing at least one domain, and parameters (e.g., weights) of the reward function (one or more) are learned to represent training samples relevant to the source domain dataset. In a specific example, for example, this model is configured to determine a policy (e.g., in a given time) based on any or all of the following: a set of internal states (e.g., collectively defining the state space) generated in a latent space representation for the vehicle; a set of actions the vehicle can take (e.g., turn right, turn left, change lanes, stop, yield, etc.); transformations (e.g., matrices) representing the probabilities of transitioning from one state to another (e.g., matrices containing the probabilities of transitioning from one state to another, thereby representing the potential modifications that may occur to the agent's state in response to performing an action); and a reward function for the vehicle (e.g., generating reward values ​​in response to the agent's state). Optionally, this reward function determination process can be followed by a reinforcement learning (RL) process (e.g., a training process) configured to generate an optimal policy as an output.To use these learnings in a new domain called the second domain, preferably, the following process is used: collecting data from the second domain and creating new deep decision network parameters (reward function) based on the parameters learned from the source domain using approximate sampling (e.g., isotropic Gaussian approximation, Fisher information matrix approximation, etc.) from the parameters of any context action pair (e.g., the rear of the model weights) aggregated from multiple contexts and actions (e.g., through manual adjustment, gradient descent-based optimization, etc.) and / or from the parameters of any context action pair (e.g., from the deep decision network associated with the source domain). The process involves initializing parameters (e.g., weights), further training these initial parameters based on data from a second region, and initializing a policy for RL training by creating a composite policy (e.g., additive, multiplicative, etc.) in which the composite policy includes a weighted sum of variances from source region policies, a function (e.g., a gating function) is determined and / or parameter values ​​(e.g., weight values ​​that determine the probability of activating each source region for a given vehicle state) can be calculated, and then performing an RL retraining session using data from the second region to obtain an optimal policy for the second region.

[0122] As an addition or alternative, this variant and / or example (and / or modified version) may be applied to models other than deep decision networks (e.g., deep orbit networks), the method may be implemented separately (e.g., without IRL, using training processes other than IRL and / or RL), and / or the method may be implemented more appropriately.

[0123] For the sake of brevity, preferred embodiments include any combination and substitution of various system components and various method processes, the method processes may be carried out sequentially or concurrently in any preferred order.

[0124] Embodiments of the system and / or method may include any combination and substitution of various system components and various method processes, and one or more examples of the methods and / or processes described herein may be carried out asynchronously (e.g., sequentially), concurrently (e.g., concurrently, in parallel, etc.), or in any other preferred order by one or more examples of the systems, elements, and / or entities described herein and / or using them. The following system and / or method components and / or processes may be used in addition to, instead of, or separately integrated with, all or part of the systems and / or methods disclosed in the above-mentioned application, each of which is incorporated in whole by this reference.

[0125] Additional or alternative embodiments implement the above method and / or processing module in a non-disclosed temporary computer-readable medium that stores computer-readable instructions. Instructions may be executed by a computer-executable component integrated with the computer-readable medium and / or processing system. The computer-readable medium may include any suitable computer-readable medium, such as RAM, ROM, flash memory, EEPROM, optical devices (CD or DVD), hard drives, floppy drives, non-disclosed temporary computer-readable medium, or any suitable device. The computer-executable component may include a computing system and / or processing system connected to the non-disclosed temporary computer-readable medium (for example, including one or more co-located or distributed, remote or local processors), such as a CPU, GPU, TPU, microprocessor, or ASIC, but instructions may, as an alternative or addition, be executed by any suitable dedicated hardware device.

[0126] As those skilled in the art will recognize from the preceding detailed description and from the figures and claims, modifications and changes can be made to preferred embodiments of the invention without departing from the scope of the invention as defined in the following claims.

Claims

1. A method performed by a computer to extend the operational design domain of an autonomous vehicle from a first set of fixed routes to a second set of fixed routes, wherein the method is: - Determining an initial set of models for the operation of the autonomous vehicle along a first set of fixed routes, wherein the initial set of models is - A first set of multiple action models, each of which receives a context related to the environment of the autonomous vehicle and determines a set of actions for the autonomous vehicle based on the context related to the environment of the autonomous vehicle, - A first set of multiple trajectory models, each of which the first set of multiple trajectory models receives a set of outputs from the first set of multiple action models and determines a trajectory for the autonomous vehicle based on the set of outputs from the first set of multiple action models. This includes determining the initial set of models, Based on the first set of the plurality of action models and the first set of the plurality of trajectory models, the autonomous vehicle is operated along one of the first set of fixed routes. - Extending the initial set of models for the operation of the autonomous vehicle along a second set of fixed routes, - To determine a second set of multiple action models, ● Determine a new set of contexts related to the second set of fixed routes, - Selecting a subset of action models from the first set of multiple action models based on common route features shared between the first set of fixed routes and the second set of fixed routes, wherein each action model in the first set of multiple action models relates to context. - To create a first set of aggregated weights, the model weights associated with the subset of the action model are aggregated, - To create a second set of the multiple action models by improving the first set of aggregated weights based on a set of sensor data collected while the autonomous vehicle is traveling along the second set of fixed routes. This involves determining a second set of multiple action models, - This involves determining a second set of multiple orbital models, ● Determine a new set of actions related to the second set of fixed routes, - Selecting a subset of orbital models from the first set of multiple orbital models based on actions common to the first set of fixed routes and the second set of fixed routes, - To create a second set of aggregated weights, the model weights associated with the subset of the trajectory model are aggregated, - To create a second set of the multiple trajectory models by improving the second set of aggregated weights based on the aforementioned set of sensor data. This involves determining a second set of multiple orbital models, including Extending the aforementioned initial set of models, Based on the second set of the plurality of action models and the second set of the plurality of trajectory models, the autonomous vehicle is operated along one of the second set of fixed routes. Methods that include...

2. Operating the autonomous vehicle along a fixed route from a first set of fixed routes based on the first set of multiple action models and the first set of multiple trajectory models means that, while the autonomous vehicle is traveling along the fixed route, at each of the set of multiple time points, Based on the location of the autonomous vehicle, extract the context related to the location from the labeled map, - Mapping the aforementioned context to one action model from the first set of the aforementioned multiple action models, - Evaluating the aforementioned action model in order to create an action, - Mapping the aforementioned action to one of the first set of the multiple trajectory models, - To create an orbit, evaluate the aforementioned orbital model, - Operating the autonomous vehicle according to the aforementioned track and The method according to claim 1, including the method described in claim 1.

3. Operating the autonomous vehicle along a fixed route from the second set of fixed routes based on the second set of multiple action models and the second set of multiple trajectory models means that, while the autonomous vehicle is traveling along the fixed route, at each of the set of multiple time points, - Based on the location of the autonomous vehicle, retrieve the context associated with the location from the labeled map, such that the new set of context labels includes the context. - Mapping the aforementioned context to a second action model among the second set of the aforementioned multiple action models, - Evaluating the aforementioned second action model in order to generate actions, - Mapping the aforementioned action to a second orbital model among the second set of the plurality of orbital models, - To evaluate the aforementioned second orbital model in order to create a second orbit, - Operating the autonomous vehicle according to the second trajectory The method according to claim 2, including the method described in claim 2.

4. The method according to claim 1, wherein each of the first set of the plurality of action models, the second set of the plurality of action models, the first set of the plurality of trajectory models, and the second set of the plurality of trajectory models is a machine learning model.

5. The method according to claim 4, wherein each of the first set of the plurality of action models, the second set of the plurality of action models, the first set of the plurality of trajectory models, and the second set of the plurality of trajectory models includes a deep neural network.

6. The method according to claim 1, wherein each of the first set of the plurality of action models, the second set of the plurality of action models, the first set of the plurality of trajectory models, and the second set of the plurality of trajectory models receives an environmental representation related to the autonomous vehicle as input, and the environmental representations of the first set of the plurality of action models and the second set of the plurality of action models are more comprehensive than the environmental representations of the first set of the plurality of trajectory models and the second set of the plurality of trajectory models.

7. The method according to claim 6, wherein the environmental representations of the first set of the plurality of trajectory models and the second set of the plurality of trajectory models are improved based on the direction of action selected for the autonomous vehicle.

8. The method according to claim 7, wherein the direction includes the orientation of the autonomous vehicle as defined by the action.

9. The method according to claim 7, wherein the environmental representations of the first set of the plurality of action models, the second set of the plurality of action models, the first set of the plurality of trajectory models, and the second set of the plurality of trajectory models include a latent space representation, the latent space representation defines a set of abstract features relating to the environment of the autonomous vehicle, and the set of abstract features is determined based on sensor data collected by the autonomous vehicle.

10. The method according to claim 1, wherein the second set of fixed routes is separate from and different from the first set of fixed routes.

11. The method according to claim 1, wherein the subset of the action model includes a plurality of models.

12. The method according to claim 11, wherein the subset of the trajectory models includes a plurality of models.

13. The method according to claim 1, wherein selecting a subset of the action models from a first set of the plurality of action models includes at least one of a rule-based process and a programmed process.

14. The method according to claim 1, wherein each of the first set of fixed routes and the second set of fixed routes relates to a delivery use case of the autonomous vehicle, and each of the first set of fixed routes and the second set of fixed routes is arranged among any or all of a set of distribution centers, a set of warehouses, and a set of retailers.

15. The method according to claim 14, wherein the new set of actions stops at a loading dock associated with one or more of the new distribution centers from the set of distribution centers, the new warehouses from the set of warehouses, and the new retailers from the set of retailers.

16. The aforementioned common route features are, - Common context labels, or • Set of road features The method according to claim 1, comprising at least one of the following.

17. The method according to claim 16, wherein the route feature includes at least one of an intersection, a pedestrian crossing, and a bicycle lane.

18. The method according to claim 1, wherein the common route feature includes the direction of movement.

19. The method according to claim 1, wherein the process of creating a first set of aggregated weights and the process of creating a second set of aggregated weights include evaluation of an algorithm.

20. The method according to claim 19, wherein the algorithm is a trained algorithm.

21. A method performed by a computer to extend the operational design domain of an autonomous vehicle from a first set of fixed routes to a second set of fixed routes, wherein the method is: - Determining an initial set of multiple models for the operation of the autonomous vehicle along a first set of fixed routes, wherein the initial set of multiple models is - A first set of multiple action models, which receives the driving context of the autonomous vehicle and determines a set of actions based on the driving context, - A first set of multiple trajectory models, which receives an output created using one or more of the first sets of multiple action models, and determines a set of trajectories based on the output, This involves determining an initial set of multiple models, Based on the set of tracks, the autonomous vehicle is operated along one of the first set of fixed routes, - Extending the operation of the autonomous vehicle to a second set of fixed routes, - To determine the first set of route features of the first set of fixed routes, - To determine a second set of route features for the second set of fixed routes, - Selecting a subset of action models containing multiple action models from the first set of multiple action models based on the route features common to the first set of route features and the second set of route features, - Selecting a subset of orbital models containing multiple orbital models from the first set of multiple orbital models based on the route features common to the first set of route features and the second set of route features, - To create a first set of aggregated weights, model weights are aggregated from a subset of the action model, - To create a second set of aggregated weights, model weights are aggregated from a subset of the trajectory models, - Training a second set of multiple action models based on the first set of aggregated weights and a set of data collected while the autonomous vehicle travels along the second set of fixed routes, - Training a second set of multiple trajectory models based on the second set of aggregated weights and the set of data. Including, extending, - Using a second set of the multiple action models and a second set of the multiple trajectory models, the autonomous vehicle is operated along one of the second set of fixed routes. Methods that include...

Images