World model based adaptive fine-tuning planning for autonomous driving

Abstract

Selective fine-tuning of autonomous driving models. Autonomous driving models employ prediction models and planning strategies, where fine-tuning includes determining whether the prediction model or the planning strategy is the primary cause of the performance degradation, and modifying the prediction model or the planning strategy based on the determination, e.g., by retraining the model or adjusting the strategy.

Description

Technical Field

[0001] This invention relates to systems and methods for training world-model-based reinforcement learning models, which are trained with adaptive fine-tuning and applied to autonomous vehicles. Background Technology

[0002] Autonomous driving has garnered widespread attention and demonstrated numerous potential advantages. Consequently, significant resources and research and development have been focused on autonomous driving technology. However, autonomous driving also faces substantial and complex challenges. Beyond overcoming the inherent challenges and immense complexity of autonomous driving systems themselves, these systems must be accurate and robust enough to gain considerable public trust. Artificial intelligence appears to be a powerful tool for addressing the challenges and complexities of autonomous driving, but it also presents its own challenges and complexities. Solutions capable of overcoming these challenges with sufficient accuracy and robustness are still needed. Summary of the Invention

[0003] A method for training an autonomous vehicle system is provided. The method may include: pre-training a first component and a second component using a dataset (e.g., offline); applying the pre-trained first and second components (i.e., a pre-trained model) to an operating environment to identify primary sources of error; and updating the first or second component based on the identified primary sources of error. The first component may be a prediction component configured to predict future events based on inputs such as sensor inputs. The second component may be a planning component for determining one or more actions based on future events predicted by the prediction component. After pre-training, the first and second components can form a pre-trained model. The operating environment (e.g., an online environment) may provide sensor inputs, such as inputs from a vehicle sensor system.

[0004] Identifying the primary error sources can include comparing a first error associated with a first component and a second error associated with a second component. The first component can be a prediction component, such as a world model employing a recurrent neural network, and the second component can be a planning component, such as a planning strategy. The first error can be a construction error associated with the prediction component, and the second error can be a distribution shift associated with the planning component, such that if the construction error dominates, the prediction component is retrained using sensor inputs from the operating environment, and if the distribution shift dominates, the strategy is adjusted. The prediction world model component can simulate and predict future states, while the planning strategy component can determine one or more actions based on the world simulation and predicted states of the prediction world model. Updating the prediction component can include low-rank fine-tuning techniques, such as decomposing one or more model parameters into two lower-dimensional vectors. Sensor inputs can be received, or configured to receive sensor inputs from one or more sensors of the vehicle system. After updating, the prediction and planning components can provide actions to the vehicle control system to control or drive one or more actuators of the vehicle.

[0005] A training system for training autonomous vehicle systems is also provided. The training system includes one or more computing devices for pre-training an artificial intelligence (AI) model with a first component and a second component using a dataset to form a pre-trained model; applying the pre-trained model to one or more tasks, such that the model receives one or more observations; identifying, based on the one or more observations and in response to applying the pre-trained model, whether the first component or the second component is the primary cause of model performance degradation; and selectively fine-tuning the first component or the second component based on whether the first component or the second component is the primary cause of model performance degradation.

[0006] Selective fine-tuning may include: retraining the first component of the pre-trained model based on one or more observations if the first component is the primary cause of performance degradation; and adjusting the second component if the second component is the primary cause of performance degradation. The first component may be a prediction component, and the second component may be a planning policy component. The primary cause can be determined by comparing the total variance distance of the prediction component with the total variance distance of the planning component. The total variance distance of the prediction component can indicate the gap between the predicted state and the actual state of the first component. The total variance distance of the second component can indicate the gap between the optimal planning policy and the current planning policy. A threshold may be applied to the total variance distance of the planning policy component before comparing it with the total variance distance of the prediction component. This threshold may be greater than 1, at least 2, at least 10, or at least 50. The prediction component may be a world model configured to simulate a real-world environment to predict outcomes. One or more observations may be observations received from the vehicle's sensor input system.

[0007] A non-transient computer-readable medium is provided for storing instructions for training an artificial intelligence (AI) system. This AI system can be used for autonomous driving. These instructions can be executed by one or more processors to optimize an autonomous driving model. These instructions can, in response to identifying errors associated with a pre-trained model's world model, retrain the world model based on data from the operating environment, and can, in response to detecting a distribution shift associated with the pre-trained model's planning policy, adjust the planning policy based on data from the operating environment. Attached Figure Description

[0008] Figure 1 An autonomous vehicle was shown.

[0009] Figure 2 An example block diagram of the training system is shown.

[0010] Figure 3 The performance degradation of various fine-tuning strategies is shown.

[0011] Figures 4A-4D The performance degradation of various fine-tuning strategies based on a specific task is shown.

[0012] Figures 5A-5C The total variation of each component of the model using different fine-tuning strategies is shown.

[0013] Figures 6A-6C The total variation of each component of the model is shown when an adaptive fine-tuning strategy with different thresholds is employed.

[0014] Figure 7 It is a method for training artificial intelligence models.

[0015] Figure 8 Examples of systems that can be used to train and / or run one or more models and / or components described herein are shown.

[0016] Figure 9 An embodiment of a computing platform that can be used to implement the models and / or methods described herein is shown. Detailed Implementation

[0017] This document describes embodiments of the present disclosure. However, it should be understood that the disclosed embodiments are merely examples, and other embodiments may take various alternative forms. The drawings are not drawn to scale; certain features may be enlarged or reduced to show details of specific components. Therefore, the specific structural and functional details disclosed herein should not be construed as limiting, but merely as a representative basis for those skilled in the art to apply these embodiments in various ways. As will be understood by those skilled in the art, the various features shown and described with reference to any of the drawings may be combined with features shown in one or more other drawings to produce embodiments not explicitly illustrated or described. The combinations of features shown provide representative embodiments of typical applications. However, for a particular application or implementation, various combinations and modifications of features consistent with the present disclosure may be required.

[0018] See Figure 1 The figure illustrates a vehicle 100 with an autonomous driving system. In various embodiments, the vehicle can be a land, sea, air, or extraterrestrial vehicle, such as a car, train, ship, airplane, spacecraft, and / or an unmanned vehicle, such as a drone or rocket. In one or more embodiments, the autonomous driving system may include an end-to-end autonomous driving stack 106. The end-to-end autonomous driving stack 106 may be modular. For example, the autonomous driving system may include a perception section 108, a prediction section 110, and / or a planning section 112, which work together to provide autonomous driving. In improved embodiments, the autonomous driving system may or may not include interpretation and reasoning capabilities. In various embodiments, the autonomous driving stack 106 works in conjunction with various sensors 102 and one or more controllers 104 via a communication network 105 to achieve autonomous driving. Sensors can be used to detect and understand the environment surrounding the vehicle. For example, the perception section 108 of the autonomous driving system stack 106 may receive input from one or more sensors 102, while the planning section 112 may pass decisions / tasks / actions to one or more controllers 104 to control or drive actuators. For example, these decisions / tasks / actions may include actuating the throttle or accelerator pedal to accelerate, rotating the steering column to turn, and / or actuating the brakes to, for example, enter a highway, change lanes, or avoid pedestrians.

[0019] Autonomous driving systems must accurately predict future situations and take actions based on these predictions. Therefore, autonomous driving systems can employ artificial intelligence (AI), such as machine learning. Figure 2 As shown, AI model 200 may include a prediction component 204 to predict future events and a planning component 202 to guide decision-making and / or control. For example, an autonomous driving system may include unsupervised learning, such as a reinforcement learning model 206. For instance, a reinforcement learning model may incorporate a Markov decision process (MDP), defined as shown in Equation 1: (1), Where S represents the state space, A represents the action space, P represents the transition probability, and r represents the reward. This represents the discount factor.

[0020] Reinforcement learning models use agents to learn through trial and error rather than labeled datasets. For example, reinforcement learning can employ methods such as... Figure 2 The actor-critic feedback strategy is shown. During training, the agent adjusts Model 200, for example, by modifying the planning strategy using rewards (and penalties) associated with success and failure in performing tasks. 202. In various embodiments, the reinforcement learning model may employ a predictive latent model to predict latent events and adopt a planning strategy. To guide action (a). For example, this strategy can be represented by Formula 2: (2), Where 'a' represents an action and 's' represents an observation. In various embodiments, the autonomous vehicle system may include a world-model-based reinforcement learning model. In at least some embodiments, the world model... And / or the prediction component 204 can be a recursive model, such as a recurrent neural network (RNN). The world model aims to understand the environment, the relationships between objects, and to model future outcomes. Furthermore, the world model can generalize to handle new tasks or situations. For example, the world model can use latent variables to handle uncertainty and represent unknown information affecting the prediction. In other words, the world model models how actions affect future states. In various embodiments, the world model includes an encoder component and a prediction component. The encoder transforms observations into latent representations. The model can be an autoregressive model that considers historical states when determining the predicted state. The model is an action-conditional prediction model in one or more components. World Model This represents the potential dynamics of the operating environment and predicts the reward (r) and future potential states (z). The world model... Incorporating it into a reinforcement learning model allows the system to internally simulate possible future scenarios.

[0021] In one or more embodiments, the model can be trained using a pre-training-fine-tuning paradigm. This may, for example, be superior to learning a world model from scratch. 204 and strategy 202, especially for complex tasks such as autonomous driving (which may require a large state space (S)). In various embodiments, the world model and strategy Components 204 and 202 can be pre-trained using offline data 208 (e.g., a dataset) for initialization. The pre-trained dynamic model can be represented as... The dynamic model of the new task can be represented as ,in and This represents the corresponding probability transition matrix. After training, the dynamic model can guide policy learning. For example, a planning policy can be learned by maximizing a value function, such as that represented by Equation 3. : (3), The first term is the cumulative reward from step K, and the second term is the expected cumulative reward of the action, represented by a Q function such as Equation 4: (4), in Indicates an action, x t This represents the state. Therefore, predictions from the dynamic model inform policy learning.

[0022] However, generalizations from pre-training may not accurately reflect the specific task and reality, leading to errors and distribution shifts. In one or more embodiments, construction errors can be identified by comparing predictions from prediction part 204 with actual results observed in the runtime or current environment to detect significant deviations between model predictions and reality. These deficiencies can be improved by retraining the model. Distribution shifts can be determined by detecting differences between the data distribution from pre-training and the data distribution from the runtime or current task. For example, performance gaps can be defined according to Equation 5: (5), in The initial state distribution is given. The second term applies the pre-trained model and policy to a dynamically distributed system. The expected return of the new task.

[0023] Therefore, after pre-training for general features and tasks, fine-tuning can be applied to more specific tasks. This pre-trained model can then be further initialized online using a runtime environment (e.g., one or more real-world environments or situations). Automatic encoding can be used based on sensor input (o) t The latent state is learned from the observations of s. For example, the observations s can be used to learn the latent state. t Encode the latent states as represented by Equation 6: (6), Where z tLet Z represent the latent state, and R represent the latent space. d Represents a d-dimensional real number / vector. Encoder This can be expressed by Formula 7: (7), in Indicates encoder, h t Indicates a hidden state, o t Represents the observed input. Model state x t This can be expressed by Formula 8: (8), in This represents the parameters. Then, the model uses the current model state to determine the action (...). Predict the next recursive state (h) t+1 The corresponding sequence model can be represented by Equation 9: (9).

[0024] The next step can be represented by Equation 10: (10), Where h t+1 This includes information about the potential representation of this step. The model also uses model state (x) t To predict the reward represented by Formula 11: (11), And the round continuation flag represented by Formula 12: (12).

[0025] The agent uses a pre-trained world model and policy to simulate a task and executes the plan with the highest predicted reward. For example, the world model... It can be used to generate virtual trajectories within the range K>0, as shown in Formula 13: (13), Where x t This is the current state. According to the strategy The current action of the simulation. However, simply performing online initialization can lead to a significant and detrimental performance degradation.

[0026] In various embodiments, fine-tuning can be selectively tailored to specific components of the model (e.g., 202, 204) (i.e., adaptive / selective fine-tuning) to more effectively mitigate performance degradation. Some systems can indiscriminately employ fine-tuning techniques on one or more components (e.g., model-only fine-tuning, policy-only fine-tuning, or alternating between model and policy fine-tuning). For example, Figure 3 Various indiscriminate and selective fine-tuning strategies described herein are illustrated. For example, these tasks are performed in a CARLA environment where the agent traverses the area performing actions (…). ) tasks and receive rewards (t) t At each time step (t), the agent receives a bird's-eye view observation. In some embodiments, the reward can be based on attributes associated with crossing waypoints, such as safety, comfort, driving time, speed, and distance. For example, weights of these attributes can be used. Pre-training can be performed for 12 hours on a task from the CARLA leaderboard v2. Fine-tuning of the pre-trained model and policy can then be performed for 1 hour. For example, a traditional system might indiscriminately apply fine-tuning only to the prediction component, only to the planning component, or both (e.g., alternately).

[0027] However, as Figures 4A-4D As shown, selectively or purposefully applying the fine-tuning 212 to the most needed components (i.e., adaptive fine-tuning) can more effectively mitigate performance degradation.

[0028] Figure 4A The performance degradation of different fine-tuning techniques (e.g., model-only fine-tuning, policy-only fine-tuning, and adaptive fine-tuning) for roundabout tasks is shown under moderate traffic flow. Figure 4B The performance degradation of different fine-tuning techniques for right-turn tasks is shown under heavy traffic conditions. Figure 4C The performance degradation of different fine-tuning techniques for left-turn tasks is shown under moderate traffic flow conditions. Figure 4D The diagram illustrates the performance degradation of different fine-tuning techniques for left-turn tasks under heavy traffic conditions. For example, applying fine-tuning to the predictive component (e.g., world model) or the planning component (e.g., policy) based on which component is the primary cause of performance degradation or impairment may be more effective than indiscriminate fine-tuning.

[0029] In one or more embodiments, fine-tuning step 212 may include identifying errors in the task associated with components (e.g., multiple components, such as a prediction component and a planning component) and targeting modifications to the components that most significantly impact performance degradation. Adaptive fine-tuning identifies and addresses the largest biases in the model by targeting specific components of the model. In improvement schemes, the errors associated with prediction component 204 and planning component 202 may be compared to determine which error is dominant. In various embodiments, prediction component 204 may be modified in response to the dominant error of prediction component 204 (e.g., greater than the error of planning component 202), or planning component 202 may be modified in response to the dominant error of planning component 202 (e.g., greater than the error of prediction component 204). For example, prediction component 204 may be targeted for fine-tuning 212, such as by retraining prediction component 204 using new online data from the current task and / or operating environment.

[0030] In one variation, the construction error of the predicted component 204 and the distribution offset of the planned component 202 can be considered. In other words, the construction error of the predicted component (e.g., a world model) can be compared with the distribution offset of the planned component to determine which one is dominant. For example, a world model can be used. Evaluate the state prediction error at each step (e.g., k=1,2,...,K). The state prediction error can be defined according to Equation 14: (14), Where x k It is the actual or true state of the new task. The state is predicted. The state prediction error can be decomposed. In various embodiments, the prediction error is decomposed such that each item corresponds to a different factor, such as the distributional offset between tasks and the limitations of generalization, which can lead to model inaccuracies. In improved schemes, these items can therefore correspond to different components of the overall model. For example, the state prediction error can be decomposed into two items, associated with either the policy component or the prediction component. In one or more embodiments, the decomposition can be represented by Equation (15): (15), in This corresponds to the true state during planning in the pre-training task. The first term can correspond to the distribution offset between tasks, thus representing the difference between the true state in the current task and the true state in the pre-training task. The second term can correspond to the prediction error of the pre-trained model on the pre-training task.

[0031] In various embodiments, the primary cause can be determined by calculating the total variation (TV) distance associated with these components. For example, the true state probability (P) and the predicted dynamics (P) can be determined. The TV distance between them. The upper limit of the TV distance is... It can be expressed by formula 16: (16).

[0032] The upper limit of the prediction error can be expressed by Equation 17: (17).

[0033] In some embodiments, the upper limit of performance degradation can be characterized by Equation 18: (18), Where L Q Equation 19 represents: (19).

[0034] Pre-training strategy With the optimal strategy The policy offset between them can be represented by Equation 20: (20).

[0035] Therefore, the upper limit of the performance gap can be expressed by Equation 21: (twenty one), in Equation 22 represents: (twenty two).

[0036] The upper limit of the performance gap (e.g., Equation 21) indicates the two main reasons affecting performance and causing performance degradation. For example, and This can represent a mismatch in dynamic models, while This can represent a mismatch between the prediction model and the rule / policy. Therefore, a mismatch between the prediction model and the rule / policy can be compared to identify the main causes of performance degradation. Fine-tuning can then be more precisely focused on the key components affecting performance degradation, i.e., the prediction component or the policy / rule component.

[0037] In various embodiments, it may be necessary to normalize the errors or mismatches associated with each component to each other in order to accurately determine the primary causes and their contribution to performance degradation. In improved implementations, this can be achieved using a threshold (e.g., 2, 10, or 50) that normalizes the errors associated with the dynamic model and those associated with the policy. Therefore, the primary causes and selective fine-tuning can be determined by the relationship in Equation 23: (twenty three), Here, C represents a threshold such that if true, the dynamic model is fine-tuned or updated, for example, by retraining on data from the current task. However, if false, the policy can be fine-tuned or updated. In improved schemes, the threshold (C) can be a function of certain criteria. In various embodiments, this parameter can be determined through trial and error (i.e., grid search), comparison / analysis with other fine-tuning strategies (e.g., model-only, policy-only, or alternating iterative fine-tuning). For example, the threshold (C) can be a function of C1 and C2, which can be associated with certain criteria to accurately determine the dominant mismatch. Function C1 can be represented by Equation 24: (twenty four), Function C2 can be represented by formula 25: (25).

[0038] In various embodiments, this criterion can be summarized by the following formulas 26 and 27: (26), and (27).

[0039] Therefore, if the criterion represented by Equation 26 is true, the error associated with the prediction model is likely the main cause of performance degradation, making fine-tuning the prediction model more effective in improving the system; if the criterion represented by Equation 27 is true, suboptimal policy decisions are likely the main cause of performance degradation, making fine-tuning the policy more effective in improving the system. In various embodiments, simpler criteria can be used to reduce computational complexity.

[0040] For example, Figure 6A The TV distances of the dynamic and policy components of a model with adaptive fine-tuning and a threshold of 2 are shown. Figure 6B The TV distances of the dynamic and policy components of the model with adaptive fine-tuning and a threshold of 10 are shown. Figure 6CThe TV distance between the dynamic and policy components of a model employing adaptive fine-tuning with a threshold of 50 is demonstrated. In the improved scheme, this threshold can be greater than 1, 1.5, 2, 5, 9, 10, 25, 45, or 50, and / or less than 100, 75, 55, 30, 15, or 5. In other words, the threshold can be at least 2, at least 5, at least 10, at least 25, or at least 50, and / or not greater than 5, 15, 30, 55, 75, or 100.

[0041] In various embodiments, the fine-tuning phase may include a variety of fine-tuning techniques, such as low-rank (LoRa) fine-tuning strategies. Alternatively or in combination, full or end-to-end fine-tuning, parameter-efficient fine-tuning (PEFT), adapter layers (e.g., AdapterFusion and Houlsby adapters), prefix or soft-cue fine-tuning, bias term fine-tuning (BitFit), quantized LoRa (QLoRa), and / or human-feedback-based reinforcement learning (RLHF) may also be used. In one or more embodiments, LoRa fine-tuning may employ gentler variations, thus being more effective because it avoids drastic or excessive changes. In improved schemes, fine-tuning of the prediction component may, for example, include decomposing the model parameters into (two) lower-dimensional vectors, such as the latent representation basis vector Z and the vector... In at least some embodiments, only the weights B of the basis vectors can be updated. For example, such fine-tuning can be represented by Equation 28: (28).

[0042] In various embodiments, this phase may include alignment with the new data distribution. In improved implementations, fine-tuning the planning components may, for example, involve adjusting the policy network. Decomposed into sub-units The weighted combination is represented by Equation 29: (29).

[0043] In some embodiments, only the weights are updated during fine-tuning. .

[0044] In one or more embodiments, centralized or distributed fine-tuning techniques may be used. For example, fine-tuning may be performed on one or more local devices of one or more users, or on a centralized location (e.g., one or more servers, such as one or more web servers). In one variant, an aggregated dataset may be used for training and / or fine-tuning. In improved embodiments, commercial or other available autonomous driving datasets may be used, such as Waymo, nuScenes, Argoverse, LyftLevel5, KITTI, ApolloScape, BDD100K, and / or PandaSet datasets. In various embodiments, fine-tuning may not include injecting noise into the fine-tuning or training updates / gradients. In one or more embodiments, all, some, or at least one parameter of the prediction or planning component may be updated during fine-tuning.

[0045] See Figure 7 A method 700 for training an artificial intelligence model is provided. In various embodiments, the training strategy includes a pre-training-fine-tuning paradigm. Method 700 includes: pre-training a prediction model to derive a policy on an autonomous driving dataset (i.e., step 702); applying the trained model and policy in a runtime environment (i.e., step 704); identifying primary error sources (i.e., step 706); and fine-tuning the prediction model or policy based on the primary error sources (i.e., step 708). In one or more embodiments, the prediction model may be a world model implementing a recurrent neural network. Pre-training can be performed using an available autonomous driving dataset for at least 8 hours, at least 10 hours, or at least 12 hours. In various embodiments, errors can be identified as described herein. In improved embodiments, fine-tuning may include retraining the model using observations from the runtime environment. In various embodiments, fine-tuning may be repeated for at least 30 minutes, at least 45 minutes, or at least 1 hour.

[0046] Figure 8 A system 800 is illustrated that can be used to train and / or run one or more models and / or components described herein. System 800 may include at least one computing system and / or device 802. Computing system 802 may include at least one processor 804, which is connected to memory unit 808 during operation. Processor 804 may be one or more integrated circuits that implement the functionality of a central processing unit (CPU) 806. Alternatively, processor 804 may be designed to implement the functionality of a graphics processing unit (GPU). CPU 806 may be a commercially available processing unit that implements an instruction set such as x86, ARM, Power, or MIPS instruction set families.

[0047] During operation, CPU 806 can execute stored computer-readable / machine / program instructions retrieved from memory unit 808. These stored computer-readable / machine / program instructions may include software that controls the operation of CPU 806 to perform the operations described herein. In some embodiments, processor 804 may be a system-on-a-chip (SoC) that integrates the functionality of CPU 806, memory unit 808, network interface, and input / output interface into a single integrated device. Computing system 802 may implement an operating system for managing various aspects of operation.

[0048] Storage unit 808 may include volatile and non-volatile memory for storing instructions and data. Non-volatile memory may include solid-state memory, such as NAND flash memory, magnetic storage media, and optical storage media, or any other suitable storage device that retains data when the computing system 802 is disabled or powered off. Volatile memory may include static and dynamic random access memory (RAM) that stores computer-readable / machine / program instructions and data. For example, storage unit 808 may store a machine learning model 810 or algorithm, a training dataset 812 of the machine learning model 810, and / or raw source data 815.

[0049] The computing system 802 may include a network interface device 822 configured to communicate with external systems and devices. For example, the network interface device 822 may include wired and / or wireless Ethernet interfaces defined by the Institute of Electrical and Electronics Engineers (IEEE) 802.11 family of standards. The network interface device 822 may include a cellular communication interface for communicating with cellular networks (e.g., 3G, 4G, 5G). The network interface device 822 may be further configured to provide a communication interface with an external network 824 or the cloud.

[0050] The external network 824 can be the World Wide Web or the Internet. The external network 824 can establish standard communication protocols between computing devices. The external network 824 allows for easy exchange of information and data between computing devices and the network. One or more servers 830 can communicate with the external network 824.

[0051] The computing system 802 may include an input / output (I / O) interface 820, which may be configured to provide digital and / or analog inputs and outputs. The I / O interface 820 may include an additional serial interface for communicating with external devices. For example, the I / O interface 820 may be configured to receive data from a sensor providing sensing signals.

[0052] The computing system 802 may include a human-machine interface (HMI) device 818, which may include any device that enables the system 800 to receive control input. Examples of input devices may include HMI inputs such as a keyboard, computer mouse, touchscreen, microphone or other voice input device, vehicle infotainment screen (which may include multiple buttons, knobs and / or touchscreens, microphones), and other similar devices. The computing system 802 may include a display device 832. The computing system 802 may include hardware and software for outputting graphical and textual information to the display device 832. The display device 832 may include an electronic display screen, projector, printer, or other suitable device for displaying information to a user or operator. The computing system 802 may be further configured to allow interaction with a remote HMI and a remote display device via a network interface device 822.

[0053] System 800 can be implemented using one or more computing systems. Although the example depicts a single computing system 802 implementing all the described features, in practice, various features and functions can be decoupled and implemented by multiple interconnected computing units. The specific system architecture chosen can depend on a variety of factors.

[0054] System 800 may implement machine learning algorithm 810, which is configured to analyze dataset 815. Dataset 815 may include an input dataset for the machine learning system. In one or more embodiments, the dataset may include raw data or sensor data that may represent the input dataset. In some examples, machine learning algorithm 810 may be a recurrent neural network algorithm designed to perform a predetermined function. For example, the neural network may be used in conjunction with the embodiments described herein.

[0055] System 800 may store a training dataset 812 for machine learning algorithm 810 (e.g., CARLA, an urban driving dataset for autonomous driving). Training dataset 812 may represent a set of previously constructed data used to train machine learning algorithm 810. Machine learning algorithm 810 can use training dataset 812 to learn weight factors associated with the machine learning algorithm and / or planning strategies. Training dataset 812 may include a set of source data corresponding to the results that machine learning algorithm 810 attempts to replicate through the learning process.

[0056] In one or more embodiments, the methods, systems, and models described herein may be embodied in algorithms, for example, on non-transient computer-readable media, which may reside in one or more computing platforms or devices, such as... Figure 8 As shown.

[0057] Computing platforms, such as Figure 9The computing platform 900 shown can be used to implement the models and / or methods described herein. The models and / or methods described herein can be implemented as part of a computing software suite. The computing platform 900 may include memory 902, processor 904, and non-volatile storage device 906. Processor 904 may include one or more devices selected from high-performance computing (HPC) facilities, such as high-performance cores, microprocessors, microcontrollers, digital signal processors, microcomputers, central processing units, graphics processing units, tensor processing units, field-programmable gate arrays, programmable logic devices, state machines, logic circuits, analog circuits, digital circuits, or any other device that processes signals (analog or digital) based on computer-executable instructions residing in memory 902. High-performance computing facilities may include advanced computing hardware and software configured to perform computationally intensive tasks at speeds far exceeding those of typical desktop or server computers. High-performance computing facilities can provide up to one million random access input / output operations per second (1) of a kind.

[0058] Memory 902 may include a single storage device or multiple storage devices, including but not limited to random access memory (RAM), volatile memory, non-volatile memory, static random access memory (SRAM), dynamic random access memory (DRAM), flash memory, cache, or any other device capable of storing information. Non-volatile storage device 906 may include one or more persistent data storage devices, such as hard disk drives, optical drives, tape drives, non-volatile solid-state devices, cloud storage devices, or any other device capable of persistently storing information.

[0059] Processor 904 may be configured to read the contents of memory 902 and execute computer-executable instructions residing in software modules 908 and 910 of non-volatile storage device 906, which embody models and / or methods of one or more embodiments. Software modules 908 and 910 may include operating systems and applications. Software modules 908 and 910 may be compiled or interpreted from computer programs created using various programming languages ​​and / or technologies, including but not limited to Java, C, C++, C#, Objective C, Fortran, Pascal, JavaScript, Python, Perl, and PL / SQL, whether used alone or in combination.

[0060] After execution by processor 904, the computer-executable instructions of software modules 908 and 910 enable computing platform 900 to implement one or more models and / or methods of one or more embodiments disclosed herein. Non-volatile storage device 906 may also include data 912 and 914 supporting the functions, features, and processes of one or more embodiments described herein.

[0061] Program code embodying the algorithms and / or methods described herein can be distributed individually or collectively as a program product in various different forms. The program code can be distributed via a computer-readable storage medium carrying computer-readable program instructions for causing a processor to execute aspects of one or more embodiments. Computer-readable storage media are inherently non-transient and can include tangible media that are volatile and non-volatile, removable and non-removable, employing any method or technology to store information, such as computer-readable instructions, data structures, program modules, or other data. Computer-readable storage media can also include random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other solid-state memory technologies, portable optical disc read-only memory (CD-ROM) or other optical storage devices, magnetic tape, magnetic tape, disk storage devices or other magnetic storage devices, or any other medium that can be used to store desired information and is readable by a computer. Computer-readable program instructions can be downloaded from the computer-readable storage medium to a computer, other types of programmable data processing equipment or other devices, or downloaded via a network to an external computer or external storage device.

[0062] Computer-readable program instructions stored in a computer-readable medium can be used to direct a computer, other type of programmable data processing device, or other device to operate in a particular manner, such that the instructions stored in the computer-readable medium can produce an article of art including instructions that implement the functions, behaviors, and / or operations specified in the flowchart or schematic diagram. In some alternative embodiments, the functions, behaviors, and / or operations specified in the flowchart and schematic diagram can be reordered, processed sequentially, and / or processed in parallel, consistent with one or more embodiments. Furthermore, any flowchart and / or schematic diagram may include more or fewer nodes or modules than shown, consistent with one or more embodiments.

[0063] In one or more embodiments described herein, the model described herein can be used in conjunction with a control system (e.g., a vehicle control system) to, for example, operate a vehicle. The vehicle control system can receive information from the model and provide information to the model. For example, the vehicle control system may include one or more sensors and provide sensing data or other data to a machine learning system. The vehicle control system may also receive output from the model, such as decisions / tasks / actions guided by a planning strategy. This output, or the inferences derived therefrom, can serve as, or initiate, the transmission of, one or more drive signals to control or drive equipment (e.g., equipment of a robot, robotic arm, or vehicle, such as throttle, steering system, brakes, signal lights, etc.). Figure 1As shown, a vehicle or vehicle control system may include various sensors to acquire data / measurements. The control system may be configured to determine or acquire drive signals from the model or system described herein. These drive signals can be used to control vehicle equipment. The drive signals may be provided to or transmitted to vehicle equipment. Vehicle equipment can be activated and controlled using the drive signals. For example, vehicle equipment may include one or more actuators for accommodating or inhibiting motion, and the drive signals may control one or more actuators.

[0064] Unless otherwise explicitly stated in the examples, all numerical values ​​indicating amounts of material or reaction and / or conditions of use in this specification should be understood as modified from the word "about" to describe the broadest scope of the invention. Practice is generally preferred within the numerical limits mentioned.

[0065] The initial definition of acronyms or other abbreviations applies to all subsequent use of the same abbreviations herein, and is analogous to the normal grammatical variations applicable to the initially defined abbreviations. Unless otherwise expressly stated, the measurement of attributes is determined by the same technique as for the same attributes mentioned previously or subsequently.

[0066] It must also be noted that, unless the context clearly specifies otherwise, the singular forms “a,” “an,” and “the” in the specification and appended claims all refer to the plural. For example, referring to a component in the singular form means referring to multiple components.

[0067] As used herein, "substantially," "usually," or "approximately" means that the quantity or value in question can be a specific value or another value close to it. Typically, the term "approximately" indicates a specific value, meaning that the value is within + / - 5%. For example, "approximately 100" means a range of 100 + / - 5, i.e., a range of 95 to 105. Generally, when the term "approximately" is used, it is expected that similar results or effects can be obtained within + / - 5% of the indicated value according to the invention. The term "substantially" may modify the value or relative feature disclosed or claimed in this disclosure. In this case, "substantially" may mean that the modified value or relative feature is a fraction of the original value or relative feature. Within the range of 0%, 0.1%, 0.5%, 1%, 2%, 3%, 4%, 5%, or 10%.

[0068] It should also be noted that the integer range explicitly includes all intermediate integers. For example, the integer range 1-10 explicitly includes 1, 2, 3, 4, 5, 6, 7, 8, 9, and 10. Similarly, the range 1 to 100 includes 1, 2, 3, 4, ..., 97, 98, 99, and 100. Likewise, when any range needs to be specified, the intermediate number, which is the increment of the difference between the upper and lower limits divided by 10, can be used as the alternative upper or lower limit. For example, if the range is 1.1 to 2.1, the following numbers could be chosen as the lower or upper limits: 1.2, 1.3, 1.4, 1.5, 1.6, 1.7, 1.8, 1.9, and 2.0. Similarly, wherever integers are listed in this document, it should be understood that the list of integers explicitly includes the range between any two integers in the list.

[0069] As used herein, the term "and / or" means that all or only one element of the group may be present. For example, "A and / or B" means "A only, or B only, or both A and B are present." In the case of "A only," the term also covers the case where B is not present, i.e., "A only, but not B." It should also be understood that the invention is not limited to the specific embodiments and methods described below, as specific components and / or conditions may, of course, vary. Furthermore, the terminology used herein is only for describing specific embodiments of the invention and is not intended to limit the invention in any way.

[0070] The term “comprising” is synonymous with “including,” “having,” “containing,” or “characterized by.” These terms are inclusive and open-ended, and do not exclude other unlisted elements or method steps. The term “consisting of” excludes any element, step, or component not specified in the claim. When this phrase appears in a clause of the body of a claim, rather than immediately following the preamble, it defines only the element described in that clause; other elements are not excluded from the claim as a whole. The term “substantially constitutes” limits the scope of the claim to the specified material or step, as well as those that do not substantially affect the essential and novel features of the claimed subject matter. The term “one or more” means “at least one,” and the term “at least one” also means “one or more.” Both the terms “one or more” and “at least one” include “a plurality” as a subset thereof.

[0071] While exemplary embodiments have been described above, these embodiments are not intended to describe all possible forms covered by the claims. The terminology used in this specification is descriptive and not restrictive, and it should be understood that various modifications may be made without departing from the spirit and scope of this disclosure. As previously stated, features of various embodiments may be combined to form other embodiments of the invention, which may not be explicitly described or illustrated in the specification. While various embodiments may be described as superior to other embodiments or the prior art in one or more desired characteristics, those skilled in the art will recognize that compromises may be made on one or more features or characteristics to achieve the desired overall system properties, and these properties depend on the specific application and implementation. These properties include, but are not limited to, cost, strength, durability, lifecycle cost, marketability, appearance, packaging, size, maintainability, weight, manufacturability, ease of assembly, etc. Therefore, if any embodiment is described as inferior to other embodiments or the prior art in one or more characteristics, such embodiments do not exceed the scope of this disclosure and may be desirable in a particular application.

Claims

1. A method for training an autonomous vehicle system, the method comprising: The first component and the second component are pre-trained using a dataset. The first component is configured to predict future events based on sensor inputs, and the second component is configured to determine one or more actions based on the future events. After pre-training, the first component and the second component form a pre-trained model. The pre-trained model is applied in an operating environment that provides sensor input to identify the main sources of error; as well as In response to identifying the primary source of error, the first component or the second component is updated based on the primary source of error.

2. The method as described in claim 1, wherein, Identifying the primary sources of error includes comparing a first error associated with the first component and a second error associated with the second component.

3. The method as described in claim 2, wherein, The first component is a prediction component, and the second component is a planning component.

4. The method of claim 3, wherein, The first error is a construction error associated with the prediction component, and the second error is a distribution offset associated with the planning component, such that if the construction error is dominant, the prediction component is retrained using sensor input from the operating environment, and if the distribution offset is dominant, the planning component is adjusted.

5. The method of claim 4, wherein, The prediction component is a world model, and the planning component is a planning strategy for determining one or more actions based on the predictions of the world model.

6. The method of claim 5, wherein, The world model includes recurrent neural networks.

7. The method of claim 1, wherein, Updating the first component includes low-rank fine-tuning techniques.

8. The method of claim 7, wherein, The low-rank fine-tuning technique involves decomposing the model parameters into two lower-dimensional vectors.

9. The method of claim 1, further comprising: In response to updating the first component or the second component, control one or more actuators of the vehicle based on one or more actions of the second component.

10. A training system for training an autonomous vehicle system, the training system comprising: One or more computing devices for: Use the dataset to pre-train an artificial intelligence model with a first component and a second component to form a pre-trained model; The pre-trained model is applied to one or more tasks to receive one or more observations; In response to the application of the pre-trained model, identifying whether the first component or the second component is the main cause of performance degradation based on the one or more observations; as well as Based on whether the first component or the second component is the main cause of performance degradation, selectively fine-tune the first component or the second component.

11. The system of claim 10, wherein, Selective fine-tuning includes: if a first component of the pre-trained model is the primary cause of performance degradation, then retraining the first component based on one or more observations; if a second component of the pre-trained model is the primary cause of performance degradation, then adjusting the second component.

12. The system of claim 10, wherein, The first component is a prediction component, and the second component is a planning component.

13. The system of claim 12, wherein, The primary cause is determined by comparing the total variation distance of the prediction component with the total variation distance of the planning component.

14. The system of claim 13, wherein, The total variation distance of the prediction component represents the difference between the predicted state and the actual state of the first component, while the total variation distance of the planning component represents the difference between the optimal planning strategy and the current planning strategy.

15. The system of claim 14, wherein, A threshold is applied to the total variation distance of the planning component before comparing it with the total variation distance of the prediction component.

16. The system of claim 15, wherein, The threshold is greater than 1.

17. The system of claim 15, wherein, The threshold is at least 10.

18. The system of claim 12, wherein, The prediction component is a world model configured to simulate real-world environments in order to predict outcomes.

19. The system of claim 12, wherein, The one or more observations are sensor inputs from the vehicle control system.

20. A non-transient computer-readable medium storing instructions that are executed by one or more processors to optimize an autonomous driving model by performing the following functions: In response to the identified error, the world model of the pre-trained model is retrained based on data from the operating environment; and In response to the detection of a distribution shift, the strategy of the pre-trained model is adjusted based on data from the operating environment.

Images