Bayesian Global Optimization-Based Parameter Tuning for Vehicle Motion Controllers

Through Bayesian global optimization and Gaussian process regression model, the parameters of autonomous driving vehicle controllers are iteratively optimized, solving the problems of low efficiency and high cost in the existing technology, and achieving more efficient controller parameter optimization.

CN113525406BActive Publication Date: 2024-11-26BAIDU USA LLC
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202011476364.9
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Priority Date
2020-04-15
Filing Date
2020-12-14
Publication Date
2024-11-26
Estimated Expiration
2040-12-14

AI Technical Summary

Technical Problem

When tuning the parameters of autonomous driving vehicle controllers, the prior art relies on human experience or exhaustive search, which is inefficient and has high calculation cost, making it difficult to ensure optimization results.

Method used

Bayesian global optimization combined with Gaussian process regression model is used to iteratively generate and optimize controller parameter samples to find the best parameter configuration to optimize controller performance.

Benefits of technology

The total cost of controller parameter optimization is reduced, efficiency is improved, and the optimization effect of controller parameters is ensured.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN113525406B_ABST
    Figure CN113525406B_ABST
Patent Text Reader

Abstract

The present disclosure provides parameter tuning based on Bayesian global optimization for a vehicle motion controller. In one embodiment, a method for optimizing a controller for an autonomous driving vehicle (ADV) includes: obtaining a plurality of samples, each sample having a parameter set, iteratively performing the following operations until a predetermined condition is satisfied: determining a score for each sample according to a configuration of the controller based on the parameter set of the sample; applying a machine learning model to the samples and the corresponding scores to determine a mean function and a variance function, generating a new sample as a minimum of a function of the mean function and the variance function with respect to an input space of the parameter set, and adding the new sample to the several samples; and outputting the new sample as an optimal sample, wherein the parameters of the optimal sample are used to configure the controller to autonomously drive the ADV.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] Embodiments of the present disclosure generally relate to operating an autonomous vehicle. More particularly, embodiments of the present disclosure relate to Bayesian global optimization based parameter tuning for a vehicle motion controller. Background Art

[0002] A vehicle operating in an autonomous mode (e.g., without human control) can relieve occupants, particularly the driver, of some driving-related responsibilities. When operating in an autonomous mode, the vehicle can navigate to various locations using onboard sensors, allowing the vehicle to travel with minimal human interaction or in some cases without any passengers.

[0003] A vehicle controller of an autonomous vehicle may generate control commands to cause the vehicle to move according to a desired path or route. For example, a model predictive controller (MPC) may generate a sequence of commands to be applied at a future time frame that will cause the controlled object to move along a predicted path. The MPC may be configured according to one or more control parameters to optimize the command sequence so that the controller can control the autonomous vehicle (ADV) to track along a target path at a target speed.

[0004] Tuning control parameters plays an important role in controller design. Conventionally, the parameters of a controller can be tuned by a human observer (e.g., by an engineer in a controlled setting such as a laboratory) or by an exhaustive search using a grid search algorithm. One problem with the first solution is that the optimization of the control parameters may not be guaranteed because the solution is highly dependent on human experience and intuition. For the second solution, such an exhaustive search has low efficiency, especially when the configuration space becomes relatively large, which may increase the computational cost exponentially. Summary of the invention

[0005] In a first aspect, there is provided a method for optimizing a controller of an autonomous vehicle ADV, comprising:

[0006] Obtain a plurality of samples, each sample having a parameter set;

[0007] Iteratively perform the following operations until a predetermined condition is met:

[0008] determining a score for each sample based on a configuration of the controller based on the set of sample-based parameters;

[0009] Apply the machine learning model to multiple samples and corresponding scores to determine the mean and variance functions,

[0010] Produces new samples as the minimum of a function of the mean and variance of the input space with respect to the parameter set, and

[0011] adding a new sample to the plurality of samples; and

[0012] The new sample is output as the best sample, where the parameters of the best sample are used to configure the controller to autonomously drive the ADV.

[0013] In a second aspect, a non-transitory machine-readable medium is provided having instructions stored therein, which when executed by a processor causes the processor to perform the method for optimizing a controller of an autonomous vehicle (ADV) as described in the first aspect.

[0014] In a third aspect, there is provided a data processing system comprising:

[0015] Processor; and

[0016] The memory is coupled to the processor and stores instructions, which, when executed by the processor, cause the processor to perform the method for optimizing a controller of an autonomous vehicle (ADV) as described in the first aspect.

[0017] In a fourth aspect, a computer program product is provided, comprising a computer program, which, when executed by a processor, implements the method for optimizing a controller of an autonomous driving vehicle according to the first aspect.

[0018] According to the present disclosure, controller parameters are optimized by reducing the overall cost. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Various aspects are illustrated by way of example and not by way of limitation in the figures of the accompanying drawings, in which like reference numerals denote similar elements. It should be noted that references to "one" or "an" aspect of the present disclosure are not necessarily to the same aspect, and that they mean at least one. Furthermore, for the sake of brevity and to reduce the total number of figures, a given figure may be used to illustrate features of more than one aspect, and not all elements in a figure may be required for a given aspect.

[0020] Figure 1 is a block diagram illustrating a networking system according to one embodiment.

[0021] Figure 2 is a block diagram illustrating an example of an autonomous vehicle according to one embodiment.

[0022] Figure 3 is a block diagram illustrating an example of a controller parameter tuner according to one embodiment.

[0023] Figure 4 is a flow chart illustrating a process for tuning parameters of a controller according to one embodiment.

[0024] Figure 5A-Figure 5Bis a block diagram illustrating an example of a perception and planning system for use with an autonomous vehicle, according to one embodiment. DETAILED DESCRIPTION

[0025] Several embodiments of the present disclosure are now described with reference to the accompanying drawings. Whenever the shapes, relative positions and other aspects of the components described in a given aspect are not clearly defined, the scope of the present disclosure is not limited to the components shown, which are intended for illustration purposes only. In addition, although many details are set forth, it should be understood that some aspects can be practiced without these details. In other examples, well-known circuits, structures and technologies are not shown in detail to avoid confusing the understanding of this specification. In addition, unless the meaning is obviously contrary, all ranges listed here are deemed to include the endpoints of each range.

[0026] References in the specification to "one embodiment" or "an embodiment" mean that a particular feature, structure, or characteristic described in connection with the embodiment can be included in at least one embodiment of the present disclosure. The phrase "in one embodiment" appearing in various places in the specification does not necessarily refer to the same embodiment.

[0027] The present disclosure solves the problem of optimizing controller parameters by applying Bayesian global optimization and Gaussian Process Regression (GPR) to parameter tuning of controllers, such as motion controllers used by autonomous vehicles (ADVs). In particular, the present disclosure describes the use of Bayesian optimization techniques to find the best controller (parameter) configuration by using a GPR model as (an approximation of) an objective function (e.g., a proxy model). For example, a sample of controller parameters to be tuned is obtained (e.g., randomly generated). A score (or label) is generated for each sample by simulating a controller configured according to one or more parameters of the sample. The sample set and the corresponding scores are fitted to a machine learning model (such as a GPR model) to derive a mean prediction function and a variance function. Using the derived function, the next most promising sampling point is determined. The new sample is added to the sample set and the process is repeated (e.g., for a predetermined number of iterations) to define a sufficient exploration of the sample space. Once the process has been executed for a sufficient number of iterations, the Bayesian global optimization based on the GPR model outputs the best sample. The parameters of the best sample can be used to configure the controller to autonomously drive the ADV. Therefore, the present disclosure optimizes controller parameters by reducing the overall cost (or negative control profile score).

[0028] According to some embodiments, a computer-implemented method for optimizing a controller for an ADV includes: obtaining a plurality of samples, each sample having a parameter set; iteratively performing the following operations until a predetermined condition is met: determining a score for each sample based on a configuration of the controller based on the parameter set of the sample, applying a machine learning model to the plurality of samples and the corresponding scores to determine a mean function and a variance function, generating a new sample that is a minimum of a utility function that combines the mean function and the variance function with respect to an input space of the parameter set, and adding the new sample to the plurality of samples; outputting the new sample as an optimal sample, wherein the parameters of the optimal sample are used to configure the controller to autonomously drive the ADV.

[0029] In one embodiment, obtaining a plurality of samples comprises randomly generating at least some of the samples. In another embodiment, the machine learning model is a Gaussian process regression (GPR) model. In some embodiments, the machine learning model is a Tree Parzen Estimator (TPE) model. In one embodiment, determining a score comprises: for each sample, configuring a controller with one or more parameters in a parameter set of the sample; and simulating the performance of the configured controller, wherein the score represents the simulated performance. In another embodiment, each parameter in the parameter set is a weight associated with a cost term of a cost function, and a motion controller of an autonomous vehicle (ADV) uses the cost function to generate control commands to autonomously navigate the ADV.

[0030] In one embodiment, each new sample is generated based on a previously generated new sample added to a number of samples. In another embodiment, the predetermined condition is a predetermined number of new samples to be generated.

[0031] In another embodiment of the present disclosure, a non-transitory machine-readable medium and a data processing system perform at least some of the processes described herein.

[0032] Figure 1 is a block diagram illustrating an autonomous vehicle network configuration according to one embodiment of the present disclosure. Figure 1, network configuration 100 includes an autonomous driving vehicle (ADV) 101, which can be communicatively coupled to one or more servers 103-104 via a network 102. Although one autonomous vehicle is shown, multiple autonomous vehicles can be coupled to each other and / or to servers 103-104 via network 102. Network 102 can be any type of wired or wireless network, such as a local area network (LAN), a wide area network (WAN) such as the Internet, a cellular network, a satellite network, or a combination thereof. (One or more) servers 103-104 can be any type of server or server cluster, such as a web or cloud server, an application server, a back-end server, or a combination thereof. Servers 103-104 can be data analysis servers, content servers, traffic information servers, maps and points of interest (MPOI) servers or location servers, etc.

[0033] An autonomous vehicle refers to a vehicle that is capable of being configured in an autonomous mode in which the vehicle navigates through an environment with little or no driver input. Such an autonomous vehicle may include a sensor system having one or more sensors configured to detect information about the environment in which the vehicle is operating. The vehicle and its associated controller(s) use the detected information to navigate through the environment. Autonomous vehicle 101 may operate in a manual mode, a fully autonomous mode, or a partially autonomous mode.

[0034] In one embodiment, autonomous vehicle 101 includes, but is not limited to, perception and planning system 110, vehicle control system 111, wireless communication system 112, user interface system 113, and sensor system 115. Autonomous vehicle 101 may also include certain common components included in ordinary vehicles, such as an engine, wheels, a steering wheel, a transmission, etc., which may be controlled by vehicle control system 111 and / or perception and planning system 110 using various communication signals and / or commands (e.g., acceleration signals or commands, deceleration signals or commands, steering signals or commands, braking signals or commands, etc.).

[0035] The components 110-115 may be communicatively coupled to each other via an interconnect, a bus, a network, or a combination thereof. For example, the components 110-115 may be communicatively coupled to each other via a controller area network (CAN) bus. The CAN bus is a vehicle bus standard designed to allow microcontrollers and devices to communicate with each other in applications without a host computer. It is a message-based protocol that was originally designed for multiplexed electrical wiring within vehicles, but is also used in many other environments.

[0036] Reference now Figure 2In one embodiment, the sensor system 115 includes, but is not limited to, one or more cameras 211, a global positioning system (GPS) unit 212, an inertial measurement unit (IMU) 213, a radar unit 214, and a light detection and range (LIDAR) unit 215. The GPS system 212 may include a transceiver operable to provide information about the location of the autonomous vehicle. The IMU unit 213 may sense the position and orientation changes of the autonomous vehicle based on inertial acceleration. The radar unit 214 may represent a system that uses radio signals to sense objects within the local environment of the autonomous vehicle. In some embodiments, in addition to sensing objects, the radar unit 214 may additionally sense the speed and / or heading of the object. The LIDAR unit 215 may use lasers to sense objects in the environment in which the autonomous vehicle is located. The LIDAR unit 215 may include one or more laser sources, a laser scanner, and one or more detectors, among other system components. The camera 211 may include one or more devices to capture images of the environment surrounding the autonomous vehicle. The camera 211 may be a static camera and / or a video camera. The camera may be mechanically movable, for example by mounting the camera on a rotating and / or tilting platform.

[0037] The sensor system 115 may also include other sensors, such as sonar sensors, infrared sensors, steering sensors, throttle sensors, brake sensors, and audio sensors (e.g., microphones). The audio sensor may be configured to capture sounds from the environment surrounding the autonomous vehicle. The steering sensor may be configured to sense the steering angle of the steering wheel, wheels, or a combination thereof of the vehicle. The throttle sensor and the brake sensor sense the throttle position and brake position of the vehicle, respectively. In some cases, the throttle sensor and the brake sensor may be integrated into an integrated throttle / brake sensor.

[0038] In one embodiment, the vehicle control system 111 includes, but is not limited to, a steering unit 201, a throttle unit 202 (also referred to as an acceleration unit), and a brake unit 203. The steering unit 201 is used to adjust the direction or heading of the vehicle. The throttle unit 202 is used to control the speed of the motor or engine, which in turn controls the speed and acceleration of the vehicle. The brake unit 203 decelerates the vehicle by providing friction to slow down the wheels or tires of the vehicle. Note that Figure 2 The components shown may be implemented in hardware, software or a combination thereof.

[0039] Return to reference Figure 1, the wireless communication system 112 allows communication between the autonomous vehicle 101 and external systems, such as devices, sensors, other vehicles, etc. For example, the wireless communication system 112 can wirelessly communicate with one or more devices (such as servers 103-104 on the network 102) directly or via a communication network. The wireless communication system 112 can use any cellular communication network or wireless local area network (WLAN), such as using WiFi to communicate with another component or system. The wireless communication system 112 can communicate directly with devices (e.g., a passenger's mobile device, a display device, a speaker within the vehicle 101) using, for example, an infrared link, Bluetooth, etc. The user interface system 113 can be part of a peripheral device implemented within the vehicle 101, including, for example, a keyboard, a touch screen display device, a microphone, and a speaker, etc.

[0040] Some or all of the functions of autonomous vehicle 101 may be controlled or managed by perception and planning system 110, particularly when operating in an autonomous driving mode. Perception and planning system 110 includes the necessary hardware (e.g., processor(s), memory, storage devices) and software (e.g., operating system, planning and routing programs) to receive information from sensor system 115, control system 111, wireless communication system 112, and / or user interface system 113, process the received information, plan a route or path from a starting point to a destination point, and then drive vehicle 101 based on the planning and control information. Alternatively, perception and planning system 110 may be integrated with vehicle control system 111.

[0041] For example, a user as a passenger may specify a starting location and a destination for a trip, for example, via a user interface. The perception and planning system 110 obtains data related to the trip. For example, the perception and planning system 110 may obtain location and route information from an MPOI server, which may be part of the servers 103-104. The location server provides location services, and the MPOI server provides map services and POIs for certain locations. Alternatively, such location and MPOI information may be cached locally in a permanent storage device of the perception and planning system 110.

[0042] As the autonomous vehicle 101 moves along the route, the perception and planning system 110 may also obtain real-time traffic information from a traffic information system or server (TIS). Note that the servers 103-104 may be operated by a third-party entity. Alternatively, the functionality of the servers 103-104 may be integrated with the perception and planning system 110. Based on the real-time traffic information, MPOI information, and location information, as well as real-time local environmental data (e.g., obstacles, objects, nearby vehicles) detected or sensed by the sensor system 115, the perception and planning system 110 may plan an optimal route and, for example, drive the vehicle 101 according to the planned route via the control system 111 to reach a designated destination safely and efficiently.

[0043] The server 103 may be a data analysis system for performing data analysis services for various clients. In one embodiment, the data analysis system 103 includes a data collector 121 and a machine learning engine 122. The data collector 121 collects driving statistics 123 from various vehicles (autonomous vehicles or conventional vehicles driven by human drivers). The driving statistics 123 include information indicating driving commands issued (e.g., throttle, brake, steering commands) and responses of the vehicle captured by the vehicle's sensors at different points in time (e.g., speed, acceleration, deceleration, direction). The driving statistics 123 may also include information describing the driving environment at different points in time, such as routes (including starting and destination locations), MPOI, road conditions, weather conditions, etc.

[0044] Based on the driving statistics 123, the machine learning engine 122 generates or trains a set of rules, algorithms, and / or prediction models 124 for various purposes. In one embodiment, the prediction models 124 may include parametric and / or non-parametric models. For example, the models may include a Gaussian process regression (GPR) model and / or a tree Parsen estimator (TPE) model, each of which may be used for parameter tuning as described herein.

[0045] As shown, the machine learning engine 122 includes a controller parameter tuner 125, which is configured to perform parameter tuning based on Bayesian global optimization as described herein. In one embodiment, the tuner 125 can be configured to tune one or more parameters of any type of controller (e.g., a motion controller). For example, the tuner can be configured to tune parameters of a proportional-integral-derivative (PID) controller, a linear quadratic regulator (LQR) controller, or a model predictive control (MPC) controller.

[0046] In one embodiment, the controller parameter (or parameters) can be any type of parameter (or setting) of the controller. For example, a PID controller can have at least three parameters, one for proportional gain, one for integral gain, and one for derivative gain. As another example, the parameter can be a weight associated with and applied to a cost term of a cost function, and a motion controller (e.g., an MPC controller) of the ADV uses the cost function to generate control commands to autonomously navigate the ADV. More information about motion controllers is described herein.

[0047] The algorithm 124 may then be uploaded onto the ADV for real-time use in autonomous driving. In one embodiment, one or more controller parameters that are optimally tuned, as described herein, may be uploaded onto the ADV for real-time use by a motion controller operating therein.

[0048] Figure 3 1 is a block diagram illustrating an example of a controller parameter tuner according to one embodiment. As described herein, the controller parameter tuner 125 may be executed by the server 103 (e.g., one or more processors of the server) to optimally tune one or more controller parameters that may be used by the motion controller of the ADV. The tuner includes a configuration generator 305, a configuration parser 310, a modeler 315, and a decision logic 320. The generator is configured to generate one or more (initial) samples of the controller parameters. For example, the generator may generate x m samples, where each x sample includes d parameters for which the tuner will optimally tune. Thus, the generator generates a d-dimensional vector x m ∈Rd, where each training point represents a control parameter setting. For example, for a PID controller, m=3. In one embodiment, the generator can randomly generate samples x m In another embodiment, the generator may obtain one or more samples from a library or from user input.

[0049] The configuration profiler 310 is configured to obtain a pool of samples (e.g., initial samples from the configuration generator 305), and is configured to determine a score (or label) y for each sample based on the configuration of the controller based on the parameters of the sample. Specifically, the profiler configures the controller with one or more parameters of the sample (e.g., associated with d parameters). The profiler then evaluates the profiled response of the configuration. Specifically, the profiler simulates the performance of the configured controller and generates a score indicative of the simulated performance. Thus, the profiler generates a score for each sample, thereby generating y m∈R, which represents a negative control profile score. In one embodiment, a low score (e.g., below a threshold) indicates a low (or less satisfactory) performance of the controller, while a high score (e.g., above a threshold) indicates a high (or more desirable) performance of the controller.

[0050] The modeler 315 is configured to obtain x m and m , and is configured to be applied to the obtained samples and scores (or to train a machine learning model such as a GPR model, for example, with the obtained samples and scores as training data). Once applied, the Gaussian process can be updated to predict the score y* for the input sample x*. In one embodiment, the input sample is a test input sample during a first pass through the Gaussian process. In another embodiment, the input sample can be a new sample generated by a previous iteration of Bayesian global optimization as described herein. The output of the GPR process is the mean function h(x*) and the variance function v(x*), which defines the prediction Gaussian process as

[0051]

[0052]

[0053] Where X is the entire input space, x* is the input sample, is the variance of the noise, I is the identity matrix, y is one or more scores, and K is the kernel function.

[0054] The modeler 315 is configured to generate new samples X*, such as

[0055]

[0056] where the weighting factor k is chosen to weigh the trade-off between exploring more potentially good points with high uncertainty (e.g., with high variance) or exploiting the knowledge found so far about the current best point (with the best configuration). It is called the utility function.

[0057] Therefore, the new sample X* is the minimum of the utility function that combines the mean function and the variance function with respect to the input space X of the parameter set of the input sample. As described herein, the new sample is determined using the Gaussian model described herein. In one embodiment, the sample can be determined using any type of model (e.g., a TPE model) to derive the new sample from the mean function and the variance function.

[0058] Decision logic 320 is configured to obtain new samples X* from the modeler and determine whether the tuner has performed a sufficient amount of exploration. In one embodiment, decision logic 320 can determine whether a predetermined (end) condition is met. For example, the predetermined condition can be a predetermined number of iterations or new samples to be generated (e.g., 100 new samples). In another embodiment, the determination can be based on an error threshold between the new samples. For example, if the error (or difference) between at least some of the new samples is below a threshold (e.g., 0.1%), it can be determined that there is a sufficient amount of exploration and that the latest sample is the best.

[0059] If not, the new sample is added to the current pool of samples (e.g., for the second pass, the pool is now x m+1 ), thereby increasing the sample pool to continue exploration. In particular, as long as the end condition is not met, the tuner iteratively performs at least some of the operations of the configuration analyzer 310 and / or the modeler 315 to generate new samples, wherein each new sample is generated based on a previously generated new sample added to the sample pool. In one embodiment, to perform the next iteration, the latest sample generated (e.g., in the previous iteration) can be the input sample of the current iteration. Therefore, the tuner will continue this loop until the decision logic 320 determines that sufficient exploration has been performed (e.g., after the new sample x is generated). m+n At this point, decision logic 320 outputs the best sample. In one embodiment, the best sample is the latest sample to be generated (e.g., x m+n ). In another embodiment, the sample that is output as the best sample is selected based on some criteria (e.g., below and / or above a threshold). In one embodiment, outputting the best sample may include sending the best sample as a controller configuration (e.g., via a wireless network) to one or more ADVs to configure the controller with the best sample to autonomously drive the ADV.

[0060] Figure 4 4 is a flow chart of a process 400 for tuning parameters of a controller according to one embodiment. Specifically, the process 400 determines an optimal sample of one or more controller parameters for configuring a controller (such as a motion controller of an ADV). At least some of the operations described herein may be performed by Figure 3 The controller parameter tuner 125 of the server 103 is shown to execute.

[0061] The process 400 obtains a number of samples (as a sample pool), each sample having a parameter set (at box 401). For example, the configuration generator 305 can randomly generate samples, each sample having parameters to be optimally tuned. The process 400 determines a score for each sample (at box 402) based on the configuration of the controller based on the parameter set of the sample. For example, the configuration analyzer 310 simulates the performance of the controller based on the configuration based on each sample to generate a corresponding score. The process 400 applies a machine learning model (e.g., a GPR model) to the samples and the corresponding scores to determine a mean function and a variance function (at box 403). For example, the modeler 315 trains the model and determines a function relative to an input sample. In one embodiment, the input sample can be a random test sample. In another embodiment, the input sample can be determined based on an acquisition function (such as a desired improvement). In some embodiments, the input sample can be a newly generated sample of a previous iteration as described herein.

[0062] The process 400 generates a new sample that is the minimum of a function that is a function of a mean function and a variance function of an input space with respect to a set of parameters (at box 404). The process 400 adds the new sample to a sample pool (at box 405). The process 400 determines whether more new samples should be generated (at decision box 406). As described herein, the decision logic 320 can determine whether an end condition is met. If so, the process 400 returns to box 402 to repeat the Bayesian global optimization of the GPR model. However, if no new samples are to be generated, the process 400 outputs the (e.g., most recently generated) new sample as the best sample, where the parameters of the best sample can be used to configure the controller to autonomously drive the ADV (at box 407).

[0063] Some embodiments perform variations of process 400. For example, certain operations of the process may not be performed in the exact order shown and described. Certain operations may be performed in a continuous series of operations, some operations may be omitted, and different certain operations may be performed in different embodiments.

[0064] Figure 5A and Figure 5B is a block diagram illustrating an example of a perception and planning system for use with an autonomous vehicle according to one embodiment. System 500 may be implemented as Figure 1 A portion of an autonomous vehicle 101 includes, but is not limited to, a perception and planning system 110, a control system 111, and a sensor system 115. Figure 5A-Figure 5B The perception and planning system 110 includes but is not limited to a positioning module 501 , a perception module 502 , a prediction module 503 , a decision module 504 , a planning module 505 , a control module 506 , a routing module 507 and one or more motion controllers 508 .

[0065] Some or all of the modules 501-508 may be implemented in software, hardware, or a combination thereof. For example, these modules may be installed in permanent storage device 552, loaded into memory 551, and executed by one or more processors (not shown). Note that some or all of these modules may be communicatively coupled to Figure 2 Some or all modules of the vehicle control system 111 or Figure 2 Some or all modules of the vehicle control system 111 are integrated. Some of the modules 501-508 can be integrated together as integrated modules.

[0066] The positioning module 501 determines the current location of the autonomous vehicle 500 (e.g., using the GPS unit 212) and manages any data related to the user's trip or route. The positioning module 501 (also known as the map and route module) manages any data related to the user's trip or route. The user can log in and specify the starting location and destination of the trip, for example, via a user interface. The positioning module 501 communicates with other components of the autonomous vehicle 500 (such as map and route information 511) to obtain data related to the trip. For example, the positioning module 501 can obtain location and route information from a location server and a map and POI (MPOI) server. The location server provides location services, and the MPOI server provides map services and POIs of certain locations, which can be cached as part of the map and route information 511. When the autonomous vehicle 500 moves along the route, the positioning module 501 can also obtain real-time traffic information from a traffic information system or server.

[0067] Based on the sensor data provided by the sensor system 115 and the positioning information obtained by the positioning module 501, the perception of the surrounding environment is determined by the perception module 502. The perception information can represent the situation around the vehicle that the driver is driving as perceived by an ordinary driver. The perception can include lane configuration, traffic light signals, relative positions of another vehicle, pedestrians, buildings, crosswalks or other traffic-related signs (e.g., stop signs, yield signs), etc., such as in the form of objects. The lane configuration includes information describing one or more lanes, such as, for example, the shape of the lane (e.g., straight or curved), the width of the lane, the number of lanes in the road, one-way or two-way lanes, merging or splitting lanes, exiting lanes, etc.

[0068] The perception module 502 may include a computer vision system or functionality of a computer vision system to process and analyze images captured by one or more cameras in order to identify objects and / or features in the environment of the autonomous vehicle. Objects may include traffic signals, roadway boundaries, other vehicles, pedestrians, and / or obstacles, etc. The computer vision system may use object recognition algorithms, video tracking, and other computer vision techniques. In some embodiments, the computer vision system may map the environment, track objects, and estimate the speed of objects, etc. The perception module 502 may also detect objects based on other sensor data provided by other sensors such as radar and / or LIDAR.

[0069] For each of the objects, the prediction module 503 predicts what behavior the object will exhibit in the environment. The prediction is performed based on the perception data of the driving environment perception at that point in time, according to a set of map / route information 511 and traffic rules 512. For example, if the object is a vehicle in the opposite direction and the current driving environment includes an intersection, the prediction module 503 will predict whether the vehicle will likely move straight forward or turn. If the perception data indicates that there is no traffic light at the intersection, the prediction module 503 can predict that the vehicle may have to come to a complete stop before entering the intersection. If the perception data indicates that the vehicle is currently in a left-turn-only lane or a right-turn-only lane, the prediction module 503 can predict that the vehicle will be more likely to turn left or right, respectively.

[0070] For each of the objects, the decision module 504 makes a decision about how to handle the object. For example, for a particular object (e.g., another vehicle in an intersecting path) and its metadata describing the object (e.g., speed, direction, angle), the decision module 504 decides how to encounter the object (e.g., overtake, yield, stop, pass). The decision module 504 can make these decisions based on a set of rules (such as traffic rules or driving rules 512), which can be stored in the persistent storage device 552.

[0071] The routing module 507 is configured to provide one or more routes or paths from the starting point to the destination point. For a given trip from the starting position to the destination position, for example, received from a user, the routing module 507 obtains the route and map information 511 and determines all possible routes or paths from the starting position to the destination position. The routing module 507 can generate a reference line in the form of a topographic map for each route determined from the starting position to the destination position. The reference line refers to an ideal route or path without any other interference such as other vehicles, obstacles or traffic conditions. That is, if there are no other vehicles, pedestrians or obstacles on the road, the ADV should follow the reference line accurately or closely. Then, the topographic map is provided to the decision module 504 and / or the planning module 505. The decision module 504 and / or the planning module 505 check all possible routes to select and modify one of the best routes based on other data provided by other modules (such as traffic conditions from the positioning module 501, the driving environment perceived by the perception module 502, and the traffic conditions predicted by the prediction module 503). Depending on the particular driving circumstances at a point in time, the actual path or route used to control the ADV may be close to or different from the reference line provided by the routing module 507 .

[0072] Based on the decision for each of the perceived objects, the planning module 505 uses the reference line provided by the routing module 507 as a basis to plan the path or route of the autonomous vehicle and the driving parameters (e.g., distance, speed and / or turning angle). That is, for a given object, the decision module 504 decides what to do with the object, and the planning module 505 determines how to do it. For example, for a given object, the decision module 504 can decide to pass through the object, and the planning module 505 can determine whether to pass on the left or right side of the object. Planning and control data are generated by the planning module 505, including information describing how the vehicle 500 will move in the next movement cycle (e.g., the next route / path segment). For example, the planning and control data can indicate that the vehicle 500 moves 10 meters at a speed of 30 miles per hour (mph), and then changes to the right lane at a speed of 25mph.

[0073] Based on the planning and control data, the control module 506 controls and drives the autonomous vehicle by sending appropriate commands or signals to the vehicle control system 111 according to the route or path defined by the planning and control data. The planning and control data includes sufficient information to drive the vehicle from a first point on the route or path to a second point using appropriate vehicle settings or driving parameters (e.g., throttle, brake, steering commands) at different points in time along the route or path.

[0074] In one embodiment, the planning phase is performed in multiple planning cycles, which are also referred to as driving cycles, such as in each time interval of 100 milliseconds (ms). For each planning cycle or driving cycle, one or more control commands will be issued based on the planning and control data. That is, for every 100ms, the planning module 505 plans the next route segment or path segment, for example, including the target position and the time required for the ADV to reach the target position. Alternatively, the planning module 505 can also specify a specific speed, direction and / or steering angle, etc. In one embodiment, the planning module 505 plans the route segment or path segment for the next predetermined time period (such as 5 seconds). For each planning cycle, the planning module 505 plans the target position for the current cycle (e.g., the next 5 seconds) based on the target position planned in the previous cycle. Then the control module 506 generates one or more control commands (e.g., throttle control command, brake control command, steering control command) based on the planning and control data of the current cycle.

[0075] Note that decision module 504 and planning module 505 can be integrated into an integrated module. Decision module 504 / planning module 505 can include a navigation system or the functionality of a navigation system to determine a driving path for an autonomous vehicle. For example, the navigation system can determine a series of speeds and direction headings to affect the movement of the autonomous vehicle along a path that substantially avoids perceived obstacles while generally advancing the autonomous vehicle along a roadway-based path to a final destination. The destination can be set based on user input via user interface system 113. The navigation system can dynamically update the driving path while the autonomous vehicle is operating. The navigation system can incorporate data from a GPS system and one or more maps to determine a driving path for the autonomous vehicle.

[0076] The motion controller 508 has an optimizer and a vehicle model. The optimizer can use a cost function and a vehicle model to generate a series of control commands (e.g., throttle, steering and / or braking commands) that track the vehicle path along the target vehicle trajectory. These commands are generated when optimizing for different cost items (e.g., lateral trajectory error, driving direction error, speed, steering, acceleration, steering rate of change and / or acceleration rate of change). Each of the cost items can be represented in the cost function to penalize undesirable behavior. Controller parameters 513 (or weights) can be associated with each item and applied to each item (e.g., multiplication) to modify the impact of each item on the overall computational cost. In one embodiment, controller parameters 513 are optimization parameters tuned by controller parameter tuner 125 for motion controller 508.

[0077] A general example of a cost function for an MPC controller is shown below, where J is the total computational cost and w xis the weight corresponding to the term x (x=1, 2, ...) and N is the point along the target trajectory of the ADV.

[0078]

[0079] These items can be optimized by minimizing the computational cost J. These items can include at least one of the following: lateral trajectory error (penalty ADV distance from target trajectory), driving direction error (penalty ADV driving direction and the error between the target trajectory direction at a certain point), speed cost (penalty speed change), steering cost (penalty steering change), acceleration cost (penalty acceleration change), steering rate of change (penalty steering change how fast), braking cost (penalty braking) and acceleration rate of change (penalty acceleration change how fast). In some embodiments, the cost function includes at least two of the above items. In other embodiments, the cost function includes all of the above items. While considering the above items, a sequential control command (throttle, steering, braking) can be generated to optimally track the target trajectory.

[0080] Note that some or all of the components shown and described above may be implemented in software, hardware, or a combination thereof. For example, these components may be implemented as software installed and stored in a permanent storage device, which may be loaded and executed by a processor (not shown) in a memory to perform the processes or operations described throughout the present application. Alternatively, these components may be implemented as executable code programmed or embedded into dedicated hardware, such as an integrated circuit (e.g., a dedicated IC or ASIC), a digital signal processor (DSP), or a field programmable gate array (FPGA), which may be accessed via a corresponding driver and / or operating system from an application. In addition, these components may be implemented as a specific hardware logic or processor core in a processor or as part of an instruction set accessible via one or more specific instruction software components.

[0081] Some portions of the foregoing detailed description have been presented in terms of algorithms and symbolic representations of operations on data bits within a computer memory. These algorithmic descriptions and representations are the means used by those skilled in the data processing arts to most effectively convey the substance of their work to others skilled in the art. An algorithm is here, and generally, conceived to be a self-consistent sequence of operations leading to a desired result. The operations are those requiring physical manipulations of physical quantities.

[0082] It should be remembered, however, that all of these and similar terms are associated with the appropriate physical quantities and are merely convenient labels applied to these quantities. Unless otherwise specifically stated, it should be apparent from the above discussion that discussions throughout the specification using terms such as those set forth in the appended claims refer to the actions and processes of a computer system or similar electronic computing device that manipulates and transforms data represented as physical (electronic) quantities within the computer system's registers and memories into other data similarly represented as physical quantities within the computer system's memories or registers or other such information storage, transmission or display devices.

[0083] Embodiments of the present disclosure also relate to apparatus for performing the operations herein. Such a computer program is stored in a non-transitory computer-readable medium. A machine-readable medium includes any mechanism for storing information in a form readable by a machine (e.g., a computer). For example, a machine-readable (e.g., computer-readable) medium includes a machine (e.g., computer) readable storage medium (e.g., a read-only memory ("ROM"), a random access memory ("RAM"), a magnetic disk storage medium, an optical storage medium, a flash memory device).

[0084] The processes or methods described in the foregoing figures may be performed by processing logic including hardware (e.g., circuits, dedicated logic, etc.), software (e.g., embodied on a non-transitory computer-readable medium), or a combination of both. Although the processes or methods are described above with respect to some sequential operations, it should be understood that some of the operations described may be performed in a different order. In addition, some operations may be performed in parallel rather than sequentially.

[0085] The embodiments of the present disclosure are not described with reference to any particular programming language. It will be appreciated that a variety of programming languages ​​may be used to implement the teachings of the embodiments of the present disclosure as described herein.

[0086] In the foregoing description, embodiments of the present disclosure have been described with reference to specific exemplary embodiments of the present disclosure. Obviously, various modifications may be made thereto without departing from the broader spirit and scope of the present disclosure as set forth in the appended claims. Therefore, the description and the accompanying drawings should be considered illustrative rather than restrictive.

[0087] In some embodiments, the disclosure may include language, such as, “at least one of [element A] and [element B]”. The language may refer to one or more elements. For example, “at least one of A and B” may refer to “A”, “B”, or “A and B”. Specifically, “at least one of A and B” may refer to “at least one of A and at least one of B” or “at least one of A or B”. In some embodiments, the disclosure may include language, such as, “[element A], [element B], and / or [element C]”. The language may refer to any one of the elements or any combination thereof. For example, “A, B, and / or C” may refer to “A”, “B”, “C”, “A and B”, “A and C”, “B and C”, or “A, B, and C.”

Claims

1. A method for optimizing a controller of an autonomous vehicle (ADV), comprising: Obtaining a plurality of samples, each sample having a parameter set; wherein each parameter of the parameter set represents a weight associated with a cost term of the cost function; Iteratively perform the following operations until a predetermined condition is met: determining a score for each sample based on a configuration of the controller based on the set of sample-based parameters; Apply the machine learning model to multiple samples and corresponding scores to determine the mean and variance functions, Produces new samples as the minimum of a function of the mean and variance of the input space with respect to the parameter set, and adding a new sample to the plurality of samples; and In response to the predetermined condition being met, outputting the most recently generated new sample as the best sample, wherein the parameters of the best sample are used to configure a controller of the ADV, and the configured controller of the ADV uses the cost function to generate a control command to autonomously drive the ADV, wherein the cost function includes at least one of the following cost items: lateral trajectory error, driving direction error, steering cost, steering change rate, and braking cost. The method of claim 1 , wherein obtaining a plurality of samples comprises randomly generating at least some of the plurality of samples.

3. The method of claim 1, wherein the machine learning model comprises a Gaussian process regression model.

4. The method of claim 1, wherein the machine learning model comprises a tree Parzen estimator model.

5. The method of claim 1, wherein determining a score for each sample comprises, configuring the controller with one or more parameters from the sample's parameter set; and The performance of the controller of the configuration is simulated, where the score indicates the simulated performance. The method of claim 1 , wherein each new sample is generated based on a previously generated new sample added to the plurality of samples. The method of claim 1 , wherein the predetermined condition includes a predetermined number of new samples to be generated.

8. A non-transitory machine-readable medium having instructions stored therein, which when executed by a processor causes the processor to perform the method for optimizing a controller of an autonomous driving vehicle as described in any one of claims 1-7.

9. A data processing system comprising Processor; and A memory is coupled to the processor and stores instructions, which, when executed by the processor, cause the processor to perform the method for optimizing a controller of an autonomous driving vehicle as described in any one of claims 1 to 7.

10. A computer program product comprising a computer program, which, when executed by a processor, implements the method for optimizing a controller of an autonomous driving vehicle according to any one of claims 1 to 7.