Dynamic Model with Learning-Based Position Correction System

By using a localization prediction and correction model in autonomous vehicles and training a dynamic model with a large amount of driving statistics, the localization mismatch problem caused by sensor measurement errors is solved, thereby improving the vehicle localization accuracy and the reliability of autonomous driving algorithms.

CN112698645BActive Publication Date: 2024-11-19BAIDU USA LLC
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202010519694.5
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Priority Date
2019-10-21
Filing Date
2020-06-09
Publication Date
2024-11-19
Estimated Expiration
2040-06-09

AI Technical Summary

Technical Problem

In autonomous vehicles, positioning mismatch caused by sensor measurement errors affects the accuracy of autonomous driving algorithms.

Method used

A positioning prediction model and a positioning correction model are adopted. By receiving vehicle status and control command parameters, the vehicle position is determined, and a dynamic model is trained using a large amount of driving statistics to correct positioning errors. The positioning prediction and correction models are integrated to improve positioning accuracy.

Benefits of technology

By simulating and calibrating models, the positioning accuracy of autonomous vehicles has been improved, the impact of sensor errors has been reduced, and the reliability of autonomous driving algorithms has been enhanced.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN112698645B_ABST
    Figure CN112698645B_ABST
Patent Text Reader

Abstract

In one embodiment, a set of parameters representing a first state of an autonomous driving vehicle (ADV) to be simulated and a set of control commands to be issued at a first point in time are received. In response, a positioning prediction model is applied to the set of parameters to determine a first position of the ADV (e.g., (x, y)). A positioning correction model is applied to the set of parameters to determine a set of positioning correction factors (e.g., (Δx, Δy)). The correction factor may represent the error between the predicted position of the ADV predicted by the positioning prediction model and the ground truth measured by the vehicle's sensors. Based on the first position of the ADV and the correction factor, a second position of the ADV is determined as the simulated position of the ADV.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] Embodiments of the present disclosure generally relate to operating an autonomous vehicle. More specifically, embodiments of the present disclosure relate to determining a location of a vehicle. Background Art

[0002] A vehicle operating in an automated driving mode (e.g., without human control) can relieve occupants, particularly the driver, from some driving-related responsibilities. When operating in an automated driving mode, the vehicle can navigate to various locations using onboard sensors, allowing the vehicle to travel with minimal human interaction or in some cases without any passengers.

[0003] Motion planning and control is a key operation in autonomous driving. As part of motion planning and control, the position or positioning of the autonomous driving vehicle needs to be determined. This position or positioning of the autonomous driving vehicle can be determined based on the output of various sensors such as inertial measurement units (IMUs) or GPS sensors. However, due to the measurement errors of the sensors, there is a mismatch between the sensor output and the positioning result (e.g., the x, y position of the vehicle). This mismatch may affect the autonomous driving algorithm. Summary of the invention

[0004] According to one aspect of the present disclosure, a computer-implemented method for simulating an autonomous vehicle is provided, the method comprising: receiving a set of parameters, the set of parameters representing a first state of an autonomous vehicle to be simulated and a set of control commands to be issued at a first point in time; applying a positioning prediction model to the set of parameters to determine a first position of the autonomous vehicle; applying a positioning correction model to the set of parameters to determine a set of positioning correction factors; and determining a second position of the autonomous vehicle based on the first position and the positioning correction factors, wherein the second position is used as a simulated position of the autonomous vehicle at the second point in time. According to another aspect of the present disclosure, a non-transitory machine-readable medium storing instructions is provided, the instructions causing the processor to perform operations when executed by a processor, the operations comprising: receiving a set of parameters, the set of parameters representing a first state of an autonomous vehicle to be simulated and a set of control commands to be issued at a first point in time; applying a positioning prediction model to the set of parameters to determine a first position of the autonomous vehicle; applying a positioning correction model to the set of parameters to determine a set of positioning correction factors; and determining a second position of the autonomous vehicle based on the first position and the positioning correction factors, wherein the second position is used as a simulated position of the autonomous vehicle at the second point in time.

[0005] According to another aspect of the present disclosure, a data processing system is provided, which includes a processor and a memory, wherein the memory is connected to the processor and stores instructions, and the instructions, when executed by the processor, cause the processor to perform operations, the operations including: receiving a set of parameters, wherein the set of parameters represents a first state of an autonomous driving vehicle to be simulated and a set of control commands to be issued at a first point in time; applying a positioning prediction model to the set of parameters to determine a first position of the autonomous driving vehicle; applying a positioning correction model to the set of parameters to determine a set of positioning correction factors; and determining a second position of the autonomous driving vehicle based on the first position and the positioning correction factors, wherein the second position is used as a simulated position of the autonomous driving vehicle at a second point in time. BRIEF DESCRIPTION OF THE DRAWINGS

[0006] Embodiments of the present disclosure are illustrated by way of example, and not by way of limitation, in the figures of the accompanying drawings in which like references indicate similar elements.

[0007] Figure 1 is a block diagram illustrating a networked system according to one embodiment.

[0008] Figure 2 An exemplary dynamic model according to an embodiment is shown.

[0009] Figure 3 A method for training according to an embodiment is shown. Figure 2 An exemplary system 300 of a dynamic model is shown.

[0010] Figure 4 An exemplary flow chart for training a dynamic model according to an embodiment is shown.

[0011] Figure 5 is a block diagram illustrating an example of a dynamic model according to one embodiment.

[0012] Figure 6 is a flow chart illustrating an example of a process for simulating an autonomous vehicle according to one embodiment.

[0013] Figure 7 is a block diagram illustrating an autonomous driving vehicle according to one embodiment.

[0014] Figure 8 is a block diagram illustrating an example of an autonomous driving vehicle according to one embodiment.

[0015] Fig. 9 is a block diagram illustrating an example of a perception and planning system for use with an autonomous vehicle, according to one embodiment. DETAILED DESCRIPTION

[0016] Various embodiments and aspects of the present disclosure will be described with reference to the details discussed below, and the accompanying drawings will illustrate the various embodiments. The following description and the accompanying drawings are illustrative of the present disclosure and should not be construed as limiting the present disclosure. Many specific details are described to provide a comprehensive understanding of the various embodiments of the present disclosure. However, in some cases, well-known or conventional details are not described to provide a concise discussion of the embodiments of the present disclosure.

[0017] References in this specification to "one embodiment" or "an embodiment" mean that a particular feature, structure, or characteristic described in conjunction with the embodiment can be included in at least one embodiment of the present disclosure. The appearances of the phrase "in one embodiment" in various places in this specification are not necessarily all referring to the same embodiment.

[0018] According to one embodiment, a set of parameters representing a first state of an autonomous driving vehicle (ADV) to be simulated and a set of control commands to be issued at a first point in time are received. In response, a positioning prediction model is applied to the set of parameters to determine a first position of the ADV (e.g., (x, y)). A positioning correction model is applied to the set of parameters to determine a set of positioning correction factors (e.g., (Δx, Δy)). The correction factor may represent the error between the predicted position of the ADV predicted by the positioning prediction model and the ground truth measured by the vehicle's sensors. Based on the first position of the ADV and the correction factor, a second position of the ADV is determined as the simulated position of the ADV.

[0019] In one embodiment, the state of the vehicle includes the speed / acceleration and angular velocity of the vehicle at a specific point in time. The control command may be a throttle command, a brake command, or a steering command. The positioning correction model is trained based on a large amount of driving statistics collected from multiple vehicles. The driving statistics include the vehicle state (e.g., speed, position, acceleration) of the vehicle in response to different control commands issued at different points in time. In one embodiment, the positioning correction model is configured to determine the difference between the expected position of the vehicle and the actual position of the vehicle, wherein the expected position of the vehicle is determined based on the positioning process of the autonomous driving system, and the actual position of the vehicle is determined based on sensor data obtained from various sensors installed on the vehicle. The first position determined by the positioning prediction model is calibrated using the difference between the expected position and the actual position. The positioning prediction model and the positioning correction model can be integrated to represent a dynamic model.

[0020] Figure 1 is a block diagram showing a network configuration of an autonomous driving vehicle according to one embodiment of the present disclosure. Figure 1, the network configuration 100 includes an autonomous vehicle 101 that can be communicatively connected to one or more servers 103 to 104 via a network 102. Although one autonomous vehicle is shown, multiple autonomous vehicles can be connected to each other and / or to servers 103 to 104 via the network 102. The network 102 can be any type of network, for example, a wired or wireless local area network (LAN), a wide area network (WAN) such as the Internet, a cellular network, a satellite network, or a combination thereof. Servers 103 to 104 can be any type of server or server cluster, such as a web or cloud server, an application server, a back-end server, or a combination thereof. Servers 103 to 104 can be data analysis servers, content servers, traffic information servers, maps and points of interest (MPOI) servers, or location servers, etc.

[0021] An autonomous vehicle refers to a vehicle that can be configured to be in an autonomous mode in which the vehicle navigates through an environment with little or no input from a driver. Such an autonomous vehicle may include a sensor system having one or more sensors configured to detect information related to the environment in which the vehicle operates. The vehicle and its associated controller use the detected information to navigate through the environment. The autonomous vehicle 101 may operate in a manual mode, in a fully autonomous mode, or in a partially autonomous mode.

[0022] In one embodiment, the autonomous vehicle 101 includes, but is not limited to, a perception and planning system 110, a vehicle control system 111, a wireless communication system 112, a user interface system 113, an infotainment system (not shown), and a control system 115. The autonomous vehicle 101 may also include certain common components included in an ordinary vehicle, such as: an engine, wheels, a steering wheel, a transmission, etc., which may be controlled by the vehicle control system 111 and / or the perception and planning system 110 using a variety of communication signals and / or commands, such as acceleration signals or commands, deceleration signals or commands, steering signals or commands, braking signals or commands, etc.

[0023] Components 110 to 115 may be communicatively coupled to each other via an interconnect, a bus, a network, or a combination thereof. For example, components 110 to 115 may be communicatively coupled to each other via a controller area network (CAN) bus. The CAN bus is a vehicle bus standard designed to allow microcontrollers and devices to communicate with each other in applications without a host. It is a message-based protocol originally designed for multiplexed electrical wiring within an automobile, but is also used in many other environments.

[0024] In one embodiment, the sensor system 115 includes one or more cameras, a global positioning system (GPS) unit, an inertial measurement unit (IMU), a radar unit, and a light detection and ranging (LIDAR) unit. The system 115 may also include other sensors, such as: a sonar sensor, an infrared sensor, a steering sensor, a throttle sensor, a brake sensor, and an audio sensor (e.g., a microphone). The audio sensor can be configured to collect sound from the environment around the autonomous driving vehicle. The steering sensor can be configured to sense the steering angle of the steering wheel, the wheels of the vehicle, or a combination thereof. The throttle sensor and the brake sensor respectively sense the throttle position and the brake position of the vehicle. In some cases, the throttle sensor and the brake sensor can be integrated into an integrated throttle / brake sensor.

[0025] In one embodiment, the vehicle control system 111 includes a steering unit, a throttle unit (also referred to as an acceleration unit), and a brake unit. The steering unit is used to adjust the direction or forward direction of the vehicle. The throttle unit is used to control the speed of the motor or engine, which in turn controls the speed and acceleration of the vehicle. The brake unit decelerates the vehicle by providing friction to decelerate the wheels or tires of the vehicle.

[0026] In one embodiment, the wireless communication system 112 allows communication between the autonomous vehicle 101 and external systems such as devices, sensors, other vehicles, etc. For example, the wireless communication system 112 can communicate directly with one or more devices wirelessly, or wirelessly via a communication network, such as communicating with servers 103 to 104 through the network 102. The wireless communication system 112 can use any cellular communication network or wireless local area network (WLAN), for example, using WiFi, to communicate with another component or system. The wireless communication system 112 can communicate directly with a device (e.g., a passenger's mobile device, a display device, a speaker in the vehicle 101) using, for example, an infrared link, Bluetooth, etc. The user interface system 113 can be part of a peripheral device implemented in the vehicle 101, including, for example, a keyboard, a touch screen display device, a microphone, and a speaker.

[0027] Some or all of the functions of the autonomous vehicle 101 may be controlled or managed by the perception and planning system 110, particularly when operating in an autonomous mode. The perception and planning system 110 includes the necessary hardware (e.g., processor, memory, storage device) and software (e.g., operating system, planning and routing program) to receive information from the sensor system 115, the control system 111, the wireless communication system 112, and / or the user interface system 113, process the received information, plan a route or path from a starting point to a destination point, and then drive the vehicle 101 based on the planning and control information. Alternatively, the perception and planning system 110 may be integrated with the vehicle control system 111.

[0028] As the autonomous vehicle 101 moves along the route, the perception and planning system 110 may also obtain real-time traffic information from a traffic information system or server (TIS). It should be noted that the servers 103 to 104 may be operated by a third-party entity. Alternatively, the functionality of the servers 103 to 104 may be integrated with the perception and planning system 110. Based on real-time traffic information, MPOI information, and location information, as well as real-time local environmental data (e.g., obstacles, objects, nearby vehicles) detected or sensed by the sensor system 115, the perception and planning system 110 may plan an optimal route and drive the vehicle 101 according to the planned route, for example, via the control system 111, to safely and efficiently reach a designated destination.

[0029] The server 103 may be a data analysis system that performs data analysis services for various clients. In one embodiment, the data analysis system 103 includes a data collector 121 and a machine learning engine 122. The data collector 121 collects driving statistics 123 from various vehicles (autonomous driving vehicles or conventional vehicles driven by human drivers). The driving statistics 123 include information indicating driving commands issued (e.g., throttle commands, brake commands, control commands) and responses of the vehicle (e.g., speed, acceleration, deceleration, direction) captured by the vehicle's sensors at different time points. The driving statistics 123 may also include, for example, information describing the driving environment at different time points, such as routes (including starting and destination locations), MPOI, road conditions, weather conditions, etc.

[0030] Based on the driving statistics data 123, the machine learning engine 122 generates or trains a set of rules, algorithms and / or prediction models for various purposes. For example, the machine learning engine 122 can train a dynamic model. The dynamic model can include a positioning prediction model and a positioning correction model integrated therein. An automatic dynamic model training model can be provided in the server 103 to control the extraction of training data from the driving statistics data 123 and the use of the extracted driving statistics data to train the dynamic model.

[0031] Figure 2 An exemplary dynamic model according to an embodiment is shown. In this example, the dynamic model 200 can be a multi-layer perceptron (MLP) neural network model trained using real-world data collected by sensors in the vehicle. The dynamic model includes a collection of connected neurons designed to simulate the human brain. The neurons in the dynamic model can be fully connected, and each node in one layer is connected to each node in the next layer with a certain weight. During the training of the dynamic model, the dynamic model changes the connection weights after processing each piece of data based on the amount of error of the output compared to the expected result.

[0032] In one embodiment, the dynamic model 200 may include an input layer 205 that receives input data 201, an output layer 209 that makes a decision or prediction or classification about the input data, and any number of hidden layers (e.g., hidden layer 207) between the input layer 205 and the output layer 209. The hidden layer is the computational engine of the dynamic model 200. The neurons in the hidden layer 207 may include an activation function for transforming the input data 201. Rectified linear unit (ReLu) 202 indicates an exemplary activation function used in the dynamic model 200. If any negative input is received, ReLu 202 returns 0; and for any returned positive value, it returns the value. Although Figure 200 shows one hidden layer 207 and one output layer 209, the dynamic model 200 may include multiple hidden layers and multiple output layers.

[0033] In an embodiment, input data 201 for the neural network model 200 includes the state of the ADV during a first driving cycle (e.g., speed, acceleration, and angular velocity) and control commands (e.g., throttle commands, brake commands, and steering commands). The input data 201 can be processed by one or more hidden layers and converted into output data 204, which is the expected state of the ADV during a second driving cycle. Based on the acceleration and angular velocity over time, the speed of the ADV during the second driving cycle can be calculated. The input data represents multiple characteristic scenarios (i.e., states) of the ADV.

[0034] Figure 3 A method for training according to an embodiment is shown. Figure 2 An exemplary system 300 of a dynamic model is shown. Figure 3 As shown, the exemplary dynamic training system 300 includes the automatic dynamic training module 124 , a feature extraction module 307 , a training controller 309 , historical driving statistics 322 , and driving statistics 303 of controlled test scenarios.

[0035] Historical driving statistics 322 include real-world data collected by sensors (e.g., IMU and GPS) to record the real-time dynamics (e.g., status) of the vehicle in various scenarios that may cover various road types, obstacle types, and road environments. TM The driving statistics 303 of the controlled test scenario are real-world data generated by vehicles driven by human drivers.

[0036] In one embodiment, the training controller 309 may implement an algorithm to automate the process of extracting training data, training the dynamic model using the training data, and iteratively evaluating the dynamic model until the control model meets the specified requirements.

[0037] In one embodiment, the training controller 309 may automatically generate a predetermined equally spaced range of values ​​for each of the plurality of features. As described above, features and driving parameters may be used interchangeably in the present disclosure. For example, for the driving parameter of speed, the training controller 309 may generate the following ranges of values: 0-5 mps, 5-10 mps, 10-15 mps, 15-20 mps, and 20-25 mps. Based on each equally spaced range of values, the training controller 309 may extract a value from the historical driving statistics data 322. In one embodiment, the historical driving statistics data 322 may be Baidu's TM A subset of the Apollo Open Data Platform and can be stored in a data storage device synchronized with the Apollo Open Data Platform. Therefore, there can be sufficient data points from each value range.

[0038] Once the value range of each driving parameter is determined, the training controller 309 can extract data from each range of the driving parameter to create multiple feature scenarios, each feature scenario representing a value of the driving parameter or a combination of values ​​of multiple driving parameters (also referred to as a feature or driving feature in this disclosure).

[0039] As an illustrative example, the training controller 309 can create feature scenarios based on three driving parameters such as steering angle, speed, and acceleration. For each driving parameter, the training controller 309 can create multiple equally spaced ranges. For example, the simulator controller 309 can create 12 equally spaced ranges for steering angle, 5 equally spaced ranges for speed, and 7 equally spaced ranges for acceleration. In one embodiment, a total of 420 feature scenarios (i.e., 12*5*7=420) will be created using values ​​from each of the above ranges. A different number of feature scenarios can be created in other embodiments.

[0040] In one embodiment, the training controller 309 may call the feature extraction module 307 to extract data for creating feature scenarios. With the training data for each feature scenario, the training controller 309 may call services, functions, routes, libraries, and / or application programming interfaces (APIs) provided by the machine learning engine 122 to train the dynamic model 301. The training controller 309 may then evaluate the dynamic model 301 by performing inference using data from the driving statistics data 303. In one embodiment, the training controller 309 may call the dynamic model evaluation module 305 to evaluate the dynamic model 301.

[0041] like Figure 3As shown, the dynamic model 301 may receive data generated from a plurality of controlled test scenarios from driving statistics data 303. Unlike test data generated by a vehicle operating in a natural manner, the data generated from the controlled test scenarios may be data generated by the vehicle in various driving scenarios or driving conditions that are specifically created to generate test data for these driving conditions or driving scenarios.

[0042] In one embodiment, the test data may include ground truth for a number of carefully designed controlled test scenarios to test the performance of the trained dynamic model 300. A controlled test scenario is a feature scenario. One or more controlled test scenarios are designed for each feature.

[0043] For example, one controlled test scenario would be turning the steering wheel 180 degrees to the left, another controlled test scenario would be turning the steering wheel 180 degrees to the right, and another controlled test scenario would be driving from point A to point B in one minute. For each controlled test scenario, the ground truth is recorded in driving statistics 303, which also stores the output of the dynamic model 301 for each controlled test scenario.

[0044] In one embodiment, the driving statistics may include the current state of the vehicle and one or more control commands. Examples of current states include speed, acceleration, and angular velocity of the ADV during the first driving cycle; examples of control commands include throttle commands, brake commands, and steering commands during the first driving cycle. Each current state may have a corresponding expected future state 308 and actual future state 306 for the second driving cycle. The corresponding expected future state may be the output of the dynamic model 301, and the actual future state 306 may be the ground truth from real-world data generated under a controlled test scenario.

[0045] In one embodiment, the dynamic model evaluation module 305 may use the loss function 310 to compare each actual future state with each expected future state to generate a comparison result 311. The dynamic model evaluation module 305 may grade the reasoning performance of the dynamic model 301 for each feature scenario based on the comparison result 311 and / or the loss function 310.

[0046] In one embodiment, each feature scenario may include values ​​for multiple features or a single feature. For example, a feature scenario may be a speed of 5 miles per second; a speed of 5 miles per second, a steering angle of 5 degrees to the left; or a speed of 5 miles per second, a steering angle of 5 degrees to the left, and a throttle rate of 15%. Thus, each feature scenario may be associated with one or more features. For all feature scenarios associated with a particular feature, a root mean square error may be calculated based on a comparison between the ground truth value and the expected value from the dynamic model. A performance score may be calculated using a predetermined algorithm for each root mean square error for each feature.

[0047] Based on the performance scores, the plurality of features associated with the controlled test scenario may be ranked. The training controller 309 may identify one or more features that received a score below a predetermined threshold and extract additional data from the historical driving data 322 using the feature extraction module 307 to retrain the dynamic model 301 for the one or more features. After retraining using the additional data, the dynamic model evaluation module 305 may be called again to evaluate the performance of the dynamic model 301 for the one or more features. This process may be automatically repeated until the dynamic model 301 has met the predetermined requirements for each feature used to train the dynamic model 301 and controlled test scenarios have been designed for those features.

[0048] In one embodiment, the above process of identifying one or more features that receive a score below a predetermined threshold can be used to identify one or more feature scenarios that receive a score below a predetermined threshold. Similarly, the feature extraction module 307 can extract additional data from the one or more feature scenarios to retrain the dynamic model 301. After retraining with the additional data, the dynamic model evaluation module 305 can be called again to evaluate the performance of the dynamic model 301 for the one or more feature scenarios. This process can be automatically repeated until the dynamic model 301 has met the predetermined requirements for each feature scenario previously created.

[0049] Figure 4 An exemplary flow chart 400 for training a dynamic model according to an embodiment is shown. Figure 4 As shown, the automatic loop for training the dynamic model may include a vehicle data generation phase 406 and an offline data pipelining & model training phase 410. Once the dynamic model is fully trained and meets all predetermined requirements, the dynamic model may be put into a simulator to fine-tune actuators and design control algorithms for real-time autonomous driving in an online simulation & grading phase 420.

[0050] More specifically, during the vehicle data generation phase 406, the external partner or dynamic model developer 401 may collect real-world driving statistics by manually driving the vehicle on the road 405, and record the driving statistics in a machine learning training dataset storage device, such as Baidu TM The Apollo Open Data Platform.

[0051] During the offline data pipeline technique and model training phase 410, data 409 for predefined feature scenarios associated with a set of predefined features are extracted for training a plurality of dynamic models 411. Once the dynamic models are trained, the dynamic models can be evaluated offline using historical driving statistics. Based on the outputs 413 of the dynamic models and the ground truth, the dynamic models can be ranked and the dynamic model with the highest score can be selected 415.

[0052] In one embodiment, the selected dynamic model may again be evaluated against driving statistics from a plurality of controlled test scenarios so that the dynamic model may be further improved. The controlled test scenarios may represent various combinations of values ​​of driving parameters from the ADV for which the dynamic model has been trained.

[0053] Examples of driving parameters may include braking, acceleration, idling speed, reverse driving, straight driving, left or right turns, U-turns, lane changes, and parking driving. Each driving parameter may have multiple values. A combination of values ​​from one parameter or values ​​from multiple parameters constitutes a controlled test scenario.

[0054] Inference may be performed by the dynamic model using driving statistics from controlled test scenarios. For each controlled test scenario, the output of the dynamic model may be compared to ground truth. The performance of the dynamic model on each controlled test scenario may be graded, and those controlled test scenarios that receive scores below a predetermined threshold will be identified.

[0055] In addition, based on the performance scores of the controlled test scenarios, the driving parameters or features used to train the dynamic model can also be ranked, and one or more features for which the dynamic model fails to meet a predetermined performance threshold can be identified. A first step for ranking the features includes determining all controlled feature scenarios for the features; comparing the ground truth value of each controlled feature scenario with an expected value generated by the dynamic model in response to receiving the controlled feature scenario as input; calculating the root mean square error or a transformed value thereof (e.g., a performance score based on the root mean square error); ranking the features based on the root mean square error or the transformed value; and identifying one or more features that each received a performance score below a predetermined threshold.

[0056] After identifying one or more features or one or more feature scenarios, the automatic loop of training the dynamic model continues by extracting 409 additional data from historical driving statistics, such as those recorded by data log 407, for retraining the dynamic model. The above process can be repeated iteratively until the dynamic model performs satisfactorily according to a set of predetermined requirements, for example, no feature scenario receives a score below 90 out of 100.

[0057] The dynamic model is specially trained for autonomous driving of a specific ADV or a specific type or model of ADV. The dynamic model may be a neural network model, which may be represented by a linear regression, multi-layer perception, or recursive neural network model. The driving statistics collected from various vehicles may include various control commands (e.g., throttle commands, brake commands, steering commands) issued at different time points and the responses or states of the vehicles.

[0058] like Figure 4 As shown, in the online simulation and classification phase 420, the trained dynamic model can be placed in a simulator, in which a simulation with a loop control process 419 can be implemented to identify issues with the output 417 from the dynamic model for determining whether future training of the dynamic model is required. Note that the vehicle data generation phase 406, the offline model training phase 410, and the simulation phase 420 can be performed in different computing nodes (e.g., different cloud servers). Alternatively, they can be performed by different nodes of the cluster (e.g., a backend server behind an application server or a network server).

[0059] Figure 5 is a block diagram illustrating an example of a dynamic model according to one embodiment. The dynamic model 500 may be trained using at least some of the above techniques. Figure 5 , the dynamic model 500 includes a positioning prediction model or module 501 and a positioning correction model or module 502. Each of these models or modules 501 to 502 can be trained individually or together using at least some of the above-mentioned techniques. The function of the dynamic model 500 is to predict the next state of the vehicle (e.g., the speed, acceleration and angular velocity of the next planning cycle) based on the previous state of the vehicle (e.g., the state of the current planning cycle) and the control command (e.g., throttle command, brake command, steering command).

[0060] The positioning prediction model 501 can be a longitudinal model (e.g., linear interpolation), a lateral model (also called a bicycle model), a neural network (e.g., a feedforward neural network, a recursive neural network). In one embodiment, the positioning prediction model 501 can predict the acceleration of the vehicle based on the state of the vehicle. Based on the acceleration, the speed and position of the vehicle can be determined by first-order integration and second-order integration. However, due to the double integral calculation, there is a mismatch between the output of the positioning prediction model 501 and the actual position of the vehicle.

[0061] In one embodiment, the positioning correction model 502 is trained to determine the error between the predicted position of the vehicle and the actual position or ground truth of the vehicle using an autonomous driving algorithm (e.g., utilized by the positioning prediction model 501). The positioning correction model 502 can be trained using a large amount of driving statistics collected from a large number of vehicles. Typically, in an ideal situation, using the prediction model, the vehicle should end up at the predicted position. However, due to various environmental factors in different driving scenarios or environments (e.g., road conditions, vehicle type, weather conditions), the actual vehicle position may deviate from the expected or predicted position.

[0062] In one embodiment, the positioning correction model 502 is designed to determine the difference between the expected position of the vehicle in response to the control command and the actual position of the vehicle. The actual position of the vehicle can be determined based on sensor data obtained from sensors such as IMU and / or GPS sensors. By training the positioning correction model 502 with a large amount of driving statistics data, the positioning correction model 502 can be used in simulations based on the state and control commands of the simulated vehicle. The final predicted position of the vehicle can be determined based on the predicted position (x, y) of the positioning prediction model 501 and the difference (Δx, Δy) determined by the positioning correction model 502. For example, the final predicted position of the vehicle can be represented by (x+Δx, y+Δy). As a result, the simulated vehicle position is closer to the actual vehicle position in the same or similar driving scene or driving environment.

[0063] Figure 6 600 is a flow chart illustrating an example of a process for simulating an autonomous vehicle according to one embodiment. Process 600 may be performed by processing logic that may include software, hardware, or a combination thereof. For example, process 600 may be performed as part of dynamic model 500. Figure 6At block 601, processing logic receives a set of parameters representing a first state of an ADV to be simulated and a set of control commands to be issued at a first point in time. At block 602, processing logic applies a positioning prediction model to the set of parameters to determine a first position of the ADV. At block 603, processing logic applies a positioning correction model to the set of parameters to determine a set of positioning correction factors. At block 604, processing logic determines a second position of the ADV based on the first position and the positioning correction factors. The second position is used as the simulated position of the ADV at a second point in time.

[0064] Figure 7 is a block diagram showing an autonomous driving vehicle according to one embodiment of the present disclosure. Figure 7 , the autonomous driving vehicle 601 may be communicatively connected to one or more servers via a network, which may be any type of network, whether wired or wireless, such as a local area network (LAN), a wide area network (WAN) such as the Internet, a cellular network, a satellite network, or a combination thereof. The server may be any type of server or server cluster, such as a network or cloud server, an application server, a backend server, or a combination thereof. The server may be a data analysis server, a content server, a traffic information server, a map and point of interest (MPOI) server, or a location server, etc.

[0065] An autonomous vehicle refers to a vehicle that can be configured to be in an autonomous mode, in which the vehicle navigates through an environment with little or no input from a driver. Such an autonomous vehicle may include a sensor system having one or more sensors configured to detect information related to the environment in which the vehicle is operating. The vehicle and its associated controller use the detected information to navigate through the environment. The autonomous vehicle 601 can operate in a manual mode, in a fully autonomous mode, or in a partially autonomous mode.

[0066] In one embodiment, the autonomous vehicle 601 includes, but is not limited to, a perception and planning system 610, a vehicle control system 611, a wireless communication system 612, a user interface system 613, and a sensor system 615. The autonomous vehicle 601 may also include certain common components included in an ordinary vehicle, such as: an engine, wheels, a steering wheel, a transmission, etc., which may be controlled by the vehicle control system 611 and / or the perception and planning system 610 using a variety of communication signals and / or commands, such as acceleration signals or commands, deceleration signals or commands, steering signals or commands, braking signals or commands, etc.

[0067] Components 610 to 615 may be communicatively coupled to each other via an interconnect, a bus, a network, or a combination thereof. For example, components 610 to 615 may be communicatively coupled to each other via a controller area network (CAN) bus. The CAN bus is a vehicle bus standard designed to allow microcontrollers and devices to communicate with each other in applications without a host. It is a message-based protocol originally designed for multiplexed electrical wiring in automobiles, but is also used in many other environments.

[0068] Reference now Figure 8 In one embodiment, the sensor system 615 includes, but is not limited to, one or more cameras 711, a global positioning system (GPS) unit 712, an inertial measurement unit (IMU) 713, a radar unit 714, and a light detection and ranging (LIDAR) unit 715. The GPS system 712 may include a transceiver operable to provide information about the location of the autonomous vehicle. The IMU unit 713 may sense the position and orientation changes of the autonomous vehicle based on inertial acceleration. The radar unit 714 may represent a system that uses radio signals to sense objects within the local environment of the autonomous vehicle. In some embodiments, in addition to sensing objects, the radar unit 714 may additionally sense the speed and / or heading of the object. The LIDAR unit 715 may use lasers to sense objects in the environment of the autonomous vehicle. The LIDAR unit 715 may also include, among other system components, one or more laser sources, a laser scanner, and one or more detectors. The camera 711 may include one or more devices for capturing images of the environment surrounding the autonomous vehicle. The camera 711 may be a still camera and / or a video camera. The camera may be mechanically movable, for example, by mounting the camera on a rotating and / or tilting platform.

[0069] The sensor system 615 may also include other sensors, such as: a sonar sensor, an infrared sensor, a steering sensor, a throttle sensor, a brake sensor, and an audio sensor (e.g., a microphone). The audio sensor may be configured to collect sound from the environment surrounding the autonomous vehicle. The steering sensor may be configured to sense the steering angle of the steering wheel, the wheels of the vehicle, or a combination thereof. The throttle sensor and the brake sensor sense the throttle position and the brake position of the vehicle, respectively. In some cases, the throttle sensor and the brake sensor may be integrated into an integrated throttle / brake sensor.

[0070] In one embodiment, the vehicle control system 611 includes, but is not limited to, a steering unit 701, a throttle unit 702 (also referred to as an acceleration unit), and a brake unit 703. The steering unit 701 is used to adjust the direction or forward direction of the vehicle. The throttle unit 702 is used to control the speed of the motor or engine, which in turn controls the speed and acceleration of the vehicle. The brake unit 703 decelerates the vehicle by providing friction to decelerate the wheels or tires of the vehicle. It should be noted that, as Figure 8 The components shown may be implemented in hardware, software or a combination thereof.

[0071] Back to Figure 7 , the wireless communication system 612 allows communication between the autonomous driving vehicle 601 and external systems such as devices, sensors, other vehicles, etc. For example, the wireless communication system 612 can communicate directly with one or more devices wirelessly, or wirelessly via a communication network. The wireless communication system 612 can use any cellular communication network or wireless local area network (WLAN), for example, using WiFi, to communicate with another component or system. The wireless communication system 612 can communicate directly with a device (e.g., a passenger's mobile device, a display device, a speaker in the vehicle 601) using, for example, an infrared link, Bluetooth, etc. The user interface system 613 can be part of a peripheral device implemented in the vehicle 601, including, for example, a keyboard, a touch screen display device, a microphone, and a speaker.

[0072] Some or all of the functions of the autonomous vehicle 601 may be controlled or managed by the perception and planning system 610, particularly when operating in an autonomous mode. The perception and planning system 610 includes the necessary hardware (e.g., processor, memory, storage device) and software (e.g., operating system, planning and routing program) to receive information from the sensor system 615, the control system 611, the wireless communication system 612, and / or the user interface system 613, process the received information, plan a route or path from a starting point to a destination point, and then drive the vehicle 601 based on the planning and control information. Alternatively, the perception and planning system 610 may be integrated with the vehicle control system 611.

[0073] For example, a user as a passenger can specify the starting location and destination of the trip, for example, via a user interface. The perception and planning system 610 obtains trip-related data. For example, the perception and planning system 610 can obtain location and route information from an MPOI server. The location server provides location services, and the MPOI server provides map services and POIs of certain locations. Alternatively, such location and MPOI information can be locally cached in a permanent storage device of the perception and planning system 610.

[0074] As the autonomous vehicle 601 moves along the route, the perception and planning system 610 may also obtain real-time traffic information from a traffic information system or server (TIS). It should be noted that the server may be operated by a third-party entity. Alternatively, the functionality of the server may be integrated with the perception and planning system 610. Based on real-time traffic information, MPOI information, and location information, as well as real-time local environmental data (e.g., obstacles, objects, nearby vehicles) detected or sensed by the sensor system 615, the perception and planning system 610 may plan an optimal route and drive the vehicle 601 according to the planned route, for example, via the control system 611, to safely and efficiently reach a designated destination.

[0075] Fig. 9 is a block diagram illustrating an example of a perception and planning system for use with an autonomous vehicle according to one embodiment. System 800 may be implemented as Figure 7 A portion of an autonomous vehicle 601 includes, but is not limited to, a perception and planning system 610, a control system 611, and a sensor system 615. Fig. 9 The perception and planning system 610 includes but is not limited to a positioning module 801 , a perception module 802 , a prediction module 803 , a decision module 804 , a planning module 805 , a control module 806 , a route module 807 and a data collector 808 .

[0076] Some or all of the modules 801 to 805 may be implemented in software, hardware, or a combination thereof. For example, these modules may be installed in permanent storage 852, loaded into memory 851, and executed by one or more processors (not shown). It should be noted that some or all of these modules may be communicatively coupled to Figure 7 Some or all modules of the vehicle control system 611 of the embodiment of the present invention may be integrated with them. Some of the modules 801 to 808 may be integrated together into an integrated module.

[0077] The positioning module 801 determines the current location of the autonomous vehicle (e.g., using the GPS unit 712) and manages any data related to the user's itinerary or route. The positioning module 801 (also referred to as the map and route module) manages any data related to the user's itinerary or route. The user can log in, for example, via a user interface and specify the starting location and destination of the itinerary. The positioning module 801 communicates with other components of the autonomous vehicle, such as the map and route information 811, to obtain itinerary-related data. For example, the positioning module 801 can obtain location and route information from a location server and a map and POI (MPOI) server. The location server provides location services, and the MPOI server provides map services and POIs at certain locations, which can be cached as part of the map and route information 811. As the autonomous vehicle moves along the route, the positioning module 801 can also obtain real-time traffic information from a traffic information system or server.

[0078] Based on the sensor data provided by the sensor system 615 and the positioning information obtained by the positioning module 801, the perception module 802 determines the perception of the surrounding environment. The perception information can represent what an average driver would perceive around the vehicle that the driver is driving. The perception can include, for example, lane configuration in the form of objects, traffic light signals, the relative position of another vehicle, pedestrians, buildings, crosswalks or other traffic-related signs (e.g., stop signs, yield signs), etc. The lane configuration includes information describing one or more lanes, such as the shape of the lane (e.g., straight lanes or curved lanes), the width of the lane, how many lanes there are in the road, one-way lanes or two-way lanes, merging lanes or split lanes, exit lanes, etc.

[0079] The perception module 802 may include a computer vision system or the functionality of a computer vision system to process and analyze images captured by one or more cameras to identify objects and / or features in the autonomous vehicle environment. The objects may include traffic signals, road boundaries, other vehicles, pedestrians, and / or obstacles, etc. The computer vision system may use object recognition algorithms, video tracking, and other computer vision techniques. In some embodiments, the computer vision system may map the environment, track objects, and estimate the speed of objects, etc. The perception module 802 may also detect objects based on other sensor data provided by other sensors such as radar and / or LIDAR.

[0080] For each object, the prediction module 803 predicts how the object will behave in the environment. The prediction is performed based on the perception data of the driving environment at that point in time, according to a set of map / route information 811 and traffic rules 812. For example, if the object is a vehicle in the opposite direction and the current driving environment includes an intersection, the prediction module 803 will predict whether the vehicle is likely to move directly forward or turn. If the perception data indicates that there is no traffic light at the intersection, the prediction module 803 can predict that the vehicle may have to come to a complete stop before entering the intersection. If the perception data indicates that the vehicle is currently in a left-turn-only lane or a right-turn-only lane, the prediction module 803 can predict that the vehicle will be more likely to make a left turn or a right turn, respectively.

[0081] For each object, the decision module 804 makes a decision about how to handle the object. For example, for a particular object (e.g., another vehicle in an intersection) and its metadata describing the object (e.g., speed, direction, turn angle), the decision module 804 decides how to encounter the object (e.g., pass, yield, stop, overtake). The decision module 804 may make such decisions based on a set of rules (e.g., traffic rules or driving rules 812) that may be stored in the persistent storage device 852.

[0082] The route module 807 is configured to provide one or more routes or paths from the starting point to the destination point. For a given trip from the starting position to the destination position, for example, received from a user, the route module 807 obtains the route and map information 811 and determines all possible routes or paths from the starting position to the destination position. The route module 807 can generate a reference line in the form of a topographic map for each route determined from the starting position to the destination position. The reference line refers to an ideal route or path without any other interference from other vehicles, obstacles or traffic conditions. That is, if there are no other vehicles, pedestrians or obstacles on the road, the ADV should accurately or closely follow the reference line. The topographic map is then provided to the decision module 804 and / or the planning module 805. The decision module 804 and / or the planning module 805 check all possible routes to select and modify one of the best routes based on other data provided by other modules, such as traffic conditions from the positioning module 801, the driving environment perceived by the perception module 802, and traffic conditions predicted by the prediction module 803. Depending on the particular driving environment at that point in time, the actual path or route used to control the ADV may be close to or different from the reference line provided by the route module 807 .

[0083] Based on the decision for each perceived object, the planning module 805 uses the reference line provided by the route module 807 as a basis to plan the path or route of the autonomous vehicle and the driving parameters (e.g., distance, speed, and / or turning angle). That is, for a given object, the decision module 804 decides what to do with the object, and the planning module 805 determines how to do it. For example, for a given object, the decision module 804 may decide to pass through the object, and the planning module 805 may determine whether to pass through the object on the left or right. Planning and control data is generated by the planning module 805, and the planning and control data includes information describing how the vehicle will move in the next movement cycle (e.g., the next route / path segment). For example, the planning and control data may indicate that the vehicle moves at a speed of 30 miles per hour (mph) for 10 meters and then changes to the right lane at a speed of 25mph.

[0084] Based on the planning and control data, the control module 806 controls and drives the autonomous vehicle according to the route or path defined by the planning and control data by sending appropriate commands or signals to the vehicle control system 611. The planning and control data includes sufficient information to drive the vehicle from a first point on the route or path to a second point using appropriate vehicle settings or driving parameters (e.g., throttle commands, brake commands, steering commands) at different points in time along the route or path.

[0085] In one embodiment, the planning phase is performed in multiple planning cycles (also referred to as driving cycles, for example, each time interval is 100 milliseconds (ms)). For each planning cycle or driving cycle, one or more control commands will be issued based on the planning and control data. That is, for every 100ms, the planning module 805 plans the next route segment or path segment, for example, including the target location and the time required for the ADV to reach the target location. Alternatively, the planning module 805 can further specify a specific speed, direction and / or steering angle, etc. In one embodiment, the planning module 805 plans the route segment or path segment for the next predetermined time period (for example, 5 seconds). For each planning cycle, the planning module 805 plans the target position of the current cycle (for example, the next 5 seconds) based on the target position planned in the previous cycle. Then, the control module 806 generates one or more control commands (for example, throttle command, brake command, steering control command) based on the planning and control data of the current cycle.

[0086] Note that the decision module 804 and the planning module 805 can be integrated into an integrated module. The decision module 804 / planning module 805 can include a navigation system or the functionality of a navigation system to determine a driving path for an autonomous vehicle. For example, the navigation system can determine a series of speeds and directions for enabling the autonomous vehicle to move along a path that substantially avoids perceived obstacles while allowing the autonomous vehicle to move along a roadway-based path to a final destination. The destination can be set based on user input via the user interface system 613. The navigation system can dynamically update the driving path while the autonomous vehicle is running. The navigation system can combine data from a GPS system and one or more maps to determine the driving path for the autonomous vehicle.

[0087] According to one embodiment, the data collector 808 is configured to collect driving statistics data and store it in the permanent storage device 852 as part of the driving statistics data 313. The driving statistics data 313 can then be used to train the dynamic model as described above.

[0088] Some portions of the foregoing detailed description have been presented in terms of algorithms and symbolic representations of operations on data bits within a computer memory. These algorithmic descriptions and representations are the means used by those skilled in the data processing arts to most effectively convey the substance of their work to others skilled in the art. An algorithm is generally considered herein to be a self-consistent sequence of operations leading to a desired result. These operations are those requiring physical manipulation of physical quantities.

[0089] It should be borne in mind, however, that all of these and similar terms are intended to be associated with the appropriate physical quantities and are merely convenient labels applied to these quantities. Unless otherwise expressly indicated in the above discussion, it should be understood that throughout the specification, discussions utilizing terms such as those set forth in the appended claims refer to the actions and processes of a computer system or similar electronic computing device that manipulates data represented as physical (electronic) quantities within the computer system's registers and memories and transforms the data into other data similarly represented as physical quantities within the computer system's memories or registers or other such information storage, transmission, or display devices.

[0090] Embodiments of the present disclosure also relate to devices for performing the operations herein. Such a computer program is stored in a non-transitory computer-readable medium. A machine-readable medium includes any mechanism for storing information in a form readable by a machine (e.g., a computer). For example, a machine-readable (e.g., computer-readable) medium includes a machine (e.g., computer) readable storage medium (e.g., a read-only memory ("ROM"), a random access memory ("RAM"), a magnetic disk storage medium, an optical storage medium, a flash memory device).

[0091] The processes or methods depicted in the foregoing figures may be performed by processing logic including hardware (e.g., circuits, dedicated logic, etc.), software (e.g., embodied on a non-transitory computer-readable medium), or a combination of both. Although the processes or methods are described above in terms of some sequential operations, it should be understood that some of the operations may be performed in a different order. In addition, some operations may be performed in parallel rather than sequentially.

[0092] The embodiments of the present disclosure are not described with reference to any particular programming language. It will be appreciated that a variety of programming languages ​​may be used to implement the teachings of the embodiments of the present disclosure as described herein.

[0093] In the above description, embodiments of the present disclosure have been described with reference to specific exemplary embodiments of the present disclosure. It will be apparent that various modifications may be made to the present disclosure without departing from the broader spirit and scope of the present disclosure set forth in the appended claims. Therefore, the present description and the accompanying drawings should be understood in an illustrative sense rather than a restrictive sense.

Claims

1. A computer-implemented method for simulating an autonomous vehicle, the method comprising: receiving a set of parameters representing a first state of an autonomous vehicle to be simulated and a set of control commands to be issued at a first point in time; Applying a positioning prediction model to the set of parameters to determine a first position of the autonomous vehicle, the first position being used as a position to be calibrated of the autonomous vehicle at a second time point; applying a positioning correction model to the set of parameters to determine a set of positioning correction factors, the positioning correction model being used to determine a difference between an expected position of a vehicle and an actual position of the vehicle; as well as A second position of the autonomous vehicle is determined based on the first position and the positioning correction factor, wherein the second position is used as a simulated position of the autonomous vehicle at a second point in time.

2. The method of claim 1, wherein: The first state of the autonomous driving vehicle includes the speed, acceleration and angular velocity of the autonomous driving vehicle at the first time point.

3. The method of claim 1, wherein: The set of control commands includes at least one of a throttle command, a brake command, and a steering command.

4. The method of claim 1, wherein: The positioning correction model is trained based on a large amount of driving statistics collected from multiple vehicles.

5. The method of claim 4, wherein: The driving statistics data include vehicle states of the vehicle in response to different control commands issued at different time points over a period of time.

6. The method of claim 4, wherein: The expected position of the vehicle is determined based on a positioning process of an autonomous driving system, and the actual position of the vehicle is determined based on sensor data obtained from a plurality of sensors mounted on the vehicle.

7. The method of claim 6, wherein: The difference between the expected position and the actual position is used to calibrate the first position determined by the positioning prediction model.

8. A non-transitory machine-readable medium storing instructions that, when executed by a processor, cause the processor to perform operations comprising: receiving a set of parameters representing a first state of an autonomous vehicle to be simulated and a set of control commands to be issued at a first point in time; Applying a positioning prediction model to the set of parameters to determine a first position of the autonomous vehicle, the first position being used as a position to be calibrated of the autonomous vehicle at a second time point; applying a positioning correction model to the set of parameters to determine a set of positioning correction factors, the positioning correction model being used to determine a difference between an expected position of a vehicle and an actual position of the vehicle; as well as A second position of the autonomous vehicle is determined based on the first position and the positioning correction factor, wherein the second position is used as a simulated position of the autonomous vehicle at a second point in time.

9. The machine-readable medium of claim 8, wherein: The first state of the autonomous driving vehicle includes the speed, acceleration and angular velocity of the autonomous driving vehicle at the first time point.

10. The machine-readable medium of claim 8, wherein: The set of control commands includes at least one of a throttle command, a brake command, and a steering command.

11. The machine-readable medium of claim 8, wherein: The positioning correction model is trained based on a large amount of driving statistics collected from multiple vehicles.

12. The machine-readable medium of claim 11, wherein: The driving statistics data include vehicle states of the vehicle in response to different control commands issued at different time points over a period of time.

13. The machine-readable medium of claim 11, wherein: The expected position of the vehicle is determined based on a positioning process of an autonomous driving system, and the actual position of the vehicle is determined based on sensor data obtained from a plurality of sensors mounted on the vehicle.

14. The machine-readable medium of claim 13, wherein: The difference between the expected position and the actual position is used to calibrate the first position determined by the positioning prediction model.

15. Data processing system, including: processor; as well as a memory coupled to the processor and storing instructions that, when executed by the processor, cause the processor to perform operations comprising: Receiving a set of parameters, the set of parameters representing a first state of an autonomous vehicle to be simulated and a set of control commands to be issued at a first point in time; applying a positioning prediction model to the set of parameters to determine a first position of the autonomous vehicle, the first position being used as a position to be calibrated of the autonomous vehicle at a second point in time; applying a positioning correction model to the set of parameters to determine a set of positioning correction factors, the positioning correction model being used to determine a difference between an expected position of a vehicle and an actual position of the vehicle; and A second position of the autonomous vehicle is determined based on the first position and the positioning correction factor, wherein the second position is used as a simulated position of the autonomous vehicle at a second point in time.

16. The system of claim 15, wherein: The first state of the autonomous driving vehicle includes the speed, acceleration and angular velocity of the autonomous driving vehicle at the first time point.

17. The system of claim 15, wherein: The set of control commands includes at least one of a throttle command, a brake command, and a steering command.

18. The system of claim 15, wherein: The positioning correction model is trained based on a large amount of driving statistics collected from multiple vehicles.

19. The system of claim 18, wherein: The driving statistics data include vehicle states of the vehicle in response to different control commands issued at different time points over a period of time.

20. The system of claim 18, wherein: The expected position of the vehicle is determined based on a positioning process of an autonomous driving system, and the actual position of the vehicle is determined based on sensor data obtained from a plurality of sensors mounted on the vehicle.