Method and system for operating an autonomous vehicle

Abstract

Described herein are methods and systems for operating an autonomous driving vehicle to guarantee safety of a control level of the ADV when at least a portion of a planned path generated by a planning module of the ADV is uncertain due to traffic and / or road condition changes. The planning module, in generating the path, also generates a confidence level for each segment of the path based on one or more of perception data, map information, or traffic rules. The farther away from the ADV, the lower the confidence level. When the control module of the ADV obtains the path and the associated confidence levels, the control module issues control commands that only track one or two segments whose confidence levels exceed a threshold and issues default control commands for the rest of the path.

Description

Technical Field

[0001] Embodiments of this disclosure generally relate to operating autonomous vehicles. More specifically, embodiments of this disclosure relate to methods and systems for operating autonomous vehicles, which process planned paths based on confidence levels to ensure safety. Background Technology

[0002] Vehicles operating in autonomous mode (e.g., driverless) can reduce some of the driving-related responsibilities for passengers, especially the driver. When operating in autonomous mode, the vehicle can use onboard sensors to navigate to various locations, allowing the vehicle to operate with minimal human-machine interaction or even without any passengers.

[0003] Motion planning and control are key operations for autonomous driving. However, traditional motion planning operations rely on the perceived environment to generate paths, and control operations generate control commands to enable the vehicle to follow the path.

[0004] However, in some driving environments, traffic changes rapidly, and a planned path a few seconds earlier based on the existing traffic and / or conditions at that time may not be the optimal path for the vehicle based on the current traffic and / or road conditions. For the planned path at a later time, some parts may be invalid when the vehicle arrives. Therefore, if a vehicle continues to travel along the entire path without considering changes in traffic and / or road conditions, the vehicle may be exposed to hazardous situations during the journey because it is effectively traveling along a path that does not reflect the actual driving environment. Summary of the Invention

[0005] On one hand, embodiments of this disclosure provide a method for operating an autonomous driving vehicle (ADV), comprising:

[0006] ADV generates a path that includes multiple segments, each with a different confidence level;

[0007] The ADV generates a first sequence of control command sets to drive the ADV to track one or more of the plurality of segments, each of which has a confidence level exceeding a predetermined threshold.

[0008] The ADV adjusts each segment of the remaining segments of the path, including correcting one or more parameters of the segment; and generates a second sequence of control commands to drive the ADV to track each adjusted segment of the path.

[0009] On the other hand, embodiments of this disclosure provide a non-transitory machine-readable medium having instructions stored thereon for operating an autonomous driving vehicle (ADV), wherein when executed by a processor, the instructions cause the processor to perform operations as described above.

[0010] On the other hand, embodiments of this disclosure provide a data processing system, including:

[0011] Processor; and

[0012] A memory coupled to the processor stores instructions for operating an autonomous vehicle (ADV), which, when executed by the processor, cause the processor to perform operations as described above.

[0013] On the other hand, embodiments of this disclosure provide a computer program product including a computer program that, when executed by a processor, causes the processor to perform the method described above.

[0014] According to this disclosure, the planned path is processed based on the confidence level of the planned path, thereby ensuring the driving safety of ADV. Attached Figure Description

[0015] Embodiments of this disclosure are shown by way of example and are not limited to the figures in the accompanying drawings, in which the same reference numerals denote similar elements.

[0016] Figure 1 This is a block diagram illustrating a networking system according to one embodiment.

[0017] Figure 2 This is a block diagram illustrating an example of an autonomous driving vehicle according to one embodiment.

[0018] Figures 3A-3B This is a block diagram illustrating an example of an autonomous driving system used with an autonomous vehicle according to one embodiment.

[0019] Figure 4 This is a block diagram illustrating an example of a decision-making and planning system according to one embodiment.

[0020] Figure 5 This is a block diagram illustrating a station-lateral distance graph according to one embodiment.

[0021] Figure 6A and 6B This is a block diagram illustrating a station-time graph according to some embodiments.

[0022] Figure 7 This illustrates a system for security assurance control when performing uncertainty planning according to one embodiment.

[0023] Figure 8A and 8B Examples of two planning paths with different confidence levels are shown.

[0024] Figure 9 This illustrates a rule-based algorithm for generating confidence scores for planned paths, based on an embodiment.

[0025] Figure 10 A rule-based path planning processing algorithm 705 according to an embodiment is shown.

[0026] Figure 11 This illustrates a process for providing security assurance controls according to one embodiment. Detailed Implementation

[0027] Various embodiments and aspects of this disclosure will be described with reference to the details of the following discussion, and the accompanying drawings will illustrate various embodiments. The following description and drawings are illustrative of this disclosure and should not be construed as limiting it. Numerous specific details are described to provide a full understanding of the various embodiments of this disclosure. However, in some cases, well-known or conventional details have not been described in order to provide a brief discussion of embodiments of this disclosure.

[0028] References to "an embodiment" or "embodiment" in the specification mean that a particular feature, structure, or characteristic described in connection with that embodiment may be included in at least one embodiment of this disclosure. The phrase "in an embodiment" appearing in various places in the specification does not necessarily refer to the same embodiment.

[0029] According to some embodiments, this document describes methods and systems for ensuring the safety of the control level of an ADV when at least a portion of a planned path generated by the planning module of the ADV becomes uncertain due to changes in traffic and / or road conditions. When generating a path, the planning module also generates a confidence level for each segment of the path based on one or more perception data, map information, or traffic rules. When the ADV's control module obtains the path and the associated confidence level, the control module issues control commands to track only one or two segments whose confidence levels exceed a threshold, and issues default control commands for the remaining portion of the path.

[0030] In the various embodiments described in this invention, the terms "path" and "trajectory" are used interchangeably. The various embodiments can handle planning uncertainties within the control module, thereby ensuring safer driving when planning uncertainties exist.

[0031] In one embodiment, the one or more parameters being corrected include one or more of the speed, heading, or acceleration of the ADV at each reference point on the segment in which the adjustment is performed. Each of the one or more parameters is corrected to a predetermined constant value.

[0032] In one embodiment, the confidence score for each of the multiple segments of the path is generated using a set of rules based on perceptual data, map information, and path rules, or using a trained neural network model. The neural network model generates a confidence score only for the first segment of the path, and the planning module provides a default confidence score for the rest of the path.

[0033] In one embodiment, the path and the confidence level of each segment of the path are passed from the planning module of ADV to the control module of ADV using a data structure, such as a linked list.

[0034] The above embodiments are not a detailed description of all aspects of the invention. It is conceivable that the invention includes all embodiments that can be practiced from all suitable combinations of the various embodiments summarized above, as well as those disclosed below.

[0035] autonomous vehicles

[0036] Figure 1 This is a block diagram illustrating an autonomous vehicle network configuration according to one embodiment of the present disclosure. (See reference...) Figure 1 Network configuration 100 includes an autonomous vehicle (ADV) 101, which can be communicatively coupled to one or more servers 103-104 via network 102. Although only one ADV is shown, multiple ADVs can be coupled to each other and / or to servers 103-104 via network 102. Network 102 can be any type of network, such as a local area network (LAN), a wide area network (WAN) such as the Internet, a cellular network, a satellite network, or a combination thereof, wired or wireless. Servers 103-104 can be any type of server or server cluster, such as a web or cloud server, an application server, a backend server, or a combination thereof. Servers 103-104 can be data analytics servers, content servers, traffic information servers, map and point of interest (MPOI) servers, or location servers, etc.

[0037] ADV refers to a vehicle that can be configured to operate in an autonomous mode, in which the vehicle navigates its environment with little or no driver input. Such an ADV may include a sensor system with one or more sensors configured to detect information about the environment in which the vehicle operates. The vehicle and its associated controller use the detected information to navigate through the environment. ADV 101 can operate in manual mode, fully autonomous mode, or partially autonomous mode.

[0038] In one embodiment, ADV 101 includes, but is not limited to, an autonomous driving system (ADS) 110, a vehicle control system 111, a wireless communication system 112, a user interface system 113, and a sensor system 115. ADV 101 may further include certain common components found in ordinary vehicles, such as an engine, wheels, steering wheel, transmission, etc., which can be controlled by the vehicle control system 111 and / or ADS 110 using various communication signals and / or commands (e.g., acceleration signals or commands, deceleration signals or commands, steering signals or commands, braking signals or commands, etc.).

[0039] Components 110-115 can be communicatively coupled to each other via interconnect, bus, network, or a combination thereof. For example, components 110-115 can be communicatively coupled to each other via a Controller Area Network (CAN) bus. The CAN bus is a vehicle bus standard designed to allow microcontrollers and devices to communicate with each other in masterless applications. It is a message-based protocol originally designed for multiplexing electrical wiring within vehicles, but is also used in many other environments.

[0040] Now for reference Figure 2 In one embodiment, the sensor system 115 includes, but is not limited to, one or more cameras 211, a Global Positioning System (GPS) unit 212, an Inertial Measurement Unit (IMU) 213, a radar unit 214, and a light detection and range (LIDAR) unit 215. The GPS system 212 may include a transceiver operable to provide information about the ADV's location. The IMU unit 213 may sense changes in the ADV's position and orientation based on inertial acceleration. The radar unit 214 may represent a system that uses radio signals to sense objects within the ADV's local environment. In some embodiments, in addition to sensing objects, the radar unit 214 may additionally sense the velocity and / or heading of objects. The LIDAR unit 215 may use lasers to sense objects in the ADV's environment. The LIDAR unit 215 may include one or more laser sources, a laser scanner, and one or more detectors, as well as other system components. The camera 211 may include one or more devices to capture images of the environment surrounding the ADV. The camera 211 may be a still camera and / or a video camera. The camera may be mechanically movable, for example, by mounting the camera on a rotating and / or tilting platform.

[0041] The sensor system 115 may further include other sensors, such as sonar sensors, infrared sensors, steering sensors, throttle sensors, brake sensors, and audio sensors (e.g., microphones). The audio sensor may be configured to capture sound from the environment surrounding the ADV. The steering sensor may be configured to sense the steering angle of the steering wheel, the vehicle's wheels, or a combination thereof. The throttle and brake sensors sense the vehicle's throttle and brake positions, respectively. In some cases, the throttle and brake sensors may be integrated into an integrated throttle / brake sensor.

[0042] In one embodiment, the vehicle control system 111 includes, but is not limited to, a steering unit 201, a throttle unit 202 (also referred to as an acceleration unit), and a braking unit 203. The steering unit 201 is used to adjust the direction or heading of the vehicle. The throttle unit 202 is used to control the speed of a motor or engine, which in turn controls the speed and acceleration of the vehicle. The braking unit 203 decelerates the vehicle by providing friction to slow down the wheels or tires. Note that... Figure 2 The components shown can be implemented in hardware, software, or a combination thereof.

[0043] Return to reference Figure 1 The wireless communication system 112 allows communication between ADV 101 and external systems, such as devices, sensors, other vehicles, etc. For example, the wireless communication system 112 can communicate wirelessly with one or more devices directly or via a communication network, such as communicating with servers 103-104 via network 102. The wireless communication system 112 can use any cellular communication network or wireless local area network (WLAN), such as using WiFi to communicate with another component or system. The wireless communication system 112 can communicate directly with devices (e.g., passenger mobile devices, display devices, speakers within vehicle 101), for example, using infrared links, Bluetooth, etc. The user interface system 113 can be part of peripheral devices implemented within vehicle 101, including, for example, a keyboard, touchscreen display device, microphone, and speakers.

[0044] Some or all of the functions of ADV 101 can be controlled or managed by ADS 110, especially when operating in autonomous driving mode. ADS 110 includes the necessary hardware (e.g., processor, memory, storage devices) and software (e.g., operating system, planning and routing programs) to receive information from sensor system 115, control system 111, wireless communication system 112, and / or user interface system 113, process the received information, plan a route or path from the origin to the destination, and then drive vehicle 101 based on the planning and control information. Alternatively, ADS 110 can be integrated with vehicle control system 111.

[0045] For example, a passenger can specify the start and destination of their trip via a user interface. The ADS 110 obtains trip-related data. For instance, the ADS 110 can obtain location and route data from an MPOI server, which may be part of servers 103-104. The location server provides location services, and the MPOI server provides map services and points of interest (POIs) for certain locations. Alternatively, this location and MPOI information can be locally cached in the ADS 110's persistent storage.

[0046] As ADV 101 moves along the route, ADS 110 can also obtain real-time traffic information from a traffic information system or server (TIS). Note that servers 103-104 can be operated by a third-party entity. Alternatively, the functionality of servers 103-104 can be integrated with ADS 110. Based on real-time traffic information, MPOI information, and location information, as well as real-time local environmental data (e.g., obstacles, objects, nearby vehicles) detected or sensed by sensor system 115, ADS 110 can plan an optimal route and, for example, drive vehicle 101 according to the planned route via control system 111 to safely and efficiently reach the designated destination.

[0047] Server 103 may be a data analysis system used to perform data analysis services for various clients. In one embodiment, data analysis system 103 includes a data collector 121 and a machine learning engine 122. Data collector 121 collects driving statistics 123 from various vehicles (ADVs or conventional vehicles driven by human drivers). Driving statistics 123 include information indicating issued driving commands (e.g., accelerator, brake, steering commands) and vehicle responses (e.g., speed, acceleration, deceleration, direction) captured by vehicle sensors at different points in time. Driving statistics 123 may further include information describing the driving environment at different points in time, such as route (including starting and destination locations), MPOI, road conditions, weather conditions, etc.

[0048] Based on driving statistics 123, machine learning engine 122 generates or trains a set of rules, algorithms, and / or prediction models 124 for various purposes. Algorithm 124 can then be uploaded to ADV for use in real time during autonomous driving.

[0049] Figure 3A and 3B This is a block diagram illustrating an example of an autonomous driving system used with an ADV according to one embodiment. System 300 can be implemented as follows: Figure 1The ADV 101 includes, but is not limited to, ADS 110, control system 111, and sensor system 115. (See reference...) Figures 3A-3B The ADS 110 includes, but is not limited to, a positioning module 301, a sensing module 302, a prediction module 303, a decision-making module 304, a planning module 305, a control module 306, and a routing module 307.

[0050] Some or all of modules 301-307 may be implemented in software, hardware, or a combination thereof. For example, these modules may be installed in permanent storage device 352, loaded into memory 351, and executed by one or more processors (not shown). Note that some or all of these modules may be communicatively coupled to... Figure 2 Some or all of the modules of the vehicle control system 111, or integrated therewith. Some of the modules 301-307 can be integrated together as an integrated module.

[0051] The positioning module 301 determines the current location of the ADV 300 (e.g., using GPS unit 212) and manages any data related to the user's trip or route. The positioning module 301 (also called the map and route module) manages any data related to the user's trip or route. The user can log in, for example, via a user interface and specify the start and destination of the trip. The positioning module 301 communicates with other components of the ADV 300, such as map and route data 311, to obtain trip-related data. For example, the positioning module 301 can obtain location and route data from a location server and a map and POI (MPOI) server. The location server provides location services, and the MPOI server provides map services and POIs for certain locations, which can be cached as part of the map and route data 311. As the ADV 300 moves along the route, the positioning module 301 can also obtain real-time traffic information from a traffic information system or server.

[0052] Based on sensor data provided by sensor system 115 and positioning information obtained by positioning module 301, perception module 302 determines the perception of the surrounding environment. The perception information can represent the situation around the vehicle being driven by a typical driver. Perception may include lane configuration, traffic light signals, and the relative positions of objects such as other vehicles, pedestrians, buildings, crosswalks, or other traffic-related signs (e.g., stop signs, yield signs). Lane configuration includes information describing one or more lanes, such as, for example, the shape of the lane (e.g., straight or curved), the width of the lane, the number of lanes in the road, one-way or two-way lanes, merging or separating lanes, lane exits, etc.

[0053] The perception module 302 may include a computer vision system or the functionality of a computer vision system to process and analyze images captured by one or more cameras to identify objects and / or features in the ADV's environment. Objects may include traffic signals, lane boundaries, other vehicles, pedestrians and / or obstacles, etc. The computer vision system may use object recognition algorithms, video tracking, and other computer vision techniques. In some embodiments, the computer vision system may map the environment, track objects, and estimate the velocity of objects, etc. The perception module 302 may also detect objects based on additional sensor data provided by other sensors such as radar and / or LIDAR.

[0054] For each object, prediction module 303 predicts how the object will behave in the environment. Given a set of map / route information 311 and traffic rules 312, predictions are performed based on perceived data of the driving environment at a given point in time. For example, if the object is a vehicle traveling in the opposite direction and the current driving environment includes an intersection, prediction module 303 will predict whether the vehicle is likely to move straight ahead or turn. If the perceived data indicates that there are no traffic lights at the intersection, prediction module 303 can predict that the vehicle may have to come to a complete stop before entering the intersection. If the perceived data indicates that the vehicle is currently in a left-turn-only lane or a right-turn-only lane, prediction module 303 can predict that the vehicle is more likely to make a left turn or a right turn, respectively.

[0055] For each object, decision module 304 makes a decision about how to handle that object. For example, given a specific object (e.g., another vehicle at an intersection) and metadata describing that object (e.g., speed, direction, steering angle), decision module 304 decides how to encounter the object (e.g., overtake, yield, stop, pass). Decision module 304 may make these decisions based on a set of rules, such as traffic rules or driving rules 312, which may be stored in permanent storage device 352.

[0056] The routing module 307 is configured to provide one or more routes or paths from the origin to the destination. For a given trip from the origin to the destination received from the user, for example, the routing module 307 obtains route and map information 311 and determines all possible routes or paths from the origin to the destination. The routing module 307 can generate reference lines in the form of topographic maps for each route it determines from the origin to the destination. The reference lines refer to ideal routes or paths free from any interference from other vehicles, obstacles, or traffic conditions. That is, if there are no other vehicles, pedestrians, or obstacles on the road, the ADV should follow the reference lines precisely or closely. The topographic map is then provided to the decision module 304 and / or the planning module 305. The decision module 304 and / or the planning module 305 examine all possible routes to select and refine one of the optimal routes based on other data provided by other modules (such as traffic conditions from the positioning module 301, the driving environment perceived by the perception module 302, and traffic conditions predicted by the prediction module 303). Depending on the specific driving conditions at a given point in time, the actual path or route used to control the ADV may be close to or different from the reference line provided by the routing module 307.

[0057] Based on the decision for each perceived object, the planning module 305 uses reference lines provided by the routing module 307 as a basis to plan the path or route for ADV, along with driving parameters (e.g., distance, speed, and / or steering angle). That is, for a given object, the decision module 304 decides what to do with that object, while the planning module 305 determines how to do it. For example, for a given object, the decision module 304 might decide to pass the object, while the planning module 305 might determine whether to pass to the left or right of the object. Planning and control data is generated by the planning module 305 and includes information describing how the vehicle 300 will move in the next movement cycle (e.g., the next route / path segment). For example, the planning and control data might instruct the vehicle 300 to move 10 meters at 30 miles per hour (mph) and then change lanes to the right at 25 mph.

[0058] Based on planning and control data, control module 306 controls and drives the ADV by sending appropriate commands or signals to vehicle control system 111 according to the route or path defined by the planning and control data. The planning and control data includes sufficient information to drive the vehicle from one point to another along the route or path at different times using appropriate vehicle settings or driving parameters (e.g., throttle, braking, steering commands).

[0059] In one embodiment, the planning phase is performed within multiple planning cycles (also referred to as driving cycles, such as in each time interval of 100 milliseconds (ms)). For each planning cycle or driving cycle, one or more control commands are issued based on the planning and control data. That is, for every 100 ms, the planning module 305 plans the next route segment or path segment, including, for example, the target location and the time required for the ADV to reach the target location. Alternatively, the planning module 305 may further specify specific speeds, directions, and / or steering angles, etc. In one embodiment, the planning module 305 plans a route segment or path segment for the next predetermined time period, such as 5 seconds. For each planning cycle, the planning module 305 plans a target location for the current cycle (e.g., the next 5 seconds) based on the target location planned in the previous cycle. The control module 306 then generates one or more control commands (e.g., throttle, braking, steering control commands) based on the planning and control data of the current cycle.

[0060] Note that the decision module 304 and the planning module 305 can be integrated into an integrated module. The decision module 304 / planning module 305 may include a navigation system or the functionality of a navigation system to determine a driving path for the ADV. For example, the navigation system may determine a series of speed and heading parameters to influence the movement of the ADV along a path that substantially avoids perceived obstacles, while generally guiding the ADV along a road-based path leading to the final destination. The destination can be set based on user input via the user interface system 113. The navigation system can dynamically update the driving path while the ADV is in operation. The navigation system may incorporate data from a GPS system and one or more maps to determine the driving path for the ADV.

[0061] Figure 4 This is a block diagram illustrating an example of a decision-making and planning system according to one embodiment. System 400 can be implemented as follows: Figures 3A-3B It is part of the autonomous driving system 300 used to perform path planning and speed planning operations. (Reference) Figure 4 The decision and planning system 400 (also known as planning and control or PnC system or module) includes, among others, a routing module 307, a positioning / sensing data 401, a path decision module 403, a velocity decision module 405, a path planning module 407, a velocity planning module 409, an aggregator 411, and a trajectory calculator 413.

[0062] Route decision module 403 and speed decision module 405 can be implemented as part of decision module 304. In one embodiment, route decision module 403 may include a route state machine, one or more route traffic rules, and a station-lateral distance graph generator. Route decision module 403 may use dynamic programming to generate coarse route profiles as initial constraints for route / speed planning modules 407 and 409.

[0063] In one embodiment, the route state machine includes at least three states: cruising state, lane-changing state, and / or idle state. The route state machine provides previous planning results and important information, such as whether the ADV is cruising or changing lanes. Route traffic rules may be... Figure 3A Part of the driving / traffic rules 312 includes traffic rules that may affect the results of the route decision module. For example, route traffic rules may include traffic information such as nearby building traffic signs, which the ADV may avoid lanes with such building traffic signs. The route decision module 403 may determine how to handle perceived obstacles (i.e., ignore, overtake, give way, stop, or pass) based on the status, traffic rules, reference lines, and obstacles perceived by the ADV's perception module 302, as part of a coarse route profile.

[0064] For example, in one embodiment, the coarse path profile is generated by a cost function based on the following cost components: path curvature and the distance from a reference line and / or reference point to an obstacle. Points on the reference line are selected and moved to the left or right of the reference line as candidate moves representing path candidates. Each candidate move has a corresponding cost. The associated costs of candidate moves for one or more points on the reference line can be solved for optimal costs sequentially using dynamic programming, one point at a time.

[0065] In one embodiment, a station-lateral distance (SL) map generator (not shown) generates an SL map as part of a coarse path profile. The SL map is a two-dimensional geometry (similar to an xy-coordinate plane) that includes obstacle information perceived by the ADV. From the SL map, the path decision module 403 can lay out the ADV path based on obstacle decisions. Dynamic programming (also known as dynamic optimization) is a mathematical optimization method that decomposes the problem to be solved into a family of value functions, each of which needs to be solved only once, and stores their solutions. The next time the same value function appears, only the previously calculated solution needs to be looked up, rather than recalculating its solution, thus saving computation time.

[0066] The speed decision process 405 or speed decision module includes a speed state machine, speed traffic rules, and a station time (ST) diagram generator (not shown). The speed decision process 405 or speed decision module can use dynamic programming to generate a coarse speed profile as initial constraints for the path / speed planning modules 407 and 409. In one embodiment, the speed state machine includes at least two states: an acceleration state and / or a deceleration state. The speed traffic rules may be... Figure 3A As part of the driving / traffic rules 312, the speed decision module 405 includes traffic rules that may affect the results of the speed decision module. For example, speed traffic rules may include traffic information such as red / green lights, another vehicle on an intersecting road, etc. From the state of the speed state machine, the speed traffic rules, the coarse path profile / SL diagram generated by the decision module 403, and perceived obstacles, the speed decision module 405 can generate a coarse speed profile to control when to accelerate and / or decelerate the ADV. The SL diagram generator can generate a station time (ST) diagram as part of the coarse speed profile.

[0067] In one embodiment, the path planning module 407 includes one or more SL diagrams, a geometric smoother, and a path cost module (not shown). The SL diagram may include one generated by the SL diagram generator of the path decision module 403. The path planning module 407 may use a coarse path profile (e.g., an SL diagram) as initial constraints to recalculate the optimal reference line using quadratic programming. Quadratic programming (QP) involves minimizing or maximizing an objective function (e.g., a quadratic function with several variables) constrained by bounds, linear equality, and inequalities.

[0068] One difference between dynamic programming and quadratic programming is that quadratic programming optimizes all candidate motions for all points on the reference line in a single operation. A geometric smoother can apply smoothing algorithms (such as B-splines or regression) to the output SL plot. The path cost module can recalculate the reference line using a path cost function to optimize the total cost for candidate motions for the reference points, for example, using QP optimization performed by the QP module (not shown). For example, in one embodiment, the total path cost function can be defined as follows:

[0069] Path cost = ∑ 点 (course) 2 +∑ 点 (curvature) 2 +∑ 点 (distance) 2 ,

[0070] The path cost is summed for all points on the reference line. The heading represents the difference in radial angle (e.g., direction) of a point relative to the reference line. The curvature represents the difference in curvature of the curves formed by these points relative to the reference line at that point. The distance represents the lateral (perpendicular to the reference line) distance from the point to the reference line. In some embodiments, the distance represents the distance from the point to a target location or intermediate point on the reference line. In another embodiment, the curvature cost is the variation in curvature values ​​between the curves formed at adjacent points. Note that points on the reference line can be selected as those equidistant from their neighbors. Based on the path cost, the path cost module can recalculate the reference line by minimizing the path cost using quadratic programming optimization (e.g., via a QP module).

[0071] The velocity planning module 409 includes an ST plot, a sequence smoother, and a velocity cost module. The ST plot may include an ST plot generated by the ST plot generator of the velocity decision module 405. The velocity planning module 409 can use a coarse velocity profile (e.g., the ST plot) and the results of the path planning module 407 as initial constraints to compute the optimal ST curve. The sequence smoother can apply smoothing algorithms (such as B-splines or regression) to the time series of points. The velocity cost module can recompile the ST plot using a velocity cost function to optimize the total cost of motion candidates (e.g., acceleration / deceleration) at different time points.

[0072] For example, in one embodiment, the total speed cost function could be:

[0073] Speed ​​cost = ∑ 点 (speed') 2 +∑ 点 (speed") 2 +(distance) 2 ,

[0074] Here, the speed cost is summed at all time progression points, where speed' represents the acceleration value or the cost of changing speed between two adjacent points, speed" represents the acceleration value, or the derivative of the acceleration value, or the cost of changing acceleration between two adjacent points, and distance represents the distance from the ST point to the destination location. Here, the speed cost module calculates the ST graph by minimizing the speed cost through quadratic programming optimization (e.g., via the QP module).

[0075] Aggregator 411 performs the function of aggregating path and velocity planning results. For example, in one embodiment, aggregator 411 can combine a two-dimensional ST map and a SL map into a three-dimensional SLT map. In another embodiment, aggregator 411 can interpolate (or fill in additional points) based on two consecutive points on the SL reference line or ST curve. In yet another embodiment, aggregator 411 can convert reference points from (S, L) coordinates to (x, y) coordinates. Trajectory generator 413 can calculate the final trajectory to control ADV 510. For example, based on the SLT map provided by aggregator 411, trajectory generator 413 calculates a list of (x, y, T) points indicating when ADV should pass through specific (x, y) coordinates.

[0076] Therefore, the path decision module 403 and the speed decision module 405 are configured to generate coarse path profiles and coarse speed profiles that take into account obstacles and / or traffic conditions. For all path and speed decisions regarding obstacles, the path planning module 407 and the speed planning module 409 optimize the coarse path profiles and coarse speed profiles using QP programming based on the obstacles to generate the optimal trajectory that minimizes path cost and / or speed cost.

[0077] Figure 5 This is a block diagram illustrating an SL diagram according to one embodiment. (Reference) Figure 5 The SL coordinate system (S, L) has a horizontal axis, or station, and a vertical axis, or lateral distance. As mentioned above, the SL coordinate system is a relative geometric coordinate system that references a specific stationary point on a reference line and follows the reference line. For example, the coordinate system (S, L) = (1, 0) can represent a stationary point (i.e., the reference point) on the reference line, one meter in front and with a lateral offset of zero meters. The reference point A(S, L) = (2, 1) can represent a stationary reference point two meters in front and with a lateral offset of 1 meter perpendicular to the reference line, such as a left offset.

[0078] refer to Figure 5The SL coordinate system 500 includes a reference line 501 and obstacles 503-509 sensed by the ADV 510. In one embodiment, obstacles 503-509 can be sensed in a different coordinate system by the radar or LIDAR unit of the ADV 510 and converted to the SL coordinate system. In another embodiment, obstacles 503-509 can be artificially created obstacles as constraints so that the decision-making and planning modules will not search in a constrained geometric space. In this example, the path decision module can generate decisions for each obstacle 503-509, such as avoiding obstacles 503-508 and gently pushing (very close) obstacles 509 (i.e., these obstacles may be other vehicles, buildings, and / or structures). The path planning module can then fine-tune the reference line 501 based on path costs, using QP planning to minimize the total cost as described above, given obstacles 503-509, recalculating or optimizing the reference line 501. In this example, the ADV gently pushes or gets very close to obstacle 509 from the left side.

[0079] Figure 6A and 6B This is a block diagram illustrating ST diagrams according to some embodiments. (Reference) Figure 6A ST diagram 600 has a station (or S) vertical axis and a time (or T) horizontal axis. ST diagram 600 includes curve 601 and obstacles 603-607. As previously mentioned, curve 601 on the ST diagram indicates when and how far the ADV is from the station. For example, (T, S) = (10000, 150) can indicate that after 10000 milliseconds, the ADV is 150 meters away from the stationary point (i.e., the reference point). In this example, obstacle 603 could be a building / structure to be avoided, and obstacle 607 could be an artificial obstacle corresponding to the decision to overtake a moving vehicle.

[0080] refer to Figure 6B In this scenario, an artificial obstacle 605 is added to ST diagram 610 as a constraint. The artificial obstacle can be an example of a red light or a pedestrian in a lane at a distance of approximately S2 from the station reference point, as perceived by the ADV. Obstacle 605 corresponds to the decision to "stop" the ADV until the artificial obstacle is removed at a later time (e.g., the traffic light changes from red to green, or the pedestrian is no longer in the lane).

[0081] Ensure safe control

[0082] Figure 7 A system for ensuring safety during uncertainty planning, according to one embodiment, is illustrated. Figure 7 As shown, a confidence generator 701 can be provided in the planning module 305 to generate confidence for each segment of the path planned by the planning module 305.

[0083] In one embodiment, the planning module 305 can generate a planned path (also called a planned path) for a subsequent time interval (e.g., the next 8 seconds). Each planned path can be a data structure (e.g., a linked list) that includes information describing multiple parameters for each reference point on the planned path. Parameters for each reference point can include velocity, curvature, and heading. The parameters specify the expected action or state of the ADV at each reference point.

[0084] In this embodiment, when generating the planned path 707, the planning module 305 can divide the planned path into multiple segments, each with a different confidence level. For example, the planned path 707 is divided into segments A 709, B 711, and C 713. The confidence level of segment A 709 is 100%, the confidence level of segment B 711 is 90%, and the confidence level of segment C 713 is 60%. These confidence levels are merely illustrative examples. In many cases, the planned path can have a confidence level that does not change much from one segment to the next. For example, a planned path with three segments can have three confidence levels: 100%, 99%, and 95%.

[0085] In one embodiment, the higher the confidence level of a specific segment of the path, the less likely the planning module 305 is to correct that specific segment when the ADV reaches that segment when a new trajectory is generated in a future planning cycle.

[0086] For example, for segment A with 100% confidence, ADV will not correct the segment and will therefore issue control commands (e.g., throttle, brake, steering control commands) to follow the segment. For segment C with 60% confidence, ADV is likely to correct the path segment when it reaches that segment.

[0087] In one embodiment, the confidence level of each planned path is a reflection of the traffic and / or road conditions of the physical road segment corresponding to that planned path segment. In real-world driving environments, such as on busy streets with constantly changing traffic and / or road conditions, the planned path generated at a specific time (e.g., t0) will be affected by changes in traffic and / or road conditions at another time (e.g., t1). 0+5 This may not be the optimal path. The further away the corresponding physical road segment is from the time when ADV generates the planned path, the lower the confidence of the corresponding path segment, because the traffic and / or road conditions will be more uncertain when ADV arrives at that road segment.

[0088] In one embodiment, the confidence generator 701 may implement an algorithm that takes perceived data, map information, and traffic rules as input to generate confidence scores for the planned path 707. Alternatively, a trained neural network model may be used to generate the confidence scores for the planned path 707.

[0089] The confidence scores of the planned path 707 and its different segments 709, 711 and 713 can be passed to the control module 306 via 703. The control module 306 can then implement the rule-based path processing algorithm 705 to process the planned path 707.

[0090] For example, control module 306 can issue multiple sets of control commands sequentially so that ADV can only track path segments at the lowest cost when the relevant confidence of a path segment exceeds a threshold (e.g., 95%). For path segments whose confidence does not exceed the threshold, control module 306 issues multiple sets of default control commands sequentially.

[0091] Therefore, by tracking high-confidence path segments, ADV can reduce the dangers it faces, thereby improving its safety.

[0092] Figure 8A and 8B Two examples of planned paths with different confidence levels are shown. Figure 8A The path shown has three segments. The first segment starts at t0501 and ends at t5503 with a confidence level of 100%; the second segment starts at t5503 and ends at t7505 with a confidence level of 80%; and the third segment starts at t7505 and ends at t8507 with a confidence level of 40%. As shown in the figure, the confidence level of a path segment decreases as the path segment is further away from the time t0 from which the path was generated.

[0093] exist Figure 8B In practice, due to changes in traffic and / or road conditions, the confidence distribution of the planned path changes accordingly. Although the planned path still has three segments: the first segment is between t0509 and t5511 with a confidence level of 100%; the second segment is between t5511 and t7513 with a confidence level of 80%; and the third segment is between t7513 and t8515 with a confidence level of 40%, the segment with a confidence level of 100% is much shorter than [the previous segment]. Figure 8A The corresponding segments, while segments with a confidence level of 40% are compared to Figure 8A The corresponding segments are much longer. This confidence distribution may reflect heavy traffic and / or poor road conditions, which introduces more uncertainty into the planned route.

[0094] In one embodiment, Figure 8A and Figure 8B The paths shown can be generated in different planning cycles, where Figure 8B The paths in the process are generated in later planning cycles.

[0095] Figure 9A rule-based algorithm for generating confidence scores for planned paths is illustrated according to an embodiment. This algorithm can be implemented using software, hardware, or a combination of both.

[0096] like Figure 9 As shown, in operation 901, the confidence generator 701 examines the perception data, map information and / or traffic rules for each reference point on the planned path.

[0097] In operation 903, the confidence generator 701 identifies segments of the planned path in which traffic and road conditions are unlikely to change, and assigns 100% confidence to these segments. In this operation, the confidence generator 701 may first determine whether there is any traffic in the perceived driving environment. If there is no traffic in the perceived driving environment, the confidence generator 701 can determine that traffic conditions are unlikely to change and can calculate the segments of the planned path to which 100% confidence should be assigned. In one embodiment, to calculate the length of a segment of the planned path, the confidence generator 701 may determine a time period based on the ADV's perception range and the ADV's maximum possible relative speed in the driving environment. Using the calculated time period, the confidence generator 701 can calculate the distance along the planned path based on this time period.

[0098] In one embodiment, the time period calculated based on the ADV's sensing range and maximum relative speed is the time period during which no traffic can interfere with the ADV's operation.

[0099] In one embodiment, the sensing range is the maximum detection distance of a LiDAR device, camera unit, or radar unit mounted on the ADV, and the maximum relative speed of the ADV is the sum of the speed limit of the driving environment and the current speed of the ADV. In another embodiment, the maximum relative speed represents the speed of the ADV relative to another vehicle entering the sensing range from the opposite direction.

[0100] For example, if an ADV is traveling at 33 mph on a road segment with a speed limit of 35 mph, and the ADV's sensors, with a perception range of 500 meters (approximately 0.310686 miles), do not detect any traffic, the confidence generator 701 can calculate the time period as 0.310686 / (35+33) = 0.00456891176 hours = 16.45 seconds. Therefore, the confidence generator can assign 100% confidence to a segment of the planned path within the first 16 seconds.

[0101] If there is traffic within the perception area, the confidence generator 701 can determine whether the ADV is traveling in a lane with solid lane lines. If so, the confidence generator 701 can also assign 100% confidence to the segments of the planned path in lanes with solid lane lines.

[0102] If there is traffic within the perception range and the ADV is not traveling in a lane with solid lane markings, the confidence generator 701 can assign 100% confidence to a predetermined segment of the planned path (e.g., the first two seconds of the planned path). It is assumed that the first two seconds of the planned path are always reliable regardless of traffic and road conditions.

[0103] In operation 905, confidence generator 701 identifies the next segment of the route where traffic or road conditions may change and assigns an 80% confidence level to that route segment.

[0104] In one embodiment, traffic conditions may change when there is traffic in the sensing area and the ADV is not traveling in a lane with solid lane markings. In this case, the confidence generator 701 can identify the next four segments as the next segment.

[0105] In operation 907, confidence generator 107 can allocate 60% of the confidence to the remaining part of the planning segment.

[0106] In the example algorithm above, the confidence level assigned to each segment is for illustrative purposes only and may be changed based on the actual implementation.

[0107] The algorithm described above is one method for generating confidence scores for planned routes, and many other algorithms or methods can be used based on perception data, map data, and traffic rules.

[0108] For example, another algorithm can divide any planned path into only two segments, with the first segment having a 100% confidence level and the remainder of the planned path having a 50% confidence level. The first segment is determined as described in Operation 901. The second segment is the remaining part of the planned path after subtracting the first segment.

[0109] In another embodiment, the algorithm for generating confidence scores for the planned path can be entirely based on prediction module 303. If the prediction module can accurately predict the movement of obstacles around the ADV within a specific segment, then that segment of the planned path is given 100% confidence. For the remaining portion of the planned path where obstacles around the ADV cannot be accurately predicted, a 50% confidence score is given.

[0110] In another embodiment, the neural network model, such as a convolutional neural network, can be trained based on perception data, map information, and traffic rules collected from a specific road segment. The trained neural network model can be trained to recognize only the first segment of the planned path (e.g., the first 2 seconds), assigning 100% confidence to the first segment and a default confidence of 50% to the remaining path.

[0111] Figure 10 A rule-based path planning processing algorithm 705 according to an embodiment is shown. This algorithm can be implemented using software, hardware, or a combination of both.

[0112] In one embodiment, during operation 1001, after receiving the planned path and the confidence level of each segment from the planning module 305, the control module 306 can perform analysis of the information, including reference points and their associated parameters, as well as the path confidence level. The control module 306 can determine the number of reference points in each segment during this operation.

[0113] In operation 1003, control module 306 can identify one or more segments of the planned path whose confidence exceeds a threshold. For example, if the planned path includes three segments with confidence levels of 100%, 94%, and 50%, respectively, control module 306 can set the threshold to 90%. Therefore, segments with a confidence level of 100% and segments with a confidence level of 94% can be identified.

[0114] In Operation 1005, the control module can issue multiple sets of control commands sequentially to drive the ADV to track one or more segments of the planned path.

[0115] In operation 1007, control module 306 can maintain constant parameters (e.g., constant speed, constant curvature, and constant heading) for the remainder of the planned path. These constant parameters can be predetermined based on previously collected driving statistics. For example, for each remaining segment, control module 306 can issue commands to maintain a constant speed of 35 mph, zero curvature, and a heading for ADV at the end of the final segment with 100% confidence. Thus, ADV effectively ignores the remaining segments of the planned path.

[0116] In one embodiment, ignoring the remaining segments of the planned path does not pose a danger to the ADV, because a new planned path has already been generated to guide the ADV before it reaches any remaining segments.

[0117] In one embodiment, ignoring the planned path segment means that the control module can dynamically adjust the control parameters based on the confidence level of the planned path segment. The control module is a model predictive control, so it can predict a predetermined number of seconds (e.g., 3 seconds or 5 seconds) along the planned path and take action in advance based on the confidence level of the path segment at the predetermined number of seconds.

[0118] As an illustrative example, if the planning module generates a planned path that includes a path segment with 100% confidence (e.g., the next 5 seconds), the control module can issue a control command to increase acceleration to drive the vehicle to the speed limit once it receives the planned path. However, if the same path segment has only 40% confidence, meaning the vehicle is only 40% confident there will be no obstacle in the next 5 seconds, the control module can begin decelerating before entering the path segment and prepare to brake quickly in case an obstacle appears.

[0119] Therefore, the control module can issue control commands to bring the vehicle into a desired driving state on a path segment where the confidence level exceeds a predetermined threshold (e.g., desired speed, desired heading, etc.). The control module can also adjust the control commands in advance for path segments where the confidence level is below the predetermined threshold to prevent dangerous driving situations. As discussed above, one method of adjusting the control commands is to issue a constant control command before entering a low-confidence path segment.

[0120] Figure 11 A process for providing security-enhancing controls according to one embodiment is illustrated. This process can be executed by processing logic, which may include software, hardware, or a combination thereof. For example, the process may be performed by… Figure 7 The planning module 305 and control module 306 described herein are executed.

[0121] like Figure 11 As shown, in operation 1101, the processing logic generates a path comprising multiple segments, each segment having a different confidence level. In operation 1103, the processing logic generates a first sequence of control commands to drive the ADV to track one or more segments of the multiple segments, each of which has a confidence level exceeding a predetermined threshold. In operation 1105, the processing logic adjusts each of the remaining segments of the path, including correcting one or more parameters of the segment. In operation 1107, the processing logic generates a second sequence of control commands to drive the ADV to track each adjusted segment of the path.

[0122] Note that some or all of the components shown and described above can be implemented in software, hardware, or a combination thereof. For example, these components can be implemented as software installed and stored in a permanent storage device, which can be loaded and executed in memory by a processor (not shown) to perform the processes or operations described throughout this application. Alternatively, these components can be implemented as executable code programmed or embedded in dedicated hardware such as integrated circuits (e.g., application-specific ICs or ASICs), digital signal processors (DSPs), or field-programmable gate arrays (FPGAs), accessible via corresponding drivers and / or operating systems from the application. Furthermore, these components can be implemented as specific hardware logic within a processor or processor core as part of an instruction set accessible via one or more specific instruction software components.

[0123] Some parts of the foregoing detailed description of algorithms and symbolic representations for operations on data bits within computer memory have already been presented. These algorithmic descriptions and representations are the most efficient way for those skilled in the art of data processing to communicate the substance of their work to others skilled in the art. Algorithms here and generally are considered to be self-consistent sequences of operations that lead to desired results. These operations are those that require physical manipulation of physical quantities.

[0124] However, it should be remembered that all these and similar terms are associated with appropriate physical quantities and are merely convenient notations applied to those quantities. Unless otherwise stated, it is obvious from the above discussion that it should be understood that throughout the specification, the use of terms such as those set forth in the appended claims refers to the actions and processes of a computer system or similar electronic computing device that manipulates and transforms data represented as physical (electronic) quantities in the registers and memories of the computer system into other data similarly represented as physical quantities in the computer system's memory or registers or other such information storage, transmission, or display devices.

[0125] Embodiments of this disclosure also relate to means for performing the operations described herein. Such a computer program is stored in a non-transitory computer-readable medium. Machine-readable media include any mechanism for storing information in a machine-readable (e.g., computer-readable) form. For example, machine-readable (e.g., computer-readable) media include machine-readable (e.g., computer-readable) storage media (e.g., read-only memory (“ROM”), random access memory (“RAM”), disk storage media, optical storage media, flash memory devices).

[0126] The processes or methods described in the foregoing figures can be performed by processing logic including hardware (e.g., circuitry, dedicated logic, etc.), software (e.g., embodied on a non-transitory computer-readable medium), or a combination of both. Although the processes or methods have been described above according to some sequential operations, it should be understood that some of the operations can be performed in different orders. Furthermore, some operations can be performed in parallel rather than sequentially.

[0127] The embodiments disclosed herein are not described with reference to any particular programming language. It will be understood that the teachings of the embodiments of this disclosure as described herein can be implemented using various programming languages.

[0128] In the foregoing description, embodiments of the present disclosure have been described with reference to specific exemplary embodiments. It will be apparent that various modifications may be made thereto without departing from the broader spirit and scope of the present disclosure as set forth in the appended claims. Therefore, the description and drawings should be considered illustrative rather than restrictive.

Claims

1. A method for operating an autonomous vehicle (ADV), comprising: ADV generates a path that includes multiple segments, each with a different confidence level; The ADV generates a first sequence of control command sets to drive the ADV to track one or more of the plurality of segments, each of which has a confidence level exceeding a predetermined threshold. The ADV adjusts each segment of the remaining segments of the path, including correcting one or more parameters of the segment; as well as The ADV generates a second sequence of control commands to drive the ADV to track each adjusted segment of the path.

2. The method of claim 1, wherein the one or more parameters being modified include one or more of the speed, heading, or acceleration of the ADV at each point on the adjusted segment.

3. The method of claim 2, wherein each of the one or more parameters is modified to a predetermined constant value.

4. The method of claim 1, wherein a set of rules is used to generate the confidence level of each segment of the path based on perception data, map information, and traffic rules.

5. The method of claim 1, wherein a trained neural network model is used to generate the confidence level of each segment of the plurality of segments of the path.

6. The method of claim 5, wherein the neural network model is trained to generate confidence for a first segment of the path and to provide default confidence for the remainder of the path.

7. The method of claim 1, wherein the ADV includes a planning module and a control module, wherein the path and the confidence level of each segment of the path are passed from the planning module of the ADV to the control module of the ADV.

8. A non-transitory machine-readable medium storing instructions for operating an autonomous driving vehicle (ADV), wherein when executed by a processor, the instructions cause the processor to perform the operation of the method as described in any one of claims 1 to 7.

9. A data processing system, comprising: processor; as well as A memory coupled to a processor stores instructions for operating an autonomous driving vehicle (ADV), which, when executed by the processor, cause the processor to perform the operation of the method as described in any one of claims 1 to 7.

10. A computer program product comprising a computer program that, when executed by a processor, causes the processor to perform the method as described in any one of claims 1 to 7.

Images