Method and apparatus for autonomous vehicle execution

Abstract

Embodiments of the present disclosure disclose a method and apparatus for autonomous vehicle execution, and a data processing system. In one embodiment, a process is performed during control of an autonomous vehicle (ADV). A plurality of point confidence scores are determined, each point confidence score defining the reliability of a corresponding point on a trajectory of a moving obstacle. At least one of the point confidence scores is determined based on a) an overall trajectory confidence score and b) at least one environmental factor of the obstacle. The ADV is controlled based on the trajectory of the moving obstacle and at least one of the plurality of point confidence scores.

Description

Technical Field

[0001] Embodiments of the present disclosure generally relate to operating autonomous vehicles. More particularly, embodiments of the present disclosure relate to methods and apparatus and a data processing system for autonomous driving vehicle (ADV) execution. Background Art

[0002] A vehicle operating in an autonomous mode (e.g., without human control) can relieve occupants, particularly the driver, of some driving-related responsibilities. When operating in an autonomous mode, the vehicle can navigate to various locations using onboard sensors, allowing the vehicle to travel with minimal human interaction or in some cases without any passengers.

[0003] Motion planning and control are key operations in autonomous driving. However, traditional motion planning operations mainly estimate the difficulty of completing a given path from the curvature and speed of the given path, without considering the differences in the characteristics of different types of vehicles. The same motion planning and control are applied to all types of vehicles, which may not be precise and smooth in some cases.

[0004] Planning decisions made by an autonomous vehicle (ADV) can be based on map data and traffic regulations. Moving obstacles such as other vehicles, cyclists, and pedestrians can change the route of the ADV. The trajectory of such moving obstacles can be predicted to help adjust the route of the ADV. Because the trajectory is a prediction of how the obstacle will move, it is uncertain that the actual path of the obstacle will match the predicted path. Therefore, planning decisions can be made based on the ADV's confidence level in the prediction. Summary of the invention

[0005] Embodiments of the present disclosure disclose a method and apparatus for autonomous driving vehicle execution, and a data processing system.

[0006] In a first aspect of an embodiment of the present disclosure, there is provided a method for autonomous driving vehicle execution, comprising: determining a current state confidence score, the current state confidence score representing a confidence level related to the accuracy of a current state of a moving obstacle, the current state of the moving obstacle including one or more of the speed, position, heading, acceleration or type of the moving obstacle; generating a trajectory predicting a path of the moving obstacle based on the current state of the moving obstacle, map data and traffic rules; determining a trajectory confidence score representing a confidence level of the trajectory on which the moving obstacle has a possibility of moving based on the current state confidence score and the current state of the moving obstacle; and planning a path in view of the trajectory and the trajectory confidence score to control and drive the ADV.

[0007] In a second aspect of an embodiment of the present disclosure, there is provided an apparatus for autonomous driving vehicle execution, comprising: a first determination module for determining a current state confidence score, the current state confidence score representing a confidence level related to the accuracy of a current state of a moving obstacle, the current state of the moving obstacle comprising one or more of the speed, position, heading, acceleration or type of the moving obstacle; a generation module for generating a trajectory predicting a path of the moving obstacle based on the current state of the moving obstacle, map data and traffic rules; a second determination module for determining a trajectory confidence score representing a confidence level of a trajectory on which the moving obstacle has a possibility of moving based on the current state confidence score and the current state of the moving obstacle; and a planning module for planning a path to control and drive the ADV in view of the trajectory and the trajectory confidence score.

[0008] In a third aspect of an embodiment of the present disclosure, a non-transitory machine-readable medium is provided, in which instructions are stored. When the instructions are executed by a processor, the processor executes the method as described in the first aspect of the embodiment of the present application.

[0009] In a fourth aspect of an embodiment of the present disclosure, a data processing system is provided, comprising: a processor; and a memory coupled to the processor to store instructions, which, when executed by the processor, causes the processor to execute the method described in the first aspect of the embodiment of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] Embodiments of the present disclosure are illustrated by way of example, and not by way of limitation, in the figures of the accompanying drawings in which like references indicate similar elements.

[0011] Figure 1 is a block diagram illustrating a networking system according to one embodiment.

[0012] Figure 2 is a block diagram illustrating an example of an autonomous vehicle according to one embodiment.

[0013] Figure 3A-3B is a block diagram illustrating an example of a perception and planning system for use with an autonomous vehicle, according to one embodiment.

[0014] Figure 4 A process for generating point confidence scores according to one embodiment is shown.

[0015] Figure 5 A block diagram illustrating a process for generating point confidence scores according to one embodiment.

[0016] Figure 6-8Shown are environmental factors surrounding a trajectory that influence point confidence scores according to some embodiments. DETAILED DESCRIPTION

[0017] Various embodiments and aspects of the present disclosure will be described with reference to the details discussed below, and the accompanying drawings will illustrate various embodiments. The following description and the accompanying drawings are illustrative of the present disclosure and should not be construed as limiting the present disclosure. Many specific details are described to provide a comprehensive understanding of various embodiments of the present disclosure. However, in some cases, in order to provide a brief discussion of embodiments of the present disclosure, known or conventional details are not described.

[0018] References in the specification to "one embodiment" or "an embodiment" mean that a particular feature, structure, or characteristic described in conjunction with the embodiment may be included in at least one embodiment of the present disclosure. The phrase "in one embodiment" appearing in various places in the specification does not necessarily refer to the same embodiment.

[0019] According to some embodiments, a process may determine multiple confidence scores along a single predicted trajectory of a moving obstacle. The confidence score indicates the confidence level that a particular data is considered to be determined. This process may include determining a current state confidence level associated with a current state of the moving obstacle, the current state of the moving obstacle including one or more of the speed, position, heading, acceleration, or type of the moving obstacle. Generate a trajectory that predicts the path of the moving obstacle. The trajectory may be generated based on the current state of the moving obstacle, map data, and traffic rules. Generate a single trajectory confidence score and associate it with the trajectory of the moving obstacle. Determine the trajectory confidence score based on the current state confidence score and the current state of the moving obstacle. The trajectory confidence score defines the overall confidence level of the trajectory. Determine multiple point confidence scores, each point confidence score defining the reliability of a corresponding point on the trajectory. Determine at least one of the point confidence scores based on a) the trajectory confidence score and b) at least one environmental factor of the obstacle. Control the ADV based on at least one of the multiple point confidence scores of the trajectory of the moving obstacle. For example, the ADV may change the route based on the point confidence score of the trajectory. In the event that the point confidence score is low, the ADV may choose not to change the route, but in the event that the point confidence score is high, the ADV may change the route to prevent possible negative interactions (such as collisions) with obstacles.

[0020] Figure 1 is a block diagram showing an autonomous vehicle network configuration according to one embodiment of the present disclosure. Figure 1, network configuration 100 includes an autonomous vehicle 101, which can be communicatively coupled to one or more servers 103-104 via a network 102. Although one autonomous vehicle is shown, multiple autonomous vehicles can be coupled to each other and / or to servers 103-104 via network 102. Network 102 can be any type of network, such as a local area network (LAN), a wide area network (WAN) such as the Internet, a cellular network, a satellite network, or a combination thereof, wired or wireless. Servers 103-104 can be any type of server or server cluster, such as a web or cloud server, an application server, a back-end server, or a combination thereof. Servers 103-104 can be data analysis servers, content servers, traffic information servers, maps and points of interest (MPOI) servers, or location servers, etc.

[0021] An autonomous vehicle refers to a vehicle that can be configured to be in an autonomous mode, in which the vehicle navigates through an environment with little or no input from a driver. Such an autonomous vehicle may include a sensor system having one or more sensors configured to detect information about the environment in which the vehicle operates. The vehicle and its associated controller use the detected information to navigate through the environment. Autonomous vehicle 101 may operate in a manual mode, a fully autonomous mode, or a partially autonomous mode.

[0022] In one embodiment, autonomous vehicle 101 includes, but is not limited to, perception and planning system 110, vehicle control system 111, wireless communication system 112, user interface system 113, and sensor system 115. Autonomous vehicle 101 may also include certain common components included in ordinary vehicles, such as an engine, wheels, a steering wheel, a transmission, etc., which can be controlled by vehicle control system 111 and / or perception and planning system 110 using various communication signals and / or commands (e.g., acceleration signals or commands, deceleration signals or commands, steering signals or commands, braking signals or commands, etc.).

[0023] The components 110-115 may be communicatively coupled to each other via an interconnect, a bus, a network, or a combination thereof. For example, the components 110-115 may be communicatively coupled to each other via a controller area network (CAN) bus. The CAN bus is a vehicle bus standard designed to allow microcontrollers and devices to communicate with each other in applications without a host computer. It is a message-based protocol that was originally designed for multi-way electrical wiring in automobiles, but is also used in many other environments.

[0024] Reference now Figure 2In one embodiment, the sensor system 115 includes, but is not limited to, one or more cameras 211, a global positioning system (GPS) unit 212, an inertial measurement unit (IMU) 213, a radar unit 214, and a light detection and range (LIDAR) unit 215. The GPS system 212 may include a transceiver operable to provide information about the location of the autonomous vehicle. The IMU unit 213 may sense the position and orientation changes of the autonomous vehicle based on inertial acceleration. The radar unit 214 may represent a system that uses radio signals to sense objects within the local environment of the autonomous vehicle. In some embodiments, in addition to sensing objects, the radar unit 214 may additionally sense the speed and / or heading of the object. The LIDAR unit 215 may use lasers to sense objects in the environment in which the autonomous vehicle is located. The LIDAR unit 215 may include one or more laser sources, a laser scanner, and one or more detectors, among other system components. The camera 211 may include one or more devices to capture images of the environment surrounding the autonomous vehicle. The camera 211 may be a static camera and / or a video camera. The camera may be mechanically movable, for example by mounting the camera on a rotating and / or tilting platform.

[0025] The sensor system 115 may also include other sensors, such as sonar sensors, infrared sensors, steering sensors, throttle sensors, brake sensors, and audio sensors (e.g., microphones). The audio sensor may be configured to capture sounds from the environment surrounding the autonomous vehicle. The steering sensor may be configured to sense the steering angle of the steering wheel, the wheels of the vehicle, or a combination thereof. The throttle sensor and the brake sensor sense the throttle position and the brake position of the vehicle, respectively. In some cases, the throttle sensor and the brake sensor may be integrated into an integrated throttle / brake sensor.

[0026] In one embodiment, the vehicle control system 111 includes, but is not limited to, a steering unit 201, a throttle unit 202 (also referred to as an acceleration unit), and a brake unit 203. The steering unit 201 is used to adjust the direction or heading of the vehicle. The throttle unit 202 is used to control the speed of the motor or engine, which in turn controls the speed and acceleration of the vehicle. The brake unit 203 decelerates the vehicle by providing friction to slow down the wheels or tires of the vehicle. Note that Figure 2 The components shown may be implemented in hardware, software or a combination thereof.

[0027] Return to reference Figure 1, the wireless communication system 112 allows communication between the autonomous vehicle 101 and external systems, such as devices, sensors, other vehicles, etc. For example, the wireless communication system 112 can wirelessly communicate with one or more devices (such as servers 103-104 on the network 102) directly or via a communication network. The wireless communication system 112 can use any cellular communication network or wireless local area network (WLAN), such as using WiFi to communicate with another component or system. The wireless communication system 112 can communicate directly with a device (e.g., a passenger's mobile device, a display device within the vehicle 101, a speaker) using, for example, an infrared link, Bluetooth, etc. The user interface system 113 can be part of a peripheral device implemented within the vehicle 101, including, for example, a keyboard, a touch screen display device, a microphone, and a speaker, etc.

[0028] Some or all functions of autonomous vehicle 101 may be controlled or managed by perception and planning system 110, particularly when operating in an autonomous driving mode. Perception and planning system 110 includes necessary hardware (e.g., processor, memory, storage device) and software (e.g., operating system, planning and routing programs) to receive information from sensor system 115, control system 111, wireless communication system 112, and / or user interface system 113, process the received information, plan a route or path from a starting point to a destination point, and then drive vehicle 101 based on the planning and control information. Alternatively, perception and planning system 110 may be integrated with vehicle control system 111.

[0029] For example, a user as a passenger may specify a starting location and a destination of a trip, for example, via a user interface. The perception and planning system 110 obtains trip-related data. For example, the perception and planning system 110 may obtain location and route information from an MPOI server, which may be part of the servers 103-104. The location server provides location services, and the MPOI server provides map services and POIs for certain locations. Alternatively, such location and MPOI information may be cached locally in a permanent storage device of the perception and planning system 110.

[0030] As the autonomous vehicle 101 moves along the route, the perception and planning system 110 may also obtain real-time traffic information from a traffic information system or server (TIS). Note that the servers 103-104 may be operated by a third-party entity. Alternatively, the functionality of the servers 103-104 may be integrated with the perception and planning system 110. Based on the real-time traffic information, MPOI information, and location information, as well as real-time local environment data (e.g., obstacles, objects, nearby vehicles) detected or sensed by the sensor system 115, the perception and planning system 110 may plan an optimal route and, for example, drive the vehicle 101 according to the planned route via the control system 111 to reach a designated destination safely and efficiently.

[0031] The server 103 may be a data analysis system for performing data analysis services for various clients. In one embodiment, the data analysis system 103 includes a data collector 121 and a machine learning engine 122. The data collector 121 collects driving statistics 123 from various vehicles (autonomous vehicles or conventional vehicles driven by human drivers). The driving statistics 123 include information indicating driving commands issued (e.g., throttle, brake, steering commands) and responses of the vehicle (e.g., speed, acceleration, deceleration, direction) captured by the vehicle's sensors at different points in time. The driving statistics 123 may also include information describing the driving environment at different points in time, such as routes (including starting and destination locations), MPOIs, road conditions, weather conditions, etc.

[0032] Based on the driving statistics 123, the machine learning engine 122 generates or trains a set of rules, algorithms, and / or prediction models 124 for various purposes. For example, the algorithms 124 may include algorithms or functions to calculate a confidence score for the current state of an obstacle, a trajectory predicted for the obstacle, and a trajectory point of the trajectory, as will be further described below. The algorithms 124 may then be uploaded to the ADV for use in real time during autonomous driving.

[0033] Figure 3A and 3B is a block diagram illustrating an example of a perception and planning system for use with an autonomous vehicle according to one embodiment. System 300 may be implemented as Figure 1 Portions of autonomous vehicle 101 include, but are not limited to, perception and planning system 110, control system 111, and sensor system 115. Figure 3A-3B The perception and planning system 110 includes but is not limited to a positioning module 301 , a perception module 302 , a prediction module 303 , a decision module 304 , a planning module 305 , a control module 306 , and a routing module 307 .

[0034] Some or all of the modules 301-307 may be implemented in software, hardware, or a combination thereof. For example, these modules may be installed in permanent storage device 352, loaded into memory 351, and executed by one or more processors (not shown). Note that some or all of these modules may be communicatively coupled to Figure 2 Some or all modules of the vehicle control system 111 may be integrated with or integrated with the vehicle control system 111. Some of the modules 301-307 may be integrated together as an integrated module.

[0035] The positioning module 301 determines the current location of the autonomous vehicle 300 (e.g., using the GPS unit 212) and manages any data related to the user's trip or route. The positioning module 301 (also known as the map and route module) manages any data related to the user's trip or route. The user can log in and specify the starting location and destination of the trip, for example, via a user interface. The positioning module 301 communicates with other components of the autonomous vehicle 300, such as map and route information 311, to obtain data related to the trip. For example, the positioning module 301 can obtain location and route information from a location server and a map and POI (MPOI) server. The location server provides location services, and the MPOI server provides map services and POIs for certain locations, which can be cached as part of the map and route information 311. When the autonomous vehicle 300 moves along the route, the positioning module 301 can also obtain real-time traffic information from a traffic information system or server.

[0036] Based on the sensor data provided by the sensor system 115 and the positioning information obtained by the positioning module 301, the perception of the surrounding environment is determined by the perception module 302. The perception information can represent the situation around the vehicle that the driver is driving as perceived by an ordinary driver. The perception can include lane configuration, traffic light signals, relative positions of another vehicle, pedestrians, buildings, crosswalks or other traffic-related signs (e.g., stop signs, give way signs), etc., such as in the form of objects. The lane configuration includes information describing one or more lanes, such as, for example, the shape of the lane (e.g., straight or curved), the width of the lane, the number of lanes in the road, one-way or two-way lanes, merging or splitting lanes, exit lanes, etc.

[0037] The perception module 302 may include a computer vision system or functionality of a computer vision system to process and analyze images captured by one or more cameras in order to identify objects and / or features in the environment of the autonomous vehicle. Objects may include, among others, traffic signals, roadway boundaries, other vehicles, pedestrians, and / or other obstacles. The computer vision system may use object recognition algorithms, video tracking, and other computer vision techniques. In some embodiments, the computer vision system may map the environment, track objects, and estimate the speed of objects, among others. The perception module 302 may also detect objects based on other sensor data provided by other sensors such as radar and / or LIDAR.

[0038] For each object, the prediction module 303 predicts what the object will behave in this situation. Predictions are performed based on sensory data that senses the driving environment at a point in time, given a set of map / route information 311 and traffic rules 312. For example, if the object is a vehicle in the opposite direction and the current driving environment includes an intersection, the prediction module 303 will predict whether the vehicle will likely move straight ahead or turn. If the sensory data indicates that the intersection does not have a traffic light, the prediction module 303 can predict that the vehicle may have to come to a complete stop before entering the intersection. If the sensory data indicates that the vehicle is currently in a left-turn-only lane or a right-turn-only lane, the prediction module 303 can predict that the vehicle will be more likely to make a left turn or a right turn, respectively.

[0039] In some aspects, the prediction module generates a predicted trajectory of an obstacle, such as a vehicle other than the ADV, a pedestrian, or a cyclist, the predicted trajectory predicting the path of the moving obstacle in at least an area believed to be relevant to the current path of the ADV. The predicted trajectory can be generated based on the current state of the moving obstacle (e.g., the speed, position, heading, acceleration, or type of the moving obstacle), map data, and traffic regulations.

[0040] For example, the ADV may identify an obstacle as a vehicle sensed to be traveling in a driving lane on a map based on the obstacle's heading and position and reference to map data, including a lane position and orientation that confirms that the obstacle appears to be traveling in the driving lane based on the obstacle's heading and position. Assume that the map data indicates that this is a right-turn-only lane. Based on traffic rules (e.g., an obstacle of type "vehicle" in a right-turn-only lane must turn right), a trajectory including coordinates and / or a mathematical representation of a line predicting the movement of the obstacle may be generated.

[0041] In some embodiments, when predicting the movement trajectory of an obstacle, the prediction system or module divides the obstacle trajectory prediction into two parts: 1) longitudinal movement trajectory generation and 2) lateral movement trajectory generation. These parts can be combined to form the predicted trajectory of the obstacle.

[0042] In some embodiments, generating a lateral movement trajectory (also referred to as a lateral trajectory for short) includes optimizing the trajectory using a first polynomial function. Generating a longitudinal movement trajectory (also referred to as a longitudinal trajectory for short) includes optimizing the trajectory using a second polynomial function. Optimization is performed based on the current state of the obstacle as an initial state and the predicted end state of the obstacle as a set of constraints so that the trajectory is at least smoothly aligned with the current heading of the obstacle. The end state is determined in view of the shape of the lane to which the obstacle is predicted to move. Once the longitudinal movement trajectory and the lateral movement trajectory have been defined and generated, the final predicted trajectory of the obstacle can be determined by combining the longitudinal movement trajectory and the lateral movement trajectory. As a result, the predicted trajectory of the obstacle is more accurate based on the current state of the obstacle and the shape of the lane.

[0043] According to some aspects, an obstacle is predicted to move from a starting point to an end point based on perception data of a driving environment around an ADV driving within a lane. A longitudinal movement trajectory from the starting point to the end point is generated in view of the shape of the lane. A lateral movement trajectory from the starting point to the end point is generated, including optimizing the shape of the lateral movement trajectory using a first polynomial function. Then, the longitudinal movement trajectory and the lateral movement trajectory are combined to form a final predicted trajectory that predicts how the obstacle will move. A path is generated to control the ADV to move in view of the predicted trajectory of the obstacle, for example to avoid a collision with the obstacle.

[0044] For each of the objects, the decision module 304 makes a decision about how to handle the object. For example, for a particular object (e.g., another vehicle in an intersecting path) and metadata describing the object (e.g., speed, direction, steering angle), the decision module 304 determines how to encounter the object (e.g., overtake, yield, stop, pass). The decision module 304 can make these decisions based on a set of rules, such as traffic rules or driving rules 312, which can be stored in a persistent storage device 352.

[0045] The routing module 307 is configured to provide one or more routes or paths from the starting point to the destination point. For a given trip from the starting position to the destination position, for example, received from a user, the routing module 307 obtains the route and map information 311 and determines all possible routes or paths from the starting position to the destination position. The routing module 307 can generate a reference line in the form of a topographic map for each route determined by it from the starting position to the destination position. The reference line refers to an ideal route or path without any interference from others such as other vehicles, obstacles or traffic conditions. In other words, if there are no other vehicles, pedestrians or obstacles on the road, the ADV should follow the reference line accurately or closely. Then, the topographic map is provided to the decision module 304 and / or the planning module 305. The decision module 304 and / or the planning module 305 check all possible routes to select and modify one of the best routes in view of other data provided by other modules (such as traffic conditions from the positioning module 301, the driving environment perceived by the perception module 302, and the traffic conditions predicted by the prediction module 303). Depending on the particular driving circumstances at a point in time, the actual path or route used to control the ADV may be close to or different from the reference line provided by the routing module 307 .

[0046] Based on the decision for each perceived object, the planning module 305 uses the reference line provided by the routing module 307 as a basis to plan a path or route for the autonomous vehicle and driving parameters (e.g., distance, speed, and / or steering angle). That is, for a given object, the decision module 304 decides what to do with the object, and the planning module 305 determines how to do it. For example, for a given object, the decision module 304 may decide to pass through the object, and the planning module 305 may determine whether to pass on the left or right side of the object. Planning and control data are generated by the planning module 305 and include information describing how the vehicle 300 will move in the next movement cycle (e.g., the next route / path segment). For example, the planning and control data may indicate that the vehicle 300 moves at a speed of 30 miles per hour (mph) for 10 meters and then changes to the right lane at a speed of 25mph.

[0047] Based on the planning and control data, the control module 306 controls and drives the autonomous vehicle by sending appropriate commands or signals to the vehicle control system 111 according to the route or path defined by the planning and control data. The planning and control data includes sufficient information to drive the vehicle from a first point on the route or path to a second point using appropriate vehicle settings or driving parameters (e.g., throttle, brake, steering commands) at different points in time along the route or path.

[0048] In one embodiment, the planning phase is performed over multiple planning cycles (also referred to as driving cycles, such as within each time interval of 100 milliseconds (ms)). For each planning cycle or driving cycle, one or more control commands will be issued based on the planning and control data. That is, for every 100ms, the planning module 305 plans the next route segment or path segment, for example, including the target position and the time required for the ADV to reach the target position. Alternatively, the planning module 305 can also specify a specific speed, direction and / or steering angle, etc. In one embodiment, the planning module 305 plans the route segment or path segment for the next predetermined time period, such as 5 seconds. For each planning cycle, the planning module 305 plans the target position for the current cycle (e.g., the next 5 seconds) based on the target position planned in the previous cycle. The control module 306 then generates one or more control commands (e.g., throttle, brake, steering control commands) based on the planning and control data of the current cycle.

[0049] Note that decision module 304 and planning module 305 can be integrated into an integrated module. Decision module 304 / planning module 305 may include a navigation system or the functionality of a navigation system to determine a driving path for an autonomous vehicle. For example, the navigation system may determine a series of speeds and direction headings to affect the movement of the autonomous vehicle along a path that substantially avoids perceived obstacles while generally advancing the autonomous vehicle along a roadway-based path to a final destination. The destination may be set based on user input via user interface system 113. The navigation system may dynamically update the driving path while the autonomous vehicle is operating. The navigation system may incorporate data from a GPS system and one or more maps to determine a driving path for the autonomous vehicle.

[0050] Polynomial optimization or polynomial fitting refers to optimizing the shape of a curve (e.g., the trajectory of an obstacle) represented by a polynomial function (e.g., a quintic or quartic polynomial function) so that the curve is continuous along the curve (e.g., the derivative at the joint of two adjacent segments is available). In the field of autonomous driving, the polynomial curve from the starting point to the end point is divided into multiple segments (or blocks), each segment corresponding to a control point (or reference point). This segmented polynomial curve is called a piecewise polynomial. When optimizing a piecewise polynomial, in addition to the set of initial state constraints and end state constraints, it is also necessary to satisfy the set of joint constraints and the set of boundary constraints between two adjacent segments.

[0051] The set of joint constraints includes that the position (x, y), velocity, heading, and acceleration of adjacent segments need to be the same. For example, the end position of the first segment (e.g., the leading segment) and the start position of the second segment (e.g., the following segment) need to be the same or within a predetermined proximity. The velocity, heading, and acceleration of the end position of the first segment and the corresponding velocity, heading, and acceleration of the start position of the second segment need to be the same or within a predetermined range. In addition, each control point is associated with a predetermined boundary (e.g., 0.2 meters to the left and right around the control point). The polynomial curve needs to pass through each control point within its corresponding boundary. When these two sets of constraints are met during optimization, the polynomial curve representing the trajectory should be smooth and continuous.

[0052] refer to Figure 4 , shows a process 400 that can be performed for an autonomous vehicle (ADV) during driving. As a result of the process, a trajectory of a moving obstacle is generated, with the trajectory having multiple confidence levels at different points along the trajectory.

[0053] At block 401, the process includes determining a current state confidence score (also referred to as a confidence level) associated with a current state of a moving obstacle, the current state of the moving obstacle including one or more of the speed, position, heading, acceleration, or type of the moving obstacle. The current state confidence score may be a value representing the degree of confidence that the ADV has in the current state of the vehicle. The confidence score may also be interpreted as representing the probability that the moving obstacle will take a particular path or move to a particular location.

[0054] In some embodiments, the current state confidence score can be a percentage. If the confidence in the current state is low due to the factors discussed in other sections, the level can be set to 20%. If the confidence is high due to these factors, the level can be set to 99%. The current state and the confidence associated with the current state can change from one frame to the next as different factors change (e.g., sensing of an obstacle becomes less obstructed). In some embodiments, the current state includes all of the speed, position, heading, acceleration, and type of moving obstacles.

[0055] At box 402, the process includes generating a trajectory of the path of the predicted mobile obstacle, and the generation of the trajectory is based on the current state of the mobile obstacle, map data and traffic rules. As described in other parts, the trajectory of the obstacle can be formed by the coordinates and / or mathematical representation of the line. This can involve combining the lateral movement trajectory of the obstacle with the longitudinal movement trajectory. Before the combination, the lateral and longitudinal movement trajectories can be optimized using polynomial optimization. The current state of the mobile obstacle (e.g., heading, speed, position, acceleration) can be used as a constraint to optimize the corresponding polynomial function. The polynomial function can represent the predicted trajectory formed based on map data and traffic rules. For example, if the current state limits the obstacle to the type of vehicle that sails down the driving lane, and the driving lane is straight without any turns (based on map data), the trajectory of the vehicle can be directed to the driving lane with reference to the traffic rules.

[0056] At block 403, the process includes determining a trajectory confidence score associated with the trajectory of the mobile obstacle, the trajectory confidence score being determined based on the current state confidence score and the current state of the mobile obstacle. Based on certain factors, a single trajectory confidence score may be determined for the entire trajectory. This may be described as a global trajectory confidence score and may be used in the next block to determine a point confidence score for the trajectory. Factors that affect the global confidence score of a trajectory are further discussed in other sections.

[0057] At box 404, the process includes determining a plurality of point confidence scores, each point confidence score defining the reliability of a corresponding point on the trajectory, at least one of the point confidence scores being determined based on a) the trajectory confidence score and b) at least one environmental factor of the obstacle. The ADV is controlled based on the trajectory of the moving obstacle and at least one of the plurality of point confidence scores. For example, based on the trajectory of the obstacle, the planning may adjust the path of the ADV to avoid the predicted path of the obstacle. In some cases, the planning of the ADV may determine whether to yield, stop, or overtake the obstacle. In some cases, the ADV may determine whether to adjust the path of the ADV based on whether the confidence score of the obstacle meets some threshold criteria. For example, if the confidence score is less than 5%, the ADV may maintain the current path. The control module of the ADV may issue steering, acceleration, and braking commands to the control system of the ADV to move the ADV accordingly. Environmental factors that affect each point confidence score may include any combination of a) the distance between the corresponding point and the starting position of the moving obstacle, b) traffic regulations, c) map data, or d) the type of the moving obstacle (e.g., a vehicle, pedestrian, or cyclist).

[0058] Due to the confidence scores of the predicted trajectory and objects, such a process can help the ADV adjust the route and therefore adjust other vehicle controls such as steering, braking and / or acceleration. The confidence scores along different points can vary so that the ADV can plan the route and control based on the confidence level of the ADV in the predicted trajectory at a specific point. As opposed to making driving decisions based solely on the overall confidence score of the trajectory, this provides more refined control and decision making. In some embodiments, at least one of the point confidence scores is determined based on a) the predicted interaction between objects, traffic regulations, or map data in the environment of the ADV and b) the moving obstacles at corresponding points on the trajectory. This is further described in other sections.

[0059] refer to Figure 5 , showing further detailed description Figure 4 Flowchart of the features of the process 400 shown, in the perception module 302, the sensor system 115 senses an obstacle, such as a vehicle 114. A current state is generated based on the sensed data, including the speed, position, heading, acceleration and / or type of the mobile obstacle (in this case, the vehicle 114). A confidence score associated with the current state can be determined based on many factors. In some embodiments, the confidence score of the current state is determined based on the distance between the ADV and the mobile obstacle. The farther the mobile obstacle is, the lower the confidence given to the sensing of the mobile obstacle. Therefore, although the sensor of the sensor system 115 may appear to sense an obstacle, when the obstacle is relatively far away (e.g., 30-50 meters), the distance factor can reduce the confidence score of the current state. As the distance between the obstacle and the ADV decreases, the confidence score can increase because the sensor is closer to the obstacle, and therefore, the sensed data is likely to be more reliable.

[0060] In some embodiments, the current state confidence score is further determined based on the object recognition accuracy. For example, the object recognition block 702 can use computer vision and machine learning algorithms, such as trained neural networks that have been trained using data sets, to identify sensed obstacles. For example, images sensed by the sensor system 115 can be processed at the object recognition block 702 to identify moving obstacles in the image, such as vehicles (cars, motorcycles, trucks, etc.), pedestrians or cyclists. However, object recognition is not perfect. Depending on the clarity of the image, or the degree of recognizability of the shape of the obstacle based on past training data sets, the object recognition block 702 can determine the confidence or reliability of the recognition of the object in those images. The score can help determine whether the recognition of the object is true or false positive. A neural network API (e.g., Tensorflow object detection API) can be used to provide various reliability metrics indicating the confidence of object recognition (e.g., PASCAL VOC 2010 detection metrics, weighted PASCAL VOC detection metrics, PASCAL VOC 2010 instance segmentation metrics, COCO detection metrics, COCO mask metrics, open image V2 detection metrics, etc.).

[0061] In some embodiments, the current state confidence score is determined based on whether the line of sight between one or more sensors of the ADV and the mobile obstacle is at least partially blocked. The blocking of the mobile obstacle can affect the object recognition metric. As a supplement or alternative to the metric, if the mobile obstacle is at least partially blocked, the object recognition block 702 can add a penalty to the current state confidence score. The penalty can be increased or decreased according to the amount of blocking (for example, the greater the blocking, the higher the penalty). Therefore, as the blocking increases, the confidence score of the current state decreases, and conversely, as the blocking decreases, the confidence score of the current state increases.

[0062] It should be understood that in the present disclosure, factors may increase or decrease the confidence score of the current state, obstacle track, or point along the obstacle track. These factors may be applied by arithmetic, such as addition, subtraction, multiplication, and / or division.

[0063] In some aspects, the current state confidence score is further determined based on the type of mobile obstacle, such as a vehicle, pedestrian, or cyclist. Based on different types of obstacles, the sensing of the current state information may be considered more reliable. For example, due to the size, detectable surface, and speed differences of vehicles, pedestrians, and cyclists, sensor information that senses the heading, speed, and position of a vehicle may be considered more accurate than sensor information that senses the heading, speed, and position of a pedestrian or cyclist. The ADV may require multiple frames to determine the heading, speed, and / or acceleration. The vehicle may tend to have a more constant heading, speed, and acceleration from one frame to the next, making the heading information more reliable than the heading information of a pedestrian.

[0064] At block 704, a single current state confidence score may be determined based on one or more of the factors discussed (e.g., obstacle type, obstacle, and / or machine learning object recognition accuracy), and the single current state confidence score is associated with the current state of the obstacle. As described above, the current state and associated confidence score may change frame by frame, for example, as the obstacle becomes closer or farther away from the ADV, or becomes more or less obstructed. The current state 706 and the associated current state confidence score 708 may be provided to the prediction module 303.

[0065] The trajectory generation block 714 generates trajectories based on the map data from the map and route information 311. Traffic rules 312 may be referenced to predict how the obstacle will move relative to the map data. For example, if the obstacle is a vehicle, one or more trajectories may be determined for the vehicle. Figure 5 As shown in the example in , the map data may indicate an intersection detected in front of the vehicle. Traffic rules may indicate that in the detected lane of the vehicle, the vehicle is allowed to go straight or make a right turn. Therefore, trajectories 710 can be generated to predict the movement of the vehicle along those trajectories. A confidence score 712 can be generated and associated with each trajectory of the vehicle. The confidence score can describe the confidence that the ADV system has when adopting the corresponding trajectory. In another example, if the obstacle is a pedestrian detected as walking along a sidewalk, the map data specifies where the sidewalk will end. Based on the reference traffic rules, multiple trajectories of the pedestrian can be determined, for example, a first trajectory can be generated so that the pedestrian is predicted to move across the sidewalk in a first direction, a second trajectory can be determined in which the pedestrian moves across the sidewalk in a second direction, and a third trajectory can be determined in which the pedestrian goes around the corner of the sidewalk and continues to walk on the sidewalk.

[0066] In some embodiments, a single track confidence score is associated with each track. The track confidence score can be determined based on the current state confidence score of the obstacle and the current state. For example, if the confidence score for the current state is 80%, this can be carried forward as the baseline track confidence score. Confidence based on data in the current state (e.g., speed, heading, acceleration, and type) can be added to this baseline, but this confidence is determined relative to the track. For example, reference Figure 5 , if the vehicle's speed is high, and / or no deceleration is detected, this may indicate that the vehicle intends to move straight rather than turn right. In this case, the confidence score associated with the straight trajectory may be higher than the confidence score associated with the right turn trajectory. As indicated, in some embodiments, the trajectory confidence score is determined based on map data (e.g., knowing where sidewalks, intersections, roads, and / or lanes are) and traffic regulations (knowing what is allowed for each obstacle type on a given road or intersection, which may indicate how the obstacle intends to move).

[0067] In some embodiments, the trajectory confidence score is further determined based on machine learning. For example, a neural network can be trained based on multiple training data indicating different situations (such as vehicles, pedestrians, and / or cyclists approaching an intersection at a given heading, speed, and acceleration). Other training data may indicate when an obstacle may change lanes or change direction based on changes in heading, speed, and / or acceleration. Based on these trained neural networks, at box 714, the heading information of the obstacle may be processed to determine a confidence score 712 for the obstacle trajectory. It should be understood that multiple obstacles can be processed simultaneously. One or more trajectories are generated for each obstacle, each trajectory representing a predicted path that the obstacle may take.

[0068] At block 716, point confidence scores are determined. Point confidence scores 715 are confidence scores that vary at different corresponding points of the obstacle trajectory 712. These points may be spaced at different intervals (e.g., every 1, 5, 10, 20 meters), depending on the application, obstacle type, obstacle speed, trajectory length, and can be determined through routine testing and experimentation. Each point confidence score for the trajectory may be determined based on a) using the trajectory confidence score for the trajectory as a baseline, and b) one or more environmental factors.

[0069] In some embodiments, the at least one environmental factor includes one or more of a) the distance between the corresponding point and the starting position of the moving obstacle, b) traffic rules, or c) map data. Figure 6, a trajectory S is generated for an obstacle 802 (e.g., a vehicle, a pedestrian, or a cyclist). The point confidence scores at corresponding points S1-S8 can be determined based on the distance from the current position of the obstacle (which can be specified in the current state of the obstacle) to a farther distance along the trajectory. As the distance from the current position along the trajectory increases, the confidence C at each point S1-S8 decreases. Although the slope is shown as straight, it should be understood that the slope can be straight or curved and can change from one obstacle type to the next obstacle type. For example, compared with a vehicle, the confidence of a pedestrian can vary more (e.g., decrease faster) relative to the distance, considering that a pedestrian may be more likely to get lost from the sidewalk or enter a store, contrary to a vehicle that tends to be constrained by the road. Appropriate slope values ​​can be determined based on conventional testing and repetition. In some cases, the relationship between confidence and distance can be nonlinear.

[0070] In some embodiments, at least one of the point confidence scores is determined based on a predicted interaction between a) an object, traffic regulation, or map data in the environment of the ADV and b) a moving obstacle at a corresponding point on the trajectory. For example, if an object such as a trash can is sensed in the middle of a road, the confidence score at a point of the trajectory of the vehicle traveling along the road may be reduced near the trash can because it is uncertain how the vehicle may move to avoid the trash can.

[0071] In another example, if Figure 7 As shown, confidence scores at different points of a trajectory can be determined based on map data and traffic rules. The map data may indicate that an obstacle 902, such as a vehicle, is located at a driving lane 904 and is traveling up the driving lane. The map data may indicate that an intersection 906 is located in front of the obstacle. Traffic rules may indicate that a vehicle driving in the driving lane is allowed to continue straight or turn right at the intersection. A first trajectory S may be generated to represent a straight trajectory, while a second trajectory S' may be generated to represent a right turn trajectory. Point confidence scores at points S1-S4 may be determined for trajectory S. Similarly, point confidence scores at points S'1-S'4 may be determined for trajectory S'. In this example, the confidence scores at S1 and S'1 and S2 and S'2 match because S and S' share the same trajectory at those points. The point confidence scores at points S3 and S4 drop after the intersection (compared to the confidence at points S1 and S2) because there is uncertainty about whether the vehicle will turn or go straight. Similarly, due to the same uncertainty, the confidence scores at points S'3 and S'4 drop after the intersection (compared to the confidence at points S'1 and S'2).

[0072] In one embodiment, based on the current state of the obstacle, points S'3 and S'4 have a lower point confidence score than S3 and S4. For example, the speed, heading and / or acceleration of vehicle 902 may indicate that the vehicle intends to continue to go straight. If the vehicle speed remains above a threshold, maintains heading and / or does not decelerate, this may indicate that the vehicle does not intend to decelerate to turn. In this case, S3 and S4 may have a higher confidence than S'3 and S'4. On the other hand, if the vehicle decelerates, has a speed below a turning threshold, or changes a heading change, this may indicate that the vehicle intends to turn right. In this case, S'3 and S'4 may have a higher point confidence score than S3 and S4. However, in both cases, the point after the intersection has a lower confidence score than the point before the intersection. This is because the intersection indicates uncertainty in the trajectory of the obstacle.

[0073] In some aspects, environmental factors include the type of mobile obstacle (e.g., whether the obstacle is a vehicle, a pedestrian, or a cyclist). The type of mobile obstacle can be referenced against map data and traffic rules to determine confidence scores at different points of the trajectory. For example, as discussed, the point confidence score based on the distance of a pedestrian trajectory can be significantly lower than the point confidence score of a vehicle trajectory because pedestrians can walk into stores, homes, throw away trash at trash cans, chase friends, etc. In contrast, vehicles may be more restricted by road rules (e.g., only right turns, only straight, etc.).

[0074] In some embodiments, the object that potentially interferes with the obstacle may be another obstacle, such as a pedestrian, a vehicle, or a cyclist. As the predicted trajectories of two obstacles get closer to each other, the confidence score at points along the trajectories where the trajectories are close or potentially intersecting will decrease due to uncertainty about how the obstacles will interfere with each other.

[0075] In another example, Figure 8 As shown, trajectories A and A' may be generated for vehicle 1002, and trajectory B may be determined for vehicle 1004. Based on the map data and traffic rules, vehicle 1002 may remain straight or turn right. Trajectories A and A' may have points A1 and A2 that share a first confidence score. After the intersection, points A3, A'3, A4, and A'4 have reduced confidence scores due to uncertainty as to whether vehicle 1002 will turn right or remain straight. In contrast, trajectory B remains constant because vehicle 1002 navigates along a road that only turns right. Therefore, points B1-B4 may have the same confidence score because the traffic rules associated with trajectory B do not create ambiguity at the intersection.

[0076] In addition, Figure 8In the illustrated embodiment, the distance from the starting point is not considered, as shown by the flat slope of the point confidence scores for track B. Instead, the distance from the starting point is factored into the Figure 7 In the point confidence scores shown, as shown by the slightly downward slope of the point confidence scores for trajectories S and S'.

[0077] It should be understood that moving obstacles may include other moving objects besides vehicles, pedestrians or cyclists. Figure 8 The examples discussed in are intended to be illustrative and not limiting. There are many relationships between map data, traffic rules, and obstacle types that can affect the point confidence score of the predicted trajectory of an obstacle. However, based on the features discussed in the present disclosure, the point confidence score along the predicted trajectory can be increased or decreased based on specific map details, specific traffic rules, specific obstacle types, and other detected objects near the obstacle (e.g., within 0-30 meters) that may affect the trajectory of the obstacle. In addition, Figure 8 The graphs shown in are shown to illustrate the concepts described herein, rather than to show exact values ​​of the confidence scores.

[0078] refer to Figure 5 , the decision module 304 and the planning module 305 can adjust the route of the ADV based on the trajectory of the obstacle and one or more point confidence scores of the trajectory, thereby generating planning and control data. In some embodiments, the route is adjusted only when the point confidence score of the trajectory meets a threshold (e.g., 5%, 10%, 25%, etc.). The control module 306 uses the planning and control data to generate control commands, such as steering and acceleration. These control commands are used by the control system 111 to move the ADV along the adjusted route.

[0079] Note that some or all of the components shown and described above may be implemented in software, hardware, or a combination thereof. For example, these components may be implemented as software installed and stored in a permanent storage device, which may be loaded and executed in a memory by a processor (not shown) to perform the processes or operations described throughout the present application. Alternatively, these components may be implemented as executable code programmed or embedded into dedicated hardware, such as an integrated circuit (e.g., a dedicated IC or ASIC), a digital signal processor (DSP), or a field programmable gate array (FPGA), which may be accessed via a corresponding driver and / or operating system from an application. In addition, these components may be implemented as specific hardware logic in a processor or processor core as part of an instruction set accessible via one or more specific instruction software components.

[0080] Some portions of the foregoing detailed description have been presented in terms of algorithms and symbolic representations of operations on data bits within a computer memory. These algorithmic descriptions and representations are the means used by those skilled in the data processing arts to most effectively convey the substance of their work to others skilled in the art. An algorithm is here, and generally, conceived to be a self-consistent sequence of operations leading to a desired result. The operations are those requiring physical manipulations of physical quantities.

[0081] It should be remembered, however, that all of these and similar terms are associated with the appropriate physical quantities and are merely convenient labels applied to these quantities. Unless otherwise stated, as will be apparent from the above discussion, it should be understood that throughout the specification, discussions using terms such as those set forth in the appended claims refer to the actions and processes of a computer system or similar electronic computing device that manipulates and transforms data represented as physical (electronic) quantities within the computer system's registers and memories into other data similarly represented as physical quantities within the computer system's memories or registers or other such information storage, transmission or display devices.

[0082] Embodiments of the present disclosure also relate to devices for performing the operations herein. Such a computer program is stored in a non-transitory computer-readable medium. A machine-readable medium includes any mechanism for storing information in a form readable by a machine (e.g., a computer). For example, a machine-readable (e.g., computer-readable) medium includes a machine (e.g., computer) readable storage medium (e.g., a read-only memory ("ROM"), a random access memory ("RAM"), a disk storage medium, an optical storage medium, a flash memory device).

[0083] The processes or methods described in the foregoing figures may be performed by processing logic including hardware (e.g., circuits, dedicated logic, etc.), software (e.g., embodied on a non-transitory computer-readable medium), or a combination of both. Although the processes or methods are described above in terms of some sequential operations, it should be understood that some of the operations described may be performed in a different order. In addition, some operations may be performed in parallel rather than sequentially.

[0084] The embodiments of the present disclosure are not described with reference to any particular programming language. It will be appreciated that a variety of programming languages ​​may be used to implement the teachings of the embodiments of the present disclosure as described herein.

[0085] In the foregoing description, embodiments of the present disclosure have been described with reference to specific exemplary embodiments of the present disclosure. Obviously, various modifications may be made thereto without departing from the broader spirit and scope of the present disclosure as set forth in the appended claims. Therefore, the description and the accompanying drawings should be considered illustrative rather than restrictive.

Claims

1. A method for an autonomous driving vehicle (ADV), the method comprising: determining a current state confidence score, the current state confidence score representing a confidence level related to the accuracy of a current state of a moving obstacle, the current state of the moving obstacle comprising one or more of a speed, a position, a heading, an acceleration, or a type of the moving obstacle; generating a trajectory predicting a path of the moving obstacle based on the current state of the moving obstacle, map data, and traffic rules, wherein the trajectory of the path of the moving obstacle is generated by generating a longitudinal moving trajectory and a lateral moving trajectory and combining the longitudinal moving trajectory and the lateral moving trajectory, wherein the lateral moving trajectory is optimized using a first polynomial function and the longitudinal moving trajectory is optimized using a second polynomial function, and the optimization is performed based on the current state of the moving obstacle as an initial state and the predicted end state of the moving obstacle as a set of constraints; determining a trajectory confidence score representing a confidence level of a trajectory on which the mobile obstacle has a likelihood of moving based on the current state confidence score and the current state of the mobile obstacle; and planning a path to control and drive the ADV given the trajectory and the trajectory confidence score; The method further comprises: determining a plurality of point confidence scores based on the trajectory confidence score and at least one environmental factor of the moving obstacle, each point confidence score defining the reliability of a corresponding point on the trajectory, wherein the path is also planned based on the point confidence scores of the trajectory, wherein the at least one environmental factor comprises a) a distance between a corresponding point and a starting position of the moving obstacle, or, b) one or more of the traffic rules, and wherein at least one of the point confidence scores is determined based on a predicted interaction between the traffic rule and the moving obstacle at the corresponding point on the trajectory; where as the trajectories of two obstacles approach, the confidence scores at points along the trajectories where they potentially intersect decrease; wherein the current state confidence score is determined based on the distance between the ADV and the moving obstacle; The trajectory confidence score is further determined based on the map data and the traffic rules. The method of claim 1 , wherein the at least one environmental factor further comprises the map data.

3. The method of claim 1, wherein the at least one environmental factor includes the type of the moving obstacle.

4. The method of claim 1, wherein at least one of the point confidence scores is determined based on a predicted interaction between an object in the environment of the ADV or the map data and the moving obstacle at a corresponding point on the trajectory. The method of claim 1 , wherein the current state confidence score is determined based on a distance between the ADV and the moving obstacle.

6. An apparatus for autonomous driving vehicle (ADV), comprising: a first determination module for determining a current state confidence score, the current state confidence score representing a confidence level related to the accuracy of a current state of a moving obstacle, the current state of the moving obstacle comprising one or more of a speed, a position, a heading, an acceleration, or a type of the moving obstacle; a generating module for generating a trajectory predicting a path of the moving obstacle based on the current state of the moving obstacle, map data, and traffic rules, wherein the trajectory of the path of the moving obstacle is generated by generating a longitudinal moving trajectory and a lateral moving trajectory and combining the longitudinal moving trajectory and the lateral moving trajectory, wherein the lateral moving trajectory is optimized using a first polynomial function and the longitudinal moving trajectory is optimized using a second polynomial function, and the optimization is performed based on the current state of the moving obstacle as an initial state and the predicted end state of the moving obstacle as a set of constraints; a second determination module for determining a trajectory confidence score representing a confidence level of a trajectory on which the mobile obstacle has a possibility of moving based on the current state confidence score and the current state of the mobile obstacle; and a planning module for planning a path to control and drive the ADV in view of the trajectory and the trajectory confidence score; The first determination module is further configured to determine a plurality of point confidence scores based on the trajectory confidence score and at least one environmental factor of the moving obstacle, each point confidence score defining the reliability of a corresponding point on the trajectory, wherein the path is also planned based on the point confidence scores of the trajectory, wherein the at least one environmental factor comprises a) a distance between a corresponding point and a starting position of the moving obstacle, or, b) one or more of the traffic rules, and wherein at least one of the point confidence scores is determined based on a predicted interaction between the traffic rule and the moving obstacle at a corresponding point on the trajectory; where as the trajectories of two obstacles approach, the confidence scores at points along the trajectories where they potentially intersect decrease; wherein the current state confidence score is determined based on the distance between the ADV and the moving obstacle; The trajectory confidence score is further determined based on the map data and the traffic rules.

7. A non-transitory machine-readable medium having instructions stored therein, which when executed by a processor cause the processor to perform the method according to any one of claims 1-5.

8. A data processing system comprising: processor; as well as A memory coupled to the processor to store instructions which, when executed by the processor, cause the processor to perform a method according to any one of claims 1 to 5.

9. A computer program product, comprising a computer program, which, when executed by a processor, implements the method according to any one of claims 1 to 5.

Images