PLATFORM FOR DEVELOPING A ROUTE PLANNING SYSTEM FOR AN AUTOMATED DRIVING SYSTEM

Description

TECHNICAL FIELD

[0001] The present invention relates to a method for performance evaluation and development of a path planning module of a vehicle equipped with an Automated Driving System (ADS). In particular, the present invention relates to open-loop evaluation of a path planning module of a vehicle, and subsequent updating thereof.BACKGROUND

[0002] During the last few years, the research and development activities related to autonomous vehicles has exploded in number and many different approaches are being explored. An increasing portion of modern vehicles have advanced driver-assistance systems (ADAS) to increase vehicle safety and more generally road safety. ADAS - which for instance may be represented by adaptive cruise control, ACC, collision avoidance system, forward collision warning, etc. - are electronic systems that may aid a vehicle driver while driving. Today, there is ongoing research and development within a number of technical areas associated to both the ADAS and Autonomous Driving (AD) field. ADAS and AD will herein be referred to under the common term Automated Driving System (ADS) corresponding to all of the different levels of automation as for example defined by the SAE J3016 levels (0 - 5) of driving automation, and in particular for level 4 and 5.

[0003] In a not too distant future, ADS solutions are expected to have found their way into a majority of the new cars being put on the market. An ADS may be construed as a complex combination of various components that can be defined as systems where perception, decision making, and operation of the vehicle are performed by electronics and machinery instead of a human driver, and as introduction of automation into road traffic. This includes handling of the vehicle, destination, as well as awareness of surroundings. While the automated system has control over the vehicle, it allows the human operator to leave all or at least some responsibilities to the system. An ADS commonly combines a variety of sensors to perceive the vehicle's surroundings, such as e.g. radar, LIDAR, sonar, camera, navigation system e.g. GPS, odometer and / or inertial measurement units (IMUs), upon which advanced control systems may interpret sensory information to identify appropriate navigation paths, as well as obstacles, free-space areas, and / or relevant signage.

[0004] Much of the current efforts for development of ADSs revolves around safely launching a first system to the market. However, once that is achieved it will be paramount to improve the system in a safe and efficient manner, both to achieve cost reductions as well as performance improvements. Generally, there are significant costs associated with the development, testing, and validation of safety of the ADS (or of "ADS features"), especially related to field tests and the understanding of how the system behaves in traffic. Moreover, there are additional challenges in terms of managing the immense amounts of data generated by ADS equipped vehicles in order to develop, test and verify various ADS features, not only from a data storage, processing and bandwidth perspective, but also from a data security / privacy perspective.

[0005] There is accordingly a need in the art for new solutions for facilitating development, testing, and / or validation of ADSs in order to continuously be able to provide safer and more performant systems. As always, the improvements shall preferably be made without significant impact on the size, power consumption and cost of the on-board system or platform.

[0006] US 2020 / 0307561 A1 discloses systems and methods for pre-decision manoeuvre risk planning, and in particular for generating fuller representations of uncertainty indicators for use in pre-decision manoeuvre risk planning under conditions of reduced sensed information. In more detail, the system of US 2020 / 0307561 A1 includes one or more modules configured to apply functions to sensed data of the environment for configuring a machine learning (ML) model of decision making behaviour of the vehicle by an action risk assessment model trained use semi-supervised ML techniques by online and offline training for mapping function to candidate actions to determine with risk factors a learned drivable path.

[0007] US 2019 / 0302767 A1 discloses a temporal prediction model for semantic intent understanding. In more detail, an agent (e.g., a moving object) in an environment can be detected in sensor data collected from sensors on a vehicle. Computing device(s) associated with the vehicle can determine, based partly on the sensor data, attribute(s) of the agent (e.g. classification, position, velocity, etc.), and can generate, based partly on the attribute(s) a temporal prediction model, semantic intent(s) of the agent (e.g., crossing a road, staying straight, etc.), which can correspond to candidate trajectories of the agent. These candidate trajectories can be associated with weight(s) representing likelihood(s) that the agent will perform respective intent(s). The computing device(s) of the vehicle can use one or more of the candidate trajectories to determine a vehicle trajectory along which a vehicle is to drive. WO 2020 / 056331 A1 discloses a system and method for obtaining training data. In more detail, the method includes receiving sensor data and applying a neural network to the sensor data. A trigger classifier is applied to an intermediate result of the neural network to determine a classifier score for the sensor data. Based at least in part of the classifier score, a determination is made whether to transmit via a computer network at least a portion of the sensor data. Upon a positive determination, the sensor data is transmitted and used to generate training data.SUMMARY

[0008] It is therefore an object of the present invention to provide a method performed by an in-vehicle computing system for automated development of a path planning module of a vehicle, a computer-readable storage medium, an apparatus for automated development of a path planning module of a vehicle, and a vehicle comprising such an apparatus, which alleviate, mitigate or completely eliminate all or at least some of the drawbacks of presently known solutions.

[0009] It is also an object of the present invention to provide solutions for facilitating development, testing, and / or validation of ADSs in order to continuously be able to provide safer and more performant systems.

[0010] These objects are achieved by means of a method performed by an in-vehicle computing system for automated development of a path planning module of a vehicle, a computer-readable storage medium and an apparatus for automated development of a path planning module of a vehicle as defined in the appended independent claims. The term exemplary is in the present context to be understood as serving as an instance, example or illustration.

[0011] The invention is set out in the appended independent claims.

[0012] Further embodiments of the invention are defined in the dependent claims.

[0013] It should be emphasized that the term "comprises / comprising" when used in this specification is taken to specify the presence of stated features, integers, steps, or components. It does not preclude the presence or addition of one or more other features, integers, steps, components, or groups thereof.

[0014] The herein presented method provides for a learning platform for autonomous vehicles where the production system and sensors of the production ADS are utilized to carry out federated learning of next versions of path planning features for ADSs. Thereby readily providing advantages in terms of cost and time for development, testing, and validation of path planning features for autonomous vehicles.

[0015] The term "non-transitory," as used herein, is intended to describe a computer-readable storage medium (or "memory") excluding propagating electromagnetic signals, but are not intended to otherwise limit the type of physical computer-readable storage device that is encompassed by the phrase computer-readable medium or memory. For instance, the terms "non-transitory computer readable medium" or "tangible memory" are intended to encompass types of storage devices that do not necessarily store information permanently, including for example, random access memory (RAM). Program instructions and data stored on a tangible computer-accessible storage medium in non-transitory form may further be transmitted by transmission media or signals such as electrical, electromagnetic, or digital signals, which may be conveyed via a communication medium such as a network and / or a wireless link. Thus, the term "non-transitory", as used herein, is a limitation of the medium itself (i.e., tangible, not a signal) as opposed to a limitation on data storage persistency (e.g., RAM vs. ROM).

[0016] Features and advantages of the present invention will in the following be further clarified with reference to the embodiments described hereinafter.BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Further objects, features and advantages of embodiments of the invention will appear from the following detailed description, reference being made to the accompanying drawings, in which: Fig. 1 is a schematic flow chart representation of a method performed by an in-vehicle computing system for automated development of a path planning module of a vehicle in accordance with an embodiment of the present invention. Fig. 2 is a schematic top-view illustration of a post-processing method in accordance with an embodiment of the invention in the form of a series of scenes depicting a temporal development of a vehicle approaching an object. Fig. 3a is a schematic top view of a path evaluation process in view of a baseline worldview in accordance with an embodiment of the present invention. Fig. 3b is a schematic top view of a path evaluation process in view of a baseline worldview in accordance with an embodiment of the present invention. Fig. 4 is a schematic block diagram representation of a system for automated development a path planning module of a vehicle in accordance with an embodiment of the present invention. Fig. 5 is a schematic side-view of a vehicle comprising an apparatus for automated development a path planning module of a vehicle in accordance with an embodiment of the present invention. DETAILED DESCRIPTION

[0018] Those skilled in the art will appreciate that the steps, services and functions explained herein may be implemented using individual hardware circuitry, using software functioning in conjunction with a programmed microprocessor or general purpose computer, using one or more Application Specific Integrated Circuits (ASICs) and / or using one or more Digital Signal Processors (DSPs). It will also be appreciated that when the present invention is described in terms of a method, it may also be embodied in one or more processors and one or more memories coupled to the one or more processors, wherein the one or more memories store one or more programs that perform the steps, services and functions disclosed herein when executed by the one or more processors.

[0019] Fig. 1 is a schematic flow chart representation of a method 100 performed by an in-vehicle computing system for automated development of a path planning module of a vehicle. The vehicle is equipped with an Automated Driving System (ADS), which in the present context comprises all of the different levels of automation as for example defined by the SAE J3016 levels (0 - 5) of driving automation, and in particular level 4 and 5. In other words, the term ADS encompasses both Advanced Driver-Assistance System (ADAS) and Autonomous Driving (AD).

[0020] More specifically, the method 100 relates to open-loop evaluation of a path planning module of a vehicle, and subsequent updating thereof. The term "open loop" evaluation may also be referred to as "shadow mode" or "sandbox mode", and may accordingly be understood as evaluation of the output of the path planning module without actuation of the generated candidate paths.

[0021] The method 100 comprises obtaining 101 a candidate path from the path planning module. The term "path planning module" may also be referred to as a "Module Under Test" (MUT), or path planning development-module, meaning that is a "new" (under development) software and / or hardware of a path planning feature for automotive applications. In other words, the path planning module may in the present context be understood as software and / or hardware configured to generate a candidate path based on perception data (e.g. raw sensor data or processed sensor data), where the path planning module is currently "under development", and not yet "in production" (e.g. not verified / validated). The vehicle or more precisely, the ADS of the vehicle, may naturally be equipped with a "production path planner", i.e. a path planning module that is part of the production platform / system that is configured to generate paths that are to be executed by the ADS of the vehicle. The term obtaining is herein to be interpreted broadly and encompasses receiving, retrieving, collecting, acquiring, and so forth.

[0022] Accordingly, the path planning module is configured to generate the candidate path for execution the vehicle based on a path planning model (e.g. a machine learning algorithm, artificial neural network, or the like) and data indicative of the surrounding environment of the vehicle. Thus, the data indicative of the surrounding environment (e.g. perception data) serves as input to the path planning model, and the output of the path planning model is one or more candidate paths. The data indicative of the surrounding environment of the vehicle may for example be perception output (e.g. fused sensor data) generated by a perception system of the vehicle. However, in some embodiments, the data indicative of the surrounding environment of the vehicle may be sensor data obtained directly from one or more vehicle-mounted sensors. Furthermore, the path planning model may also utilize the actuation capabilities of the vehicle as input. The term "actuation capability" as used herein may include one or more of a braking capacity of the vehicle, an acceleration capacity of the vehicle, a steering capacity of the vehicle, and so forth as readily appreciated by the skilled person in the art.

[0023] Further, the method 100 comprises obtaining 102 a reference framework for evaluating the candidate path. The reference framework is configured to indicate one or more risk values associated with the candidate path when the candidate path is applied in the reference framework. The reference framework may be understood as a framework where the evaluation of the candidate path is performed in the open-loop testing. The reference framework is in the form of a baseline worldview generated from a post-processing of the output of the production system's perception system. The post-processing and the baseline worldview are discussed in further detail in reference to Figs. 5, 6a, and 6b below.

[0024] The method 100 further comprises evaluating 103 the obtained candidate path by applying the candidate path in the reference framework in order to determine 114 a cost function based on the one or more risk values. The cost function is accordingly indicative of a performance of the path planning module within the reference framework.

[0025] Further, the method 100 comprises updating 104 one or more parameters of the path planning model by means of an optimization algorithm (e.g. gradient-based optimizers or derivative-free optimizers) configured to optimize the determined cost function. In the present disclosure, the terms cost function, loss function, and error function are used interchangeably. The purpose of the cost function as defined herein, is accordingly to provide a means to update the path planning model so to maximize desirable output and to minimize unwanted output from the path planning model. In more detail, the cost function is preferably set in relation to one or more predefined goals (i.e. predefined target values). For example, a defined risk value threshold may be used to form a "goal" for the cost function. In other words, a purpose of the cost function is to minimize the one or more risk values associated with the candidate path until a certain point. This is in order to avoid potential situations where the path planning is optimized to the level of "realizing" that the "safest" option is to stand still. However, one may also impose some fixed constraints (e.g. vehicle must move from A to B) in order to avoid such situations.

[0026] An equivalent solution not covered by the scope of protection of this invention would be to determine 114 a corresponding "reward function" in the evaluation 103 step and update 104 the one or more model parameters of the path planning model by means of an optimization algorithm configured to maximize the determined "reward function".

[0027] The development, testing, and validation of path planning algorithms for ADSs are generally thought of as an expensive and time-consuming endeavour as the testing needs to be safe and it requires a huge effort for validation. Both of these stages (development and testing) are costly and prolongs the lead-time of delivering new path planning solutions to the market. However, it was realized by the present inventors that many of the costly steps could potentially be relieved by building on top of already released production vehicles because parts of the development, testing, and validation can be done in the production vehicles without the need for dedicated vehicles with testing personnel. Accordingly, the method and apparatus as proposed herein provides a fully-automated solution for performing open-loop development, testing, and / or validation of new path-planning modules in a sufficiently safe manner.

[0028] In other words, the present invention provides for a learning platform for autonomous vehicles where the production system and sensors of the production ADS are utilized to carry out federated learning of next versions of path planning features for ADSs. Thereby providing advantages in terms of cost and time for development, testing, and / or validation of path planning features for autonomous vehicles.

[0029] The open-loop evaluation as proposed herein may advantageously be allowed to continue iteratively as long as it appears that the open-loop learning continues to improve the path planning module at a sufficient rate. For example, one could run the open-loop evaluation until the performance of the path planning module is such that the majority (over a set period of time) of the candidate paths would be allowed to be executed with respect to their estimated risk in the reference framework. Additionally, or alternatively, the open-loop evaluation could be used until the estimated risk of the candidate paths in the reference framework (over a set period of time) is at least on par with the path planner of the production platform (or at least within some tolerance margin).

[0030] Accordingly, some of the technical advantages of the invention as disclosed herein are: Cost-effective and time-efficient development, testing, and / or validation of new path planning features. It is possible to capitalise on the available production resources in the launched cars to further develop, test, and / or validate the path planning system.

[0031] Further, in the scheme of federated learning, the updated 104 parameters of each of a plurality of vehicles may advantageously be consolidated in a central or cloud unit, whereby a "global update" may be pushed to the entire fleet of vehicles. Therefore, in some embodiments, the method 100 further comprises transmitting 105 the one or more updated 104 parameters of the path planning model to a remote entity (e.g. a back-office or fleet management system). Moreover, the method 100 may comprise receiving 106 a set of globally updated parameters of the path planning model of the path planning module from the remote entity. The set of globally updated parameters are accordingly based on information obtained from a plurality of vehicles comprising a corresponding path planning module. Then, the method 100 may comprise updating 107 the path planning model of the path planning module based on the received set of globally updated parameters.

[0032] As mentioned, the reference framework is in the form of a baseline world generated from a post-processing of the output of the production system's perception system. Thus, in some embodiments, the step of obtaining 102 a reference framework comprises storing 111, during a time period, a set of perception data obtained from a perception system of the vehicle. The perception system being configured to generate the set of perception data based on sensor data obtained from one or more vehicle-mounted sensors during the time period. A perception system (of the production platform / system) is in the present context to be understood as a system responsible for acquiring raw sensor data from on-board sensors such as cameras, LIDARs and RADARs, ultrasonic sensors, and converting this raw data into scene understanding.

[0033] The set of perception data may for example be stored or saved in a data buffer (not shown), where this set of perception data may be understood as data indicative of the vehicle's surroundings. This may for example be detected objects or objects' states and / or vehicle localization, and / or statistical and physical model predictions of future states, derived continuously and / or intermittently from a first time point T1 to a second time point T2. The time period - and correspondingly the length of the optional data buffer - may be of any arbitrary size deemed feasible, e.g. in consideration of data capacity restraints and / or characteristics of the ADS, and may for instance range from under a second up to several minutes, or more preferred, from a few seconds up to less than a minute.

[0034] Further, the step of obtaining 102 a reference framework comprises forming 112, by post-processing the set of perception data, a baseline worldview indicative of a scenario in the surrounding environment of the vehicle during the time period. Accordingly, this baseline worldview forms the reference framework. A "scenario" may be one or more momentary scenes at one or more points in time during the time period including the positions of detected objects, object classes / types, positions of lane markers, extensions of lane markers, free-space detections, and / or trajectories of detected objects in the surrounding environment of the vehicle. It should be noted that this list merely serves to exemplify the parameters included in a "scenario" and may include other parameters detectable by the vehicle's perception module as readily understood by the skilled person in the art.

[0035] The post-processing step 112 will now be further exemplified in reference to Fig. 2, which depicts a series (a) - (d) of schematic top-view illustrations of a vehicle 1 moving a road portion towards an external object 24. Each illustration is associated with a point in time within the time period 21 ranging from a first moment in time T1 to a second moment in time T2.

[0036] In the first illustration (a) the vehicle 1 (may also be referred to as ego-vehicle 1) is moving towards an external object, here in the form of a truck 24, that is traveling in the same direction on an adjacent lane on the road portion. However, due to the distance to the truck 24, the vehicle's perception system / module may not be able to determine, with a sufficiently high level of accuracy, the position of the external object, and to classify it as a truck. This is indicated by the box 22a enclosing the truck 24, which serves to schematically indicate the "uncertainties" of the detection and classification.

[0037] At a subsequent moment in time, i.e. illustration (b) of Fig. 2, the vehicle 1 is closer to the external object, and the uncertainties regarding the external object's 24 position and class / type are reduced, as indicated by the reduced size of the box 22b as compared to the first box 22a.

[0038] At yet another subsequent moment in time, i.e. illustration (c) of Fig. 2, the vehicle's 1 perception system / module is able to accurately determine the external object's 2 position and classify it as a truck 2. More specifically, the ego-vehicle 1 is now sufficiently close to the truck 2 to be able to classify it and estimate the truck's position on the road with a higher level of accuracy as compared to when the ego-vehicle 1 was located further away from the truck.

[0039] Then, by means of a suitable filtering technique and based on the temporal development of the "scenario", one is able to establish a "baseline worldview" at an intermediate point 23 in time between T1 and T2, as indicated in the bottom illustration in Fig. 2, i.e. in illustration (d) of Fig. 2. In more detail, the filtering may for example be based on the temporal development of the trajectories, positions, etc. in combination with predefined models (e.g. motion models) of the vehicle 1 and external objects 2. This established baseline worldview may subsequently used as a "ground truth" for training and / or validation of the output obtained from the path planning module.

[0040] In accordance with some embodiments, the step of forming the baseline worldview comprises determining, based on post-processing a portion of the set of perception data ranging back from the second time point to an intermediate time point between the first time point T1 and second time point T2 the baseline worldview indicative of the surrounding environment of the vehicle. The baseline worldview accordingly being conditional on the portion of the set of perception data. Moreover, in accordance with some embodiments, the post-processing of the portion of the set of perception data comprises running the portion of the set of perception data through a backwards filter. Here, the backwards filter is configured to align a set of perceptive parameters of the set of perception data at the intermediate time point based on a development of the state(s) of the set of perceptive parameters from the intermediate time point to the second time point T2. A perceptive parameter may in the present context be an object detection estimation, an object classification estimation, an object state estimation, a road reference feature estimation, a free-space estimation, a road friction estimation, an object trajectory estimation, and / or a drivable-area estimation.

[0041] In other words, with the increased knowledge of vehicle 1 surroundings as time passes from the intermediate time point 23 to the second time point T2 and by analyzing data in reverse temporal direction, one may be able to determine, with a higher level of accuracy, the "state" (i.e. classes, positions, trajectories, etc.) of the objects in the vehicle's 1 surroundings at the intermediate time point, than it was able to do at "run-time". Thus, the post-processing may be understood as a type of automated "ex-post-facto" analysis of data indicative of the surrounding environment of the vehicle. In more detail, the post processing may for example comprise running the set of perception data through a backwards filter configured to align e.g. the objects current and predicted future states with what happened in the future - i.e. from the intermediate time point to the second time point T2. The post-processing may include further processing steps than running it through a backwards filter. More specifically, the post-processing may include fusion of data from various sensors, as well as applying backward and forward filtering on the fused information. Suitable filters for this purpose may for example be Particle filters or different types of Kalman filters (e.g. extended Kalman filters).

[0042] Thus, by performing the above-described post-processing 112 method for series of "intermediate time points", one can establish a more reliable description of the temporal evolution of the "state" of the objects in the ego-vehicle's surroundings than is possible to do at "run-time". This post-processed "description" of the surrounding environment accordingly constitutes the baseline worldview, which can then be used to evaluate the obtained 101 candidate path. Preferably, the predicted duration of the candidate path is comprised within the "duration" of the formed baseline worldview. For example, if the generated baseline worldview is indicative of a scenario in the surrounding environment of the vehicle during a time period extending from T1 to T5, the duration of the candidate path may be from T2 to T4.

[0043] According to the invention, the step of evaluating 103 the obtained 101 candidate path comprises comparing 113 the candidate path with the baseline worldview in order to obtain the one or more risk values. Then, a cost function is determined 114 based on the obtained one or more risk values, where each risk value is indicative of a temporal evolution of a collision threat measure for the candidate path during at least a portion of the time period.

[0044] A pair of illustrative examples of a candidate path evaluation in view of a baseline worldview are provided in Figs. 3a - 3b. In these example embodiments (Figs. 3a - 3b), Post Encroachment Time (PET) is used to define the collision threat measure. Post Encroachment Time (PET) may be understood as the time difference between a vehicle leaving the area of encroachment and a conflicting vehicle entering the same area. In other words, PET at any time point may be construed as the time gap (difference) between two objects occupying any overlap in space. Accordingly, in the present case PET is the time between the moment that the ego-vehicle leaves the path of an external vehicle and the moment that the external vehicle reaches the path of the ego-vehicle, or vice versa.

[0045] Thus, Figs. 3a and 3b are schematic top-view illustrations of an ADS-equipped vehicle 1 comprising an apparatus for automated development of a path planning module in accordance with an embodiment of the invention. More specifically, Figs. 3a and 3b depict two different situations where PET is used to perform a pass / fail evaluation of a plurality of candidate paths 61, 62 generated by the path planning module of the vehicle 1.

[0046] However, PET may be used as a factor in order to determine a cost function using PET so to maximize the PET up until a set level as it may not be desirable to maximize PET infinitely. Thus, the cost may be defined as max(-PET, -PET max, value ) or -min(PET, PET max, value ).

[0047] When utilizing the baseline worldview as the reference framework, a defined performance value (i.e. the collision threat measure) is required to perform the evaluation. This performance value is accordingly calculated after the situation has been post-processed in accordance with the methodology described in the foregoing. It should be noted that PET is only one example of an applicable performance value / collision threat measure, and that other metrics may be utilized such as e.g. Brake-Threat Number (BTN) and Steer-Threat Number (STN) as readily understood by the skilled artisan.

[0048] In the first situation, depicted in Fig. 3a, the ego-vehicle's 1 path planning module 1 (i.e. the Module-Under-Test) generates a number of candidate paths 61, 62 based on its path planning model and data indicative of the surrounding environment of the vehicle. These candidate paths may then be stored (in e.g. a data buffer of a suitable length) similarly as the perception data generated by the production platform's perception system that is to be used for the post-processing and generation of the baseline worldview. Once the baseline worldview has been formed over a suitable time period, the candidate paths 61, 62 are evaluated based on a PET threshold τ. As indicated in Fig. 3a depicting the "first" situation, one of the candidate paths 61 fails the evaluation, while the remaining paths 62 pass the evaluation.

[0049] In the "second" situation, depicted in Fig. 3b, the external vehicle 2a traveling in the neighbouring lane is traveling at a higher speed as compared to the first situation depicted in Fig. 3a. Thus, the candidate paths that involve a change of lanes to the left-most lane are affected, which is indicated by a higher number of candidate paths 61 that failed the evaluation. It should be noted that the depicted examples in Figs. 3a and 3b are simplified examples of a binary evaluation (pass / fail), and that further parameters may be used in the evaluation to determine the one or more risk values for each candidate path.

[0050] Moreover, it should be noted that in neither of the two evaluation processes (risk map / post-processing) is the candidate path actually executed by the ADS of the vehicle. Moreover, the cost function for the open loop learning may be constructed based on any combination of the results from these two evaluation processes. Thus, in some embodiments, the reference framework is in fact two reference frameworks, one defined by the risk map and one defined by the baseline worldview generated by the post-processing process.

[0051] Executable instructions for performing these functions are, optionally, included in a non-transitory computer-readable storage medium or other computer program product configured for execution by one or more processors.

[0052] Fig. 4 is a schematic block diagram representation of an apparatus or system 10 for automated development of a path planning module 73 of an ADS-equipped vehicle in accordance with an embodiment of the present invention. In general, Fig. 4 depicts the flow of information through exposure to an event in the vehicle surroundings, to the path evaluation processes, and further to the transmission and subsequent consolidation in the "back-office" 2. The apparatus / system 10 comprises various control circuitry configured to perform the functions of the methods disclosed herein, where the functions may be included in a non-transitory computer-readable storage medium or other computer program product configured for execution by the control circuitry. Fig. 4 serves to better elucidate the present invention by depicting various "modules" each of them linked to one or more specific functions described in the foregoing.

[0053] It should be noted that the "candidate path" that is compared in the "evaluation engine" 74 may comprise a plurality of candidate paths.

[0054] The system 10 has a path planning module 73 that is configured to generate the candidate path for the vehicle based on a path planning model and data indicative of the surrounding environment of the vehicle (such as e.g. perception data generated by a perception system 78 of the vehicle). Accordingly, the perception system 78 is configured to generate a perception output based on sensor data 71 obtained from one or more vehicle-mounted sensors during the time period. The sensor data 71 may for example output from one or more of a RADAR device, a LIDAR device, a camera, and ultrasonic sensor, and so forth. In other words, a "perception system" (i.e. the perception system of the production platform) is in the present context to be understood as a system responsible for acquiring raw sensor data from on-board sensors such as cameras, LIDARs and RADARs, ultrasonic sensors, and converting this raw data into scene understanding including state estimates and predictions thereof.

[0055] The vehicle's production platform / system 72 is configured to supply a reference framework for evaluating the candidate path. The reference framework is configured to indicate one or more risk values associated with the candidate path when the candidate path is applied in the reference framework. Moreover, the reference framework is in the form of a baseline worldview generated by a post-processing module 77.

[0056] Thus, in some embodiments, the perception data is stored or saved in a data buffer (not shown), where this perception data may be understood as data indicative of the vehicle's surroundings. This may for example be detected objects or objects' states and / or vehicle localization, and / or statistical and physical model predictions of future states, derived continuously and / or intermittently from a first time point T1 to a second time point T2. The time period - and correspondingly the length of the optional data buffer - may be of any arbitrary size deemed feasible, e.g. in consideration of data capacity restraints, a predicted duration of the candidate path(s), and / or characteristics of the ADS, and may for instance range from under a second up to several minutes, or more preferred, from a few seconds up to less than a minute.

[0057] Further, the production platform 72 may accordingly comprise a post-processing module 77 for forming a baseline worldview indicative of a scenario in the surrounding environment of the vehicle during the time period. The baseline worldview is formed based on the perception data generated by the perception system 78 of the production ADS. It should be noted that the term "forming, by post-processing the first set of perception data" does not necessarily mean that all of the stored data is post-processed, but should be construed as that at least a portion or at least a range of the stored perception data is post-processed.

[0058] Further, the reference framework is supplied to the evaluation engine 74 configured to evaluate the obtained candidate path by applying the candidate path in the reference framework in order to determine a cost function based on the one or more risk values. The output of the cost function is accordingly indicative of a performance of the path planning module within the reference framework, or more specifically, indicative of a level of risk associated with the candidate path. Thus, in some embodiments the cost function is indicative of whether or not the generated candidate path can be considered safe (given one or more predefined criteria).

[0059] The evaluation engine 74 is configured to evaluate the candidate path by comparing the candidate path with the baseline worldview, and determining the cost function based on the one or more risk values. Each risk value is accordingly indicative of a temporal evolution of a collision threat measure for the candidate path during at least a portion of the time period.

[0060] Further, the system 10 has a learning engine 75 configured to update one or more parameters of the path planning model by means of an optimization algorithm configured to optimize the determined cost function. Moreover, the learning engine 75 may be configured to transmit the one or more updated parameters of the path planning model of the path planning module to a remote entity 2, and then receive a set of globally updated parameters of the path planning model of the path planning module from the remote entity. The set of globally updated parameters are based on information obtained from a plurality of vehicles comprising a corresponding path planning module. Thereafter, the learning engine 75 may update the path planning model of the path planning module based on the received set of globally updated parameters. Thus may be construed as a "consolidation" process across an entire fleet of vehicles.

[0061] Fig. 5 is a schematic side view illustration of a vehicle 1 comprising an apparatus 10 for automated development of a path planning module of an ADS-equipped vehicle in accordance with an embodiment of the present invention. The vehicle 1 further comprises a perception module / system 6 (i.e. the perception system of the production platform), and a localization system 5. The localization system 5 is configured to monitor a geographical position and heading of the vehicle, and may in the form of a Global Navigation Satellite System (GNSS), such as a GPS. However, the localization system may alternatively be realized as a Real Time Kinematics (RTK) GPS in order to improve accuracy.

[0062] In more detail, the perception module / system 6 may refer to any commonly known system and / or functionality, e.g. comprised in one or more electronic control modules and / or nodes of the vehicle 1, adapted and / or configured to interpret sensory information - relevant for driving of the vehicle 1 - to identify e.g. obstacles, vehicle lanes, relevant signage, appropriate navigation paths etc. The exemplified perception system 6 may thus be adapted to rely on and obtain inputs from multiple data sources, such as automotive imaging, image processing, computer vision, and / or in-car networking, etc., in combination with sensory information. Such exemplifying sensory information may for instance be derived from one or more optional surrounding detecting sensors 6a-c comprised in and / or provided on-board the vehicle 1. The surrounding detecting sensors 6a-c may be represented by any arbitrary sensors adapted to sense and / or perceive the vehicle's 1 surroundings and / or whereabouts, and may e.g. refer to one or a combination of one or more of radar, LIDAR, sonar, camera, navigation system e.g. GPS, odometer and / or inertial measurement units.

[0063] The apparatus 10 comprises one or more processors 11, a memory 12, a sensor interface 13 and a communication interface 14. The processor(s) 11 may also be referred to as a control circuit 11 or control circuitry 11. The control circuitry 11 is configured to execute instructions stored in the memory 12 to perform a method f for automated development of a path planning module of a vehicle 1 according to any one of the embodiments disclosed herein. Stated differently, the memory 12 of the apparatus 10 can include one or more (non-transitory) computer-readable storage mediums, for storing computer-executable instructions, which, when executed by one or more computer processors 11, for example, can cause the computer processors 11 to perform the techniques described herein. The memory 12 optionally includes high-speed random access memory, such as DRAM, SRAM, DDR RAM, or other random access solid-state memory devices; and optionally includes non-volatile memory, such as one or more magnetic disk storage devices, optical disk storage devices, flash memory devices, or other non-volatile solid-state storage devices.

[0064] The control circuitry 11 is configured to obtain a candidate path from the path planning module. The path planning module is configured to generate the candidate path for the vehicle based on a path planning model and data indicative of the surrounding environment of the vehicle. The control circuitry 11 is further configured to obtain a reference framework for evaluating the candidate path. The reference framework is configured to indicate one or more risk values associated with the candidate path when the candidate path is applied in the reference framework. Further, the control circuitry 11 is configured to evaluate the obtained candidate path by applying the candidate path in the reference framework in order to determine a cost function based on the one or more risk values. The cost function is indicative of a performance of the path planning module within the reference framework. Moreover, the control circuitry 11 is configured to update one or more parameters of the path planning model by means of an optimization algorithm configured to optimize the determined cost function.

[0065] Further, the vehicle 1 may be connected to external network(s) 20 via for instance a wireless link (e.g. for retrieving map data). The same or some other wireless link may be used to communicate with other vehicles 2 in the vicinity of the vehicle or with local infrastructure elements. Cellular communication technologies may be used for long range communication such as to external networks and if the cellular communication technology used have low latency it may also be used for communication between vehicles, vehicle to vehicle (V2V), and / or vehicle to infrastructure, V2X. Examples of cellular radio technologies are GSM, GPRS, EDGE, LTE, 5G, 5G NR, and so on, also including future cellular solutions. However, in some solutions mid to short range communication technologies are used such as Wireless Local Area (LAN), e.g. IEEE 802.11 based solutions. ETSI is working on cellular standards for vehicle communication and for instance 5G is considered as a suitable solution due to the low latency and efficient handling of high bandwidths and communication channels.

[0066] The present invention has been presented above with reference to specific embodiments. However, other embodiments than the above described are possible and within the scope of the invention. Different method steps than those described above, performing the method by hardware or software, may be provided within the scope of the invention. Thus, according to an exemplary embodiment, there is provided a non-transitory computer-readable storage medium storing one or more programs configured to be executed by one or more processors of a vehicle control system, the one or more programs comprising instructions for performing the method according to any one of the above-discussed embodiments. Alternatively, according to another exemplary embodiment a cloud computing system can be configured to perform any of the methods presented herein. The cloud computing system may comprise distributed cloud computing resources that jointly perform the methods presented herein under control of one or more computer program products.

[0067] Generally speaking, a computer-accessible medium may include any tangible or non-transitory storage media or memory media such as electronic, magnetic, or optical media-e.g., disk or CD / DVD-ROM coupled to computer system via bus. The terms "tangible" and "non-transitory," as used herein, are intended to describe a computer-readable storage medium (or "memory") excluding propagating electromagnetic signals, but are not intended to otherwise limit the type of physical computer-readable storage device that is encompassed by the phrase computer-readable medium or memory. For instance, the terms "non-transitory computer-readable medium" or "tangible memory" are intended to encompass types of storage devices that do not necessarily store information permanently, including for example, random access memory (RAM). Program instructions and data stored on a tangible computer-accessible storage medium in non-transitory form may further be transmitted by transmission media or signals such as electrical, electromagnetic, or digital signals, which may be conveyed via a communication medium such as a network and / or a wireless link.

[0068] The processor(s) 11 (associated with the apparatus 10) may be or include any number of hardware components for conducting data or signal processing or for executing computer code stored in memory 12. The apparatus 10 has an associated memory 12, and the memory 12 may be one or more devices for storing data and / or computer code for completing or facilitating the various methods described in the present description. The memory may include volatile memory or non-volatile memory. The memory 12 may include database components, object code components, script components, or any other type of information structure for supporting the various activities of the present description. According to an exemplary embodiment, any distributed or local memory device may be utilized with the systems and methods of this description. According to an exemplary embodiment the memory 12 is communicably connected to the processor 11 (e.g., via a circuit or any other wired, wireless, or network connection) and includes computer code for executing one or more processes described herein.

[0069] It should be appreciated that the sensor interface 13 may also provide the possibility to acquire sensor data directly or via dedicated sensor control circuitry 6 in the vehicle. The communication / antenna interface 14 may further provide the possibility to send output to a remote location (e.g. remote operator or control centre) by means of the antenna 8. Moreover, some sensors in the vehicle may communicate with the system 10 using a local network setup, such as CAN bus, I2C, Ethernet, optical fibres, and so on. The communication interface 14 may be arranged to communicate with other control functions of the vehicle and may thus be seen as control interface also; however, a separate control interface (not shown) may be provided. Local communication within the vehicle may also be of a wireless type with protocols such as WiFi, LoRa, Zigbee, Bluetooth, or similar mid / short range technologies.

[0070] Accordingly, it should be understood that parts of the described solution may be implemented either in the vehicle, in a system located external the vehicle, or in a combination of internal and external the vehicle; for instance in a server in communication with the vehicle, a so called cloud solution. For instance, data may be sent to an external system and that system performs the steps to evaluate the candidate paths. The different features and steps of the embodiments may be combined in other combinations than those described.

[0071] It should be noted that the word "comprising" does not exclude the presence of other elements or steps than those listed and the words "a" or "an" preceding an element do not exclude the presence of a plurality of such elements. It should further be noted that any reference signs do not limit the scope of the claims, that the invention may be at least in part implemented by means of both hardware and software, and that several "means" or "units" may be represented by the same item of hardware.

[0072] Although the figures may show a specific order of method steps, the order of the steps may differ from what is depicted. In addition, two or more steps may be performed concurrently or with partial concurrence. For example, the steps of obtaining a candidate path and obtaining a reference framework may be interchanged based on a specific realization. Such variation will depend on the software and hardware systems chosen and on designer choice. Likewise, software implementations could be accomplished with standard programming techniques with rule-based logic and other logic to accomplish the various connection steps, processing steps, comparison steps and decision steps. The above mentioned and described embodiments are only given as examples and should not be limiting to the present invention. Other solutions, uses, objectives, and functions within the scope of the invention as claimed in the below described patent claims should be apparent for the person skilled in the art.

Claims

1. A method (100) performed by an in-vehicle computing system for automated development of a path planning module of a vehicle, wherein the vehicle is equipped with an Automated Driving System, ADS, the method comprising: obtaining (101) a candidate path from the path planning module, wherein the path planning module is configured to generate the candidate path for the vehicle based on a path planning model and data indicative of the surrounding environment of the vehicle; obtaining (102) a reference framework for evaluating the candidate path, the reference framework being configured to indicate one or more risk values associated with the candidate path when the candidate path is applied in the reference framework; wherein obtaining (102) the reference framework comprises: storing (111), during a time period, a set of perception data obtained from a perception system of the vehicle, the perception system being configured to generate the set of perception data based on sensor data obtained from one or more vehicle-mounted sensors during the time period, forming (112), by post-processing the set of perception data, a baseline worldview indicative of a scenario in the surrounding environment of the vehicle during the time period, and wherein the baseline worldview forms the reference framework; evaluating (103) the obtained candidate path by applying the candidate path in the reference framework in order to determine a cost function based on the one or more risk values, the cost function being indicative of a performance of the path planning module within the reference framework; wherein evaluating (103) the obtained candidate path comprises: comparing (113) the candidate path with the baseline worldview in order to obtain the one or more risk values, and determining (114) the cost function based on the obtained one or more risk values, each risk value being indicative of a temporal evolution of a collision threat measure for the candidate path during at least a portion of the time period; updating (104) one or more parameters of the path planning model by means of an optimization algorithm configured to optimize the determined cost function.

2. The method (100) according to claim 1, further comprising: transmitting (105) the one or more updated parameters of the path planning model of the path planning module to a remote entity; receiving (106) a set of globally updated parameters of the path planning model of the path planning module from the remote entity, wherein the set of globally updated parameters are based on information obtained from a plurality of vehicles comprising a corresponding path planning module; updating (107) the path planning model of the path planning module based on the received set of globally updated parameters.

3. A computer-readable storage medium storing one or more programs configured to be executed by one or more processors of an in-vehicle processing system, the one or more programs comprising instructions for performing the method according to any one of the preceding claims.

4. An apparatus (10) for automated development of a path planning module (73) of a vehicle (1), wherein the vehicle is equipped with an Automated Driving System, ADS, the apparatus comprising control circuitry (11) configured to: obtain a candidate path from the path planning module (73), wherein the path planning module is configured to generate the candidate path for the vehicle based on a path planning model and data indicative of the surrounding environment of the vehicle; obtain a reference framework for evaluating the candidate path, the reference framework being configured to indicate one or more risk values associated with the candidate path when the candidate path is applied in the reference framework; wherein obtaining (102) the reference framework comprises: storing (111), during a time period, a set of perception data obtained from a perception system of the vehicle, the perception system being configured to generate the set of perception data based on sensor data obtained from one or more vehicle-mounted sensors during the time period, forming (112), by post-processing the set of perception data, a baseline worldview indicative of a scenario in the surrounding environment of the vehicle during the time period, and wherein the baseline worldview forms the reference framework; evaluate the obtained candidate path by applying the candidate path in the reference framework in order to determine a cost function based on the one or more risk values, the cost function being indicative of a performance of the path planning module (73) within the reference framework; wherein evaluating (103) the obtained candidate path comprises: comparing (113) the candidate path with the baseline worldview in order to obtain the one or more risk values, and determining (114) the cost function based on the obtained one or more risk values, each risk value being indicative of a temporal evolution of a collision threat measure for the candidate path during at least a portion of the time period; update one or more parameters of the path planning model by means of an optimization algorithm configured to optimize the determined cost function.

5. The apparatus (10) according to claim 4, wherein the control circuitry (11) is further configured to: transmit the one or more updated parameters of the path planning model of the path planning module to a remote entity (2); receive a set of globally updated parameters of the path planning model of the path planning module from the remote entity (2), wherein the set of globally updated parameters are based on information obtained from a plurality of vehicles comprising a corresponding path planning module; update the path planning model of the path planning module based on the received set of globally updated parameters.

6. A vehicle (1) comprising: a set of vehicle-mounted sensors (6a, 6b, 6c) configured to monitor a surrounding environment of the vehicle; an automated driving system, ADS, having a perception system (6, 78) configured to generate perception data based on sensor data obtained from one or more of the set of vehicle-mounted sensors; a path planning module (73) configured to generate the candidate path for the vehicle; an apparatus (10) according to any one of claims 4 - 5.