Method of determining merge strategy for forming merge machine learning model

Abstract

The inventive concept relates to a method of determining a merge strategy for forming a merge machine learning model. Specifically, the inventive concept relates to a computer-implemented method (100) for determining a consolidation policy for forming a consolidated machine learning model from a plurality of local machine learning models of a fleet equipped with an automated driving system. The method (100) comprises: obtaining (S102) two or more model updates from one or more vehicles of the fleet, wherein each model update is a result of training a local machine learning model of a respective vehicle; merging (S106) the two or more model updates according to a candidate merging policy, thereby forming a merged machine learning model; evaluating (S110) the merged machine learning model according to evaluation criteria; and updating (S114) the candidate merge policy in view of the evaluation, thereby forming an updated candidate merge policy. The inventive concept also relates to a device thereof.

Description

Technical Field

[0001] The inventive concept relates to the field of autonomous vehicles. Specifically, the inventive concept relates to a method and apparatus for determining a consolidation strategy for forming a consolidated machine learning model. Background Art

[0002] With the development of technology in recent years, image capture and processing technologies have been widely applied in different technical fields. Specifically, vehicles produced today are usually equipped with some form of vision or perception system to enable new functions. In addition, more and more modern vehicles have advanced driver assistance systems (ADAS) to improve vehicle safety and more generally road safety. ADAS - which can be represented, for example, by adaptive cruise control (ACC), collision avoidance systems, forward collision warnings, lane support systems, etc. - are electronic systems that can assist vehicle drivers. Today, many technical fields related to the ADAS and autonomous driving (AD) areas are under research and development. ADAS and AD can also be collectively referred to as autonomous driving systems (ADS), corresponding to all different levels of automation defined, for example, by the driving automation of SAE J3016 levels (0 - 5). ADS or its perception system usually depends on machine learning models to some extent.

[0003] The continuous learning of these systems is important for providing up - to - date and high - performance functions to the market. One approach is to perform online learning in the vehicle. In other words, in - vehicle learning can be implemented to locally improve the machine learning model at the edge node (i.e., the vehicle) rather than at a central entity (such as within a remote server). Based on this, the so - called federated learning is established, which is a scheme for consolidating a "global" or "central" model from "local" models trained at the edge. Then, the remote server is responsible for aggregating or consolidating the model updates from each edge node into a consolidated model.

[0004] Although this learning scheme allows the effective use of production vehicles for continuous learning and effectively alleviates the problems of data transmission limitations and data privacy, it is not an easy task to use the proposed model updates. Since each vehicle operates within its respective geographical area and experiences different scenarios and events during each learning cycle, the updated models from each vehicle will be different and contribute differently to the performance of the consolidated model. Therefore, progress is needed in this field in terms of new and improved methods for consolidating the model updates of the federated learning scheme. Summary of the Invention

[0005] The technology disclosed herein aims to alleviate, mitigate, or eliminate one or more of the above-mentioned deficiencies and drawbacks in the prior art, and to address various problems related to the development of an autonomous driving system (ADS). In particular, the disclosed technology provides methods for improving the model performance of a combined machine learning model using a federated learning approach. More specifically, the inventors have implemented a novel and improved method for determining a merging strategy through an optimization method. The determined merging strategy can improve the model performance by combining local model updates from a fleet of vehicles in an optimized manner.

[0006] Various aspects and embodiments of the disclosed invention are defined below and in the appended independent and dependent claims.

[0007] According to a first aspect, there is provided a computer-implemented method for determining a merging strategy for forming a combined machine learning model from a plurality of local machine learning models of a fleet of vehicles equipped with an autonomous driving system. The method includes obtaining two or more model updates from one or more vehicles of the fleet. Each model update is a result of training a local machine learning model of a corresponding vehicle. The method further includes merging the two or more model updates according to a candidate merging strategy to form a combined machine learning model. The method further includes evaluating the combined machine learning model according to an evaluation criterion. The method further includes updating the candidate merging strategy in view of the evaluation to form an updated candidate merging strategy.

[0008] Possible related advantages of the disclosed technology may be that, within the limitations of learning capabilities and what model updates are available, it can improve (or even optimize) federated learning. More specifically, it can form a combined model that achieves optimal performance when using a federated learning approach on a fleet of vehicles experiencing different operating conditions and scenarios. Additionally, it can balance the impact of the process when different learning processes are merged into a single model. It can help ensure that new model updates can be appropriately incorporated into the mixture of model updates, which can help the model perform better on previously intractable or rare samples.

[0009] According to a second aspect, there is provided a computer program product including instructions that, when executed by a computing device, cause the computing device to perform the method according to any embodiment of the first aspect. According to a third aspect, there is provided a (non-transitory) computer-readable storage medium. The non-transitory computer-readable storage medium stores one or more programs configured to be executed by one or more processors of a processing system, the one or more programs including instructions for performing the method according to any embodiment of the first aspect. Where applicable, any of the above features and advantages of the first aspect also apply to the second and third aspects. For the sake of avoiding unnecessary repetition, reference is made to the above content.

[0010] According to a fourth aspect, there is provided an apparatus for determining a merging strategy for forming a merged machine learning model from a plurality of local machine learning models of a fleet equipped with an autonomous driving system. The apparatus includes a control circuit. The control circuit is configured to obtain two or more model updates from one or more vehicles of the fleet, where each model update is a result of training a local machine learning model of a corresponding vehicle. The control circuit is further configured to merge the two or more model updates according to a candidate merging strategy to form a merged machine learning model. The control circuit is further configured to evaluate the merged machine learning model according to an evaluation criterion. The control circuit is further configured to update the candidate merging strategy in view of the evaluation to form an updated candidate merging strategy. In applicable cases, any of the above features and advantages of the foregoing aspects are also applicable to the fourth aspect. To avoid unnecessary repetition, reference is made to the above content.

[0011] As used herein, the term "non-transitory" is intended to describe a computer-readable storage medium (or "memory") that does not include propagating electromagnetic signals, but is not intended to otherwise limit the type of physical computer-readable storage device encompassed by the phrase computer-readable medium or memory. For example, the term "non-transitory computer-readable medium" or "tangible memory" is intended to encompass types of storage devices that do not necessarily store information permanently, including, for example, random access memory (RAM). Program instructions and data stored in a tangible computer-accessible storage medium in non-transitory form can further be transmitted via a transmission medium or signal (such as an electrical, electromagnetic, or digital signal), which can be conveyed via a communication medium (such as a network and / or a wireless link). Thus, the term "non-transitory" as used herein is a limitation on the medium itself (i.e., tangible, rather than a signal), rather than a limitation on the persistence of data storage (e.g., RAM versus ROM).

[0012] The disclosed aspects and preferred embodiments can be appropriately combined with each other in any way obvious to those of ordinary skill in the art, such that one or more features or embodiments disclosed in relation to one aspect can also be considered to be disclosed in relation to another aspect or an embodiment of another aspect.

[0013] Further embodiments are defined in the dependent claims. It should be emphasized that the term "comprising / including" as used in this specification is used to specify the presence of the stated features, integers, steps, or components. It does not preclude the presence or addition of one or more other features, integers, steps, components, or groups thereof.

[0014] These and other features and advantages of the disclosed technology will be further elucidated below with reference to the embodiments described hereinafter. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] The above aspects, features, and advantages of the disclosed technology will be more fully understood from the following illustrative and non-limiting detailed description of example embodiments of the present disclosure when considered in conjunction with the accompanying drawings, in which:

[0016] Figure 1 is a schematic flowchart of a method for determining a merging strategy for forming a merged machine learning model from multiple local machine learning models according to some embodiments.

[0017] Figure 2 is a schematic diagram of an apparatus for determining a merging strategy for forming a merged machine learning model from multiple local machine learning models according to some embodiments.

[0018] Figure 3 is a schematic diagram of a vehicle according to some embodiments.

[0019] Figure 4 is a schematic diagram of a system according to some embodiments.

[0020] Figure 5 A flowchart of updating a merging strategy according to some embodiments is illustrated by way of example. Detailed Description

[0021] The present disclosure will now be described in detail with reference to the accompanying drawings, in which some example embodiments of the disclosed technology are shown. However, the disclosed technology may be embodied in other forms and should not be construed as limited to the disclosed example embodiments. The disclosed example embodiments are provided to fully convey the scope of the disclosed technology to those skilled in the art. Those skilled in the art will understand that the steps, services, and functions described herein can be implemented using separate hardware circuits, using software running in conjunction with a programmed microprocessor or a general-purpose computer, using one or more application-specific integrated circuits (ASICs), using one or more field-programmable gate arrays (FPGAs), and / or using one or more digital signal processors (DSPs).

[0022] It should also be understood that when the present disclosure is described in terms of a method, the present disclosure can also be embodied in a device including one or more processors and one or more memories coupled to the one or more processors, where computer code is loaded at the device to implement the method. For example, the one or more memories can store one or more computer programs, which in some embodiments, when executed by the one or more processors, cause the device to perform the steps, services, and functions disclosed herein.

[0023] It should also be understood that the terms used herein are for the purpose of describing particular embodiments only and are not intended to be limiting. It should be noted that the articles "a", "the" and "said" used in the specification and the appended claims are intended to mean that there is one or more elements, unless the context clearly dictates otherwise. Thus, for example, in some instances, reference to "a unit" or "the unit" may refer to more than one unit, and so on. In addition, words such as "comprising", "including", "containing" and the like do not exclude other elements or steps. It should be emphasized that the term "comprising / including" used in this specification is used to specify the presence of the stated features, integers, steps or components. It does not exclude the presence or addition of one or more other features, integers, steps, components or groups thereof. The term "and / or" should be construed to also mean "both" and each is an alternative.

[0024] It should also be understood that although the terms first, second, etc. may be used herein to describe various elements or features, these elements should not be limited by these terms. These terms are only used to distinguish one element from another. For example, without departing from the scope of the embodiments, the first element may be referred to as the second element, and similarly, the second element may be referred to as the first element. The first element and the second element are both elements, but they are not the same element.

[0025] As used herein, the phrase "one or more of a group of elements" (such as "one or more of A, B, and C" or "at least one of A, B, and C") should be construed as conjunctive logic or disjunctive logic. In other words, it can refer to all of the elements in a group of elements, one element, or a combination of two or more elements. For example, the phrase "one or more of A, B, and C" can be construed as A or B or C, A and B and C, A and B, B and C, or A and C.

[0026] Throughout this disclosure, reference is made to machine learning models (which may also be referred to as machine learning algorithms, networks, neural networks, deep learning networks, etc.). A machine learning model herein refers to any computational system or algorithm that is trained on data to make predictions or decisions, for example, by learning patterns and relationships from the training data and applying that knowledge to new input data.

[0027] A machine learning model can be a model configured to process any type of sensor data of a vehicle (such as image data, LIDAR data, RADAR data, etc.). The machine learning model can be, for example, a perception model, an object detection model, an object classification model, an object tracking model, a lane estimation model, a free space estimation model, a trajectory prediction model, an obstacle avoidance model, a path planning model, a scene classification model, a traffic sign classification model, etc. In addition, the machine learning model can include different architectures, including but not limited to convolutional neural networks (CNNs), recurrent neural networks (RNNs), and other existing or future alternatives.

[0028] The deployment of a machine learning model typically involves a training phase in which the model learns from labeled or unlabeled training data to make accurate predictions in subsequent inference phases. The training data (and the input data during the inference process) can be, for example, images or image sequences, LIDAR data (i.e., point clouds), radar data, etc. In addition, the training / input data can include a combination or fusion of one or more different data types. The training / input data can, for example, include an image depicting an annotated object and a corresponding LIDAR point cloud including the same annotated object.

[0029] In some embodiments, the machine learning model can be implemented using suitable software development machine learning code elements available publicly and in any manner known to be suitable for a person of ordinary skill in the art, such as those available in Pytorch, TensorFlow, and Keras, or code elements available in any other suitable software development platform.

[0030] Reference will be made below to Figures 1 to 5 Describe a method for determining a merging strategy for forming a merged machine learning model (sometimes simply referred to as a "merged model") from multiple local machine learning models and other related aspects thereof. In short, this process can be regarded as an optimization problem, the goal of which is to find the best strategy for forming the merged model. This optimization can be guided by evaluating different candidates, which can be continuously improved until the best strategy is found.

[0031] As described above, online learning of machine learning models in the vehicles of a vehicle fleet can be utilized to form a "central" or "global" model, herein referred to as a combined machine learning model, or simply a combined model. Thus, the combined model is formed by the combination of multiple local models trained at the edge (i.e., in the vehicles). This is commonly referred to as federated learning. A central server is typically used to collect the local models and combine (or aggregate) them to form the combined model. The idea is that a combined model formed by combining several local models will have better performance than can be achieved by more traditional learning schemes that train only a single model. This is partly because the local models contributing to the combined model all have different experiences and have learned from different data distributions. This diversity of data sources helps the combined model to better generalize to various patterns and conditions, which can lead to improved overall performance.

[0032] Federated learning also offers several additional advantages. First, it can improve privacy and security, as it enables local models to be trained locally on the device without transmitting any raw and potentially sensitive data to the central server. This improves privacy both because the data remains on the local device and security because the risk of unauthorized access during data transmission can be reduced. Second, it can provide better bandwidth efficiency, as only model updates are sent to the server, rather than the raw data. Thus, federated learning can reduce the amount of data transmitted over the network. Additionally, a decentralized approach leveraging edge computing can speed up model updates and reduce latency. It can also bring advantages related to resilience to device failures, data governance, robustness, and redundancy.

[0033] The prominent problem the inventors set out to solve was how to combine local models in a way that optimizes the performance of the combined model given a limited number of model updates. Considering the limited learning capabilities of machine learning models, e.g., due to in-vehicle computing and latency constraints, how to balance the use of different model updates can play an important role in achieving a high-performance combined model.

[0034] A common way to combine models using federated learning is by simply averaging the model updates of the local models, or in a slightly more complex way, weighted averaging. In the latter case, the weights can be derived, for example, based on the number of new samples incorporated in each model update. While this effectively combines the updates of all entities, in federation, since data from different entities may contribute to the final performance of the combined model to varying degrees, this can affect the performance of the combined model when relying on a fixed (simple) protocol for weighted averaging, and thus can be troublesome. Instead, the techniques of the present disclosure solve this problem in a new and improved way that utilizes an optimization mechanism to determine the combination strategy, which helps to achieve an optimal combined model within a search space that includes the number of available local model updates. The search space is explored to find candidates for the optimal combination strategy. Then, a combined model is formed based on the candidate combination strategy and subsequently evaluated. Based on the evaluation results, the candidate combination strategy can be updated. These steps can be iterated in an optimization loop until an evaluation criterion and / or a convergence criterion is met. After iteration, the final candidate combination strategy can be used to form the combined model for use in production. Thus, the present technique proposed herein provides a method for finding an optimal combined model from a learning loop that relies on federated learning. The proposed method will be explained in more detail below.

[0035] It should be noted that federated learning as described above can be referred to as a centralized federated learning scheme. However, in some embodiments, the need for a central entity can be omitted, and edge devices (such as a fleet of vehicles) can be configured to coordinate themselves to obtain a global model, namely a so-called decentralized federated learning scheme. In the following, the description is mainly given with reference to the centralized federated learning scheme. However, it is assumed that those skilled in the art will readily understand how to implement the teachings herein as a decentralized federated learning scheme, and the scope conveyed by the present invention disclosed herein includes such an implementation. Additionally, it should be noted that in this context, the term "global" does not necessarily mean "global", but should be interpreted as something "shared" among multiple "local" entities.

[0036] It should be noted that terms such as "optimal", "optimized", or similar should be understood as something that is "as good as possible" within certain limitations. The limitations can be, for example, time limitations defined by an evaluation criterion and / or a convergence criterion. These limitations may also relate to computational resources or numerical approximations. As is known in the art, it can also be "optimal" or "optimized" in the sense of being determined based on an optimization algorithm. An "optimal combination strategy" or "optimal model performance", etc. does not need to be a single optimal solution (i.e., a global minimum / maximum), but can be a solution close to a single optimal solution.

[0037] Figure 1FIG. 100 is a schematic flow chart of a method 100 for determining a merging strategy for forming a merged machine learning model from multiple local machine learning models. The multiple local machine learning models may be deployed in respective vehicles of a fleet equipped with an autonomous driving system (ADS). More specifically, the local models may be part of the ADS of the vehicles. As described above, forming a merged model (or global model) from multiple local models can be regarded as a federated learning scheme.

[0038] The method 100 may be performed offline, for example, by a server. The server may also be referred to as a remote server, cloud server, central server, backend server, fleet server, or back-end server. In this case, a centralized federated learning scheme may be implemented. Alternatively, the method 100 may be performed by one (or more) vehicles in the fleet. In this case, a decentralized federated learning scheme is implemented. This may be regarded as a swarm approach (or setting) for implementing the techniques disclosed herein.

[0039] Below, the different steps of the method 100 are described in more detail. Even though shown in a specific order, the steps of the method 100 may be performed in any suitable order and multiple times. Thus, although Figure 1 a specific order of method steps may be shown, the order of the steps may be different from the described order. Furthermore, two or more steps may be performed simultaneously or partially simultaneously. Such variations will depend on the software and hardware systems selected and on the designer's choice. For example, the steps denoted as S102 and S104 may be performed independently of each other in any order. All such variations are within the scope of the present invention. Similarly, the software implementation may be accomplished using standard programming techniques with rule-based logic and other logics to perform the respective steps. Other variations of the method 100 will become apparent from the present disclosure. The embodiments mentioned and described above are given only as examples and should not limit the present invention. Other solutions, uses, purposes, and functions within the scope of the present invention claimed by the following patent claims should be apparent to those skilled in the art.

[0040] The method 100 includes obtaining S102 two or more model updates from one or more vehicles of the fleet. In other words, in S102, at least two model updates may be obtained from at least one vehicle.

[0041] The term "obtain" should be construed broadly throughout this disclosure and includes receiving, retrieving, collecting, acquiring, etc. directly and / or indirectly between two entities that are configured to communicate with each other or further with other external entities. However, in some embodiments, the term "obtain" should be construed as determining, deriving, forming, calculating, etc. In other words, obtaining two or more model updates from one or more vehicles in a vehicle fleet can include receiving two or more model updates from the respective vehicles. Thus, each vehicle can transmit their respective model updates to the server hosting method 100. In some embodiments, for example, when method 100 is executed by a vehicle in a vehicle fleet, the respective model updates can be transmitted to the vehicle. Then, the vehicle can perform the remaining steps of method 100 as further described below. Or perform some of the subsequent steps and then transmit the data to the server to perform the remaining steps.

[0042] Each model update is the result of training a local machine learning model of the respective vehicle. More specifically, the model update is the result of an in-vehicle learning process in which the local machine learning model is trained (or retrained) on a plurality of training samples collected by the vehicle (e.g., via in-vehicle sensors). The model update should be understood herein as any information indicative of the respective updated local model. For example, each model update can include updated model parameters. The updated model parameters are updated as part of the training of the respective local model. By obtaining the model updates (e.g., via data transmission from the vehicle) as updated model parameters, an efficient utilization of bandwidth can be achieved. However, it should be noted that the model update may include the updated local model itself or an updated portion thereof.

[0043] Two or more model updates may come from different vehicles. Since each vehicle in a fleet will operate in different geographical areas, at different times of the day, and will experience different situations and rare events during operation, the local models will experience heterogeneous or differently distributed data. Thus, model updates from each vehicle will be based on data that has different value in terms of improving the performance of the combined model. Alternatively or in combination, two or more model updates may come from the same vehicle, but from different points in time (i.e., from different learning cycles). Since the same vehicle will experience different situations at different points in time, diversity in the data used to train the local model can also be achieved by taking model updates of the same local model from these different points in time (i.e., from different learning cycles). Thus, before combining S106 two or more model updates to form a combined model, obtaining S102 two or more model updates for the local model can be performed over multiple learning cycles. Then, the combination strategy can further consider from which learning cycle each model update comes. For example, compared to model updates from a more recent learning cycle, model updates from an earlier learning cycle may be associated with a lower relevance.

[0044] As used herein, the term "learning cycle" refers to the process of obtaining a model update. In other words, one learning cycle may correspond to the time during which new training samples are collected and the local model is trained on the training samples. Then, when this process terminates and a model update is obtained, the learning cycle ends. For example, the training process may be terminated when the updated local model meets a specific criterion (such as a convergence criterion), after a prescribed amount of time, or after a certain number of new training samples have been added to the updated model. The learning cycles can be synchronized across the entire fleet such that the server obtains two or more model updates at approximately the same time. Alternatively, for the vehicles in the fleet, the learning cycles can be separate.

[0045] Method 100 further includes combining S106 two or more model updates according to a candidate combination strategy. Thus, a combined machine learning model (or more specifically, a candidate combined machine learning model) is formed. Combining S106 two or more model updates (or generally speaking, how to combine multiple models into one) to form a combined model can be accomplished by combining the model parameters of the local models into one set of model parameters for the combined model according to the combination strategy. In some embodiments, the combined model can be formed in the vehicle, and then the vehicle will transmit the combined model (along with information indicating it) to the server.

[0046] (Candidate) merging strategies can include a set of blending weights. Each blending weight in the set of blending weights can be associated with a model update among two or more model updates. Thus, the merging strategy defines how different model updates should be combined to form a merged model. Blending weights can be used to assign different weights to different model updates. More specifically, blending weights can be used to assign different weights to different sets of model parameters of individual local models.

[0047] By the term "candidate", such as "candidate merging strategy" or "candidate merged machine learning model", it is meant a potential candidate for the "optimal" solution. Thus, a candidate merging strategy can be regarded as a potential merging strategy that can be used as the merging strategy for forming a final merged model.

[0048] Then, the principle of the proposed method 100 can provide a method for updating the candidate merging strategy until an ideal merging strategy is obtained (e.g., in the sense that it can produce a model with good performance). In other words, by iteration (i.e., repeating the process multiple times (also called loops)), the candidate merging strategy can be refined to be close to becoming the best merging strategy. Thus, the candidate merging strategy used in step S106 can be a merging strategy determined as a result of a previous iteration. Alternatively, the candidate merging strategy can be an initial set of merging strategies as the starting point of the iteration. How to iterate the method will be further explained below.

[0049] Method 100 further includes evaluating S110 the merged machine learning model according to evaluation criteria. Evaluating S110 candidate merged models can provide information on how the model will perform after being formed according to the candidate merging strategy. The evaluation criteria can, for example, provide information indicating the performance of the merged model or any other suitable metric. The evaluation criteria can include multiple sub-criteria. In other words, the merged machine learning model can be evaluated S110 according to a set of evaluation criteria. Each sub-criterion (i.e., each evaluation criterion in the set of evaluation criteria) can provide information indicating different performance aspects of the merged machine learning model. The evaluation criteria (or any of its sub-criteria) can, for example, include classification metrics such as accuracy, false positive / negative rate, precision, recall, log loss, F1 score, etc. Alternatively or in combination, the evaluation criteria (or any of its sub-criteria) can include regression test metrics such as coefficient of determination (or R-squared), mean squared error, mean absolute error, etc. In the case of iteratively executing method 100, the evaluation criteria can involve comparing the merged machine learning model with the machine learning model merged in a previous iteration according to a previous candidate merging strategy. In other words, the performance of the merged machine learning model can be compared with the previously merged machine learning model of the previous iteration. The term "regression test metric" here refers to a metric obtained from a regression test procedure, which includes previous test cases of previous versions on which the merged model has been tested. For example, previous test cases on which the previous version of its merged model (i.e., formed according to a previous candidate merging strategy) performed poorly can be run to see if the updated merged model (i.e., trained with an updated candidate merging strategy) performs better. More specific methods of performing evaluation S110 will be given below. In some embodiments, the evaluation step can be run on several merged models formed in parallel by different candidate merging strategies to evaluate several models at once. This may be particularly relevant when using in-vehicle evaluation to effectively reduce the time to find the next best merged model.

[0050] Method 100 further includes updating S114 the candidate merging strategy in view of the evaluation, thereby forming an updated candidate merging strategy. In other words, the candidate merging strategy can be updated in view of the evaluation criteria. The candidate merging strategy can be updated S114 in view of the evaluation so that the updated candidate merging strategy potentially leads to better model performance. The phrase "in view of the evaluation" can be understood as being based on the satisfaction of the evaluation criteria. More specifically, if the evaluation criteria are met, the candidate merging strategy can remain as it is, while if not, the candidate merging strategy can be adjusted in any way to compensate for this. In the former case, method 100 can terminate, and the candidate merging strategy (and the candidate merged model) can be used as the final merging strategy (and the final merged model). In the latter case, the updated candidate merging strategy can be used in subsequent loops of repeating method 100.

[0051] In some embodiments, method 100 further includes obtaining metadata associated with each obtained model update at S104. The metadata may be obtained at S104 along with two or more model updates obtained at S102. Then, the step of updating the candidate merge strategy at S114 may be based on the obtained metadata. Or in other words, the merge strategy may be defined by the metadata in order to combine the local models according to their respective metadata. Using the metadata may further reduce the search space, which in turn may improve convergence and reduce the search effort required to achieve such convergence. Thus, the possible mixing weights that define the merge strategy may be set based on the metadata. Thus, a higher weight may be given to a model update that seems to be more relevant to achieving a high-performance merged model according to its metadata. In some embodiments, the metadata may also be used in the step of evaluating the merged machine learning model at S110.

[0052] The metadata may include, for example, information indicating what the corresponding local model has experienced during its training, the conditions of the training, how many new data samples were trained on, the internal system state of the vehicle during training and / or data sample collection (e.g., active or not, potential error codes, etc.). The experience of the local model may include, for example, information about driving scenarios (e.g., highway driving, city driving, traffic jam), detected objects (e.g., pedestrians, certain types of vehicles), etc. The conditions of the training may include, for example, information about where the training took place (i.e., geographical location) and / or when (e.g., time of day) (or more specifically, the set of training samples used in the training).

[0053] Figure 5 A flowchart illustrating how the merge strategy may be updated is shown by way of example. A plurality of model updates 502 (ADS_1 to ADS_N) are provided as input. Here, the model updates are shown by three boxes with different patterns, where each pattern corresponds to samples associated with certain metadata, and the width of the box indicates the number of samples associated with each metadata. A candidate merge strategy 504 is also provided. The candidate merge strategy may also be defined by the metadata and the amount of data samples for each metadata. The candidate merge strategy defined by the metadata may be converted into a set of mixing weights 506, here denoted as w_i, where i ranges from 1 to N. The set of mixing weights may include one weight associated with each model update. In block 508, based on the set of mixing weights 506, a candidate merged model (ADS_consolidated) may be formed. The candidate merged model may be formed according to the following formula:

[0054]

[0055] Then, the consolidated model ADS_consolidated can be evaluated in block 510. Based on the evaluation, the candidate consolidation strategy can be updated (as shown herein), or the consolidated model can be output as the final consolidated model.

[0056] Continuing, the candidate consolidation strategy can be updated S114 by an optimization algorithm. In other words, an optimization algorithm can be used to determine the updated candidate consolidation strategy. In other words, an optimization algorithm can be used to perform the process of determining the consolidation strategy, i.e., selecting a candidate consolidation strategy from the search space, forming a consolidated model according to the candidate consolidation strategy, evaluating the consolidated model, and updating the candidate consolidation strategy based on the evaluation. The optimization algorithm can be, for example, random search, naive random search, grid search, stochastic optimization (such as genetic algorithms), Bayesian optimization, swarm optimization, or reinforcement learning.

[0057] If the evaluation criteria are met, method 100 can be terminated, and the candidate consolidation strategy can be stored as the determined consolidation strategy for forming the final consolidated machine learning model. Then, the final consolidated model can be deployed in the fleet. Thus, method 100 can include storing the candidate consolidation strategy as the determined consolidation strategy in response to meeting the evaluation criteria (and / or convergence criteria). In other words, if the evaluation criteria are met, the candidate consolidation strategy can be kept unchanged. In other words, the updated candidate consolidation strategy can be selected as the current candidate consolidation strategy. This will be further explained below.

[0058] As described above, the steps of method 100 (denoted as S102 to S114) can be repeated for the updated candidate consolidation strategy until the evaluation criteria and / or convergence criteria for the consolidated machine learning model are met. In other words, method 100 can be repeated with the updated candidate consolidation strategy as the new candidate consolidation strategy for subsequent loops / iterations. Thus, in response to not meeting the evaluation criteria and / or convergence criteria, the step of updating S114 the candidate consolidation strategy can be performed. Otherwise, the iteration may terminate. Thus, in response to meeting the evaluation criteria and / or convergence criteria, the candidate consolidation strategy can be selected as the determined consolidation strategy for forming the consolidated model. By repeating these steps, the candidate consolidation strategy can be continuously improved until the evaluation criteria and / or convergence criteria are met. In other words, the proposed solution provides for updating the candidate consolidation strategy based on the evaluation criteria in a loop to continuously improve the candidate consolidation strategy, thereby improving the performance of the consolidated model.

[0059] In response to meeting the evaluation criteria and / or convergence criteria, method 100 can further include forming a consolidated machine learning model based on the determined consolidation strategy. Alternatively, the consolidated model of the last iteration can be selected as the consolidated model to be used in production (i.e., deployed in the fleet). Thus, method 100 can be a method for forming a consolidated model.

[0060] In response to meeting the evaluation criteria and / or the convergence criteria, method 100 may further include transmitting information indicating the combined machine learning model to one or more vehicles in the fleet. In other words, the combined model may be deployed in the fleet. The information indicating the combined machine learning model may include the model itself, or more precisely, a copy of the model. Alternatively, the information may include a model update, which includes updated model parameters. Alternatively or in combination, the information may include an incremental update for indicating changes in the model (or its parameters) compared to a previously deployed model.

[0061] In some embodiments, the evaluation criteria (by themselves) may be used as a trigger to terminate the loop. In some embodiments, both the evaluation criteria and the convergence criteria may be used as triggers to terminate the loop. Then, if both are met, or if only one of the evaluation criteria and the convergence criteria is met, the loop may be terminated. In some embodiments, the convergence criteria (by themselves) may be used as a trigger to terminate the loop. Different ways of implementing the trigger for terminating the loop may be used for different purposes. For example, if model performance is the most relevant, the evaluation criteria as a trigger may be advantageous. Conversely, if a more robust model is desired, the convergence criteria may be advantageous. If both are important, their combination may be preferred.

[0062] The convergence criteria may be any suitable convergence criteria known in the art. For example, the convergence criteria may be a threshold of a loss function. In another example, the convergence criteria may monitor changes in model parameters (such as model weights). If the changes in model parameters between iterations become smaller, this may indicate convergence. In yet another example, the convergence criteria may monitor changes in model performance on a validation set. If the changes in model performance between iterations become smaller, this may also indicate convergence. It should be noted that the above examples of convergence criteria and other possible criteria may be combined. Thus, the convergence criteria may include one or more convergence metrics. Each convergence metric may correspond to a different way of measuring convergence. When one or more convergence metrics reach their respective thresholds, the convergence criteria may be met.

[0063] Evaluating the S110 combined machine learning model may include determining S112 one or more evaluation metrics associated with the evaluation criteria. Then, the S114 candidate combination strategy can be updated based on the comparison of the one or more evaluation metrics with the corresponding thresholds. The evaluation metrics can be regarded as numerical values based on which the evaluation criteria can be formulated. The evaluation metrics can correspond to different sub-criteria of the above-mentioned evaluation criteria. When one or more evaluation criteria reach the corresponding thresholds, the evaluation criteria can be met. By evaluating the combined machine learning model based on the one or more evaluation metrics, the progress of the model over time (or iterations) can be evaluated. When method 100 traverses the space of possible candidate combination strategies, the directionality of the evaluation metrics can thus be observed. Based on this, the candidate combination strategy can then be updated.

[0064] In some embodiments, method 100 further includes applying the S108 combined machine learning model on a validation dataset. In other words, the step of evaluating the S110 combined machine learning model may include applying the combined machine learning model S108 to the validation dataset. In other words, the combined machine learning model can be run on the validation dataset. Then, the evaluation criteria can indicate the performance of the combined machine learning model on the validation dataset. The validation dataset can be selected as a pre-existing dataset. The pre-existing dataset can be stored, for example, in the server that executes the method. The pre-existing dataset can be, for example, a regulatory baseline dataset or a dataset of a set of defined test scenarios on which the model should be evaluated to meet certain regulatory requirements, such as the so-called EURO-NCAP. Therefore, the step of evaluating the S110 combined model can be executed offline on the server.

[0065] However, it should be understood that method 100 or at least a part of method 100 can be executed online in the vehicle. As described above, executing the entire method 100 in the vehicle would correspond to a decentralized federated learning scheme. As an example of a way of executing a part of method 100, one or more vehicles can execute the step (or part of the step) of evaluating the S110 combined model.

[0066] A validation dataset can be formed by obtaining validation data samples and storing the validation data samples in the validation dataset in response to the validation data samples satisfying one or more validation triggers. The validation data samples can be obtained by being collected by on-vehicle sensors of a vehicle. Then, the vehicle can check whether the validation data samples satisfy one or more validation triggers. If satisfied, the validation data samples can be stored in the validation dataset. This may be repeated until validation. A certain number of validation data samples for each validation trigger or combination of validation triggers can be collected and stored until the validation dataset conforms to the validation dataset distribution (or such that the aggregated validation dataset has a certain distribution on one or more validation triggers). If executed in a server, the validation data samples can be obtained from vehicles of a fleet that collect the validation data samples by on-vehicle sensors. Then, the combined machine learning model can be applied S108 to the validation data samples that satisfy one or more validation triggers (or belong to the validation dataset). Alternatively, the vehicle can store the validation data samples found to satisfy one or more validation triggers and / or transmit them to the server to form the validation dataset. It should be understood that the check as to whether the validation data samples satisfy one or more validation triggers (and / or the aggregation of the validation dataset to have a certain distribution on one or more validation triggers) can also be executed in the server. Thus, the vehicle can transmit any data samples or only the data samples that satisfy one or more validation triggers.

[0067] In addition, a set of validation triggers can be distributed among the vehicles of a fleet together with the combined machine learning model. Here, the set of validation triggers should be regarded as conditions indicating which data samples should be included in the evaluation. The validation triggers can be defined, for example, by metadata, such as the above-mentioned metadata. In response to the vehicle encountering a scenario (i.e., a validation data sample) that satisfies the validation trigger, the vehicle can transmit the sensor data related to the scenario to the server. Alternatively or in combination, the combined machine learning model can be evaluated (i.e., in a so-called shadow mode test) in parallel with the vehicle's operating platform in this scenario (in the vehicle). The evaluation metrics can be stored in the vehicle and transmitted to the server. Then, the server can subsequently update the candidate combined strategy based on the evaluation metrics (if needed) and form a new combined machine learning model based on the updated candidate combined strategy later.

[0068] It should be noted that, as described above, either a pre-existing validation dataset or validation data samples dynamically collected based on validation triggers can be used to evaluate S110 the machine learning model.

[0069] The executable instructions for performing these functions are optionally included in a non-transitory computer-readable storage medium or other computer program products configured to be executed by one or more processors.

[0070] Generally, a computer-accessible medium can include any tangible or non-transitory storage medium or memory medium, such as an electronic, magnetic, or optical medium, e.g., a disk or a CD / DVD-ROM connected to a computer system via a bus. The terms "tangible" and "non-transitory" as used herein are intended to describe a computer-readable storage medium (or "memory") that does not include propagating electromagnetic signals, but are not intended to otherwise limit the types of physical computer-readable storage devices encompassed by the phrase computer-readable medium or memory. For example, the term "non-transitory computer-readable medium" or "tangible memory" is intended to cover types of storage devices that do not necessarily store information permanently, including, for example, random access memory (RAM). Program instructions and data stored in a tangible computer-accessible storage medium in non-transitory form can also be transmitted via a transmission medium or signal, such as an electrical, electromagnetic, or digital signal, which can be conveyed via a communication medium, such as a network and / or a wireless link.

[0071] Figure 2 is a schematic diagram of an apparatus 200 for determining a merging strategy for forming a merged machine learning model from multiple local machine learning models. The apparatus 200 can be configured to perform the method 100 in conjunction with Figure 1 the method described above.

[0072] The apparatus 200 described herein refers to any computer system or general-purpose computing device. The apparatus 200 can be a server (which can also be referred to as a remote server, cloud server, central server, backend server, fleet server, or back-end server) or a networked device configured to provide various computing services, data storage, processing capabilities, or resources to a client or user via a communication network. In this case, the term "client" refers to a networked vehicle in a fleet (such as the vehicle 300 described below). Then, the server can be, for example, the server 602 described below in conjunction with Figure 4 the description. Alternatively, the apparatus 200 can be provided as part of a vehicle, such as the vehicle described below in conjunction with Figure 3 the description. Although the apparatus 200 is shown as one apparatus herein, the apparatus 200 can be a distributed computing system formed by multiple different computing devices.

[0073] The apparatus 200 includes a control circuit 202. The control circuit 202 can physically include a single circuit device. Alternatively, the control circuit 202 can be distributed over several circuit devices.

[0074] As Figure 2As shown in the example of, the apparatus 200 may further include a transceiver 206 and a memory 208. The control circuit 202 is communicatively coupled to the transceiver 206 and the memory 208. The control circuit 202 may include a data bus, and the control circuit 202 may communicate with the transceiver 206 and / or the memory 208 via the data bus.

[0075] The control circuit 202 may be configured to perform overall control of the functions and operations of the apparatus 200. The control circuit 202 may include a processor 204, such as a central processing unit (CPU), a microcontroller, or a microprocessor. The processor 204 may be configured to execute program code stored in the memory 208 to perform the functions and operations of the apparatus 200. The control circuit 202 is configured to perform the steps of the method 100 as described above in connection with Figure 1 The steps may be implemented in one or more functions stored in the memory 208.

[0076] The transceiver 206 is configured to enable the apparatus 200 to communicate with other entities (e.g., vehicles or other servers). The transceiver 206 can both transmit data to the apparatus 200 and receive data from the apparatus 200.

[0077] The memory 208 may be a non-transitory computer-readable storage medium. The memory 208 may be one or more of a buffer, a flash memory, a hard disk drive, a removable medium, a volatile memory, a non-volatile memory, a random access memory (RAM), and other suitable devices. In a typical arrangement, the memory 208 may include non-volatile memory for long-term data storage and volatile storage that serves as the system memory of the apparatus 200. The memory 208 may exchange data with the circuit 202 via the data bus. There may also be accompanying control lines and address buses between the memory 208 and the circuit 202.

[0078] The functions and operations of the apparatus 200 may be implemented in the form of executable logic routines (e.g., lines of code, software programs, etc.) that are stored on a non-transitory computer-readable recording medium (e.g., the memory 208) of the apparatus 200 and executed by the circuit 202 (e.g., using the processor 204). In other words, when it is mentioned that the circuit 202 is configured to perform a specific function, the processor 204 of the circuit 202 may be configured to run a portion of the program code stored on the memory 208, where the stored portion of the program code corresponds to the specific function. Additionally, the functions and operations of the circuit 202 may be a stand-alone software application or form part of a software application that performs additional tasks related to the circuit 202. The described functions and operations may be regarded as methods that the corresponding apparatus is configured to perform, such as those described above in connection with Figure 1The method 100 under discussion. Additionally, while the described functions and operations can be implemented in software, these functions can also be implemented by dedicated hardware or firmware, or by some combination of one or more of hardware, firmware, and software. In the following, the functions and operations of the device 200 are described.

[0079] The control circuit 202 is configured to obtain two or more model updates from one or more vehicles in a vehicle fleet, where each model update is the result of training a local machine learning model of a corresponding vehicle. This can be performed, for example, by executing the obtaining function 210.

[0080] The control circuit 202 is further configured to merge two or more model updates according to a candidate merging strategy, thereby forming a merged machine learning model. This can be performed, for example, by executing the merging function 212.

[0081] The control circuit 202 is further configured to evaluate the merged machine learning model according to an evaluation criterion. This can be performed, for example, by executing the evaluation function 214.

[0082] The control circuit 202 is further configured to update the candidate merging strategy in view of the evaluation, thereby forming an updated candidate merging strategy. This can be performed, for example, by executing the updating function 216.

[0083] The control circuit 202 can be configured to repeat the above steps for the updated candidate merging strategy until the evaluation criterion and / or the convergence criterion is satisfied.

[0084] The control circuit 202 can also be configured to transmit information indicating the merged machine learning model to one or more vehicles in the vehicle fleet in response to satisfying the evaluation criterion and / or the convergence criterion. This can be performed, for example, by executing the transmitting function 218.

[0085] The control circuit 202 can also be configured to obtain metadata associated with each model update. This can be performed, for example, by executing the further obtaining function 220. The updated candidate merging strategy can be based on the obtained metadata. The obtaining function 210 and the further obtaining function 220 can be implemented as a common obtaining function.

[0086] The control circuit 202 can also be configured to apply the merged machine learning model to a validation dataset. This can be performed, for example, by executing the applying function 222. Then, the evaluation criterion can indicate the performance of the merged machine learning model on the validation dataset.

[0087] It should be noted that Figure 1 the principles, features, aspects, and advantages of the method 100 described above also apply to the device 200 described herein. To avoid unnecessary repetition, please refer to the above content.

[0088] Figure 3 FIG. 1 is a schematic diagram of a vehicle 300 according to some embodiments. The vehicle 300 is equipped with an autonomous driving system (ADS) 310. As used herein, "vehicle" refers to any form of motorized vehicle. For example, the vehicle 300 can be any road vehicle, such as a car (as shown herein), a motorcycle, a (freight) truck, a bus, a smart bicycle, etc.

[0089] The vehicle 300 includes a plurality of elements common in autonomous or semi-autonomous vehicles. It should be understood that the vehicle 300 can have Figure 3 any combination of the various elements shown. In addition, the vehicle 300 can include other elements in addition to Figure 3 those shown. Although the various elements shown here are located inside the vehicle 300, one or more of these elements can be located outside the vehicle 300. In addition, even though the various elements are depicted herein in a certain arrangement, as will be readily understood by those skilled in the art, the various elements can be implemented in different arrangements. It should be further noted that the various elements can be communicatively connected to each other in any suitable manner. Figure 3 The vehicle 300 shown should be considered only an illustrative example, as the elements of the vehicle 300 can be implemented in several different ways.

[0090] The vehicle 300 includes a control system 302. The control system 302 is configured to perform overall control of the functions and operations of the vehicle 300. The control system 302 includes a control circuit 304 and a memory 306. The control circuit 302 can physically include a single circuit device. Alternatively, the control circuit 302 can be distributed over several circuit devices. As an example, the control system 302 can share its control circuit 304 with other parts of the vehicle. The control circuit 302 can include one or more processors, such as a central processing unit (CPU), a microcontroller, or a microprocessor. One or more processors can be configured to execute program code stored in the memory 306 to perform the functions and operations of the vehicle 300. The processor can be or include any number of hardware components for performing data or signal processing or for executing computer code stored in the memory 306. In some embodiments, the control circuit 304 or some of its functions can be implemented on one or more so-called systems-on-chip (SoCs). As an example, the ADS 310 can be implemented on an SoC. The memory 306 optionally includes high-speed random access memory, such as DRAM, SRAM, DDR RAM, or other random access solid-state memory devices; and optionally includes non-volatile memory, such as one or more disk storage devices, optical disk storage devices, flash memory devices, or other non-volatile solid-state storage devices. The memory 306 can include database components, object code components, script components, or any other type of information structure for supporting the various activities of this specification.

[0091] In the illustrated example, the memory 306 also stores map data 308. The map data 308 can be used, for example, by the ADS 310 of the vehicle 300 to perform autonomous functions of the vehicle 300. The map data 308 can include high-definition (HD) map data. It is contemplated that even though the memory 308 is shown as a separate element from the ADS 310, it can be provided as an integral element of the ADS 310. In other words, according to some embodiments, any distributed or local memory device can be utilized when implementing the inventive concept. Similarly, the control circuit 304 can be allocated such that, for example, one or more processors of the control circuit 304 are provided as integral elements of the ADS 310 or any other system of the vehicle 300. In other words, according to an exemplary embodiment, any distributed or local control circuit device can be utilized when implementing the inventive concept.

[0092] The vehicle 300 further includes a sensor system 320. The sensor system 320 is configured to acquire sensing data regarding the vehicle itself or its surrounding environment. The sensor system 320 can include, for example, a global navigation satellite system (GNSS) module 322 (e.g., GPS), which is configured to collect geographical location data of the vehicle 300. The sensor system 320 can further include one or more sensors 324. The one or more sensors 324 can be any type of on-board sensors, such as cameras, LIDAR and RADAR, ultrasonic sensors, gyroscopes, accelerometers, odometers, etc. It should be understood that the sensor system 320 can also provide the possibility of directly acquiring or acquiring sensing data through dedicated sensor control circuits in the vehicle 300.

[0093] The vehicle 300 further includes a communication system 326. The communication system 326 is configured to communicate with external units, such as other vehicles (i.e., via vehicle-to-vehicle (V2V) communication protocols), remote servers (e.g., as described below in connection with Figure 4Further shown are cloud servers), databases or other external devices, namely vehicle-to-infrastructure (V2I) or vehicle-to-everything (V2X, vehicle-to-everything) communication protocols. The communication system 326 can communicate using one or more communication technologies. The communication system 326 can include one or more antennas. Cellular communication technologies can be used for, for example, remote communication with a remote server or a cloud computing system. Additionally, if the cellular communication technology used has low latency, it can also be used for V2V, V2I or V2X communication. Examples of cellular radio technologies are GSM, GPRS, EDGE, LTE, 4G, 5G NR, etc., and also include future cellular solutions. However, in some solutions, short- and medium-range communication technologies, such as solutions based on wireless local area network (LAN) (such as IEEE 802.11), can be used to communicate with other vehicles near the vehicle 300 or with local infrastructure elements. The ETSI (European Telecommunications Standards Institute) is developing cellular standards for vehicle communication, and 5G is considered a suitable solution, for example, due to its low latency and efficient handling of high bandwidth and communication channels.

[0094] The communication system 326 can also provide the possibility of sending outputs via one or more antennas to a remote location (such as a remote operator or a control center). Additionally, the communication system 326 can also be configured to allow various elements of the vehicle 300 to communicate with each other. As an example, the communication system can provide local network settings, such as CAN bus, I2C, Ethernet, fiber optic, etc. Local communication within the vehicle can also be of the wireless type, with protocols such as WiFi, LoRa, Zigbee, Bluetooth or similar medium / short-range technologies.

[0095] The vehicle 300 further includes a maneuvering system 320. The maneuvering system 328 is configured to control the maneuvering of the vehicle 300. The maneuvering system 328 includes a steering module 330, which is configured to control the heading of the vehicle 300. The maneuvering system 328 further includes a throttle module 332, which is configured to control the actuation of the throttle of the vehicle 300. The maneuvering system 328 further includes a braking module 334, which is configured to control the actuation of the brakes of the vehicle 300. The respective modules of the steering system 328 can receive manual inputs from the driver of the vehicle 300 (i.e., from the steering wheel, the accelerator pedal and the brake pedal respectively). However, the maneuvering system 328 can be communicatively connected to the vehicle's ADS 310 to receive instructions on how the various modules should act. Thus, the ADS 310 can control the maneuvering of the vehicle 300.

[0096] As described above, vehicle 300 includes ADS 310. ADS 310 can be part of the vehicle's control system 302. ADS 310 is configured to perform the functions and operations of the autonomous functions of vehicle 300. ADS 310 can include multiple modules, where each module is given a different function of ADS 310.

[0097] ADS 310 can include a positioning module 312 or positioning block / system. The positioning module 312 is configured to determine and / or monitor the geographical location and heading of vehicle 300, and can utilize data from the sensor system 320, such as data from the GNSS module 322. Alternatively or in combination, the positioning module 312 can utilize data from one or more sensors 324. To improve accuracy, the positioning system can alternatively be implemented as Real Time Kinematics (RTK) GPS.

[0098] ADS 310 can further include a perception module 314 or perception block / system. The perception module 314 can refer to any known module and / or function, such as being included in one or more electronic control modules and / or nodes of vehicle 300, adapted and / or configured to interpret sensing data related to the driving of vehicle 300 to identify, for example, obstacles, lanes, relevant signs, appropriate navigation paths, etc. Thus, the perception module 314 can be adapted to rely on and obtain inputs from multiple data sources (such as automotive imaging, image processing, computer vision, and / or vehicle networks, etc.), and combine, for example, sensing data from the sensor system 320.

[0099] An object detection model (or other type of machine learning model) for detecting and / or classifying objects in the environment around vehicle 300 can be part of ADS 310, or more specifically part of the perception module 314. Such a model can be a local copy of a global model managed by a remote (fleet) server, i.e., as part of a federated learning scheme. Vehicle 300 can be configured to perform the functions and principles of method 100 as described above. These functions can be implemented in a separate computing device provided in the vehicle, such as the device 200 described above in connection with Figure 2 described. The computing device can include control circuitry configured to perform the steps of method 100 as described above in connection with Figure 1 described. Alternatively, as will be readily understood by those skilled in the art, these functions can be distributed across one or more modules, systems, or components of vehicle 300. For example, the control circuitry 304 of the control system 302 can be configured to perform some of the steps of method 100.

[0100] The positioning module 312 and / or the perception module 314 may be communicatively connected to the sensor system 320 to receive sensor data from the sensor system 320. The positioning module 312 and / or the perception module 314 may also transmit control instructions to the sensor system 320.

[0101] The ADS may further include a path planning module 316. The path planning module 316 is configured to determine a planned path of the vehicle 300 based on the perception and position of the vehicle determined by the perception module 314 and the positioning module 312, respectively. The planned path determined by the path planning module 316 may be sent to the maneuvering system 328 for execution.

[0102] The ADS may further include a decision and control module 318. The decision and control module 318 is configured to execute the control of the ADS 310 and make decisions. For example, the decision and control module 318 may decide whether the planned path determined by the path planning module 316 should be executed.

[0103] It should be understood that part of the solution may be implemented in the vehicle 300, in a system located outside the vehicle, or in a combination of inside and outside the vehicle; for example, in a server communicating with the vehicle, such as the so-called cloud solution further explained below. Different features and steps of the embodiments may be combined in other combinations than the described combinations. In addition, the elements (i.e., systems and modules) of the vehicle 300 may be implemented in combinations different from those described herein. Figure 4 The system 400 according to some embodiments is illustrated by way of example. More specifically, the system 400 may be configured to implement the principles of the present disclosure technology. Therefore, the system 400 should be regarded as a non-limiting example of the implementation of the aspects of the present technology disclosed herein. For example, the system 400 may be configured to execute the method 100 described above in connection with

[0104] Figure 4 Any feature or principle described above in connection with Figure 1 Therefore, unless otherwise specified, any feature or principle described above in connection with Figure 1 also applies to the system 400 described herein, and vice versa.

[0105] The system 400 includes a server 402 (or a remote server, a cloud server, a central server, a back-end server, a fleet server, or a back-end server), hereinafter referred to as the remote server 402 or simply as the server 402. The server may be configured as described above in connection with Figure 3The device 300 described above. Thus, any feature described in connection with the device 300 can also be applicable to the server 402. As shown, the server 402 can be set up in the cloud, i.e., a server implemented as a cloud. Advantageously, the server 402 can perform more computationally intensive tasks, or manage and coordinate a fleet (to be further described below), such as aggregating data from different vehicles in the fleet or distributing data to them.

[0106] The system 400 further includes one or more vehicles 404a-c, also referred to as the fleet 404a-c. One or more vehicles 404a-c can be the vehicles described above in connection with Figure 5 the vehicles described above. Thus, one or more vehicles 404a-c can be configured to perform the method 100 (or a part of the method 100) described in connection with Figure 1 the method 100. Figure 4 The shown fleet includes three vehicles, such as the first, second, and third vehicles 404a-c. However, the system 400 can include any number of vehicles 404a-c. Hereinafter, the first vehicle 404a will be mainly referred to, however, the same aspects also apply to the other vehicles in the fleet.

[0107] One or more vehicles 404a-c are communicatively connected to the remote server 402 to transmit and / or receive data 406 between the vehicle and the server. One or more vehicles 404a-c can also be communicatively connected to each other. The data 406 can be any type of data, such as communication signals or sensor data. More specifically, the data 406 can include training samples obtained by the vehicles 404a-c, or any information indicating a determined merging strategy, a merging model based on the merging strategy, a model update provided by the vehicle, a triggering condition for collecting which data samples, etc. The communication can be performed through any suitable wireless communication protocol. The wireless communication protocol can be, for example, a long-distance communication protocol (such as cellular communication technologies (such as GSM, GPRS, EDGE, LTE, 4G, 5G NR, etc.)) or a medium- to short-range communication protocol (for example, a solution based on wireless local area network (LAN) (such as IEEE 802.11)). The server 402 includes suitable memory and control circuitry (such as one or more processors or processing circuits) and one or more other components (such as data interfaces and transceivers). The server 402 can also include software modules or other components such that the control circuitry can be configured to execute machine-readable instructions loaded from the memory to implement the steps of the method to be performed.

[0108] Hereinafter, some methods for implementing the principles of the present disclosure technology within the system 400 will be given. However, it should be understood that these examples should not be considered as limiting, because according to specific implementations, there may be several different implementation manners. For more details about different steps, reference can be made to the above Figure 1 to avoid unnecessary repetition.

[0109] Generally, the steps of method 100 for determining a merging strategy for forming a merged machine learning model can be performed by server 402. Server 402 can then receive multiple model updates from the fleet. The model updates are the result of training local copies of the model on different vehicles. Server 402 can then form a merged model by merging the received model updates according to a candidate merging strategy. Server 402 can then evaluate the merged model according to an evaluation criterion. The merged model can be evaluated with the help of the fleet by performing the evaluation in the vehicles (e.g., via shadow mode testing) or receiving validation data samples from them (e.g., based on validation triggers transmitted to the vehicles). Alternatively, the evaluation of the merged model can be performed in server 402, e.g., using a validation data set.

[0110] The above process of system 400 should be understood as a non-limiting example of the technology of the present disclosure for better understanding. Further variations are obvious from the present disclosure and are easily implemented by those skilled in the art. For example, method 100 (or parts thereof) can be executed locally within vehicles 404a-c of the fleet.

[0111] The present invention has been described above with reference to specific embodiments. However, other embodiments besides the above embodiments are also possible and within the scope of the present invention. Within the scope of the present invention, method steps different from the above method steps can be provided, and these method steps can be executed by hardware or software. Thus, according to an exemplary embodiment, a non-transitory computer-readable storage medium is provided, which stores one or more programs configured to be executed by one or more processors of a vehicle control system, and the one or more programs include instructions for executing the method according to any one of the above embodiments. Alternatively, according to another exemplary embodiment, a cloud computing system can be configured to execute any method proposed herein. The cloud computing system can include distributed cloud computing resources that jointly execute the methods proposed herein under the control of one or more computer program products.

[0112] It should be noted that any reference numerals do not limit the scope of the claims, and the present invention can be implemented at least partially by hardware and software, and the same hardware item can represent several "devices" or "units".

Claims

1. A computer-implemented method (100) for determining a merging strategy for forming a merged machine learning model from a plurality of local machine learning models of a fleet of vehicles equipped with an automated driving system, the method (100) comprising: Obtaining ( S102 ) two or more model updates from one or more vehicles of the fleet, wherein each model update is a result of training a local machine learning model of the corresponding vehicle; Merging (S106) the two or more model updates according to a candidate merging strategy to form a merged machine learning model; Evaluating (S110) the combined machine learning model according to an evaluation criterion; as well as The candidate merge strategies are updated ( S114 ) in view of the evaluation, thereby forming an updated candidate merge strategy.

2. The method (100) according to claim 1, wherein: The steps of the method (100) are repeated for the updated candidate merge strategy until the evaluation criterion and / or convergence criterion is met.

3. The method (100) according to claim 2, further comprising: In response to satisfying the evaluation criteria and / or the convergence criteria, information indicative of the merged machine learning model is transmitted (S116) to one or more vehicles of the fleet.

4. The method (100) according to any one of claims 1 to 3, wherein: The candidate merging strategy includes a set of blending weights, wherein each blending weight in the set of blending weights is associated with a model update in the two or more model updates.

5. The method (100) of claim 1, wherein: The model update of the two or more model updates comprises updated model parameters.

6. The method (100) according to claim 1, further comprising obtaining (S104) metadata associated with each model update, and in, The candidate merge strategies are updated ( S114 ) based on the obtained metadata.

7. The method (100) of claim 1, wherein: The candidate merging strategies are updated ( S114 ) by an optimization algorithm.

8. The method (100) of claim 1, wherein: Evaluating (S110) the combined machine learning model includes determining (S112) one or more evaluation metrics associated with the evaluation criteria; and Wherein, based on the comparison between the one or more evaluation indicators and corresponding thresholds, the candidate merging strategies are updated (S114).

9. The method (100) according to claim 8, wherein: When the one or more evaluation indicators reach the corresponding threshold, the evaluation criterion is met.

10. The method (100) according to claim 1, further comprising applying (S108) the combined machine learning model on a validation dataset, and in, The evaluation criteria indicate the performance of the merged machine learning model on the validation dataset.

11. The method (100) according to claim 10, wherein the validation data set is formed by: Obtaining a sample of validation data; and In response to the verification data sample satisfying one or more verification triggers, the verification data sample is stored in the verification data set.

12. A computer program product comprising instructions which, when executed by a computing device, cause the computing device to perform the method (100) according to claim 1.

13. An apparatus (200) for determining a merging strategy for forming a merged machine learning model from a plurality of local machine learning models of a fleet of vehicles equipped with an autonomous driving system, the apparatus (200) comprising a control circuit (202), the control circuit (202) being configured to: obtaining two or more model updates from one or more vehicles in the fleet, wherein each model update is a result of training a local machine learning model for the corresponding vehicle; merging the two or more model updates according to a candidate merging strategy to form a merged machine learning model; evaluating the combined machine learning model according to an evaluation criterion; as well as The candidate merge strategy is updated in view of the evaluation, thereby forming an updated candidate merge strategy.

14. The device (200) according to claim 13, wherein: The control circuit (202) is further configured to transmit information indicative of the merged machine learning model to one or more vehicles in the fleet in response to satisfying the evaluation criteria and / or the convergence criteria.

15. The device (200) according to claim 13 or 14, wherein: The control circuit (202) is further configured to obtain metadata associated with each model update, and Wherein, the candidate merging strategy is updated based on the obtained metadata.

Images