Using machine learning to control resource utilization

Abstract

Vehicle (102), comprising: at least one computer processor (114); a non-volatile computer memory (116) that is communicatively connected to the processor (114); a plurality of machine learning agents (108), each machine learning agent (108) comprising instructions stored in the computer memory (116) and executable by at least one computer processor (114) to perform a procedure (200) to: Receiving (202) a pre-trained model (304) as an output model, wherein the pre-trained model (304) was trained for each of the machine learning agents (108) on a non-identical subset of pre-training data (300, 306); Receiving (204, 206) information (302) about an operating environment of the vehicle (102), wherein the received information (302) includes resource usage information; Modifying (204, 208) a model (304) of the machine learning agent (108) based on at least some of the received information (302); and Sending (204, 210) a prediction based on the modified model (304) of the machine learning agent (108) to an aggregator (110); and wherein the aggregator (110) comprises instructions stored in the computer memory (116) and executable by the at least one computer processor (114) to perform a procedure (400) to: Receiving (402, 404) predictions from at least some machine learning agents (108) based on their modified models (304); Apply (402, 406) at least some of the received predictions to create an updated aggregator model, wherein the aggregator (110) adopts the model of the machine learning agent (108) that most frequently makes the most frequent prediction as the updated aggregator model; and Using (402, 408) the updated aggregator model to predict and control the use of a resource in the vehicle's operating environment (102), wherein controlling the use of the resource includes controlling how a limited resource is distributed among multiple devices (104, 106, 112) that compete for that resource.

Description

introduction

[0001] A modern machine control environment (i.e., an environment that includes, for example, a motor vehicle, a production plant, or a smart home) can contain many sensors and actuators. Sensors, such as a thermometer or a camera, report aspects of their environment, while actuators, such as an air conditioner or an electric vehicle charging station, modify that environment in some way. Some devices, such as a thermostat, contain both a sensor and an actuator.

[0002] Many of these sensors and actuators, collectively referred to here as "devices," utilize resources such as electrical energy from their environment to perform their functions. Many also utilize communication resources to communicate with each other via a wired, wireless, optical, or other network. These might include, for example, Internet of Things ("IoT") devices. The proliferation of such devices can lead to competition among them for resources that are scarce in their environment.

[0003] US 2020 / 0 240 637 A1 describes an ensemble control system. This system combines various types of plant control. A variety of sub-controllers issue plant control actions based on a predictor's forecast. A combiner or switch combines or switches actions to maximize predictive or control performance as the best control action based on the actions issued by each sub-controller. The sub-controllers include at least two types. A first-type sub-controller is an optimization-based sub-controller that optimizes an objective function—a cost function to be minimized for computational actions—and issues a control action.A second-type subcontrol unit is a predictive subcontrol unit that makes predictions based on machine learning models and outputs a predicted action.

[0004] The scientific article “YUAN, Weilin [et al.]: Ensemble strategy learning for imperfect information games In: Neurocomputing, Vol. 546, 2023, Article-No. 126241, pp. 1-14. ISSN 0925-2312” describes how algorithms with multiple paradigms (such as rule-based methods, game theory, and reinforcement learning) have achieved great success in solving games with incomplete information (IIGs). According to the authors, agents based on a single paradigm tend to be vulnerable in certain aspects due to the weaknesses of that paradigm. The article first introduces three basic solvers with diversified paradigms for IIGs and then combines them to design three ensemble solvers (including an attention ensemble solver, a gradient ensemble solver, and an evolution ensemble solver) to learn ensemble strategies based on the strengths of the basic solvers.The presented methods are evaluated using Leduc Poker with non-stationary opponents and limited games. According to the authors, the results show that the presented ensemble strategy Lem method effectively integrates the advantages of various advanced single algorithms and can significantly outperform them.

[0005] The scientific article “KURENKOV, Andrey [et al.]: AC-Teach: A bayesian actor-critic method for policy learning with an ensemble of suboptimal teachers. [v3] Fri, 13 Dec 2019 00:15:26 UTC. 2019-12-13. S. 1-19.” describes how the exploration mechanism used by a deep reinforcement learning (RL) agent plays a crucial role in determining its sampling efficiency. Therefore, improvement over random exploration is critical for solving tasks with long time horizons and sparse rewards. The article proposes using an ensemble of partial solutions as teachers, guiding the agent's exploration throughout the training with suggested courses of action.While the setup of learning with teachers has been previously investigated, the proposed approach—Actor-Critic with Teacher Ensembles (AC-Teach)—is the first to work with an ensemble of suboptimal teachers who may only solve part of the problem or contradict each other, and to develop a unified algorithmic solution compatible with a variety of teacher ensembles. The described AC-Teach uses a probabilistic representation of the expected outcome of teacher and student actions to guide exploration, reduce indecisiveness, and adapt to the dynamically changing quality of the learner.A variant of AC-Teach is evaluated that guides the learning of a Bayesian DDPG agent on three tasks - path following, robotic picking and dropping, and robotic sweeping of cubes with a hook - and is shown to significantly improve sampling efficiency over a range of baselines for both our target scenario with unrestricted suboptimal teachers and simpler setups with optimal or single teachers. Description

[0006] The invention is defined by the claims.

[0007] According to certain aspects of the present disclosure, an "aggregator" controls the allocation of scarce resources under competing demands within a target machine control environment. Several machine learning agents are initiated, each with its own initial model for optimizing resource utilization, based on a pre-trained model. The machine learning agents receive resource utilization information from the target environment. They then use this information to modify their models to utilize the scarce resources more optimally. Each agent sends a prediction, based on its modified model, to the aggregator. The aggregator uses the received predictions to update its own model and uses this updated aggregator model to control the allocation of scarce resources within the target environment, at least to some extent.

[0008] Aspects of the present disclosure can be applied to a number of machine control environments, e.g., a vehicle, a dwelling, an industrial site such as a factory, a farm, a computer server installation, and even a number of personal devices (e.g., smartphone, headphones, fitness monitor, etc.) worn by one or more people.

[0009] A type of resource whose use is reportable can be controlled. While this disclosure cites electrical power and energy as examples, other resources may include cooling capacity, bandwidth of a communication channel, and computing power.

[0010] In some embodiments, the updated aggregator model is trained for a specific operator within the machine control environment. For example, if the machine control environment is a motor vehicle, the aggregator attempts to optimize the use of scarce resources in the way they are generally used by a driver. For a different driver, even if it is the same vehicle, the aggregator can create a different model based on that other driver's typical resource usage.

[0011] Machine learning is often a very slow process. To accelerate the learning of the machine learning agents, and consequently the learning of the aggregator, each agent is initialized with a pre-trained model. This pre-trained model can be based on numerous simulations run in a virtual environment, attempting to account for a range of different operator preferences. Alternatively, another type of pre-trained model can be based on numerous simulations, but these simulations are chosen to mimic specific operating characteristics of the respective operator within the environment. The various agents in an environment are typically pre-trained slightly differently from one another. This variation contributes to accelerating the overall learning of the aggregator.

[0012] The aggregator has several options for creating its updated model. For example, the aggregator compares the predictions made by the machine learning agents and selects the most frequently made prediction. Using a majority vote system, the agent that most often makes the most frequent prediction is selected by the aggregator as the best agent, and that agent's model is incorporated into the aggregator's updated model. Alternatively, the aggregator runs one agent for a specific period. The aggregator repeats this process with the other agents. After each agent has run, the aggregator selects the agent whose performance was best according to a specific metric and uses its model as the updated aggregator model.

[0013] The above-mentioned procedures can be repeated as often as desired to continuously update the aggregator model. Brief description of the drawings

[0014] While the attached claims detail the features of the present techniques, these techniques, together with their objectives and advantages, can best be understood by reference to the following detailed description in conjunction with the attached drawings, in which: Fig. 1 is a simplified diagram of a representative environment in which the techniques of the present disclosure can be applied; Fig. 2 is a flowchart of a process by which a machine learning agent learns how to better model resource utilization in its environment; Fig. 3A and Fig. 3B Data flow diagrams of a machine learning agent that receives pre-trained model data; Fig. 4 is a flowchart of a process by which an aggregator can find and apply the best model among the models submitted by multiple machine learning agents; Fig. 5 is a data flow diagram of a machine learning control system that interacts with its environment; Fig. 6 is a schematic representation of an exemplary machine learning control system and its connection to its environment, implemented using deep reinforcement learning with two fully connected neural networks; and Fig. Figure 7 is a diagram showing multiple machine control environments, multiple machine learning control systems, and the model selection process.

[0015] The drawings are not necessarily to scale and may show simplified representations of various features of the present disclosure. The details associated with these features are partly determined by a specific intended application and operating environment. Detailed description

[0016] The increasing number of connected devices leads to competition for limited resources. In a server farm, for example, each additional server consumes power, communication bandwidth, and cooling capacity. IT specialists therefore design the server farm taking this competition for resources into account and, from their central control position, update the resource allocation procedures as the farm grows.

[0017] In other examples, the competition isn't so readily apparent. Several IoT devices can be casually added to a machine control environment that isn't carefully monitored by the IT department. Consider, for instance, a homeowner who installs a wireless security camera. Because the camera is wireless, it doesn't strain the smart home's power resources, but it does consume some communication bandwidth, which could cause increasingly annoying and unpredictable problems as other devices are added to the house, competing with it for bandwidth.

[0018] As a final example used in this discussion, consider a motor vehicle. Modern vehicles, even gasoline-powered ones, require significant amounts of electrical energy from the vehicle's limited power generation capacity. As with the house example, this demand increases when additional features are added. Since many of these devices make their demands without coordinating with other devices, in the worst-case scenario, resources for "secondary" purposes, such as playing the radio or using the vehicle to power an air compressor, can deplete the electrical reserves to such an extent that the "primary" goal of propelling the vehicle on the road is compromised. The scarcity of electrical energy can be exacerbated if the vehicle is electrically powered.

[0019] To counteract this possibility, aspects of the present disclosure monitor resource utilization within a machine control environment, learn from this monitoring to predict future levels of resource utilization, and use the results of this learning to more effectively balance competing resource demands.

[0020] To examine these aspects in detail, let us consider Fig. Figure 1, which shows an exemplary machine control environment 100 focusing on the use of electrical resources in a vehicle 102, is referenced. The vehicle 102 contains many devices, such as sensors 104 and actuators 106, which require resources from the vehicle 102 to function. As explained in detail in the accompanying text to the other figures, machine learning agents 108 and an aggregator 110 jointly coordinate how the devices 104 / 106 utilize the limited resources provided by the vehicle 102.

[0021] The devices to be controlled can extend beyond the sensors 104 and the actuators 106 that are actually located in the vehicle 102. To illustrate these "external" devices 112, Figure 112 shows... Fig. 1 an electric charging station 112 and a household AC power supply 112. Normally, the AC power 112 would be supplied via the local power grid (not shown), but under certain circumstances, e.g., a power outage on a particularly hot day, the homeowner can operate the AC power 112 via the battery of the vehicle 102. In this case, the household AC power 112 competes with the devices 104 / 106 in the vehicle 102 for limited power resources, and this competition can be coordinated, according to aspects of the present disclosure, by the machine learning agents 108 working in conjunction with the aggregator 110.

[0022] So far, the discussion has focused on resource consumption. To effectively control this consumption, aspects of the present disclosure, in some embodiments, also monitor current resource levels, such as the state of charge of the vehicle battery, and resource replenishment. In general, the electric charging station 112 does not literally consume resources from the vehicle 102, but rather replenishes these resources by charging the vehicle battery. Therefore, the machine learning agents 108 / the aggregator 110 can use the information that the electric charging station 112 is connected to the vehicle 102 and how quickly it is recharging the battery when coordinating competing resource demands.

[0023] In some embodiments, the machine learning agents 108 / the aggregator 110 are supported by a computer architecture that is in Fig. 1 is illustrated by a computer processor 114 and a computer memory 116. Although this computer architecture 114 / 116 is shown inside the vehicle 102, for the sake of simplicity it can be located anywhere: inside the vehicle 102, as in Fig. 1 is shown, in a local computer that is communicatively connected to the vehicle 102, or in a computer network cloud. To avoid a lengthy interruption of the narrative flow, further aspects of the computer architecture 114 / 116 will be discussed at the end of this detailed description.

[0024] The discussion now focuses on the machine learning agents 108 and the aggregator 110. Fig. Figure 2 shows an exemplary method 200 which in some embodiments can be used by the machine learning agents 108, while Fig. 4 represents the aggregator 110. The discussion on the Fig. 5, Fig. 6 to Fig. Figure 7 then shows how the machine learning agents 108 and the aggregator 110 work together as one system.

[0025] With reference to Fig. 2 In a typical embodiment, the method 200 is applied while a specific operator is working in the machine control environment 100. The method 200 is executed again for each prospective operator.

[0026] Leaving step 202 aside for the moment, step 204 is a loop that can be repeated indefinitely.

[0027] In the first step 206 of loop 204, several machine learning agents 108 receive information about the current status of resource utilization within the machine control environment 100. Returning to the example of the vehicle 102, this information can include which devices 104 / 106 / 112 are currently consuming or replenishing electrical resources. Information about the timing of this utilization is also collected, which can be used to predict historical trends in resource utilization for a specific operator. The machine learning agents 108 can gather information about the current driver's typical driving style, which is important for predicting both resource utilization (especially if the vehicle 102 is electrically powered) and resource replenishment (e.g., for an electrically powered vehicle 102 recharging its battery).Other operator-specific information includes the likelihood that the operator is operating the vehicle or the AC system 112 at home given the outside temperature and humidity, how long the operator is likely to operate the AC system, how long the operator typically parks while powering the devices 104 / 106 / 112 or a subset thereof, and similar information.

[0028] In step 208, the machine learning agents 108 apply machine learning techniques to modify their internal models of resource utilization when a particular operator is active in the machine control environment 100. Details of this machine learning are discussed below. To summarize this discussion for some embodiments, this learning involves using the data received in step 206 (and in earlier iterations of step 206, as loop 204 repeats), processing this data through an internal model to generate a prediction of future resource utilization, receiving feedback on how well this prediction matches reality, and "optimizing" the agent's internal model to bring its future predictions closer to reality.

[0029] In step 210, the predictions made by the machine learning agents 108 are sent to the aggregator 110, whose operation is described below with reference to Fig. 4 is explained.

[0030] It should be noted that in the discussion about Fig. Section 2 refers to "machine learning agents 108" in the plural. While it is true that machine learning can take place with only one machine learning agent 108, aspects of the present disclosure tend to use multiple machine learning agents 108 in parallel. This parallelization, in conjunction with the aggregator 110, significantly accelerates the learning process and makes embodiments of the present disclosure more responsive for the operator of the machine control environment 100. In some embodiments applicable to certain machine control environments 100, each resource (e.g., electrical power, communication bandwidth, cooling) is modeled by its own set of a few parallel machine learning agents 108.

[0031] At this point, the discussion returns to the first step, 202, of procedure 200. One problem with many machine learning procedures is that they use tiny, incremental steps when improving their model. As discussed in connection with step 208, for example, a machine learning agent 108 identifies the difference between its prediction and the actual outcome and "optimizes" its environmental control model so that it moves only slightly toward the actual outcome. Because of these tiny steps, this learning process makes the agent's convergence to a near-optimal model a very slow process. This slowness is useful to prevent the model from taking too large a step and thus "overshooting" the target, missing the best possible configuration.It also helps ensure that the model is robust in very different situations and not just optimal for the specific situations that Agent 108 saw and reacted to.

[0032] While the reasons mentioned above support slow learning, the slowness itself is not an advantage. That is, if the machine control environment 100 changes slightly due to the addition of a new device 104 / 106 / 112, or if a particular operator changes their behavior for any reason, a slow-learning system consisting of machine learning agents 108 and an aggregator 110 may react so slowly to these changes that it can no longer keep up and, at best, becomes worthless.

[0033] One method for accelerating learning has already been described above: using several machine learning agents 108 in parallel. Another method is the reason for step 202. Here, each machine learning agent 108 does not begin learning with a "blank slate", but is initialized with a pre-trained model that is at least reasonably suitable for the respective task.

[0034] Fig. 3A and Fig. Figure 3B shows these pre-trained models. In some embodiments, each machine learning agent 108 is first pre-trained with pre-training data 300 developed in a virtual environment. This pre-training data 300 is generated by running several scenarios that appropriately mimic the expected resource utilization in the target machine control environment 100. These simulations can cover many scenarios and reflect the expected behaviors of a range of virtual operators anticipated in the environment 100. In a standard example, several operators of a vehicle 102 are simulated in various driving and parking situations. These simulations are passed to the machine learning agent 108, which updates its model in the same way as it will later with the "live" data 302.The resulting model is improved based on learning from hundreds or thousands of simulated driving and parking hours. Therefore, when this pre-trained model is combined with the "live" learning data 302 (the focus of step 206 in . Fig. 2) When combined, the machine learning agent 108 starts with a reasonable, albeit operator-independent, model 304, and this model 304 matures much faster than it could without the pre-training data 300.

[0035] If some data already exists about the behavioral characteristics of a particular operator, whose prediction the machine learning agent 108 is trying to learn, then Fig. 3B goes a step further than the pre-training data from Fig. 3A. Here too, simulations of the machine control environment 100 are performed, this time based on the characteristics of virtual operators assumed to have similar characteristics to the target operator. Again, these simulations are used by the machine learning agent 108 to update its internal model. Since the machine learning agent 108 uses both the “generic operator” training data 300 from Fig. Since the machine learning agent model is pre-trained with both 3A and the more specific operator data 306, it begins by closely approximating the target operator and then improves its already close model with "live" data 302 (as described above with reference to Fig. 2 discussed).

[0036] To create a variety of machine learning agents 108 that increases the overall learning rate of the combined system of machine learning agents 108 and aggregator 110, the machine learning agents 108 are not pre-trained with exactly the same data 300 / 306. Instead, each machine learning agent 108 is pre-trained with a subset of the pre-training data 300 / 306. Thus, according to the aspects of the present disclosure, both the pre-training itself and the pre-training for creating a variety of machine learning agents 108 are useful tools for improving the learning rate, regardless of whether the combined control system 108 / 110 is learning about this one specific operator for the first time or whether it is modifying its learning to adapt to new features of the machine learning environment 100 or of the one specific operator.The use of various machine learning agents 108 can also improve the stability of the combined control system 108 / 110 in view of changes in the machine control environment 100.

[0037] In some embodiments, the aggregator 110 performs the method 400 of Fig. 4 through. The loop from step 402 is repeated infinitely.

[0038] In step 404, the aggregator receives 110 predictions from one or more machine learning agents. These are the predictions derived from the internal models of the machine learning agents in step 210. Fig. 2 were created.

[0039] In step 406, the aggregator 110 uses at least some of the received predictions to update its own model for controlling resource utilization within the machine control environment 100. In different embodiments, the aggregator 110 uses different techniques to update its model. In one technique, the aggregator 110 compares the predictions received from the machine learning agents 108 and selects the most frequently made prediction. Using a majority voting system, the machine learning agent 108 that most frequently makes the most frequent prediction is selected by the aggregator 110 as the best machine learning agent 108, and the model of this machine learning agent is adopted as the updated aggregator model.

[0040] In another technique, the aggregator 110 runs a machine learning agent 108 for a specific period of time. The aggregator 110 repeats this with the other machine learning agents 108. After the machine learning agents 108 have been executed, the aggregator 110 selects the machine learning agent 108 whose prediction performance was best according to a specific metric and uses that machine learning agent's model as the updated aggregator model.

[0041] The aggregator 110 uses its updated model in step 408 to control how limited resources are distributed among the devices 104 / 106 / 112 that compete for those resources. For example, if the state of charge of an electric vehicle's battery is slowly decreasing, but the aggregator's updated model predicts that, based on the historical behavior of that particular operator, the vehicle 102 is likely to have to travel a considerable distance soon, the aggregator 110 can conserve resources by rejecting resource requests from some of the devices 104 / 106 / 112 or by granting them less than the requested amount. In some embodiments, the aggregator 110 can inform the operator of the status of the monitored resource, allowing the operator, for example, to go to the electric charging station 112.

[0042] By repeating the loop of step 402, the combined control system 108 / 110 learns better about the behaviors of the respective operator and approaches the optimization of resource utilization within the machine control environment 100 to support these behaviors.

[0043] In some embodiments, the combined control system 108 / 110 can be implemented with "reinforcement learning". This reinforcement learning is schematically represented by the data flows in Fig. Figure 5 illustrates this. The 108 / 110 control system operates in its own "world" 500. As explained above in the text. Fig. As described in Figure 1, the machine control environment 100 provides the machine learning agents 108 with information 302 from the sensors 104. Furthermore, the environment 100 uses a reward 502 to inform the control system 108 / 110 how well its environment control model is performing. The reward 502 can be positive or negative. The control system 108 / 110 takes the reward 502 into account when adapting its model and uses its adapted model to control aspects of the environment 100 by controlling the actuators 106 (step 408 in Figure 1). Fig. 4) The cycle of the control system, in which it receives environmental information 302 and rewards 502 and improves its model for controlling the environment, is repeated indefinitely.

[0044] In more detail, in some embodiments, the control system 108 / 110 attempts to maximize the rewards 502 it receives over time. There are several ways to do this. One way is to maximize the rewards 502 using a discounted return: Gt≅∑i=0∞γiRt+i+1 where y is the discount rate and R is the reward at a given time. The rewards 502 are intended to induce the control system 108 / 110 to behave in a manner deemed advantageous, such as improving resource utilization efficiency and operator comfort. Conversely, negative rewards 502 are given to penalize undesirable behavior of the control system 108 / 110, such as attempting to use the electric charging station 112 when the vehicle 102 is either fully charged or unplugged, or consuming excessive amounts of a monitored resource. In other examples, the use of an electrical resource may incur at least a small negative reward 502, thus inducing the control system 108 / 110 to charge the battery pack when the vehicle 102 is not in use.The 108 / 110 control system is also encouraged to maintain a sufficient charge level for the driver's usual needs, but also a little more to ensure a comfortable margin and thus reduce range anxiety.

[0045] This code snippet shows a concrete example of how rewards can be calculated in reinforcement learning:

[0046] This is a clear example, and specific reward mechanisms can be developed specifically for each machine control environment.

[0047] The in Fig. The techniques of reinforcement learning described in section 5 and in the accompanying text can be implemented using a pair of neural networks, as in Fig. Figure 6 is shown and implemented. Here, the control system 108 / 110 is referred to as the "actor". Information 302 about the current status of the machine control environment 100 is transmitted to the control system 108 / 110 on the left side of Fig. 6. As with many deep neural networks, this information is processed through layers of weights to create a list of possible outputs. One of these options is selected and becomes the action that control system 108 / 110 uses to control an aspect of environment 100.

[0048] In the Fig. In the embodiment shown in Figure 6, when the action is received by the machine control environment 100 (the “Critic”), it is fed into another neural network, processed by the weights in that network, and a value is generated. This value can be combined with the reward 502 from Fig. 5 must be identical and is fed back as further input to the control system 108 / 110. If this process continues, a learning process is achieved when the weights in each neural network are adjusted.

[0049] Fig. 7 places the above-mentioned aspects in a broader context. 700. On the left side of Fig. Figure 7 shows illustrations from the wide variety of machine control environments 100. This variety is initially managed by a categorization 702 of the environments 100. Each category may require its own specific adaptations to the aspects of this disclosure in order to best align these aspects with the requirements of that category or each specific environment 100.

[0050] Several pre-training "worlds" 500 are set up. In each of these worlds, a control system 108 / 110 is pre-trained in a unique way. Then, the various controllers 108 / 110 are configured to operate in the selected machine control environment 100, receive status information 302 from the sensors 104 in the environment 100, and update their internal predictive models. At regular intervals, the performance of the various control systems 108 / 100 is compared, and the best one 704 is selected to control the resources within the environment 100.

[0051] As reinforcement learning progresses, each model improves, and the selection process for the best model is repeated. The unselected models can sometimes be "revived": if the circumstances in the machine control environment change, one of the unselected models might perform better than the best model before the change. In this case, the previously unselected model becomes the selected model that controls the resources, and learning continues from there.

[0052] Back to the computer processor 114 and the computer memory 116 of Fig.1. Together, they constitute a computer architecture that can support the control system of the machine learning agents 108 and the aggregator 110. In particular, the computer processor 114 can comprise one or more computer processors located locally within the machine control environment 100, remotely, as in a cloud computing scenario, or working together in a combination of these configurations. The computer memory 116 can also be local, remote, or a combination thereof. The computer processor 114 and the computer memory 116 can be connected via a local bus or via a wired, wireless, or optical communication system. Other devices, including in some cases the devices 104 / 106 / 112, can be communicatively connected to the computer processor 114 and the computer memory 116.The software running on the computer processor 114 and stored in the computer memory 116 includes an operating system and the code specific to the control system 108 / 110.

Claims

[1] Vehicle (102), comprising: at least one computer processor (114); a non-volatile computer memory (116) that is communicatively connected to the processor (114); a plurality of machine learning agents (108), each machine learning agent (108) comprising instructions stored in the computer memory (116) and executable by at least one computer processor (114) to perform a procedure (200) to: Receiving (202) a pre-trained model (304) as an output model, wherein the pre-trained model (304) was trained for each of the machine learning agents (108) on a non-identical subset of pre-training data (300, 306); Receiving (204, 206) information (302) about an operating environment of the vehicle (102), wherein the received information (302) includes resource usage information; Modifying (204, 208) a model (304) of the machine learning agent (108) based on at least some of the received information (302); and Sending (204, 210) a prediction based on the modified model (304) of the machine learning agent (108) to an aggregator (110); and wherein the aggregator (110) comprises instructions stored in the computer memory (116) and executable by the at least one computer processor (114) to perform a procedure (400) to: Receiving (402, 404) predictions from at least some machine learning agents (108) based on their modified models (304); Apply (402, 406) at least some of the received predictions to create an updated aggregator model, wherein the aggregator (110) adopts the model of the machine learning agent (108) that most frequently makes the most frequent prediction as the updated aggregator model; and Using (402, 408) the updated aggregator model to predict and control the use of a resource in the vehicle's operating environment (102), wherein controlling the use of the resource includes controlling how a limited resource is distributed among multiple devices (104, 106, 112) that compete for that resource. [2] Vehicle (102) according to claim 1, wherein the resource is selected from the group consisting of: electrical power, electrical energy, cooling, communication bandwidth and computer processing power. [3] Vehicle (102) according to claim 1, wherein the method is carried out by the plurality of machine learning agents (108) while a particular operator is operating the vehicle (102), and wherein the updated aggregator model is assigned to that particular operator. [4] Vehicle (102) according to claim 3, wherein the pre-trained model of each machine learning agent (108) is created on the basis of simulations of a plurality of virtual operators of the vehicle (102). [5] Vehicle (102) according to claim 3, wherein the pre-trained model of each machine learning agent (108) is created based on a simulation of a virtual operator of the vehicle (102), the operating characteristics of which are chosen to resemble those of a specific operator. [6] Vehicle (102) according to claim 1, wherein the application of at least some of the received predictions comprises creating an updated aggregator model: for each of the multitude of machine learning agents (108), the execution of this machine learning agent (108) in the operating environment of the vehicle (102) for a period of time; Evaluating the performance of each machine learning agent (108) over its time span; and Creating the updated aggregator model as an updated model of a machine learning agent (108) that performed best in its time span. [7] System configured to operate in a machine control environment (100), the system comprising: at least one computer processor (114); a non-volatile computer memory (116) that is communicatively connected to the computer processor (114); a plurality of machine learning agents (108), each machine learning agent (108) comprising instructions stored in the computer memory (116) and executable by at least one computer processor (114) to perform a procedure (200) to: Receiving (202) a pre-trained model as an output model, wherein the pre-trained model (304) for each of the machine learning agents (108) was trained on a non-identical subset of pre-training data (300, 306); Receiving (204, 206) information (302) about the machine control environment (100), wherein the received information includes resource utilization information; Modifying (204, 208) a model of the machine learning agent (108) based on at least some of the received information (302); and Sending (204, 210) a prediction based on the modified model of the machine learning agent (108) to an aggregator (110); and wherein the aggregator (110) comprises instructions stored in the computer memory (116) and executable by the at least one computer processor (114) to perform a procedure (400) to: Receiving (402, 404) predictions from at least some machine learning agents (108) based on their modified models; Apply (402, 406) at least some of the received predictions to create an updated aggregator model, wherein the aggregator (110) adopts the model of the machine learning agent (108) that most frequently makes the most frequent prediction as the updated aggregator model; and Using (402, 408) the updated aggregator model to predict and control the use of a resource in the machine control environment (100), wherein controlling the use of the resource includes controlling how a limited resource is distributed among multiple devices (104, 106, 112) that compete for that resource. [8] Aggregator (110) configured to operate in a machine control environment (100) comprising at least one computer processor (114) and non-volatile computer memory (116) communicatively connected to the computer processor (114), wherein the aggregator (110) comprises: Instructions stored in the computer memory (116) and executable by at least one computer processor (114) to perform a procedure (400) for: Receiving (402, 404) predictions from a variety of machine learning agents (108) based on their modified models, wherein the modified models are modified starting from pre-trained models (304) as the initial model, wherein the pre-trained model (304) for each of the machine learning agents (108) was trained on a non-identical subset of pre-training data (300, 306); Apply (402, 406) at least some of the received predictions to create an updated aggregator model, wherein the aggregator (110) adopts the model of the machine learning agent (108) that most frequently makes the most frequent prediction as the updated aggregator model; and Using (402, 408) the updated aggregator model to predict and control the use of a resource in the machine control environment (100), wherein controlling the use of the resource includes controlling how a limited resource is distributed among multiple devices (104, 106, 112) that compete for that resource.

Images