Method, device and system for network configuration

Abstract

Methods, devices, and systems for network configuration are provided. A computer-implemented method for modifying a configuration of a communication network using an ML agent includes receiving first and second network modification suggestions from a suggestion agent. The method further includes generating a third suggestion using the first and second suggestions. The method further includes predicting, using an analog communication network, an effect of the suggested implementation on a state of the communication network, wherein the analog communication network simulates the state of the communication network. The method further includes selecting one of the suggestions based on the predicted implementation impact, and in the event that third suggestions are selected, determining respective weights of first and second ones of the third suggestions. The method further includes initiating a modification of the communication network based on the selected network modification suggestion.

Description

Technical Field

[0001] The embodiments described herein relate to methods and apparatus for network configuration, and particularly for using machine learning (ML) agents to assist in the configuration of communication networks. Background Technology

[0002] Networks such as the 3GPP 5G network, the 3GPP 6G network, and networks conforming to the IEEE 802.11 standard (which may be referred to as WiFi networks) can be used to provide communication coverage of an environment. The environment can be a specific volume of space, including, for example, all or part of a country.

[0003] Any suitable type of transceiver can be used to supplement or provide network coverage. The type of transceiver can be determined based on the nature of the network; for example, in the case of a WiFi network, a suitable WiFi transceiver can be provided. In the case of a 5G or 6G network, a corresponding base station can be provided. It may be necessary or desirable to change the configuration of one or more components within the communication network to provide the desired network capabilities. For example, when the communication traffic carried by the network increases, it may be desirable to provide additional network capacity (by increasing the number of active base stations, refocusing base stations, etc.). The performance of some networks can be monitored using key performance indicators (KPIs) including latency, quality of experience (QoE), packet loss rate, etc. In the case of using KPI monitoring, network configuration changes can be used to ensure that one or more KPIs do not drop below a predetermined threshold. The predetermined threshold can be set, for example, by an internal network supervisor (which can be a human or software supervisor) or by the network operator. The predetermined threshold can be set by an intent, that is, the intent can include a target value (or target) for one or more KPIs. The process of controlling and (if necessary) changing the configuration of the network can be called network orchestration. Intent-based networking is an example of network orchestration. In some systems that implement intent-based networking, machine learning (ML) can be used to automate aspects of the process of translating intents into changes to the network. Intent-based network orchestration can combine rule-based policies, shared knowledge objects, and logical partitioning to create a sustainable, automated framework for achieving business requirements within the network.

[0004] Some existing rule-based intent networking uses intents and orchestration frameworks to implement requirements on the network. Agents can be used to generate suggestions that will be implemented on the network and report the network's response to those suggestions. Closed-loop monitoring can be used to ensure that the system receives feedback from the network and reporting agents. Automation and analytics can also be used to enable faster threat detection and reduced downtime. Kim, T. and Abed, E. H.'s "Closed-loop monitoring systems for detecting impending instability" (IEEE Transactions on Circuits and Systems, available as of March 13, 2023 at https: / / ieeexplore.ieee.org / document / 886978) discusses how closed-loop monitoring can be configured to ensure that the system receives feedback from the network and reporting agents. Typically, in existing systems, reports are provided at a high frequency, which can be computationally intensive. Problems can also occur in existing systems when multiple intents are received, especially in situations where resolvable network changes conflict. A system is desired that addresses multiple intents with potentially conflicting modifications that need to be implemented. Summary of the Invention

[0005] One object of this disclosure is to provide methods, apparatus, and computer-readable media that at least partially solve one or more of the problems discussed above. In particular, one object of this disclosure is to support the efficient implementation of network modifications to achieve the intended solution. Saving network resources by avoiding unnecessary evaluation is also an object.

[0006] This disclosure provides methods, apparatus, and systems for modifying the configuration of a communication network. Embodiments can efficiently achieve intents by combining suggestions from multiple agents so that they can be applied together. Intents from agents that may require sequential actions can be integrated for execution within the network, thereby increasing the general applicability of the embodiments. Embodiments can additionally or alternatively save computational resources (memory, processor time, etc.) by omitting one or more evaluations of the network.

[0007] An embodiment provides a computer-implemented method for modifying the configuration of a communication network using an ML agent. The method includes receiving a first network modification proposal from a first recommendation agent and a second network modification proposal from a second recommendation agent. The method further includes generating a third network modification proposal using the first and second network modification proposals. The method also includes using a simulated communication network to predict the impact of the implementation of the first, second, and third network modification proposals on the state of the communication network, wherein the simulated communication network simulates the state of the communication network. The method further includes selecting one of the network modification proposals based on the predicted implementation impact, and, if the third network modification proposal is selected, determining a corresponding weighting of the first and second network modification proposals in the third network modification proposal. The method further includes initiating a modification of the communication network based on the selected network modification proposal.

[0008] Another embodiment provides an ML agent configured to modify the configuration of a communication network using an ML model. The ML agent includes a processing circuit module and a memory containing instructions executable by the processing circuit module. The ML agent is operable to receive a first network modification proposal from a first suggestion agent and a second network modification proposal from a second suggestion agent. The ML agent is further operable to generate a third network modification proposal using the first and second network modification proposals. The ML agent is also operable to use a simulated communication network to predict the impact of the implementation of the first, second, and third network modification proposals on the state of the communication network, wherein the simulated communication network simulates the state of the communication network. The ML agent is further operable to select one of the network modification proposals based on the predicted implementation impact, and, if the third network modification proposal is selected, to determine a corresponding weighting of the first and second network modification proposals in the third network modification proposal. The ML agent is further operable to initiate modifications to the communication network based on the selected network modification proposal.

[0009] The following describes embodiments of this disclosure. The scope of this disclosure is defined by the claims. Attached Figure Description

[0010] This disclosure is described by way of example only, with reference to the following figures, in which: - Figure 1 This is a flowchart of the method according to the embodiment; Figure 2A This is a schematic diagram of an ML agent according to an embodiment; Figure 2B This is a schematic diagram of an ML agent according to an embodiment; Figure 3This is a schematic diagram of a typical RL system; Figure 4 These are schematic diagrams illustrating examples of systems according to some embodiments; and Figure 5A and Figure 5B This is a result graph illustrating the impact of an example implementation on the time taken to achieve a KPI target. Detailed Implementation

[0011] For illustrative purposes, details are set forth in the following description to provide a thorough understanding of the disclosed embodiments. However, those skilled in the art will readily recognize that embodiments may be implemented without such specific details or with equivalent arrangements.

[0012] An embodiment provides a system for predicting the potential impact of multiple proposals for achieving multiple intentions on a communication network, and then selecting a proposal to implement based on the predicted effects of the proposals, and initiating modifications to the communication network based on the selected proposal.

[0013] The method according to the embodiment is... Figure 1 Illustration, Figure 1 This is a flowchart illustrating a method for modifying the configuration of a communication network using an ML agent. The computer-implemented method can be executed by any suitable device, for example, by a device such as... Figure 2A and Figure 2B The ML agents 20A and 20B shown in Figure 2 (collectively referred to as ML agents) perform the actions. When the network is a communication network or a WiFi network (or a portion thereof), the method may be performed by a device (such as an ML agent) that is or forms part of a network node, such as a base station or core network node (or may be incorporated into a base station or core network node). The communication network may be, for example, a 5G or 6G network. In some embodiments, the ML agent may form part of a transceiver or a transceiver controller. In some embodiments, the ML agent may be used for multiple steps, and accordingly, the ML agent used may be one (or more) of one or more advisory agents, agents managing predictive ML models, agents managing weighted ML models, and agents managing implementations of ML models. Various agents and ML models are discussed in more detail below.

[0014] As in Figure 1 As shown in S102, the method includes receiving a first network modification suggestion from a first suggestion agent and receiving a second network modification suggestion from a second suggestion agent. The steps of receiving the first and second network modification suggestions can be executed according to a computer program stored in memory 23 and executed by processor 21 in conjunction with one or more interfaces 22 of ML agent 20A, such as by... Figure 2AAs illustrated. Alternatively, the steps of receiving the first and second network modification suggestions can be performed by the transceiver 24 of the ML agent 20B, as shown. Figure 2B As shown in the illustration. In some embodiments, the recommendations may be generated by first and second recommendation agents based on first and second intentions, respectively; the intentions may be received from any suitable source such as an internal network orchestrator or an external customer request. The recommendations may be received, for example, by ML agent 20, as discussed above. Additional recommendations may also be received (e.g., a third recommendation, a fourth recommendation, etc.) and processed as discussed below; however, for brevity, embodiments in which two recommendations are received are discussed here. Some embodiments may include a system comprising some or all of the recommendation agents that generate recommendations and an ML agent that receives (one or more) the recommendations. In some embodiments, the recommendation agents may generate network modification recommendations based on the expected KPIs of the communication network; for example, an intention to reduce latency received by the recommendation agent may cause the recommendation agent to generate recommendations (to address the intention) that adjust various network configuration options (activating some or all of one or more base stations, adjusting the antenna tilt settings of one or more base station antennas, etc.). In some embodiments, the recommendation agents may use rule-based agents or ML models or a combination of these or any other suitable techniques to generate recommendations.

[0015] In the case of one or more recommendation agents that generate recommendations, these ML models can be trained in any suitable manner. An example of a suitable training method is reinforcement learning (RL). RL allows ML models to learn by attempting to maximize the expected cumulative reward of a series of actions that utilize trial and error. RL agents (i.e., systems that use RL to improve the performance of ML models in a given task over time) can be closely associated with the system (environment) they are used to model / control and learn through the experience of performing actions that change the state of the environment. Additionally or alternatively, RL can be used to train ML models before they are used to control the actual environment, where simulation of the environment is used; this option can be useful when it is desirable to avoid having an untrained or partially trained ML model control the actual operating environment.

[0016] Figure 3 A schematic diagram of a typical RL system is shown. Figure 3In the architecture shown, the agent receives and transmits actions from and to the environment it is used to model / control (which can be a real or simulated environment). For time t, the agent receives information about the current state of the environment St. The agent then processes the information St and generates one or more actions to be taken; one of these actions is implemented At. The action At to be implemented is then transmitted back to the environment and implemented. The result of action At is a change in the state of the environment over time, such that at time t+1, the state of the environment is St+1. The action also produces a reward (numerical, typically scalar) Rt+1, which is a measure of the impact of action At that caused the environment state St+1. The changed state of environment St+1 is then transmitted from the environment to the agent along with the reward Rt+1. Figure 3 The diagram shows that reward Rt is sent to the agent along with state St; reward Rt is the reward generated for action At-1 performed on state St-1. When the agent receives information St+1, this information is processed in conjunction with reward Rt+1 to determine the next action At+1, and so on. The actions to be performed are selected by the agent from its available actions to maximize the cumulative reward. RL can provide a powerful solution to the problem of optimal decision-making for agents interacting with uncertain environments.

[0017] In some embodiments, the first and second recommendation agents may generate network modification recommendations based on anticipated KPIs of the communication network. Any suitable KPIs familiar to those skilled in the art may be included in the intent; examples include quality of experience (QoE) metrics, packet loss measurements, latency values, and so on.

[0018] When the first and second network modification suggestions have been received, the method continues to generate a third network modification suggestion using the first and second network modification suggestions, such as... Figure 1 As shown in step S104. The step of generating the third network modification proposal can be executed by a computer program stored in memory 23 and executed by processor 21 in conjunction with one or more interfaces 22 of ML agent 20A, such as by... Figure 2A Illustration. Alternatively, the step of generating the third network modification proposal can be performed by generator 25 of ML agent 20B, as shown. Figure 2B As shown in the diagram, the third network modification suggestion is generated using the first and second network modification suggestions; in some embodiments, the third network modification suggestion is generated by combining the first and second network modification suggestions. Alternatively, the third network modification suggestion can be obtained by analyzing the first and second network modification suggestions and generating the third network modification suggestion based on this analysis. When the third network modification suggestion is generated by combining the first and second network modification suggestions, the third network modification suggestion may be referred to as the combined network modification suggestion.

[0019] After generating a third network modification proposal (which, as discussed above, may be a combination of network modification proposals), the method then proceeds to the step of predicting the impact of each of the network modification proposals on the state of the communication network, such as... Figure 1 As shown in step S106. The step of predicting the impact of each proposal in the network modification proposals can be executed by a computer program stored in memory 23 and executed by processor 21 in conjunction with one or more interfaces 22 of ML agent 20A, such as by... Figure 2A Illustration. Alternatively, the step of predicting the impact of each proposal in the network modification proposals can be performed by the predictor 26 of the ML agent 20B, as shown. Figure 2B As shown in the image.

[0020] The predictions are made using a simulated communication network that mimics the state of a (real) communication network. This simulated communication network may be referred to as a digital twin of a real-world communication network and may be hosted, for example, in a network simulator. In this context, the state of the communication network may include one or more of the following: the configuration of one or more base stations in the communication network, the distribution of network traffic in the communication network, the network performance relative to KPIs, etc. In some embodiments, a predictive ML model may be used to predict the impact of proposed network modifications on the network; the predictive ML model may interact with or include the simulated communication network. When using a predictive ML model, this predictive ML model can be trained using KPI data from the communication network. The predictive ML model can be any type of ML model, such as a neural network, random forest, or support vector machine.

[0021] An example training process for a predictive ML model according to some embodiments is as follows. In the example, the predictive ML model is a neural network. The neural network is trained to predict the impact of each of the proposed suggestions; in this example, there are first, second, and third network modification suggestions, wherein the third network modification suggestion has been generated by combining the first and second network modification suggestions as discussed above. The predictive ML model (in this example, a neural network) is trained using KPI data from a communication network. The predictive ML model then uses the state of the communication network as simulated by the simulated communication network to (individually) predict the impact of each of the suggestions on the communication network. The output of the prediction can then be used to determine which of the network modification suggestions to implement. In an example according to some embodiments, the network modification suggestion predicted to have the most beneficial effect on the communication network is selected (e.g., it is predicted to cause the largest positive change in the KPI metric). In other embodiments, the selection of which network modification suggestion to implement can be based on different bases, such as which of the network modification suggestions would require the minimum amount of change to the network configuration or based on another base. In an embodiment where the third network modification suggestion is generated by combining the second and third network modification suggestions (as in this example), the predictive ML model can be configured to predict whether applying the combined network modification suggestion will yield a better or worse result than applying only the first or second network modification suggestion; this result can be output as a binary value (e.g., 1 if the combined suggestion is better, and 0 if the combined suggestion is worse).

[0022] Once the simulated communication network, using the state of the simulated communication network, has predicted the impact on the implementation of each of the proposed network modifications, the method then continues to select one (or more) of the proposed network modifications based on the predicted implementation impact. The selection is made in... Figure 1 Step S108 is shown. Although more than one network modification proposal may be selected and subsequently implemented, typically a single network modification proposal is selected and subsequently implemented. The step of selecting one of the network modification proposals (typically) can be executed by a computer program stored in memory 23 and executed by processor 21 in conjunction with one or more interfaces 22 of ML agent 20A, such as by... Figure 2A Illustration. Alternatively, the step of selecting one of the network modification suggestions can be performed by selector 27 of ML agent 20B, such as... Figure 2BAs shown in the diagram. The basis for making the selection (i.e., one or more criteria) can vary between embodiments, as discussed above. Typically, the basis for selection is determined by the most important factor, which is determined by the operator of the communications network. For example, if improving network performance (i.e., improving measured KPI values) is more important than minimizing the energy spent performing network modifications (such as adjusting antenna tilt values), the selection can be based on which network modification proposal is predicted to provide the best KPI improvement. Two (or more) network modification proposals may be suitable for a combination in which the proposed modifications are complementary, for example, one proposal includes activating part or all of currently deactivated base stations to provide additional service capacity, and another proposal includes redirecting the tilt on one or more existing antennas to improve coverage. In contrast, two (or more) network modification proposals may not be suitable for a combination in which the proposed modifications are contradictory, for example, one proposal includes increasing the tilt angle of antennas for a given set of base stations, and another proposal includes decreasing the tilt angle of the same base stations (this could be, for example, a proposal aimed at providing improved coverage for different geographic areas).

[0023] If the selected network modification proposal to be implemented is obtained from a single proposal agent (in the example discussed above, from the first or second proposal agent), the method can then proceed to the step of initiating modifications to the communication network based on the proposal as discussed below (see [link to relevant documentation]). Figure 1 Step S112). Alternatively, if the selected network modification proposal to be implemented is not obtained from a single proposal agent (because it is generated using network modification proposals from said multiple proposal agents, such as, in the example discussed above, where the third network modification proposal is generated by combining the first and second network modification proposals), the method may then proceed to determine the corresponding weights of the first and second network modification proposals among the generated network modification proposals. The step of determining the weights of the first and second network modification proposals is in... Figure 1 It is shown in S110.

[0024] The step of selecting one of the network modification suggestions can be executed according to a computer program stored in memory 23 and executed by processor 21 in conjunction with one or more interfaces 22 of ML agent 20A, such as by Figure 2A Illustration. Alternatively, the step of selecting one of the network modification suggestions can be performed by selector 27 of ML agent 20B, such as... Figure 2BAs shown in the figure. In some embodiments, the determination of the corresponding weights of the first and second network modifications in the selected network modification proposals may be performed by a predictive ML model. When the weight determination process is performed by a predictive ML model, this process may be combined with a step of predicting the impact of the implementation of various network modification proposals. In the combined process, the prediction of the impact of the implementation of the third network modification proposal may include testing various weights of the first and second network modification proposals in the third network modification proposal, and predicting the impact of the implementation based on the most favorable weight among the tested weights. Alternatively, in some embodiments, a weighted ML model is used to determine the corresponding weights of the first and second network modification proposals in the third network modification proposal.

[0025] Similar to predictive ML models, weighted ML models can be any suitable type of ML model. Examples of suitable types include neural networks and random forests. Similarly, weighted ML models can be trained in any suitable manner. An example of a suitable training method is using RL, as discussed above. When a weighted ML model is trained using RL, the environment is a simulation of the communication network; in some embodiments, this can be the same simulation used by the predictive ML model when both types of ML models are present. When a weighted ML model is trained using RL, it can be called a multi-objective RL model because it performs multi-objective optimization.

[0026] The actions suggested by the weighted ML model can be weights of the corresponding network modification suggestions, where the reward is based on the predicted impact on the communication network (e.g., based on predicted KPI improvement). As with other ML models discussed in this paper, the ML model can be trained on the simulated communication network before being used to determine the appropriate weights for the real communication network.

[0027] Once the proposed network modification (or, in the case of multiple recommendations) has been selected and its weighted average determined (if applicable), the method then proceeds to initiate modifications to the communication network based on the selected network modification recommendations(s). The initiation step is... Figure 1 This is shown as S112. The steps for modifying the communication network can be executed according to a computer program stored in memory 23, executed by processor 21 in conjunction with one or more interfaces 22 of ML agent 20A, such as... Figure 2A Illustration. Alternatively, the steps to modify the communication network can be performed by modifier 28 of ML agent 20B, such as... Figure 2BAs shown in the diagram. The steps of initiating modifications to the communication network may include, for example, sending instructions to one or more base stations to activate or deactivate all or part of the base stations, changing the operating parameters of one or more base stations (e.g., adjusting antenna tilt values, thereby changing the coverage provided by the base stations), etc. When the method is performed by an ML agent that is or forms part of a base station, the ML agent may directly modify the configuration of at least that base station. When the ML agent wishes to modify the configuration of other base stations (because the ML agent is not or does not form part of a base station, or because the ML agent wishes to modify the configuration of base stations other than those it is or forms part of), the steps of initiating modifications to the communication network may include transmitting instructions to the other base stations. In some embodiments, instructions from the ML agent may be distributed to one or more base stations via another component, for example, via a core network node (in these embodiments, the ML agent may be or may form part of the core network node).

[0028] In some embodiments, after initiating a modification to the communication network based on a selected modification proposal, the method may further include waiting for a determined time interval before considering further network modification proposals. Where the proposal agent forms part of a system including an ML agent, the proposal agent may be configured not to generate network modification proposals until the determined time interval has elapsed. Alternatively, where the proposal agent is separate from the ML agent and not under the control of the ML agent, the ML agent may be configured to store or ignore network modification proposals from the proposal agent and not evaluate network modification proposals until after the determined time interval has elapsed. In some embodiments, the ML agent may trigger the proposal agent to begin generating new network modification proposals after the determined time interval has elapsed.

[0029] In some embodiments, the determined time interval may be a setting value determined, for example, by the network operator. In other embodiments, the determined time interval may be determined using an implementation ML model. The determined time interval may be determined before initiating network modifications. Like the other ML models discussed above, the implementation ML model can be any suitable type of ML model and can be trained in any suitable manner. In some embodiments, the implementation ML model may use RL and be trained using input KPI data from the communication network. The determined time interval may be provided in the form of multiple evaluation steps; that is, when the recommendation agent is configured to evaluate the network at a predetermined frequency (with the aim of generating network modification recommendations), the determined time interval may indicate that a given number of evaluation steps can be omitted.

[0030] Figure 4 This is a schematic diagram illustrating an example of a system according to some embodiments. Figure 4In the system shown, communication network 401 transmits network status data (including KPI data) to system 402, which includes an ML agent, and receives network modification instructions. Figure 4 In the example shown, grounding agent 403 receives network state data and supplies it to other parts of the system, such as suggestion agent 404. In this example, the system includes suggestion agents, of which two exist. The suggestion agents receive intents from network users / customers and generate first and second network modification suggestions based on the received intents. In this example, neural network model 405 generates a third network modification suggestion by combining the first and second network modification suggestions. The generated suggestions are passed to predictive ML model 406, which then uses the simulated communication network (virtual twin) to predict the impact of implementing each network modification suggestion on the state of the communication network. In some embodiments, separate predictive agents (where the predictive agents are collectively referred to as predictive ML models) can be used to evaluate each network modification suggestion. Based on the result of this prediction, selector 407 selects the network modification suggestion to be implemented.

[0031] exist Figure 4 In the example shown, if it is determined that the proposal to be implemented is a third (combined) network modification proposal, a weighted ML model 408 is used to calculate the corresponding weights of the first and second network modification proposals in the combined network modification proposals, and then an implementation ML model 409 is used to calculate a determined time interval that should be applied before generating further network modification proposals. Alternatively, if it is determined that the proposal to be implemented is either the first or second network modification proposal, a weighted ML model is not required. Once the determined time interval has been determined by the implementation ML model, the actuation agent 410 initiates the modification of the communication network 401 by sending a network modification command.

[0032] Figure 5A and Figure 5B (Collectively referred to as Figure 5) is a result graph illustrating an example of the impact of the embodiments on the time taken to achieve the KPI target. In the example shown in Figure 5, two network modification proposals are generated by a proposal agent; the first aims to change the service priority within the network, and the second aims to increase the network's maximum bit rate (MBR). A third network modification proposal is generated by combining the first and second network modification proposals; in this example, the proposals are weighted equally. In this example, the combined proposals are determined to be applied. Figure 5A The effects of modifying the network based on the following are shown: First network modification recommendations (in Figure 5A The priority recommendation referred to as "Priority_CV" in the network; the second network modification recommendation (in Figure 5AThe MBR proposal referred to as "MBR_CV2" in the document); and the combined (third) network modification proposal (in Figure 5A It is referred to as "MBR_CV1" in China. Figure 5A The x-axis shows the number of time steps the network has been modified based on various network modification suggestions, while the two y-axes show priority and MBR, respectively. For example... Figure 5A As shown, the combined recommendations result in a faster improvement in priority and MBR compared to either the first or second network modification recommendations. This rapid improvement is due to... Figure 5B The diagram shows the number of time steps the network has been modified based on the third network modification proposal, with the x-axis representing the QoE value and the y-axis representing the QoE value. Figure 5B The illustration shows how the combined model can be used in a small number of time steps, particularly four, to achieve the target Quality of Experience (QoE) based on priority and MBR with a rapid rate of improvement. For comparison, applying the first network modification suggestion (priority suggestion) resulted in the target QoE being achieved in nine time steps, while applying the second network modification suggestion (MBR suggestion) resulted in the target QoE being achieved in seven time steps.

[0033] The embodiments can allow network performance goals (such as target KPI values) to be achieved faster than could be achieved using existing systems, particularly by supporting the simultaneous resolution of multiple proposed network modifications where appropriate. The embodiments can also allow for a reduction in network performance evaluation without being required, thereby saving network resources (such as processing resources). The embodiments can also be implemented without requiring substantial modifications to the existing network architecture.

[0034] It will be appreciated that the examples disclosed herein can be virtualized so that the methods and processes described herein can be run in a cloud environment.

[0035] The methods disclosed herein may be implemented in hardware or as software modules running on one or more processors. The methods may also be executed according to the instructions of a computer program, and this disclosure also provides a computer-readable medium having a program stored thereon for performing any of the methods described herein. The computer program implementing this disclosure may be stored on a computer-readable medium, or it may be in the form of, for example, a signal (such as a downloadable data signal provided from an Internet website), or it may be in any other form.

[0036] Generally, various exemplary embodiments may be implemented in hardware or dedicated circuitry, software, logic, or any combination thereof. For example, some embodiments may be implemented in hardware, while others may be implemented in firmware or software executable by a controller, microprocessor, or other computing device, but this disclosure is not limited thereto. While various aspects of exemplary embodiments may be illustrated and described as block diagrams, flowcharts, or using some other graphical representation, it is fully understood that the blocks, devices, systems, techniques, or methods described herein may be implemented in hardware, software, firmware, dedicated circuitry or logic, general-purpose hardware or controllers or other computing devices, or any combination thereof, as non-limiting examples. Therefore, it should be appreciated that at least some aspects of exemplary embodiments of this disclosure may be practiced in various components such as integrated circuit chips and modules. Therefore, it should be appreciated that exemplary embodiments of this disclosure may be implemented in a device implemented as an integrated circuit, wherein the integrated circuit may include circuit modules (and possible firmware) for implementing at least one or more of a data processor, digital signal processor, baseband circuit module, and radio frequency circuit module, which may be configured to operate according to exemplary embodiments of this disclosure.

[0037] It should be understood that at least some aspects of the exemplary embodiments of this disclosure may be implemented in computer-executable instructions (such as in one or more program modules) that are executed by one or more computers or other devices. Generally, program modules include routines, programs, objects, components, data structures, etc., which perform specific tasks or implement specific abstract data types when executed by a processor in a computer or other device. Computer-executable instructions may be stored on computer-readable media such as hard disks, optical disks, removable storage media, solid-state memories, RAM, etc. As those skilled in the art will appreciate, the functionality of program modules may be combined or distributed as desired in various embodiments. Additionally, the functionality may be implemented wholly or partially in firmware or hardware equivalents such as integrated circuits, field-programmable gate arrays (FPGAs), and the like. References to “an embodiment,” “embodiment,” etc., in this disclosure indicate that the described embodiment may include a particular feature, structure, or characteristic, but not every embodiment necessarily includes that particular feature, structure, or characteristic. Furthermore, such phrases do not necessarily refer to the same embodiment. Furthermore, when a particular feature, structure, or characteristic is described in conjunction with an embodiment, whether or not it is explicitly described, it is assumed that implementing such a feature, structure, or characteristic in conjunction with other embodiments is within the knowledge scope of those skilled in the art.

[0038] It should be understood that while the terms “first” and “second”, etc., may be used herein to describe various elements, these elements should not be limited by these terms. These terms are used only to distinguish one element from another. For example, without departing from the scope of this disclosure, a first element may be referred to as a second element, and similarly, a second element may be referred to as a first element. As used herein, the term “and / or” includes any and all combinations of one or more of the associated listed items. The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit this disclosure. As used herein, the singular forms “a” (“a”, “an”) and “described” are intended to also include the plural forms unless the context clearly indicates otherwise. It will be further understood that the terms “has”, “having”, “comprises”, and / or “including” as used herein specify the presence of the stated feature, element, and / or component, but do not exclude the presence or addition of one or more other features, elements, components, and / or combinations thereof. The term “connection” (“connects”, “connecting”, and / or “connected”) used in this document covers direct and / or indirect connections between two elements.

[0039] This disclosure includes any new features or combinations of features or any generalization thereof expressly disclosed herein. Various modifications and adaptations to the foregoing exemplary embodiments of this disclosure will become apparent to those skilled in the art from the foregoing description (read in conjunction with the accompanying drawings). However, any and all modifications will still fall within the non-limiting and exemplary scope of this disclosure. For the avoidance of ambiguity, the scope of this disclosure is defined by the claims.

Claims

1. A computer-implemented method for modifying the configuration of a communication network (401) using a machine learning ML agent, the method comprising: Receive (S102) a first network modification proposal from the first recommendation agent and a second network modification proposal from the second recommendation agent; Use the first network modification suggestion and the second network modification suggestion to generate (S104) a third network modification suggestion; Using a simulated communication network to predict (S106) the impact of the implementation of the first network modification proposal, the second network modification proposal, and the third network modification proposal on the state of the communication network, wherein the simulated communication network simulates the state of the communication network; Based on the predicted implementation impact, one of the network modification proposals is selected (S108), and if the third network modification proposal is selected, the corresponding weights of the first network modification proposal and the second network modification proposal in the third network modification proposal are determined (S110). as well as Based on the selected network modification proposal, initiate (S112) the modification of the communication network.

2. The method as described in claim 1, wherein, The method further includes receiving network status information from the communication network (401) and using the network status information to update the analog communication network.

3. The method of any preceding claim, further comprising using the first recommendation agent to generate the first network modification recommendation, and using the second recommendation agent to generate the second network modification recommendation.

4. The method of claim 3, wherein, The first and second suggestion agents utilize an ML model trained using reinforcement learning (RL) to generate network modification suggestions.

5. The method as described in any one of claims 3 and 4, wherein, The first and second recommendation agents generate network modification recommendations based on the key performance indicators (KPIs) of the communication network.

6. The method of claim 5, wherein, The KPIs include one or more of the following: Quality of Experience (QoE); packet loss; and latency.

7. The method as claimed in any of the preceding claims, wherein, The third network modification suggestion is generated by combining the first network modification suggestion and the second network modification suggestion.

8. The method as claimed in any of the preceding claims, wherein, The predictive ML model is used to predict the impact of the proposed network modifications.

9. The method of claim 8, wherein, The predictive ML model is trained using key performance indicator (KPI) data from the communication network (401).

10. The method of claim 9, wherein, The network modification suggestion predicted to provide the largest increase in KPI values ​​was selected.

11. The method as claimed in any one of claims 8 to 10, wherein, When the third network modification proposal is selected, the predictive ML model is used to determine the corresponding weights of the first network modification proposal and the second network modification proposal in the third network modification proposal.

12. The method as claimed in any one of claims 8 to 10, wherein, When the third network modification proposal is selected, a weighted ML model is used to determine the corresponding weights of the first network modification proposal and the second network modification proposal in the third network modification proposal.

13. The method of claim 12, wherein, The weighted ML model is trained using reinforcement learning (RL).

14. The method of any preceding claim, further comprising: When a modification to the communication network based on the selected network modification proposal has been initiated, a determined time interval is waited before considering further network modification proposals.

15. The method of claim 14, wherein, The time interval is determined using an implementation ML model.

16. The method of claim 15, wherein, The implementation ML model uses reinforcement learning (RL) and is trained using key performance indicator (KPI) data from the communication network (401).

17. The method of any one of claims 14 to 16, further comprising: When the defined time interval has elapsed, further network modification suggestions are generated.

18. The method as claimed in any of the preceding claims, wherein, The initiation of the modification to the communication network (401) includes indicating the activation and / or deactivation of all and / or a portion of one or more base stations in the communication network (401).

19. The method as claimed in any of the preceding claims, wherein, The initiation of the modification to the communication network (401) includes changing the operating parameters of one or more base stations in the communication network (401).

20. The method as claimed in any of the preceding claims, wherein, The state of the communication network (401) includes the configuration of one or more base stations in the communication network (401), the distribution of network services in the communication network (401), and / or the performance of the communication network (401) relative to the key performance indicators (KPIs).

21. A machine learning ML agent (20A) configured to modify the configuration of a communication network (401) using an ML model, the ML agent (20A) including a processing circuit module (21) and a memory (23) containing instructions executable by the processing circuit module (21), whereby the ML agent (20A) is operable to: Receive a first network modification proposal from a first recommendation agent and a second network modification proposal from a second recommendation agent; Use the first network modification suggestion and the second network modification suggestion to generate a third network modification suggestion; A simulated communication network is used to predict the impact of the implementation of the first network modification proposal, the second network modification proposal, and the third network modification proposal on the state of the communication network (401), wherein the simulated communication network simulates the state of the communication network (401); One of the network modification proposals is selected based on the predicted implementation impact, and if the third network modification proposal is selected, the corresponding weights of the first and second network modification proposals in the third network modification proposal are determined; as well as Modification of the communication network (401) is initiated based on the selected network modification proposal.

22. The ML agent (20A) of claim 21 is further configured to receive network status information from the communication network (401) and use the network status information to update the analog communication network.

23. A system comprising an ML agent (20A) as described in any one of claims 21 and 22, further comprising a first recommendation agent configured to generate the first network modification recommendation and a second recommendation agent configured to generate the second network modification recommendation.

24. The system of claim 23, wherein, The first and second suggestion agents are configured to generate network modification suggestions using an ML model trained with reinforcement learning (RL).

25. The system as claimed in any one of claims 23 and 24, wherein, The first and second recommendation agents are configured to generate network modification recommendations based on the key performance indicators (KPIs) of the communication network (401).

26. The system of claim 25, wherein, The KPIs include one or more of the following: Quality of Experience (QoE); packet loss; and latency.

27. The system or ML agent as described in any one of claims 21 to 26 (20A), wherein, The ML agent (20A) is configured to generate the third network modification proposal by combining the first network modification proposal and the second network modification proposal.

28. The system or ML agent as described in any one of claims 21 to 27 (20A), wherein, The ML agent (20A) is configured to use a predictive ML model to predict the impact of the proposed network modification.

29. The system or ML agent as described in claim 28 (20A), wherein, The predictive ML model is trained using key performance indicator (KPI) data from the communication network (401).

30. The system or ML agent of claim 29 (20A), wherein, The network modification suggestion predicted to provide the largest increase in KPI values ​​was selected.

31. The system or ML agent as described in any one of claims 28 to 30 (20A), wherein, When the third network modification proposal is selected, the ML agent is configured to use the predictive ML model to determine the corresponding weights of the first network modification proposal and the second network modification proposal in the third network modification proposal.

32. The system or ML agent as described in any one of claims 28 to 30 (20A), wherein, When the third network modification proposal is selected, the ML agent (20A) is configured to use a weighted ML model to determine the corresponding weights of the first network modification proposal and the second network modification proposal in the third network modification proposal.

33. The system or ML agent of claim 32 (20A), wherein, The weighted ML model is trained using reinforcement learning (RL).

34. The system or ML agent (20A) as described in any one of claims 21 to 33 is further configured to: wait for a determined time interval before considering further network modification proposals when a modification to the communication network (401) based on a selected network modification proposal has been initiated.

35. The system or ML agent of claim 34 (20A), wherein, The ML agent (20A) is further configured to use an implementation of the ML model to determine the time interval.

36. The system or ML agent of claim 35 (20A), wherein, The implementation ML model uses reinforcement learning (RL) and is trained using key performance indicator (KPI) data from the communication network (401).

37. The system or ML agent (20A) as described in any one of claims 34 to 36, wherein, The ML agent (20A) is further configured to trigger the generation of further network modification suggestions when a determined time interval has elapsed.

38. The system or ML agent (20A) as described in any one of claims 21 to 37, wherein, The ML agent (20A) is configured to, in the initiation of a modification to the communication network (401), indicate the activation and / or deactivation of all and / or a portion of one or more base stations in the communication network (401).

39. The system or ML agent as described in any one of claims 21 to 38 (20A), wherein, The ML agent (20A) is configured to change the operating parameters of one or more base stations in the communication network (401) during the initiation of the modification of the communication network (401).

40. The system or ML agent as described in any one of claims 21 to 39 (20A), wherein, The state of the communication network (401) includes the configuration of one or more base stations in the communication network (401), the distribution of network services in the communication network (401), and / or the performance of the communication network (401) relative to the key performance indicators (KPIs).

Images