Federal learning method, federal learning system and related device

Abstract

The invention provides a federated learning method and system and a related device, which are used for adopting adaptive data sampling strategies for different clients in a federated learning process, so that a plurality of different clients can learn high-value knowledge with diversity, and a machine learning model with a better use effect is obtained. The method comprises the steps that firstly, an acquisition strategy is received, the acquisition strategy indicates acquisition of operation period data of a communication network, such as data representing the network state in the communication network or environment data, and the acquisition strategy can be sent by a server in the federated learning process; then, acquiring operation period data based on the acquisition strategy; and then participating in at least one federation training based on the operation period data to obtain a first machine learning model which can be used for managing the communication network, thereby improving the performance of the communication network.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence, and in particular to a method, system and related devices for federated learning. Background Art

[0002] Due to the high bandwidth, interconnection of all things, and low power consumption of the fifth generation (5G) communication technology, in addition to mobile communication systems, 5G communication technology can also be extended to network operation and maintenance scenarios. At present, the network operation and maintenance architecture based on 5G communication technology generally includes three layers, namely: network element (NE), element management system (EMS), and network management system (NMS).

[0003] Network operation and maintenance includes network optimization. Traditional optimization solutions based on expert experience are difficult to achieve satisfactory results in increasingly complex communication scenarios. In order to provide customers with a better network experience and improve operation and maintenance efficiency, the introduction of artificial intelligence (AI) for network operation and maintenance has become a mainstream trend. Machine learning is a branch of artificial intelligence. Machine learning uses algorithms to analyze data, learn from it, and then make decisions and predictions about events in the real world. In the network optimization scenario, the decisions and predictions made by the machine learning model are optimization recommendations, that is, changes in the parameter values ​​of devices in the network.

[0004] However, in the actual operation and maintenance network environment, the parameter value combinations of each network element are often highly similar. For example, most parameters are factory default values. In addition, the frequency of parameter adjustment is not high, and the adjusted value is very limited relative to the value range of the parameter definition. For example, the value range of a certain parameter is 0 to 100, but in the actual operation and maintenance network, the parameter may be set to 8 to 20. This leads to a serious lack of diversity in the data generated in the actual operation and maintenance network. Even if the data is collected for a long time and the amount of data reaches a large scale, there is still a lot of similar data. It is difficult to learn high-value "knowledge" from such training data, resulting in poor performance of the machine learning model obtained after training. Summary of the invention

[0005] The present application provides a method, system and related devices for federated learning, which are used to adopt adaptive data sampling strategies for different clients during the federated learning process, so that multiple different clients can learn diverse high-value knowledge and obtain a model with better use effect.

[0006] In view of this, on the first aspect, the present application provides a federated learning method, comprising: first, receiving an acquisition strategy, which indicates the collection of runtime data of a communication network, such as data representing the network status or environmental data in the communication network, and the acquisition strategy may be sent by a server during the federated learning process; then, the runtime data may be collected based on the acquisition strategy; then, participating in at least one federated training based on the runtime data to obtain a first machine learning model, which may be used to manage the communication network.

[0007] In the implementation manner of the present application, taking the client and the server performing federated training as an example, the server can send an acquisition strategy, or a sampling strategy, to the client. The client can collect runtime data of the communication network used for federated learning based on the received sampling strategy. Therefore, different clients can collect samples that are adapted to the operating status or environment of the communication network in which they are located, so as to perform federated learning based on more adapted samples. Therefore, different clients can collect diverse runtime data and realize diverse end-side learning, thereby learning higher value "knowledge" and improving the performance of the model. And by performing federated training with higher quality training samples, the model can converge faster.

[0008] In one possible implementation, one of the at least one federated trainings mentioned above may include: after collecting runtime data, training a second machine learning model that is locally maintained or saved according to the runtime data to obtain a third machine learning model, where the second machine learning model may be a received machine learning model or an initial machine model saved locally; subsequently sending the third machine learning model, specifically, sending a set of parameters of the third machine learning model, such as weight parameters or structural parameters of the model, etc. The third machine learning model may be used to determine the first machine learning model, such as the server may fuse models sent by multiple clients to obtain the first machine learning model.

[0009] In this embodiment, during a federated learning process, the server can fuse the model parameters sent by different clients, and send the fused model parameters to each client, thereby updating the local model of each client, and using the local model to optimize the parameters of the communication network, so that each client can use the fused model parameters for the next step of training.

[0010] In one possible implementation, the first machine learning model is received, and the parameter set of the received first machine learning model can be used to update the locally maintained third machine learning model. For example, the third machine learning model can be directly replaced with the first machine learning model, or the parameter set of the first machine learning model can be fused with the parameter set of the third machine learning model, so as to make the locally maintained model the latest machine learning model.

[0011] In a possible implementation, different sent trained model parameters can be used for the server to update the acquisition strategy, and the new acquisition strategy can be received during the next federated learning process. In this implementation, the server can determine and update the adaptive acquisition strategy based on the parameters of the model sent by different clients, so each client can use the adaptive acquisition strategy to collect data to train the locally maintained model. Different clients can train to obtain local models that are adapted to the state of the communication network in which they are located or the environment in which they are located, which can improve the diversity of knowledge learned by the fused model and improve the performance of the obtained model.

[0012] In a possible implementation, the above method may also include: obtaining data features of runtime data, which may include the start time, end time, or number of samples (or quantity) in different scenarios collected by the client; sending data features to the server, which may be used to update the acquisition strategy of the server, and receive the new acquisition strategy during the next federated learning process. In this implementation, the data features of the runtime data collected locally can be obtained and uploaded to the server, so that the server can determine an adaptive acquisition strategy based on the data features uploaded by each client. Therefore, each client can use the adaptive acquisition strategy to collect data to train the locally maintained model parameters. Different clients can train to obtain local models adapted to their environment, which can improve the diversity of knowledge learned by the fused model and improve the performance of the obtained model.

[0013] In a possible implementation manner, the aforementioned runtime data of the communication network includes data representing the operating status of the communication network and / or data indicating the environment in which the devices in the communication network are located.

[0014] In one possible implementation, the runtime data may include the network configuration parameters of the managed network elements in the communication network, the network status of the managed network elements before the network configuration parameters are configured, and the network status of the managed network elements after the network configuration parameters are configured. Therefore, in the implementation of the present application, the runtime data may reflect the changes in the network status of the managed network elements after the network configuration parameters are executed, that is, the locally maintained model is trained based on the network changes, and the locally maintained model may be trained so that the network configuration parameters it outputs can make the network status of the communication network better. This is equivalent to reversely updating the machine learning model by observing the changes in the network status after the network elements are configured with the parameters output by the machine learning model, so that the parameters output by the machine learning model are more adapted to the environment in which each device is located, thereby optimizing the network status of the communication network.

[0015] In a possible implementation, the aforementioned communication network may include a network cable network, and the wireless network may specifically include at least two devices in a network management system NMS, a network element management system EMS, or a network element NE. In the implementation of the present application, a federated learning method may be deployed in the NMS, EMS, or network element of the communication network, so as to learn the "knowledge" included in the data collected by each device without the need for data interaction during the operation of the communication network.

[0016] In a possible implementation, the method provided by the present application can be executed by a client, and can perform federated training together with a server. The server can be deployed in a network management system NMS, and the client can be deployed in an element management system EMS. Therefore, the method provided by the present application can be deployed in a communication network, and a model that can be used to manage devices in a communication network can be obtained through federated learning training. Through the diversified acquisition strategies of different clients, diversified management of different network elements can be achieved, thereby improving the EMS management effect in the communication network.

[0017] In a possible implementation, the method provided by the present application can be executed by a client, the server is deployed in an element management system EMS, and the client is deployed on an element NE. Therefore, the method provided by the present application can be deployed in a communication network, and a model that can be used to manage devices in a communication network can be obtained through federated learning training. Through the diversified acquisition strategies of different clients, diversified management of different network elements can be achieved, thereby improving the NE management effect in the communication network.

[0018] In a possible implementation, the acquisition strategy may indicate one or more of the following: the number of collections refers to the number of data collection results; the exploration ratio refers to the ratio of data collected based on the exploration mechanism in the data collection results, and the data collected based on the exploration mechanism refers to the data collected after configuring the communication network with configuration parameters that have not been configured before, or the ratio between the part used to learn new knowledge and the part used to consolidate the learned knowledge; the collection mode refers to the mode of data collection, which is periodic collection, specified collection, or adaptive collection, etc. Therefore, the total number of runtime data, exploration ratio, or collection mode, etc. that are adapted can be configured for each client, so as to collect data that can be used for federated training.

[0019] In a possible implementation, the acquisition strategy may further indicate a sampling period or the number of sampling scenarios, where the sampling period may include a period for collecting runtime data, and the number of sampling scenarios may include the number of samples (or the number of items) of data of each sampling scenario.

[0020] In one possible implementation, the first machine learning model may be specifically used to generate or update configuration parameters of the communication network.

[0021] In a possible implementation, when the communication network includes a wireless network, the configuration parameters may specifically include but are not limited to one or more of the following: scheduling parameters, radio frequency (RF) parameters, or switching parameters, scheduling parameters include parameters indicating allocation of resources for the wireless network, RF parameters include parameters indicating adjustment of the radio frequency of the wireless network, and switching parameters include parameters for instructing a terminal in the wireless network to perform cell switching. Therefore, the parameters of various dimensions of the communication network can be adjusted through the parameters output by the machine learning model to achieve a better network state of the communication network.

[0022] In a second aspect, the present application provides a federated learning method, comprising: first, sending an acquisition strategy, such as sending the acquisition strategy to a client that jointly performs federated training, wherein the acquisition strategy is a strategy for collecting data from a communication network to obtain data for federated training; then initiating at least one federated training based on the runtime data of the communication network collected according to the acquisition strategy to obtain a first machine learning model, which can be used to manage the communication network.

[0023] In the implementation manner of the present application, the way in which the client collects data can be controlled by issuing an acquisition strategy, so the way in which data for federated training is collected is adjustable. Taking the client and server for federated learning as an example, the server can determine an adaptive acquisition strategy based on the data characteristics of the environment in which the client is located, so that each client can collect high-value learning data, such as data representing the operating status of the communication network in which it is located or data adapted to the data characteristics of the environment in which it is located. Therefore, when each client trains a local model, it can use more adapted runtime data for training, thereby learning knowledge adapted to the data characteristics of each client, and by fusing the model parameters trained by each client, the obtained machine learning model can achieve diversified knowledge learning, thereby improving the performance of the final model.

[0024] In one possible implementation, the aforementioned jointly executing at least one federated training based on runtime data with the client may include: receiving at least one group of third machine learning models, specifically receiving a parameter set of the third machine learning model, such as the structural parameters or weight parameters of the model, etc. The at least one group of third machine learning models may include machine learning models sent by different clients, and each group of third machine learning models may be obtained by training a locally maintained model for the client based on the collected runtime data; subsequently, the trained model parameters sent by different clients are fused to obtain the first machine learning model.

[0025] In this implementation, during a federated learning process, the model parameters sent by different clients can be fused, and the fused model parameters can be sent to each client, so that each client can use the fused model parameters for the next step of training.

[0026] In one possible implementation, the first machine learning model may also be sent so that the client can update the local machine learning model based on the latest machine learning model received, so that the client's model can be kept up to date.

[0027] In a possible implementation, the method may further include: updating the acquisition strategy according to the parameters of the third machine learning model, such as updating the acquisition strategies corresponding to different clients according to the trained model parameters sent by different clients. In this implementation, the adaptive acquisition strategy can be determined and updated based on the model parameters sent by different clients, so each client can use the adaptive acquisition strategy to collect data to train the locally maintained model parameters. Different clients can train to obtain local models that are adapted to their environment, which can improve the diversity of knowledge learned by the fused model and improve the performance of the obtained model.

[0028] In a possible implementation, the method may further include: receiving data features, which may be features of runtime data collected by the client; determining an updated acquisition strategy based on the data features, and the new acquisition strategy may be used in the next federated learning process. In this implementation, the client may obtain data features of runtime data collected locally, and upload the data features to the client, so that an adaptive acquisition strategy may be determined based on the data features uploaded by each client. Therefore, each client may use an adaptive acquisition strategy to collect data to train locally maintained model parameters. Different clients may train to obtain local models adapted to their environments, which may improve the diversity of knowledge learned by the fused model and the performance of the obtained model.

[0029] In a possible implementation, the aforementioned communication network includes a wireless network, and the wireless network may include at least two devices in a network management system NMS, an element management system EMS, or a network element NE.

[0030] In a possible implementation, the method provided by the present application can be executed by a server, the server is deployed in a network management system NMS, and the client is deployed in an element management system EMS. Therefore, the method provided by the present application can be deployed in a communication network, and a model for managing devices in a communication network can be trained through federated learning, and through the diversified acquisition strategies of different clients, diversified management of different network elements can be achieved, thereby improving the EMS management effect in the communication network.

[0031] In a possible implementation, the method provided by the present application can be executed by a server, the server is deployed in an element management system EMS, and the client is deployed on an element NE. Therefore, the method provided by the present application can be deployed in a communication network, and a model for managing devices in a communication network can be trained through federated learning, and through the diversified acquisition strategies of different clients, diversified management of different elements can be achieved, thereby improving the NE management effect in the communication network.

[0032] In a possible implementation, the runtime data may include the current network status of the NE managed by the EMS, the network configuration parameters of the NE managed by the EMS, and the network status of the NE managed by the EMS after the network configuration parameters are executed. When different clients collect data through the adapted acquisition strategy, data adapted to each client can be collected. When the client trains the local model, it can be trained based on the data collected by the adapted strategy, and can learn knowledge adapted to the client environment, thereby realizing diversified management of the communication network.

[0033] In a possible implementation, the acquisition strategy may indicate one or more of the following: the number of collections, indicating the number of data collection results; the exploration ratio, indicating the proportion of data collected based on the exploration mechanism, where the data collected based on the exploration mechanism refers to the data collected after configuring the communication network with configuration parameters that have not been configured before, or the ratio of the number of parts used to learn new instructions and the part used to consolidate the learned knowledge in the data collection results; the collection mode, indicating whether the data collection mode is periodic collection, specified collection, or adaptive collection, etc. Therefore, the total number of runtime data, exploration ratio, or collection mode, etc., which are adapted for each client configuration, can be used.

[0034] In a possible implementation, the acquisition strategy also indicates a sampling period or the number of sampling scenarios. The sampling period may include a period for collecting data, i.e., a time interval between two adjacent data collection operations on the communication network. The number of sampling scenarios may include the number or number of data items to be collected in the sampling scenario.

[0035] In one possible implementation, the model obtained by federated learning can be used to generate or update the configuration parameters of the communication network, that is, the aforementioned first machine learning model can be used to generate or update the configuration parameters of the communication network.

[0036] In a possible implementation, when the communication network includes a wireless network, the configuration parameters of the communication network may specifically include, but are not limited to, one or more of the following: scheduling parameters, radio frequency (RF) parameters, or switching parameters. The scheduling parameters may include parameters for scheduling resources in the communication network, the RF parameters may include parameters for indicating wireless resources in the communication network, and the switching parameters include parameters for instructing devices in the communication network to perform resource switching. Therefore, the parameters of various dimensions of the communication network may be adjusted through the parameters output by the machine learning model to achieve a better network state of the communication network.

[0037] In a third aspect, the present application provides a federated learning method, including:

[0038] The server sends an acquisition strategy to the client, where the acquisition strategy is a strategy for collecting data from the communication network to obtain data for federated training;

[0039] The client collects the operation period data of the communication network according to the received acquisition strategy;

[0040] The server and the client jointly perform at least one federated training based on the runtime data to obtain a first machine learning model, and the first machine learning model is used to manage the communication network.

[0041] The devices in the communication network may include a server and at least one client. In the implementation of the present application, one client is used as an example for exemplary introduction. That is, the client here only refers to any client in the communication network and is not limited to

[0042] Among them, the effects achieved by the third aspect and any optional implementation of the third aspect can refer to the effects described in any optional implementation of the first aspect or the second aspect mentioned above, and will not be repeated here.

[0043] In a possible implementation, a federated training may include:

[0044] The client trains the second machine learning model based on the runtime data to obtain a third machine learning model;

[0045] The client sends a third machine learning model to the server, where the third machine learning model is used to determine the first machine learning model;

[0046] The server performs fusion processing on the third machine learning model sent by the client to obtain the first machine learning model. In a possible implementation, the aforementioned method may also include: the server sends the first machine learning model to the client.

[0047] In a possible implementation, the runtime data includes data representing an operating state of the communication network and / or data indicating an environment in which a device in the communication network is located.

[0048] In a possible implementation, the runtime data may include: network configuration parameters of managed network elements in the communication network, the network status of the managed network elements before the network configuration parameters are configured, and the network status of the managed network elements after the network configuration parameters are configured.

[0049] In a possible implementation, the above method may further include: the server updates the acquisition strategy according to the third machine learning model, and the server may send the new acquisition strategy to the client.

[0050] In a possible implementation, the above method may further include:

[0051] The client obtains data characteristics of runtime data;

[0052] The client sends data features to the server;

[0053] The server updates the acquisition strategy based on the data characteristics.

[0054] In one possible implementation, the acquisition strategy may indicate one or more of the following: collection quantity, indicating the number of data collection results; exploration ratio, which refers to the proportion of data collected based on the exploration mechanism, where the data collected based on the exploration mechanism refers to the data collected after configuring configuration parameters that have not been configured for the communication network, or indicates the ratio of the part used to learn new instructions and the part used to consolidate the learned knowledge in the data collection results; collection mode, indicating whether the data collection mode is periodic collection, specified collection, or adaptive collection, etc.

[0055] In a possible implementation, the acquisition strategy further indicates a sampling period or a scene sampling quantity, where the sampling period is the time interval between two adjacent data collections on the communication network, and the scene sampling quantity includes the quantity (or number of items) of data to be collected in the sampling scene.

[0056] In a possible implementation, the communication network includes at least two of a network management system NMS, an element management system EMS or a network element NE.

[0057] In one possible implementation, the model obtained by federated learning can be used to generate or update the configuration parameters of the communication network, that is, the aforementioned first machine learning model can be used to generate or update the configuration parameters of the communication network.

[0058] In a possible implementation, the configuration parameters of the communication network include one or more of the following: scheduling parameters, radio frequency (RF) parameters, or switching parameters, the scheduling parameters include parameters indicating allocation of resources for the wireless network, the RF parameters include parameters indicating adjustment of the radio frequency of the wireless network, and the switching parameters include parameters for instructing a terminal in the wireless network to perform cell switching.

[0059] In a fourth aspect, the present application provides a client, which can be used to execute the steps executed by the client in the first aspect and any possible implementation manner provided in the first aspect.

[0060] In a possible design, the present application may divide the client into functional modules according to the first aspect and any possible implementation method provided in the first aspect. For example, each functional module may be divided according to each function, or two or more functions may be integrated into one processing module.

[0061] Exemplarily, the present application can divide the client into a transceiver module and a processing module according to function. The description of possible technical solutions and beneficial effects executed by each functional module divided above can refer to the technical solutions provided by the first aspect or its corresponding possible implementation methods, which will not be repeated here.

[0062] In another possible design, the client includes: a memory and a processor, the memory and the processor are coupled. The memory is used to store computer instructions, and the processor is used to call the computer instructions to execute the method provided in the first aspect or its corresponding possible implementation manner.

[0063] In a fifth aspect, the present application provides a server. The server can be used to execute the steps executed by the server in the second aspect and any possible implementation method provided in the second aspect.

[0064] In a possible design, the present application may divide the server into functional modules according to the above second aspect and any possible implementation method provided in the second aspect. For example, each functional module may be divided corresponding to each function, or two or more functions may be integrated into one processing module.

[0065] Exemplarily, the present application may divide the server into a transceiver module and a processing module according to function. The description of possible technical solutions and beneficial effects executed by each functional module divided above can refer to the technical solutions provided by the second aspect or its corresponding possible implementation methods, which will not be repeated here.

[0066] In another possible design, the server includes: a memory and a processor, the memory and the processor are coupled. The memory is used to store computer instructions, and the processor is used to call the computer instructions to execute the method provided in the second aspect or its corresponding possible implementation.

[0067] In a sixth aspect, the present application provides a communication system, which may also be referred to as a federated learning system or a communication network, etc. The communication system may include a server and at least one client. The server may be used to execute the method steps of the aforementioned second aspect or any optional implementation of the second aspect, and the client may be used to execute the method steps of the aforementioned first aspect or any optional implementation of the first aspect.

[0068] In a seventh aspect, the present application provides a computer-readable storage medium, such as a non-transitory computer-readable storage medium, on which a computer program (or instruction) is stored, and when the computer program (or instruction) is executed on a computer device, the computer device executes the method provided in the first aspect or its corresponding possible implementation, or executes the method provided in the second aspect or its corresponding possible implementation.

[0069] In an eighth aspect, the present application provides a computer program product, which, when executed on a computer device, enables the method provided in the first aspect or its corresponding possible implementation manner to be executed, or enables the method provided in the second aspect or its corresponding possible implementation manner to be executed.

[0070] In the ninth aspect, the present application provides a chip system, comprising: a processor, the processor being used to call and run a computer program stored in a memory from the memory, to execute a method provided in the first aspect or its corresponding possible implementation manner, or to execute a method provided in the second aspect or its corresponding possible implementation manner.

[0071] It can be understood that any of the systems, devices, computer storage media, computer program products or chip systems provided above can be applied to the corresponding methods provided in the first aspect or the second aspect.

[0072] For the specific implementation steps of the fourth to ninth aspects of the present application and various possible implementation methods, as well as the beneficial effects brought about by each possible implementation method, please refer to the description of the various possible implementation methods in the first or second aspect, and will not be repeated here one by one. BRIEF DESCRIPTION OF THE DRAWINGS

[0073] Figure 1 A schematic diagram of the structure of a communication system provided for this application;

[0074] Figure 2 Network operation and maintenance architecture for 5G communication technology;

[0075] Figure 3 A schematic diagram of the architecture of a communication system provided for this application;

[0076] Figure 4 A schematic diagram of the architecture of another communication system provided in an embodiment of the present application;

[0077] Figure 5 A schematic diagram of the architecture of a communication system provided in an embodiment of the present application;

[0078] Figure 6 A schematic diagram of the architecture of another communication system provided in an embodiment of the present application;

[0079] Figure 7 A flowchart of a federated learning method provided in an embodiment of the present application;

[0080] Figure 8 A flowchart of another federated learning method provided in an embodiment of the present application;

[0081] Fig. 9 A flowchart of another federated learning method provided in an embodiment of the present application;

[0082] Fig.10 A flowchart of another federated learning method provided in an embodiment of the present application;

[0083] Fig.11 A flowchart of another federated learning method provided in an embodiment of the present application;

[0084] Fig.12 A flowchart of another federated learning method provided in an embodiment of the present application;

[0085] Fig.13 A flowchart of another federated learning method provided in an embodiment of the present application;

[0086] Fig.14 A flowchart of another federated learning method provided in an embodiment of the present application;

[0087] Fig.15 A schematic diagram of the structure of a client provided for this application;

[0088] Fig.16 A schematic diagram of the structure of a server provided for this application;

[0089] Fig.17 A schematic diagram of the structure of a computing device provided in this application. DETAILED DESCRIPTION

[0090] The following will describe the technical solutions in the embodiments of the present application in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0091] Federated training (or federated learning) is a distributed machine learning technology. It is a distributed machine learning paradigm in which multiple parties use their own data to collaboratively train artificial intelligence models without aggregating data from multiple parties. The traditional machine learning paradigm requires the aggregation of large amounts of data for model training, and the data used for training is likely to come from multiple different organizations or users or participants or clients. Bringing together data from multiple parties is very likely to cause the risk of data leakage, which will expose information assets to organizations and may leak personal privacy to individual users. Federated training allows multi-party data to be retained locally without multi-party data aggregation, and multiple parties can jointly train AI models by collaboratively computing (securely) interacting with intermediate calculation results. Through federated learning technology, the business data of multiple parties is protected, and the data of multiple parties can be fully utilized to collaboratively train models, thereby obtaining a more powerful model. The communication system provided in this application can be used as Figure 1As shown, the communication system may also be referred to as a federated learning system, a federated learning network, or a communication network, etc. In the present application, the federated learning process is exemplarily divided into steps executed by the client and the server. The client uses local data to train the model, and the server can aggregate and fuse the data trained by multiple clients to obtain a federated model.

[0092] See also Figure 2 As shown in Figure 1, the current network operation and maintenance architecture based on 5G communication technology generally includes three layers, namely: network element (NE), element management system (EMS), and network management system (EMS).

[0093] system, NMS).

[0094] Among them, NE refers to various network devices that constitute the network, which may include but are not limited to base stations, access points, various core network devices, etc.

[0095] Optionally, the NE can be a physical entity device or a virtual network function, which is not limited in this application. EMS can not only manage NE, but also be a provider of network operation and maintenance management services, and can perform specific network operation and maintenance management service operations according to the service call request sent by NMS. As the invoker of network operation and maintenance management services, NMS can send service call requests to EMS according to the customer's network operation and maintenance management needs, so as to call the network operation and maintenance management services in EMS that meet the customer's operation and maintenance management needs. Both EMS and NMS are composed of at least one device. The devices in EMS can be called network element management devices, and the devices in NMS can be called network management devices.

[0096] Network operation and maintenance includes network optimization, which can generally include improving user experience, reducing network energy consumption, or increasing network capacity. The embodiments of the present application do not specifically limit this, and the requirements and goals of network optimization can be set according to actual needs. Traditional optimization solutions based on expert experience are difficult to achieve satisfactory results in increasingly complex communication scenarios. In order to bring customers a better network experience and improve operation and maintenance efficiency, the introduction of artificial intelligence for network operation and maintenance has become a mainstream trend. Machine learning is a branch of artificial intelligence. Machine learning uses algorithms to parse data, learn from it, and then make decisions and predictions about events in the real world. In the network optimization scenario, the decisions and predictions made by the machine learning model are optimization suggestions, that is, the changes in the values ​​of the parameters of the equipment in the network (sometimes referred to as parameter values ​​or simply parameters in the embodiments of the present application). Taking wireless base stations as an example, these parameters include scheduling parameters, resource management parameters, radio frequency (RF) parameters, power parameters, switching parameters, and the like.

[0097] Machine learning models (sometimes also referred to as models or federated models in the embodiments of the present application) include artificial intelligence models such as neural networks.

[0098] The performance of most neural networks depends on a large amount of training data. Taking the network optimization scenario as an example to illustrate this, the actual network scenario is complex and dynamically changing, and there are many factors that affect network performance. In addition to the parameters of the network equipment, the surrounding geographical environment (topography, surrounding buildings, etc.), the distribution and behavior of users, and even the climate will affect the network performance. This leads to a huge state space and decision space faced by the machine learning model, requiring a large amount of data as training data for learning. Or it can be understood that due to the many factors that affect network performance, the machine learning model needs to learn a lot of knowledge during the iterative training process, that is, the machine learning model needs to obtain a large amount of parameter changes and performance changes (or network state changes) The relationship between the relationship between the network configuration parameters and the network performance changes. It should be noted that the model in this application includes a neural network model, and this application does not limit the specific category of the model. For example, the model in this application can be a convolutional neural network model, a recurrent neural network model, a deep residual network model, etc., and this will not be repeated below.

[0099] However, in the actual operation and maintenance network environment, the parameter (or parameter value) combinations of each network element are often highly similar. For example, most parameters are factory default values. In addition, the frequency of parameter adjustment is not high, and the adjusted value is very limited relative to the value range of the parameter definition. For example, the value range of a certain parameter is 0 to 100, while in the actual operation and maintenance network, the parameter may be set to 8 to 20. This leads to a serious lack of diversity in the generated data in the actual operation and maintenance network, or it can be understood that because the parameter change interval is very small and the change range is very concentrated, the runtime data generated based on these configurations is similar, or the diversity is seriously insufficient. In this way, even if the data is collected for a long time and the amount of data reaches a large scale, a large amount of data is similar. It is difficult for the machine learning model to learn high-value knowledge from such training data, resulting in poor performance of the trained machine learning model and inability to give good optimization suggestions, that is, to give good parameter change suggestions.

[0100] In addition, considering the importance of data privacy, data maintained locally on a device cannot be sent to devices outside the trust zone of the device. This application combines communication networks with federated learning and adopts federated training in the process of training machine learning models in communication networks, thereby protecting the privacy data of each device while achieving better management of the communication network.

[0101] Therefore, the present application provides a federated learning method that can be deployed in a communication network. During the federated learning process, an adaptive data sampling strategy can be adopted for different clients, so that multiple different clients can learn diverse high-value knowledge and obtain a model with better performance.

[0102] For example, taking the network operation and maintenance architecture based on 5G communication technology as an example, see Figure 3 , is a schematic diagram of the architecture of a communication system, which can also be called a federated learning system or a communication network, and the devices in the communication system can act as servers and clients.

[0103] The workflow mainly includes model delivery and model fusion. In the model delivery process, the server delivers the model to the client, and the client trains the model downloaded from the server using the training data stored locally (for example, Figure 3 In the example, client 1 uses local data 1 to train local model 1, and client 2 uses local data 2 to train local model 2). After training to a certain extent, the model is uploaded to the server (for example, Figure 3As shown in the figure, client 1 sends the trained local model 1 to the server, and client 2 sends the trained local model 2 to the server). Training to a certain extent can be understood as a fixed number of rounds of training. During the model fusion process, the server collects the models uploaded by each client and performs model fusion (for example, Figure 3 The server in the example performs fusion processing on local model 1 and local model 2 to obtain fusion model 1), and sends the fusion model to the client (for example Figure 3 The servers in the model will be integrated into the model 1

[0104] The fused model is the machine learning model obtained by federated learning. The model delivery process and model fusion process will be iterated repeatedly until the model converges or meets the preset conditions (for example, the number of training times reaches the preset number).

[0105] It should be noted that, for the sake of convenience, Figure 3 The solution is demonstrated by taking 2 clients and 1 server as an example, but the embodiments of the present application do not limit the number of clients and servers included in the system.

[0106] Based on the above system architecture, a federated learning method provided by the present application may include: first, the server sends an acquisition strategy to the client, and after the client receives the acquisition strategy sent by the server, it can collect runtime data based on the acquisition strategy, and the acquisition strategy is a strategy for collecting data on the communication network to obtain data for federated training, such as data representing the network status or environmental data in the communication network; then, at least one federated training is performed together with the server based on the runtime data to obtain a machine learning model, or it can also be called a federated model, which can be used to generate or update the configuration parameters of the communication network.

[0107] Based on the above description, the federated learning method provided in the present application can be applied to a communication network, which may include a variety of devices, and the server and client mentioned in the present application can be deployed in the management device or network element of the communication network. For example, in one possible implementation, the server is deployed on the NMS, and the client is deployed on the EMS. In one possible implementation, the server is deployed on the EMS, and the client is deployed on the NE. The server and multiple clients are used to perform federated training. During any training process, the server sends an acquisition strategy to the multiple clients, and the acquisition strategies received by each client may be different. Of course, there may be two or more clients receiving the same acquisition strategy. Among them, the acquisition strategy can also be referred to as a sampling strategy or a collection strategy, etc., which can be understood as being sent by the federated learning server to each client, and is used to guide each client on how to collect data and what kind of data to collect to train the local model, so as to obtain the best federated learning efficiency and network optimization effect. Each client collects training data according to the acquisition strategy received by each client, and trains the model maintained locally. In the solution provided by the embodiment of the present application, the client can collect training data based on different acquisition strategies according to the instructions of the server, or the server can send different acquisition strategies to different clients, thereby ensuring that the total training data collected by all clients is diverse. By using a large amount of such data as training data, the machine learning model can learn high-value knowledge. The trained machine learning model has good performance and can give good optimization suggestions for the current network environment, that is, give good parameter change suggestions. Based on the above research ideas, a communication network provided by the embodiment of the present application is specifically introduced below.

[0108] The federated learning method provided in this application can be applied to a variety of communication networks, such as a variety of optical networks or wireless networks. Figure 4 , is a schematic diagram of another architecture of a communication system provided in an embodiment of the present application. The communication network includes an NMS, an EMS, and network elements. Figure 4 NMS is used to indicate the equipment included in NMS, and EMS is used to indicate the equipment included in EMS. The following will not be repeated. For the relevant description of NMS, EMS and network elements, please refer to Figure 1To understand the relevant description in, NMS is responsible for the operation, management and maintenance functions of the network. EMS, that is, the network element management system, is responsible for managing one or more categories of network elements. Network element, that is, network unit. In the wireless access network, it refers to a base station or a base station controller, including but not limited to: one or more base stations in the global system for mobile communications (GSM), universal mobile telecommunications system (UMTS), long-term evolution technology (LTE), new radio (NR), and base station controllers of GSM and UMTS. In the bearer network, it refers to one or more access routers (Access, ACC), metro edge routers (metro edgegateway, mEG), metro aggregation edge gateway (metro aggregation edge gateway, mAEG), metro core routers (metro backbone, mBB), 5G core routers (5G Backbone, 5G BB), etc. In the core network, it refers to one or more of the following: mobile switching center (MSC), serving general packet radio system support node (SGSN), equipment identity register (EIR), packet data network gateway for user plane (PGW-U), access and mobility management function (AMF), unified data management (UDM), etc.

[0109] Based on the communication network described above, the solution provided in the embodiment of the present application can be jointly executed by the NMS and the EMS, or jointly executed by the EMS and the network element, and can also be deployed in other devices.

[0110] For example, see Figure 5 , which is an architecture diagram of another communication system provided in an embodiment of the present application. When the method provided in the present application is jointly executed by NMS and EMS, the NMS includes an adaptive exploration control module and a federal model management module, and the EMS includes a strategy execution module and a local model management module.

[0111] On the NMS side:

[0112] The adaptive exploration control module generates a sampling strategy for each specific EMS, which can also be called an acquisition strategy or collection strategy, and distributes it to the EMS. It improves itself through the feedback of the federated model in the federated model management module, and generates a sampling strategy based on the sample data features reported by each EMS.

[0113] The federated model management module is used by each EMS to collaboratively update the federated model. It collects and aggregates the local model parameters of each EMS to update the federated model, and then distributes the federated model parameters to the EMS.

[0114] EMS side:

[0115] The sampling strategy executor module uses uncertainty-based sampling to explore the optimization space, generate sampling actions and execute them according to the sampling strategy issued by NMS.

[0116] The local model management module incrementally trains the local model based on the sample data generated by sampling.

[0117] See also Figure 6 , which is an architecture diagram of another communication system provided in an embodiment of the present application. When the method provided in the present application is jointly executed by the EMS and the network element, the EMS includes an adaptive exploration control module and a federal model management module, and the network element includes a policy execution module and a local model management module.

[0118] EMS side:

[0119] The adaptive exploration control module generates a sampling strategy for each specific network element and distributes it to the network elements. It improves itself through the feedback of the federated model in the federated model management module and generates a sampling strategy based on the sample data features reported by each network element.

[0120] The federation model management module is used for the network elements to collaboratively update the federation model. It collects and aggregates the local model parameters of each network element to update the federation model, and then distributes the federation model parameters to the network elements.

[0121] NE side:

[0122] The sampling strategy executor module uses uncertainty-based sampling to explore the optimization space, generate sampling actions and execute them according to the sampling strategy issued by EMS.

[0123] The local model management module incrementally trains the local model based on the sample data generated by sampling.

[0124] The following introduces the federated learning method provided in this application in combination with the aforementioned federated learning system and communication network.

[0125] First, the overall process of the federated learning method provided in this application is introduced.

[0126] See also Figure 7 , a flowchart of a federated learning method provided in this application, the federated learning method provided in this application can be deployed in Figure 1 In the federated learning system shown. As described below.

[0127] 701. The server sends an acquisition strategy to the client.

[0128] The acquisition strategy can be used to guide the client on how to collect runtime data and what kind of runtime data to collect to train the local model in the process of federated learning, so as to obtain the best federated learning efficiency and network optimization effect. The acquisition strategy can also be called a sampling strategy or a collection strategy.

[0129] Exemplarily, the acquisition strategy may indicate one or more of the following: collection quantity, which refers to the number of data collection results; exploration ratio, which refers to the proportion of data collected based on the exploration mechanism in the data collection results, and the data collected based on the exploration mechanism refers to the data collected after configuring the communication network with configuration parameters that have not been configured, or the quantitative ratio between the part used to learn new knowledge and the part used to consolidate the learned knowledge (which may also be called the new-old ratio or the collection ratio, etc.); collection mode, which refers to the data collection mode, which is periodic collection, specified collection, or adaptive collection, etc.

[0130] Optionally, the acquisition strategy may also indicate a sampling period or a scene sampling quantity, where the sampling period is the time interval between two adjacent data collections on the communication network, and the scene sampling quantity includes the quantity (or number of items) of data to be collected in the sampling scene.

[0131] It should be noted that in the communication system provided by the present application, the number of clients may include one or more. The embodiment of the present application is illustrative, and only one client is used as an example for exemplary introduction, and it is not intended to be limiting.

[0132] 702. The client collects runtime data according to the acquisition strategy.

[0133] After receiving the acquisition strategy, the client collects the operation period data of the communication network under the instruction of the acquisition strategy.

[0134] Specifically, the runtime data includes data representing an operating state of the communication network and / or data indicating an environment in which a device in the communication network is located.

[0135] The operation period data may include data indicating the operation status of the communication network or data of the environment in which the equipment in the communication network is located. Specifically, the operation period data may include network configuration parameters of the managed network elements in the communication network, the network status of the managed network elements before the network configuration parameters are configured, and the network status of the managed network elements after the network configuration parameters are configured. For example, the data indicating the operation status of the communication network may include performance data, self-configuration, working parameters, alarms or measurement reports (MR) of each device in the communication network equipment, and the environmental data may include data representing the environment such as a map, building or temperature where the EMS is located.

[0136] 703. The server and the client jointly perform at least one federated training to obtain a machine learning model.

[0137] The server and the client can perform a federated training based on the collected runtime data to obtain a machine learning model. The machine learning model can be used to manage the communication network, thereby realizing the distributed learning of each client set in the communication network through federated learning, and the server aggregates the distributed learning results, so that the server can obtain a machine learning model suitable for managing the entire communication network.

[0138] Specifically, the federated training mentioned in this application is usually jointly participated in by the server and one or more clients to obtain the required machine learning model. Specifically, the server can initiate the federated training, or the server can trigger the client to participate in the federated training, or the client can actively participate in the federated training. For example, on the server side, the server can send an instruction to the client to instruct the client to participate in the federated training; on the client side, after collecting the runtime data, the client can actively or under the trigger of the server to train the locally saved model, and upload the trained model to the server.

[0139] It should be understood that the server-initiated federated training mentioned in this application may specifically include the server sending an instruction to the client to instruct it to participate in the federated training, or it may include the server sending an acquisition strategy to the client, which can be understood as the server triggering the federated training to the client. The server may initiate federated training after it first determines that it is necessary to conduct federated training with the client, or it may trigger the client to perform federated training before each federated training. It should be noted that in some scenarios, after the server sends the acquisition strategy to the client for the first time, the subsequent client may participate in the federated training under the instruction of the server, or the client may actively participate in the federated training after collecting the runtime data without the server triggering it again.

[0140] The client participating in federated training mentioned in this application means that the client uses the collected runtime data to train the locally maintained model and sends the trained model to the server.

[0141] In one possible implementation, the model obtained by federated learning can be used to generate or update the configuration parameters of the communication network, that is, the aforementioned machine learning model can be used to generate or update the configuration parameters of the communication network.

[0142] In one possible implementation, the configuration parameters of the communication network include one or more of the following: scheduling parameters, radio frequency (RF) parameters, or switching parameters, the scheduling parameters include parameters indicating allocation of resources for the wireless network, the RF parameters include parameters indicating adjustment of the radio frequency of the wireless network, and the switching parameters include parameters for instructing a terminal in the wireless network to perform cell switching.

[0143] In addition, it should be noted that when the client is deployed in a network element in a communication network, in some possible scenarios, the configuration parameters of the network element itself can also be configured through the received first machine learning model.

[0144] In one possible implementation, a federated training may specifically include: after the client collects the runtime data, it trains the locally maintained second machine learning model based on the runtime data to obtain a third machine learning model; then, the client sends the third machine learning model to the server; the server fuses the third machine learning models sent by one or more clients to obtain the first machine learning model.

[0145] Optionally, after obtaining the first machine learning model, the server can also send the first machine learning model to each client. That is, after aggregating the received machine learning models, the server can also send the latest machine learning model to each client, so that each client can save the latest machine learning model.

[0146] Of course, after the federated training, the server may not send the latest machine learning model to the client. The specific choice can be made based on the actual application scenario, and this application does not limit this.

[0147] In addition, during the federated training process, the client can obtain the data features of the runtime number, which can be used to represent the features of the data structure included in the runtime data, and send the data features to the server. The server updates the client's acquisition strategy based on the data features, thereby obtaining an acquisition strategy that is more adapted to the client's data structure features.

[0148] The following further describes the federated learning method provided in the application in combination with a specific communication network.

[0149] In the following implementation manner of the present application, the solution provided in the embodiment of the present application can be jointly executed by the NMS and the EMS or by the EMS and the network element as an example for exemplary introduction. Figure 5 and Figure 6 The network architecture of the present invention will introduce the detailed steps of these two implementation methods respectively.

[0150] (I) Joint implementation by NMS and EMS

[0151] In this implementation, the NMS and EMS jointly perform federated training. The NMS generates an acquisition strategy and sends it to the EMS for execution, thereby controlling the local model training process on each EMS, improving learning efficiency, and obtaining a high-performance machine learning model.

[0152] See also Figure 8 , is a flow chart of a method for federated training runtime data provided in an embodiment of the present application, such as Figure 8 As shown, a federated learning method provided in an embodiment of the present application may include the following steps:

[0153] 801. EMS receives the acquisition strategy sent by NMS.

[0154] This acquisition strategy is used to guide EMS on how to collect runtime data and what kind of runtime data to collect to train the local model during the federated learning process, so as to obtain the best federated learning efficiency and network optimization effect.

[0155] Exemplarily, the acquisition strategy may indicate one or more of the following: the number of collections refers to the number of data collection results; the exploration ratio refers to the proportion of data collected based on the exploration mechanism in the data collection results, and its value is usually in the range of 0-100%, or the ratio between the part used to learn new knowledge and the part used to consolidate the learned knowledge (such as the learned data category) (this may also be called the new-old ratio or the collection ratio, etc.), the data collected based on the exploration mechanism is the data collected after the communication network is configured with unused configuration parameters, and the unused configuration parameters are new configuration parameters generated for the communication network, and are usually not obtained by model reasoning; the collection mode refers to the mode of data collection, which is periodic collection, specified collection, or adaptive collection, etc. For ease of understanding, the exploration ratio may also be called the collection ratio, the new ratio, or the new-old ratio, etc.

[0156] More specifically, the exploration ratio can be understood as the ratio of the number of data collected based on the exploration mechanism to the total number of data. The total number of data usually includes: data collected based on the exploration mechanism (used to learn new knowledge) and data collected based on the utilization mechanism (used to consolidate the knowledge learned). Among them, exploration refers to trying new actions (configuration parameter combinations), and the data collected based on the exploration mechanism is the data of the communication network collected when trying to configure new configuration parameters for the communication network. The new configuration parameters can be understood to include randomly generated new configuration parameters, and can also include non-model reasoning configuration parameters such as manually configured configuration parameters. Among them, utilization refers to the action given by model reasoning (here the model is a machine learning model, which is obtained based on historical data training, and can be a model maintained locally by the client or a model sent by the server received by the client). The data collected based on the utilization mechanism is the data of the communication network collected when the configuration parameters obtained by the communication network configuration model reasoning. The data obtained based on the exploration mechanism can be understood as the new knowledge that you want to learn. The data obtained based on the utilization mechanism can be interpreted as the hope to consolidate the knowledge that you have learned.

[0157] It can be understood that, usually, the model training set uses training data to update the model parameters, and the updated parameters can be called "knowledge". In other words, the experience or method of adjusting the configuration parameters of network devices under a given network state so that the network performance reaches the best state can also be called "knowledge". "New knowledge" refers to the experience or method of adjusting configuration parameters that have not yet been mastered, or configuring a new network state or a new configuration parameter combination for the communication network. "Learning new knowledge" means learning new configuration parameter adjustment experience or method by configuring a new configuration parameter combination for the communication network to obtain runtime data, that is, using the runtime data obtained by configuring new configuration parameters for the communication network to update the model parameters. "Consolidating the learned knowledge" means using the learned configuration parameter adjustment experience or method to adjust the configuration parameters of the device to obtain runtime data to strengthen the learned experience or method, such as using the model obtained from the last model training to adjust the configuration parameters of the device to obtain runtime data to update the model parameters again, thereby strengthening the learned experience or method. Setting a certain exploration ratio can enable the model to maintain more effective output in the dimension of "learned knowledge" while learning "new knowledge".

[0158] Specifically, the acquisition strategy will be introduced in steps 802 and 803, which will not be described in detail here.

[0159] Since the NMS manages multiple EMSs, the acquisition strategies received by any two EMSs may be the same or different. For example, when the solution provided by the embodiment of the present application is executed for the first time, the acquisition strategies received by the multiple EMSs are all initial acquisition strategies. With the execution of step 803, the NMS may send acquisition strategies to different EMSs to obtain the best federated learning efficiency and network optimization effect.

[0160] In a possible implementation, the NMS may send initial model parameters to the EMS, so that the EMS uses the received initial model parameters as parameters of the initial local model, and after executing step 803, performs federated training on the initial local model.

[0161] 802. The EMS collects runtime data according to the acquisition strategy.

[0162] After receiving the acquisition strategy, the EMS parses and executes data sampling actions according to the acquisition strategy, and acquires the runtime data of the communication network.

[0163] Specifically, the runtime data may include data indicating the operating state of the communication network or data of the environment in which the equipment in the communication network is located. Here, the data of the environment in which the equipment in the communication network is located is the environmental data in which the EMS is located. Specifically, the runtime data may include network configuration parameters of the managed network elements in the communication network, the network state of the managed network elements before the network configuration parameters are configured, and the network state of the managed network elements after the network configuration parameters are configured. For example, the data indicating the operating state of the communication network may include performance data, self-configuration, working parameters, alarms or measurement reports (MR) of each device in the communication network equipment, and the environmental data may include data representing the environment such as a map, building or temperature where the EMS is located.

[0164] In order to better understand the solution provided by the embodiment of the present application, the solution is introduced below by taking the runtime data including the current network status of the NE managed by the EMS, the network configuration parameters of the NE managed by the EMS, and the network status of the NE managed by the EMS after the network configuration parameters are configured as an example.

[0165] In one possible implementation, after receiving the acquisition strategy, the EMS generates multiple groups of reference configuration parameters for the managed NE. Then, according to the uncertainty of each group of reference configuration parameters, one of the reference configuration parameters with uncertainty higher than a threshold is obtained from the multiple groups of reference configuration parameters as the first network configuration parameter. The uncertainty of the reference configuration parameter is obtained based on the uncertainty of the network state obtained after multiple simulation models perceive that the NE executes the reference configuration parameter in the current network state. Exemplarily, the network configuration parameter is the reference configuration parameter with the highest uncertainty among the multiple groups of reference configuration parameters as an example:

[0166] In one possible implementation, after receiving the acquisition strategy, the EMS generates multiple groups of reference configuration parameters through the maintained local model. The embodiment of the present application also refers to the configuration parameters as actions, that is, the EMS randomly generates multiple actions, each action includes a set of configuration information for the NE.

[0167] In one possible implementation, the EMS generates multiple uniform random actions, where the uniform random actions can be understood as the multiple actions being dispersed, so that the multiple groups of configuration information are not too similar, but have diversity. For example, Latin hypercube sampling (LHS) can be used to generate multiple actions. Next, the uncertainty of each action is determined, and the action with the largest uncertainty is determined as the action to be executed, or determined as the network configuration parameter, or determined as the network configuration parameter to be executed. The greater the uncertainty of the action, the higher the learning value of executing the action, or after executing the action, the model parameters can learn more valuable knowledge. The ensemble learning method can be used to calculate the uncertainty of each action based on the Gaussian distribution output by the simulation model. For example, using multiple simulation models, the input of each simulation model is the current environment of the NE, which may include the network status of the NE, the electronic map where the NE is located, etc., and the input also includes the action to be evaluated for certainty. The output of the simulation model is the probability distribution of the change in the network state of the NE after executing the action. For example, for the network transmission rate, the output of the simulation model is the probability distribution of the rate at each value. Alternatively, it can be understood that the simulation model can predict the change of the network state of the NE and the uncertainty of the simulation model in making this prediction. The uncertainty of each action is determined according to the outputs of the multiple simulation models.

[0168] In one possible implementation, high-risk actions can also be filtered based on the security correction strategy to avoid a significant impact on the network. For example, verification logic based on expert experience can be used to implement the security correction strategy, or a simulation model can be used to predict the impact of actions, thereby filtering out actions whose impact exceeds a certain threshold. Finally, the action with the highest uncertainty is executed, and it is ensured that the action with the highest uncertainty will not have a significant impact on the network. As mentioned above, an action is essentially a set of configuration parameter values ​​of a network element, so executing an action means changing the configuration parameters of the network element, obtaining the changed state of the network element, and obtaining a set of runtime data.

[0169] In one possible implementation, after receiving the acquisition strategy, the EMS acquires the network configuration parameters according to the current network status of the NE managed by the EMS and the model parameters maintained locally by the EMS. For example, when the solution is executed for the first time, the model parameters maintained locally by the EMS may be the initial model parameters issued by the NMS. The current network status of the NE is input into the model to predict the network configuration parameters of the NE, or the network configuration parameter information of the NE is predicted based on the current network status of the NE through the initial model parameters. This is another way to obtain runtime data.

[0170] 803. The EMS performs at least one federated training with the NMS based on the runtime data to obtain a machine learning model.

[0171] Among them, the EMS can be triggered by the NMS to participate in the federated training, or the EMS can autonomously conduct the federated training after collecting the runtime data. For example, the NMS can send instruction data to the EMS to instruct the EMS to participate in the federated training; or the EMS can autonomously conduct the federated training after collecting the runtime data according to the pre-agreed agreement.

[0172] The machine learning model can be used to manage the communication network, such as the machine learning model can be used to generate or update the configuration parameters of the communication network to improve the network quality of the communication network. It can be understood that in the federated training process, the trained machine learning model can be used to generate the configuration parameters of the communication network, and to generate or update the configuration parameters of the communication network. After the configuration parameters of the communication network are generated or updated, the next sampling and federated training process can be performed.

[0173] The configuration parameters output by the machine learning model can be used to schedule resources in the communication network or manage devices, etc. Taking the communication network as a wireless network as an example, the configuration parameters may specifically include but are not limited to one or more of the following: scheduling parameters, resource management parameters, radio frequency (RF) parameters, or switching parameters, etc., wherein the scheduling parameters may include parameters for allocating resources in the wireless network, the RF parameters may include parameters for indicating radio frequency adjustments to the wireless network (such as adjusting the azimuth angle, downtilt angle, beamforming, transmission power, etc.), and the switching parameters include parameters for instructing a terminal accessing the wireless network to perform inter-cell switching.

[0174] For example, scheduling parameters may include parameters for scheduling resources within the communication network, such as the parameter group in the NRDUCELLPDSCH field used to modify the physical downlink shared channel (PDSCH) configuration of the NR distribution unit (DU) cell, which may specifically include the DlInitRank parameter, which indicates the number of downlink initial spatial division multiplexing streams (RANK) (PDSCH spatial division multiplexing layers) before the first channel quality indicator (CQI) is reported after the first sounding reference signal (SRS) is reported after access. RF parameters such as the parameters in the NRDUCELLTRPBEAM field are used to modify the NR DU TDD cell TRP beam configuration; for example, they may specifically include the Tilt parameter to indicate the downtilt angle corresponding to the configured beamforming weight. Switching parameters such as the parameters in the NRCELLINTRAFHOMEAGRP field are used to modify a data record of a NR cell intra-frequency switching measurement parameter group; for example, they may specifically include the IntraFreqHoA3Offset parameter to indicate the offset value of the neighboring cell quality being higher than the serving cell in intra-frequency switching.

[0175] The process of one federated training can be understood by referring to the following description. The machine learning model obtained after the federated training is called the first machine learning model: the EMS trains the second machine learning model based on the runtime data, that is, the local model maintained by the EMS, to obtain the third machine learning model; the EMS sends the third machine learning model to the NMS, and the NMS fuses the third machine learning models received from one or more EMSs to obtain the first machine learning model, that is, the federated model.

[0176] After NMS obtains the first machine learning model through integration, it may not send the first machine learning model to EMS, or it may send the first machine learning model to EMS. If NMS sends the first machine learning model to EMS, EMS may update the locally maintained third machine learning model after receiving the first machine learning model. For example, the third machine learning model may be replaced with the first machine learning model, or the parameter set of the third machine learning model may be integrated with the parameter set of the first machine learning model, etc., to obtain a new locally maintained machine learning model.

[0177] It should be noted that the sending end mentioned in this application sends a machine learning model, such as sending a first machine learning model, sending a third machine learning model, etc., which can be understood as sending a parameter set of the model, such as a structural parameter or a weight parameter of the model. The receiving end receives the machine learning model, which can be understood as receiving a parameter set of the machine learning model. After receiving the parameter set of the model, the receiving end can save the received parameter set or further process the received parameter set.

[0178] Exemplarily, taking the first execution of federated training and the second execution of federated training as examples: EMS trains the model maintained locally by EMS according to the first runtime data collected based on the first acquisition strategy. For example, when the solution is executed for the first time, EMS trains the initial local model using the runtime data to obtain a trained local model. Here, the parameter set of the trained local model is referred to as the initial model parameter.

[0179] EMS sends initial model parameters to NMS;

[0180] After the NMS obtains the initial model parameters sent by multiple EMSs, it fuses the multiple initial model parameters to obtain a machine learning model parameter set, that is, a parameter set of the federated model obtained by the current training, which is referred to as the first federated model parameter set.

[0181] The NMS then sends the first federated model parameter set to multiple EMSs, and the EMS updates the local model based on the received first federated model, or uses the first federated model as a new local model.

[0182] The second federation training can be understood by referring to the following description:

[0183] The EMS trains the model parameters maintained locally by the EMS according to the second runtime data collected based on the second acquisition strategy. For example, when executing the solution for the second time, the EMS trains the new local model using the second runtime data to obtain the parameters of the trained local model, which are referred to as the second model parameters.

[0184] The EMS sends the second model parameters to the NMS;

[0185] After the NMS obtains the second model parameters sent by multiple EMSs, it performs fusion processing on the multiple second model parameters to obtain a second federation model;

[0186] The NMS then sends the second federation model to multiple EMSs.

[0187] The subsequent federated training process can be understood by referring to these two federated trainings and the federated training process described above, and will not be repeated here.

[0188] In one possible implementation, the acquisition strategy is variable. The NMS adjusts the acquisition strategy based on the model parameters obtained from different clients to better ensure the acquisition of the required training data and train a model with better performance. In addition, the NMS can also adjust the acquisition strategy based on the data features obtained from different clients, so that the next sampling can be based on the new acquisition strategy, where the data features refer to the data features of the runtime data collected by the client based on the acquisition strategy, which is explained below in conjunction with a specific implementation.

[0189] See also Fig. 9 , is a flow chart of a federated learning method provided in an embodiment of the present application, such as Fig. 9 As shown, a federated learning method provided in an embodiment of the present application may include the following steps:

[0190] 901. EMS receives the acquisition strategy sent by NMS.

[0191] 902. The EMS collects runtime data according to the acquisition strategy.

[0192] 903. The EMS performs at least one federated training with the NMS based on the runtime data to obtain a machine learning model.

[0193] Steps 901 to 903 can refer to Figure 8 Steps 801 to 803 in the corresponding embodiment are understood and will not be repeated here.

[0194] 904. The NMS updates the acquisition strategy according to the model parameters sent by different clients.

[0195] In step 903, i.e., step 803, the process of federated training is introduced. In a federated training process, after the NMS obtains the model parameters sent by multiple EMSs, the multiple model parameters are fused to obtain the machine learning model parameters.

[0196] In this embodiment, after the NMS obtains the model parameters sent by multiple EMSs, it also updates the acquisition strategy based on the model parameters sent by the multiple EMSs to optimize the acquisition strategy in real time, thereby obtaining the best federated learning efficiency and network optimization effect, so that the trained model has better performance and can configure better parameters for network elements for different network environments.

[0197] In one possible implementation, a first distance between a vector corresponding to a machine learning model parameter and a vector corresponding to a target model parameter is obtained, where the target model parameter is a machine learning model parameter obtained by the NMS during a previous federated training process;

[0198] The exploration ratio in the updated acquisition strategy is determined based on the distance to improve the second distance obtained in the next training process. The second distance is the distance between the vector corresponding to the machine learning model parameters obtained in the next training process and the vector corresponding to the machine learning model parameters obtained in this training process.

[0199] For example, an example is given below:

[0200] NMS maintains a separate multi-armed bandit (MAB) for each EMS, where each arm represents the exploration ratio in the next round of federated learning. The definition of exploration ratio has been introduced above and will not be repeated here. Assume that 7 arms are set for each MAB, representing exploration ratios of 0%, 10%, 25%, 50% and 100% respectively. Taking 10% as an example, it means that the runtime data used to train model parameters to learn new knowledge accounts for 10% of the total runtime data, then the runtime data used to train model parameters to consolidate the learned knowledge accounts for 90% of the total runtime data. Learn a strategy that maximizes the expected reward under a given number of exploration rounds K (K is a positive integer greater than 0). One round of exploration means executing arm A selected in round k (K is a positive integer greater than 0). k The maximization objective is defined as:

[0201]

[0202] The maximization goal can also be understood as maximizing the expected gain, such as maximizing network energy saving, where the reward R k It is defined as the distance (such as cosine distance) between all parameter layer weight matrices of the neural networks of two federated model instances in two consecutive rounds of federated learning. It can be understood by referring to the definition of the first distance mentioned above and will not be repeated here.

[0203] Each arm of each EMS-specific MAB maintains two parameters:

[0204] a) After k rounds, the number of times arm a is run is:

[0205] m=0

[0206] b) After k rounds, the empirical average value of arm a (new knowledge acquired) is:

[0207]

[0208] Given a parameter N (N is a positive integer greater than 0 and less than x, x is the number of arms) and a confidence upper bound μ (a real number between 0 and 1), MAB selects which arm according to the following formula:

[0209]

[0210] Each MAB in NMS can generate an acquisition strategy for each EMS according to the above formula (in this embodiment, the exploration ratio is an important field in the acquisition strategy). After this round of federated learning is completed, NMS obtains a new federated model (i.e., new machine learning model parameters). NMS calculates the cosine distance between the new and old federated model parameters (i.e., old machine learning model parameters) to set rewards for each MAB and update the parameters of MAB to improve itself. Qualitatively speaking, the core logic of its improvement is to maximize the new knowledge learned. The larger the cosine distance, the more new knowledge has been learned.

[0211] In a possible implementation, the EMS may also obtain data features of the runtime data, and the NMS may consider the data features of the runtime data when updating the acquisition strategy. This is described below in conjunction with a specific embodiment.

[0212] See also Fig.10 , which is a flowchart of a federated learning method provided in an embodiment of the present application, may include the following steps:

[0213] 1001. EMS receives the acquisition strategy sent by NMS.

[0214] 1002. EMS collects runtime data according to the acquisition strategy.

[0215] 1003. EMS performs at least one federated training with NMS based on the runtime data to obtain a machine learning model.

[0216] Steps 1001 to 1003 can refer to Figure 8 Steps 801 to 803 in the corresponding embodiment are understood and will not be repeated here.

[0217] 1004. EMS obtains data characteristics of runtime data.

[0218] Any feature of the data may be obtained. For example, in one possible implementation, data distribution features of runtime data may be obtained.

[0219] 1005. The EMS sends the data feature to the NMS.

[0220] 1006. The NMS updates the acquisition strategy according to the model parameters and data features sent by different clients.

[0221] How to update the exploration ratio in the acquisition strategy based on the trained model parameters sent by different clients has been discussed in Fig. 9 The corresponding embodiments have been introduced and will not be repeated here. Here is a method for updating the acquisition strategy using the data features sent by different clients:

[0222] The total number of samples and the sampling mode are mainly based on the sample data characteristics of each local model, so as to maximize the coverage of scenarios and data diversity while reducing ineffective learning costs. The scenarios can be identified from the data in combination with domain knowledge, and the distribution and diversity of data in different scenarios and the distribution in the time dimension can be used to determine which scenarios need to continue to supplement data, which sampling mode to use, and which scenarios have sufficient data and do not need excessive sampling. For example:

[0223] NMS determines the number distribution of runtime data collected by multiple EMSs in different scenarios based on the data characteristics sent by multiple EMSs; then determines the updated adoption strategy based on the number distribution, and the total number of runtime data collected by each EMS indicated by the updated adoption strategy is used to balance the number distribution of runtime data collected by multiple EMSs in different scenarios during the next training process. The scenario is determined by the specific business. For example, in an online network, the scenario mainly involves but is not limited to dimensions such as network standards (UMTS, NR, GSM, etc.), number of frequency points, traffic load characteristics (low traffic, medium traffic, high traffic, etc.), and geographical characteristics (suburbs, general urban areas, dense urban areas, government agencies, schools, highways, rivers, etc.). For example, GSM single-standard high traffic load scenario, GSM / LTE multi-standard multi-frequency common mode low traffic load scenario, etc. It can also be combined with machine learning algorithms to automatically divide scenarios using wireless network parameters, configurations, performance indicators, measurement reports and other data.

[0224] This application introduces active sampling technology. Based on the evaluation of the federated model, the NMS actively sends the acquisition strategy to the EMS, so that each network element generates sample data with high learning value for training, so as to improve the training efficiency of the federated model and enhance the optimization effect of the model. For example, the above embodiment can be understood with reference to the architecture diagram shown in Table 1:

[0225] Table 1:

[0226]

[0227]

[0228] For ease of understanding, the following table 1 and Fig.11 The process of federated training in the method provided in this application is exemplarily introduced.

[0229] See also Fig.11 , taking N operators’ EMS as an example, N can be any positive integer, such as Fig.11 EMS 1 to EMS N shown in the figure provide a detailed introduction to the federated training process in the federated learning provided in this application.

[0230] 1101: Based on the evaluation feedback of the federated model, the NMS self-improves and generates an acquisition strategy for each EMS.

[0231] The adaptive exploration controller determines the subsequent differentiated acquisition strategy of each EMS based on the evaluation of the effect of the current federated model and the sample data characteristics of each local model (the acquisition strategy of each EMS can be different). For example, the total number of samples in the next round of a certain EMS, the sampling mode (that is, the mode in which NMS collects data) and the exploration ratio, etc. Step 1 can refer to Figures 8 to 10 For more information, see the description of the acquisition strategy in .

[0232] 1102: The NMS sends federation model parameters to EMS 1 to EMS N.

[0233] EMS 1 to EMS N may obtain the local model based on the received parameters of the federated model, for example, the received federated model parameters may be directly used as the parameters of the local model.

[0234] The implementation form of the federated model is usually a neural network. The neural network model is a typical machine learning model, and its model parameters include the order, size or weight matrix of each layer used to characterize the model structure. The model parameters mentioned in this application can represent the weight matrix of the neural network.

[0235]

[0236] Federal model parameter explanation table:

[0237] model_version Model version number layer Neural Network Layers layer_id Layer ID layer_size Layer size weight_matrix The weight matrix of this layer

[0238] 1103: The NMS sends the acquisition policy to EMS 1 to EMS N.

[0239] Among them, step 1103 can refer to the description of the aforementioned step 801, and the similarities are not repeated here. The difference is that in the aforementioned step 801, only one EMS is introduced as an example, and in step 1101, the acquisition strategy is sent to N EMSs. If the current time is the first time to execute the method steps provided by this application, the acquisition strategy sent to the N EMSs can be the same acquisition strategy. If the current sending is not the first time to execute the method steps provided by this application, the acquisition strategy sent to the N EMSs can be the same or different.

[0240] Acquisition strategies include but are not limited to one or more of the following information combinations:

[0241]

[0242] Parameter explanation table:

[0243]

[0244]

[0245] Parameter explanation table in the example of learning configuration parameters:

[0246]

[0247] 1104: EMS 1 generates a sampling action based on the received policy and performs sampling.

[0248] 1105: EMS 1 generates a sampling action based on the received policy and performs sampling.

[0249] Step 1104 is similar to step 1105. Taking EMS as the execution subject as an example, step 1104 and step 1105 are described in combination below.

[0250] After receiving the message of the acquisition strategy, EMS uses the local model to generate sampling actions. This is divided into two parts: exploration and utilization. The sum of the number of samples in the two parts is the total number of samples.

[0251] For the exploration part, the number of sampling actions is equal to the exploration ratio multiplied by the total number of samples. Generate M uniform random actions through the local model (for example, Latin hypercube sampling is used to ensure diversity) and sort them based on uncertainty, and then select the sampling action with the largest uncertainty. The uncertainty here refers to the perceived uncertainty of the model, which can be calculated based on the Gaussian distribution of the model output using an ensemble learning method. It represents the value of the sampling action, that is, the sampling action with greater uncertainty has a higher learning value. After generating the sampling action, you can also filter high-risk sampling actions based on the security correction strategy to avoid a greater impact on the network. For example, you can use verification logic based on expert experience to implement the security correction strategy, or you can use a simulation model to predict the impact of the sampling action, thereby filtering out sampling actions whose impact exceeds a certain threshold. Finally, execute the sampling action. The sampling action is essentially a set of configuration parameter values ​​of the network element, so executing the sampling action is to change the configuration parameters of the network element.

[0252] For the part of utilization, the number of sampling actions is equal to the total number of samplings minus the number of sampling actions for exploration. Sampling actions can be directly generated by reasoning through the local model. This means that unlike exploration, which is to pursue new knowledge, utilization is to verify and consolidate existing knowledge.

[0253] 1106: EMS1 collects network status changes as learning samples, and performs incremental training based on the distributed federated model to form local model 1.

[0254] 1107: EMS1 collects network status changes as learning samples, and performs incremental training on the basis of the distributed federated model to form a local model N.

[0255] Step 1106 is similar to step 1107. Taking EMS as the execution entity as an example, step 1106 and step 1107 are described in combination below.

[0256] After the sampling action is executed, the network status will inevitably change. EMS collects the change data of the network status as training samples and stores them in the buffer. In addition, simulation training samples are generated based on the simulation model, and their number is equal to the total number of training times minus the number of sampling actions, which are also stored in the buffer. EMS obtains the current local model and uses the training samples in the buffer to update the local model in an incremental learning manner. The model training process is: EMS collects the difference in network status before and after the configuration parameters of the communication network are executed, and based on the difference, the weight parameters of the local model are reversely updated, so that the configuration parameters output by the local model are configured in the communication network. After that, the communication network can achieve a better network status. It can be understood that parameter adjustments in the communication network will affect the network status of the communication network. Multiple training samples can be obtained by adjusting the configuration parameters of the communication network multiple times.

[0257] The network status difference of the communication network can reflect the configuration parameters output by the local model after being configured in the communication network.

[0258] 1108: EMS1 uploads the parameters and sample data features of local model 1 to NMS.

[0259] 1109: EMS N uploads the parameters and sample data features of the local model N to the NMS.

[0260] Step 1108 is similar to step 1109. Taking EMS as the execution entity as an example, step 1108 and step 1109 are described in combination below.

[0261] The structure of the local model is consistent with the federal model, so the model parameters uploaded here are similar to those in step 1103. It includes but is not limited to one or more of the following information combinations:

[0262]

[0263] Parameter explanation table:

[0264]

[0265] Local model parameter explanation table:

[0266]

[0267] Sample data feature explanation table:

[0268]

[0269] 1110: NMS aggregates the local model parameters of each EMS to form a new federated model.

[0270] NMS communicates with each EMS asynchronously to monitor the local model upload status of each EMS. When the local model parameters of all EMSs are obtained, or the timeout has occurred and the successful upload ratio reaches the threshold, model aggregation is performed. This embodiment adopts the aggregation method commonly used in federated learning to perform weighted averaging of the parameters of all local models to obtain a new federated learning model. However, it is not limited to a specific model aggregation method.

[0271] Then, the NMS drives the next round of federated learning process, that is, continues to execute steps 1101 to 1110.

[0272] (II) Joint execution of EMS and network elements

[0273] See also Fig.12 , is a flow chart of a federated learning method provided in an embodiment of the present application, such as Fig.12 As shown, a federated learning method provided in an embodiment of the present application may include the following steps:

[0274] 1201. The network element receives the acquisition strategy sent by EMS.

[0275] 1202. The network element collects runtime data according to the acquisition strategy.

[0276] 1203. The network element performs at least one federated training with the EMS based on the runtime data to obtain a machine learning model.

[0277] Fig.12 The corresponding embodiments can refer to Figure 8 The difference is that the two parties of the interaction have changed. Fig.12 The network element in the corresponding embodiment is equivalent to Figure 8 The EMS in the corresponding embodiment, Fig.12 The EMS in the corresponding embodiment is equivalent to Figure 8 The NMS in the corresponding embodiment.

[0278] See also Fig.13 , which is a flowchart of a federated learning method provided in an embodiment of the present application, may include the following steps:

[0279] 1301. The network element receives the acquisition strategy sent by EMS.

[0280] 1302. The network element collects runtime data according to the acquisition strategy.

[0281] 1303. The network element performs at least one federation training together with the EMS based on the runtime data.

[0282] 1304. EMS updates the acquisition strategy according to the model parameters sent by different clients.

[0283] Fig.13 The corresponding embodiments can refer to Fig. 9 The difference is that the two parties of the interaction have changed. Fig.13 The network element in the corresponding embodiment is equivalent to Fig. 9 The EMS in the corresponding embodiment, Fig.13 The EMS in the corresponding embodiment is equivalent to Fig. 9 The NMS in the corresponding embodiment.

[0284] See also Fig.14 , is a flow chart of a federated learning method provided in an embodiment of the present application, such as Figure 4 As shown, a federated learning method provided in an embodiment of the present application may include the following steps:

[0285] 1401. The network element receives the acquisition strategy sent by EMS.

[0286] 1402. The network element collects runtime data according to the acquisition strategy.

[0287] 1403. The network element performs at least one federated training with the EMS based on the runtime data to obtain a machine learning model.

[0288] Steps 1401 to 1403 can refer to Figure 8 Steps 801 to 803 in the corresponding embodiment are understood and will not be repeated here.

[0289] 1404. The network element obtains data characteristics of the runtime data.

[0290] 1405. The network element sends the data feature to the EMS.

[0291] 1406. EMS updates the acquisition strategy according to the model parameters and data features sent by different clients.

[0292] Fig.14 The corresponding embodiments can refer to Fig.10 The difference is that the two parties of the interaction have changed. Fig.14 The network element in the corresponding embodiment is equivalent to Fig.10 The EMS in the corresponding embodiment, Fig.14 The EMS in the corresponding embodiment is equivalent to Fig.10 The NMS in the corresponding embodiment.

[0293] The above is an introduction to a federated learning method provided in an embodiment of the present application. Based on the same inventive concept of the above method embodiment, the embodiment of the present application also provides a federated learning device. Corresponding to the above embodiment, the federated learning device can be an NMS, an EMS or a network element. The NMS and the EMS are used to perform the above Figures 4 to 6 The method described in the corresponding embodiment, EMS and network element are used to perform the above Figure 10 to Figure 10 The method described in the corresponding embodiment.

[0294] The above mainly introduces the solution provided by the embodiment of the present application from the perspective of the method. In order to realize the above functions, it includes hardware structures and / or software modules corresponding to the execution of each function. Those skilled in the art should easily realize that, in combination with the units and algorithm steps of each example described in the embodiments disclosed herein, the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the present application.

[0295] The embodiments of the present application can divide the functional modules of each client or server (such as NMS, EMS or NE) according to the above method examples. For example, each functional module can be divided according to each function, or two or more functions can be integrated into one processing module. The above integrated modules can be implemented in the form of hardware or in the form of software functional modules. It should be noted that the division of modules in the embodiments of the present application is schematic and is only a logical functional division. There may be other division methods in actual implementation. The following introduces the device for executing the method flow provided by the present application.

[0296] See also Fig.15 , a schematic diagram of the structure of a client provided by the present application. The client may include:

[0297] The transceiver module 1501 and the processing module 1502. The transceiver module 1501 can be used to execute the aforementioned Figures 7 to 14 The processing module 1502 can be used to perform the sending and receiving actions performed by the client. Figures 7 to 14 The processing actions performed by the client, that is, other actions except sending and receiving actions.

[0298] Specifically, the transceiver module 1501 is used to receive an acquisition strategy sent by a server, where the acquisition strategy is a strategy for collecting data from a communication network to obtain data for federated training;

[0299] The processing module 1502 is used to collect the operation period data of the communication network according to the acquisition strategy;

[0300] The processing module is also used to participate in at least one federal training based on runtime data to obtain a first machine learning model, which is used to manage the communication network.

[0301] In one possible implementation, a federated training performed by the processing module 1502 may include: training a second machine learning model based on runtime data to obtain a third machine learning model; sending the third machine learning model, and the third machine learning model is used to determine the first machine learning model.

[0302] In a possible implementation, the transceiver module 1501 is also used to receive a parameter set of the first machine learning model.

[0303] In a possible implementation, the aforementioned communication network may include a wireless network, and the wireless network may include at least two devices in a network management system NMS, an element management system EMS, or a network element NE.

[0304] In a possible implementation, the server is deployed in a network management system NMS, and the client is deployed in an element management system EMS.

[0305] In a possible implementation manner, the server is deployed in an element management system EMS, and the client is deployed on a network element NE.

[0306] In a possible implementation manner, the aforementioned runtime data of the communication network includes data representing the operating status of the communication network and / or data indicating the environment in which the devices in the communication network are located.

[0307] In a possible implementation manner, the network configuration parameters of the managed network elements in the communication network, the network status of the managed network elements before the network configuration parameters are configured, and the network status of the managed network elements after the network configuration parameters are configured.

[0308] In one possible implementation, the first machine learning model is used to generate or update configuration parameters of the communication network.

[0309] In one possible implementation, the acquisition strategy indicates the total number of runtime data, the exploration ratio, or the mode of collecting runtime data. The exploration ratio refers to the proportion of data collected based on the exploration mechanism in the results of data collection. The data collected based on the exploration mechanism refers to the data collected after configuring configuration parameters that have not been configured for the communication network, or the ratio between the runtime data used to train model parameters to learn new knowledge and the runtime data used to train model parameters to consolidate the learned knowledge. The mode of collecting runtime data may include periodic collection, specified collection, adaptive collection, etc.

[0310] In a possible implementation, the acquisition strategy indicates a sampling period or a number of sampling scenarios, where the sampling period may include a period for collecting runtime data, and the number of sampling scenarios may include a number of samples of data for each scenario.

[0311] In one possible implementation, the first machine learning model may be specifically used to generate or update configuration parameters of the communication network.

[0312] In a possible implementation, the configuration parameters of the communication network may specifically include, but are not limited to, one or more of the following: scheduling parameters, radio frequency (RF) parameters, or switching parameters. The scheduling parameters may include parameters for scheduling resources in the communication network, the RF parameters may include parameters indicating wireless resources in the communication network, and the switching parameters include parameters for instructing devices in the communication network to perform resource switching.

[0313] See also Fig.16, a schematic diagram of the structure of a server provided in the present application, the server may include:

[0314] The transceiver module 1601 and the processing module 1602. The transceiver module 1601 can be used to execute the aforementioned Figures 7 to 14 The processing module 1602 can be used to perform the aforementioned sending and receiving actions performed by the server. Figures 7 to 14 The processing actions performed by the server in the packet, that is, other actions except sending and receiving actions.

[0315] The transceiver module 1601 is used to send an acquisition strategy, where the acquisition strategy is a strategy for collecting data from a communication network to obtain data for federated training;

[0316] The processing module 1602 is used to initiate at least one federated training based on the runtime data of the communication network collected according to the acquisition strategy to obtain a first machine learning model, and the first machine learning model is used to manage the communication network.

[0317] In one possible implementation, the processing module 1602 and the client jointly perform at least one federated training based on runtime data, which may include: receiving at least one set of third machine learning models; and fusing at least one set of third machine learning models to obtain a first machine learning model.

[0318] In one possible implementation, the transceiver module 1601 is also used to send the first machine learning model, so that the client can update the local machine learning model based on the latest machine learning model received, so that the client's model can be kept up to date.

[0319] In one possible implementation, the processing module 1602 is further used to update the acquisition strategy according to the third machine learning model.

[0320] In a possible implementation, the transceiver module 1601 is further used to receive data features, where the data features are features based on data collected under the instruction of the acquisition strategy;

[0321] The processing module 1602 is further configured to determine an update acquisition strategy according to data characteristics.

[0322] In a possible implementation, the aforementioned communication network wireless network may include at least two devices in a network management system NMS, an element management system EMS or a network element NE.

[0323] In a possible implementation, the server is deployed in a network management system NMS, and the client is deployed in an element management system EMS.

[0324] In a possible implementation manner, the server is deployed in an element management system EMS, and the client is deployed on a network element NE.

[0325] In a possible implementation manner, the runtime data includes the current network state of the managed network element in the communication network, the network configuration parameters of the managed network element, and the network state of the managed network element after the network configuration parameters are executed.

[0326] In one possible implementation, the acquisition strategy may indicate one or more of the following: collection quantity, indicating the number of data collection results; exploration ratio, indicating the proportion of data collected based on the exploration mechanism in the data collection results, where the data collected based on the exploration mechanism refers to the data collected after configuring configuration parameters that have not been configured for the communication network, or the ratio between the part used to learn new instructions and the part used to consolidate the learned knowledge; collection mode, indicating whether the data collection mode is periodic collection, specified collection, or adaptive collection, etc.

[0327] In a possible implementation, the acquisition strategy also indicates a sampling period or the number of sampling scenarios. The sampling period may include a period for collecting data, i.e., a time interval between two adjacent data collection operations on the communication network. The number of sampling scenarios may include the number or number of data items to be collected in the sampling scenario.

[0328] In one possible implementation, the model obtained by federated learning can be used to generate or update the configuration parameters of the communication network, that is, the aforementioned first machine learning model can be used to generate or update the configuration parameters of the communication network.

[0329] In a possible implementation, the configuration parameters of the communication network may specifically include, but are not limited to, one or more of the following: scheduling parameters, radio frequency (RF) parameters, or switching parameters. The scheduling parameters may include parameters for scheduling resources in the communication network, the RF parameters may include parameters for indicating wireless resources in the communication network, and the switching parameters include parameters for indicating that a device in the communication network switches resources. Therefore, the parameters of various dimensions of the communication network may be adjusted through the parameters output by the machine learning model to achieve a better network state of the communication network.

[0330] like Fig.17 , which is a schematic diagram of the hardware structure of a computer device 170 provided in an embodiment of the present application. The computer device 170 can be used to implement the functions of a server or a client in federated learning.

[0331] Fig.17The computer device 170 shown may include: a processor 1701 , a memory 1702 , a communication interface 1703 , and a bus 1704 . The processor 1701 , the memory 1702 , and the communication interface 1703 may be connected via a bus 1704 .

[0332] The processor 1701 is the control center of the computer device 170, and may be a general-purpose central processing unit (CPU) or other general-purpose processors, etc. The general-purpose processor may be a microprocessor or any conventional processor, etc.

[0333] As an example, the processor 1701 may include one or more CPUs, such as Fig.17 CPU 0 and CPU 1 are shown in .

[0334] The memory 1702 may be a read-only memory (ROM) or other type of static storage device that can store static information and instructions, a random access memory (RAM) or other type of dynamic storage device that can store information and instructions, an electrically erasable programmable read-only memory (EEPROM), a disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited to these.

[0335] In one possible implementation, the memory 1702 may exist independently of the processor 1701. The memory 1702 may be connected to the processor 1701 via a bus 1704 and used to store data, instructions, or program codes. When the processor 1701 calls and executes the instructions or program codes stored in the memory 1702, the machine learning model management method provided in the embodiment of the present application can be implemented, for example, Fig.10 The machine learning model management method shown.

[0336] In another possible implementation, the memory 1702 may also be integrated with the processor 1701 .

[0337] The communication interface 1703 is used for connecting the computer device 170 with other devices through a communication network, and the communication network may be Ethernet, a radio access network (RAN), a wireless local area network (WLAN), etc. The communication interface 1703 may include a receiving unit for receiving data and a sending unit for sending data.

[0338] The bus 1704 may be an industry standard architecture (ISA) bus, a peripheral component interconnect (PCI) bus, or an extended industry standard architecture (EISA) bus. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Fig.17 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0339] It should be pointed out that Fig.17 The structure shown in the figure does not constitute a limitation on the computer device 170, except Fig.17 In addition to the components shown, computer device 170 may include more or fewer components than shown, or combine certain components, or arrange the components differently.

[0340] Through the description of the above implementation mode, the technicians in the field can clearly understand that the present application can be implemented by means of software plus necessary general hardware, and of course, it can also be implemented by special hardware including special integrated circuits, special CPUs, special memories, special components, etc. In general, all functions completed by computer programs can be easily implemented by corresponding hardware, and the specific hardware structure used to implement the same function can also be various, such as analog circuits, digital circuits or special circuits. However, for the present application, software program implementation is a better implementation mode in more cases. Based on such an understanding, the technical solution of the present application is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a readable storage medium, such as a computer floppy disk, a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a disk or an optical disk, etc., including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0341] In the above embodiments, all or part of the embodiments may be implemented by software, hardware, firmware or any combination thereof. When implemented by software, all or part of the embodiments may be implemented in the form of a computer program product.

[0342] In an embodiment of the present application, a computer-readable storage medium is also provided, in which a program for training a model or performing an inference task is stored. When the program is run on a computer, the computer performs the above-mentioned Figures 4 to 12 All or part of the steps in the method described in the embodiment shown in FIG.

[0343] The embodiment of the present application also provides a digital processing chip. The digital processing chip integrates a circuit and one or more interfaces for implementing the above-mentioned processor or the function of the processor. When the digital processing chip integrates a memory, the digital processing chip can complete the method steps of any one or more embodiments in the above-mentioned embodiments. When the digital processing chip does not integrate a memory, it can be connected to an external memory through a communication interface. The digital processing chip implements the method steps of any one or more embodiments in the above-mentioned embodiments according to the program code stored in the external memory.

[0344] A computer program product is also provided in the embodiment of the present application, and the computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from a website site, a computer, a server, or a data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server, or data center. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a server or a data center that includes one or more available media integrations. The available medium may be a magnetic medium, (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state hard disk Solid State Disk (SSD)), etc.

[0345] A person skilled in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing the relevant hardware through a program, and the program can be stored in a computer-readable storage medium, which can include: ROM, RAM, disk or CD, etc.

[0346] The terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments described here can be implemented in a sequence other than the content illustrated or described here. The term "and / or" in the present application is only a kind of association relationship describing the associated objects, indicating that there can be three kinds of relationships, for example, A and / or B can be represented: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the associated objects before and after are an "or" relationship. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or modules is not necessarily limited to those steps or modules clearly listed, but may include other steps or modules that are not clearly listed or inherent to these processes, methods, products or devices. The naming or numbering of the steps in this application does not mean that the steps in the method flow must be executed in the time / logical sequence indicated by the naming or numbering. The named or numbered process steps can change the execution order according to the technical purpose to be achieved, as long as the same or similar technical effects can be achieved. The division of modules in this application is a logical division. There may be other division methods when it is implemented in actual applications. For example, multiple modules can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between each other shown or discussed can be through some ports, and the indirect coupling or communication connection between modules can be electrical or other similar forms, which are not limited in this application. In addition, the modules or submodules described as separate components may or may not be physically separated, may or may not be physical modules, or may be distributed in multiple circuit modules, and some or all of the modules can be selected according to actual needs to achieve the purpose of the present application solution.

Claims

1. A federated learning method, characterized in that: include: receiving an acquisition strategy, wherein the acquisition strategy is a strategy for collecting data from a communication network to obtain data for federated training; collecting runtime data of the communication network according to the acquisition strategy; Participate in at least one federated training based on the runtime data to obtain a first machine learning model, wherein the first machine learning model is used to manage the communication network.

2. The method according to claim 1, characterized in that: One federal training session includes: Training a second machine learning model based on the runtime data to obtain a third machine learning model; The third machine learning model is sent, and the third machine learning model is used to determine the first machine learning model.

3. The method according to claim 2, characterized in that The method further comprises: Receive the first machine learning model.

4. The method according to any one of claims 1 to 3, characterized in that The runtime data includes data representing an operational status of the communication network and / or data indicating an environment in which a device in the communication network is located.

5. The method according to any one of claims 1 to 4, characterized in that The runtime data includes: network configuration parameters of managed network elements in the communication network, the network state of the managed network element before being configured with the network configuration parameters, and The network state of the managed network element after being configured with the network configuration parameters.

6. The method according to any one of claims 1 to 5, characterized in that: The acquisition strategy indicates any one or more of the following: The number of collections refers to the number of results of the data collection; The exploration ratio refers to the ratio of data collected based on the exploration mechanism in the result of the data collection, and the data collected based on the exploration mechanism refers to the data collected after configuring the communication network with configuration parameters that have not been configured before and performing the data collection; The collection mode refers to the mode of data collection, which can be periodic collection, specified collection or adaptive collection.

7. The method according to claim 6, characterized in that The acquisition strategy further indicates a sampling period or a scene sampling quantity, wherein the sampling period is a time interval between two adjacent data collection operations on the communication network, and the scene sampling quantity includes the quantity of data to be collected in a sampling scene.

8. The method according to any one of claims 1 to 7, characterized in that The communication network includes a wireless network, and the wireless network includes at least two devices in a network management system NMS, an element management system EMS or a network element NE.

9. The method according to any one of claims 1 to 8, characterized in that The first machine learning model is used to generate or update configuration parameters of the communication network.

10. The method according to claim 9, characterized in that The communication network includes a wireless network, and the configuration parameters of the communication network include one or more of the following: scheduling parameters, radio frequency (RF) parameters, or switching parameters. The scheduling parameters include parameters indicating allocation of resources for the wireless network, the RF parameters include parameters indicating adjustment of the radio frequency of the wireless network, and the switching parameters include parameters for instructing a terminal accessing the wireless network to perform cell switching.

11. A federated learning method, characterized in that: include: Sending an acquisition strategy, where the acquisition strategy is a strategy for collecting data from a communication network to obtain data for federated training; Initiate at least one federated training based on runtime data of the communication network collected according to the acquisition strategy to obtain a first machine learning model, wherein the first machine learning model is used to manage the communication network.

12. The method according to claim 11, characterized in that The at least one federal training includes: receiving at least one set of third machine learning models; The at least one group of third machine learning models is fused to obtain the first machine learning model.

13. The method according to claim 12, characterized in that The method further comprises: Send the first machine learning model.

14. The method according to claim 12 or 13, characterized in that The method further comprises: The acquisition strategy is updated according to the at least one set of third machine learning models.

15. The method according to any one of claims 11 to 14, characterized in that: The method further comprises: receiving data features, the data features being features based on data collected under the instruction of the acquisition strategy; Determine and update the acquisition strategy according to the data characteristics.

16. The method according to any one of claims 11 to 15, characterized in that: The acquisition strategy indicates one or more of the following: The collection quantity refers to the total number of runtime data to be collected; The exploration ratio refers to the ratio of data collected based on the exploration mechanism in the result of the data collection, and the data collected based on the exploration mechanism refers to the data collected after configuring the communication network with configuration parameters that have not been configured before and performing the data collection; The acquisition mode refers to whether the data acquisition mode is periodic acquisition, specified acquisition, or adaptive acquisition.

17. The method according to claim 16, characterized in that The acquisition strategy further indicates a sampling period or a scene sampling quantity, wherein the sampling period is a time interval between two adjacent data collection operations on the communication network, and the scene sampling quantity includes the quantity of data to be collected in a sampling scene.

18. The method according to any one of claims 11 to 17, characterized in that The communication network includes a wireless network, and the communication network includes at least two of a network management system NMS, an element management system EMS or a network element NE.

19. The method according to any one of claims 11 to 18, characterized in that The first machine learning model is used to generate or update configuration parameters of the communication network.

20. The method according to claim 19, characterized in that The communication network includes a wireless network, and the configuration parameters of the communication network include one or more of the following: scheduling parameters, radio frequency (RF) parameters, or switching parameters. The scheduling parameters include parameters for scheduling resources in the communication network, the RF parameters include parameters indicating wireless resources in the communication network, and the switching parameters include parameters for instructing devices in the communication network to perform resource switching.

21. A federated learning method, characterized in that: include: The server sends an acquisition strategy to the client, where the acquisition strategy is a strategy for collecting data from a communication network to obtain data for federated training; The client collects the operation period data of the communication network according to the received acquisition strategy; The server and the client jointly participate in at least one federated training based on the runtime data to obtain a first machine learning model, and the first machine learning model is used to manage the communication network.

22. The method according to claim 21, characterized in that One federal training session includes: The client trains a second machine learning model based on the runtime data to obtain a third machine learning model; The client sends the third machine learning model to the server, where the third machine learning model is used to determine the first machine learning model; The server performs fusion processing on the parameter set of the third machine learning model sent by the client to obtain the first machine learning model.

23. The method according to claim 22, characterized in that The method further comprises: The server sends the first machine learning model to the client.

24. The method according to any one of claims 21 to 23, characterized in that The runtime data includes data representing an operational status of the communication network and / or data indicating an environment in which a device in the communication network is located.

25. The method according to any one of claims 21 to 24, characterized in that The runtime data includes: network configuration parameters of managed network elements in the communication network, the network state of the managed network element before being configured with the network configuration parameters, and The network state of the managed network element after being configured with the network configuration parameters.

26. The method according to claim 25, characterized in that The method further comprises: The server updates the acquisition strategy according to the third machine learning model.

27. The method according to any one of claims 21 to 26, characterized in that The method further comprises: The client obtains data characteristics of the runtime data; The client sends the data feature to the server; The server updates the acquisition strategy according to the data characteristics.

28. The method according to any one of claims 21 to 27, characterized in that The acquisition strategy indicates one or more of the following: The number of collections refers to the number of results of the data collection; The exploration ratio refers to the ratio of data collected based on the exploration mechanism in the result of the data collection, and the data collected based on the exploration mechanism refers to the data collected after configuring the communication network with configuration parameters that have not been configured before and performing the data collection; The collection mode refers to the mode of data collection, which can be periodic collection, specified collection or adaptive collection.

29. The method according to claim 28, characterized in that The acquisition strategy further indicates a sampling period or a scene sampling quantity, wherein the sampling period is a time interval between two adjacent data collection operations on the communication network, and the scene sampling quantity includes the quantity of data to be collected in a sampling scene.

30. The method according to any one of claims 21 to 29, characterized in that The communication network includes a wireless network, and the wireless network includes at least two of a network management system NMS, an element management system EMS or a network element NE.

31. The method according to any one of claims 21 to 30, characterized in that The first machine learning model is used to generate or update configuration parameters of the communication network.

32. The method according to claim 31, characterized in that The communication network includes a wireless network, and the configuration parameters of the communication network include one or more of the following: scheduling parameters, radio frequency (RF) parameters, or switching parameters. The scheduling parameters include parameters indicating allocation of resources for the wireless network, the RF parameters include parameters indicating adjustment of the radio frequency of the wireless network, and the switching parameters include parameters for instructing a terminal accessing the wireless network to perform cell switching.

33. A client, characterized in that: include: A transceiver module, used to perform the steps of receiving data or sending data as described in any one of claims 1 to 10; A processing module, used to execute the step of processing data as described in any one of claims 1-10.

34. A server, characterized in that: include: A transceiver module, used to perform the steps of receiving data or sending data as described in any one of claims 11 to 20; A processing module, configured to execute the step of processing data as claimed in any one of claims 11 to 20.

35. A communication system, characterized in that: The system comprises a server and at least one client; The server is used to execute the method according to any one of claims 11 to 20; The at least one client is used to execute the method according to any one of claims 1 to 10.

36. A client, characterized in that: include: A memory storing executable program instructions; and A processor, wherein the processor is used to be coupled with the memory, read and execute instructions in the memory, and trigger the client to implement the method according to any one of claims 1 to 10.

37. A server, characterized in that: include: A memory storing executable program instructions; and A processor, wherein the processor is used to be coupled to the memory, read and execute instructions in the memory, and trigger the server to implement the method according to any one of claims 11 to 20.

38. A computer-readable storage medium comprising instructions, which, when executed on a computer, enables the computer to execute the method according to any one of claims 1 to 10, or to execute the method according to any one of claims 11 to 20.

Images