Federated learning method, apparatus, device, and readable storage medium

Abstract

The application discloses a federated learning method, device, equipment and readable storage medium, and relates to the technical field of communication. The method comprises the following steps: predicting the computing power resources and / or network resources of at least one first client based on a timing large model; determining a task efficiency factor of the first client based on the computing power resources and / or network resources of the first client; selecting a second client from the first client based on the task efficiency factor, wherein the second client is a client participating in federated learning; receiving first model parameters sent by a third client, and performing aggregation according to the first model parameters, wherein the third client is a client selected from the second client according to the task efficiency factor, and the first model parameters are obtained by performing primary aggregation on model parameters of the third client and model parameters of other clients.

Description

Technical Field

[0001] This application belongs to the field of communication technology, specifically relating to a federated learning method, apparatus, device, and readable storage medium. Background Technology

[0002] Federated learning, as an innovative machine learning paradigm, enables multi-client joint model training through a decentralized collaborative mechanism, while ensuring that the original training data of each participant remains local and is not shared. Its core working principle is that each client uses its local dataset for model training, only uploading model parameter updates to a central server for aggregation and optimization, ultimately forming a shared global model. This mechanism not only effectively protects data privacy but also significantly reduces reliance on centralized data storage, requiring only the transmission of lightweight model update data.

[0003] As the number of participants (such as bank branches, insurance company branches, retail stores, and advertising platforms) increases, traditional federated learning requires transmitting the intermediate gradients generated by all participants to a central aggregation server for processing. This not only causes significant delays in scenarios with high timeliness requirements, such as real-time financial risk control (e.g., anti-fraud transaction interception), rapid insurance underwriting, and marketing campaigns, but also generates high bandwidth costs and puts enormous computational pressure on the central aggregation server, making it a bottleneck for system expansion and ultimately reducing the efficiency of federated learning. Summary of the Invention

[0004] This application provides a federated learning method, apparatus, device, and readable storage medium to improve federated learning efficiency.

[0005] Firstly, a federated learning method is provided for application to a central aggregation server, including:

[0006] Based on the time series large model, predict the computing power resources and / or network resources of at least one first client;

[0007] Based on the computing power resources and / or network resources of the first client, determine the task efficiency factor of the first client;

[0008] Based on the task efficiency factor, a second client is selected from the first client, wherein the second client is a client participating in federated learning;

[0009] The system receives first model parameters sent by a third client and aggregates them based on the first model parameters. The third client is selected from the second client based on a task efficiency factor, and the first model parameters are obtained by the third client through a primary aggregation of its own model parameters and those of other clients.

[0010] Optionally, the method further includes:

[0011] The proximity between clients is calculated based on the large text model, and the clients are divided into at least one client group according to the proximity, wherein the first client is a client in any client group.

[0012] Optionally, the calculation of proximity between clients based on the large text model includes:

[0013] Obtain the average heartbeat response text between the clients of the client;

[0014] Based on the aforementioned large text model, the average heartbeat response text between clients is vectorized to obtain the average heartbeat response text vector between clients.

[0015] Determine the cosine similarity between the average heartbeat response text vectors of any two clients, and use the cosine similarity as the proximity between the two clients.

[0016] Optionally, the method further includes:

[0017] Fine-tuning the aforementioned large-scale time-series model includes:

[0018] Obtain the historical computing power resources and / or historical network resources of at least one client;

[0019] Based on the historical computing power resources and / or the historical network resources, and using the training computing power resources and / or training network resources of the at least one client obtained through historical training of the time series large model, and the parameters currently used by the time series large model, the differential parameters are obtained.

[0020] The time series model is fine-tuned using the difference parameters to obtain the fine-tuned time series model.

[0021] Optionally, the difference parameter can be obtained in the following way:

[0022] Based on the historical computing power resources and / or the historical network resources, and using the training computing power resources and / or training network resources of the at least one client obtained through historical training of the time series large model, and the parameters currently used by the time series large model, backpropagation iteration is performed to obtain a first low-rank matrix and a second low-rank matrix.

[0023] The difference parameter is obtained based on the first low-rank matrix and the second low-rank matrix.

[0024] Optionally, the method further includes:

[0025] Based on the fine-tuned time-series large model, predict the computing resources and / or network resources of the first client.

[0026] Optionally, determining the task efficiency factor of the first client based on its computing power resources and / or network resources includes:

[0027] The computing power resources and / or network resources of the first client are used as inputs to the task efficiency factor model, and the output of the task efficiency factor model is used as the task efficiency factor of the first client.

[0028] Optionally, selecting a second client from the first client based on the task efficiency factor includes:

[0029] Calculate the probability of the first client participating in federated learning based on the task efficiency factor;

[0030] The first client, whose probability of participating in federated learning is greater than or equal to a first preset value, is designated as the second client.

[0031] Optionally, the method further includes:

[0032] The third client is selected from the second client, wherein the third client is used to perform primary aggregation of the model parameters of other clients and the model parameters of the third client based on the proximity between the third client and other clients.

[0033] Optionally, the method further includes:

[0034] Send the final model parameter aggregation result to the first client.

[0035] Secondly, a federated learning method is provided for application to a third client, including:

[0036] Receive model parameters sent by other clients;

[0037] The model parameters of the other clients and the model parameters of the third client are initially aggregated to obtain the first model parameters, and the first model parameters are sent to the central aggregation server.

[0038] Thirdly, a federated learning device is provided, comprising: an application to a central aggregation server, including:

[0039] The first processing module is used to predict the computing power resources and / or network resources of at least one first client based on a time series large model.

[0040] The second processing module is used to determine the task efficiency factor of the first client based on the computing power resources and / or network resources of the first client.

[0041] The third processing module is used to select a second client from the first client based on the task efficiency factor, wherein the second client is a client participating in federated learning;

[0042] The fourth processing module is used to receive the first model parameters sent by the third client and aggregate them according to the first model parameters. The third client is a client selected from the second client according to the task efficiency factor, and the first model parameters are obtained by the third client through primary aggregation of the model parameters of the third client and the model parameters of other clients.

[0043] Optionally, the device may further include: a fifth processing module, configured to calculate the proximity between clients based on a large text model, and to divide the clients into at least one client group according to the proximity, wherein the first client is a client in any client group.

[0044] Optionally, the fifth processing module is further configured to:

[0045] Obtain the average heartbeat response text between the clients of the client;

[0046] Based on the aforementioned large text model, the average heartbeat response text between clients is vectorized to obtain the average heartbeat response text vector between clients.

[0047] Determine the cosine similarity between the average heartbeat response text vectors of any two clients, and use the cosine similarity as the proximity between the two clients.

[0048] Optionally, the device may further include: a sixth processing module for fine-tuning the large time-series model, including:

[0049] Obtain the historical computing power resources and / or historical network resources of at least one client;

[0050] Based on the historical computing power resources and / or the historical network resources, and using the training computing power resources and / or training network resources of the at least one client obtained through historical training of the time series large model, and the parameters currently used by the time series large model, the differential parameters are obtained.

[0051] The time series model is fine-tuned using the difference parameters to obtain the fine-tuned time series model.

[0052] Optionally, the difference parameter can be obtained in the following way:

[0053] Based on the historical computing power resources and / or the historical network resources, and using the training computing power resources and / or training network resources of the at least one client obtained through historical training of the time series large model, and the parameters currently used by the time series large model, backpropagation iteration is performed to obtain a first low-rank matrix and a second low-rank matrix.

[0054] The difference parameter is obtained based on the first low-rank matrix and the second low-rank matrix.

[0055] Optionally, the first processing module is further configured to predict the computing resources and / or network resources of the first client based on the fine-tuned time-series large model.

[0056] Optionally, the second processing module is further configured to:

[0057] The computing power resources and / or network resources of the first client are used as inputs to the task efficiency factor model, and the output of the task efficiency factor model is used as the task efficiency factor of the first client.

[0058] Optionally, the third processing module is further configured to:

[0059] Calculate the probability of the first client participating in federated learning based on the task efficiency factor;

[0060] The first client, whose probability of participating in federated learning is greater than or equal to a first preset value, is designated as the second client.

[0061] Optionally, the device may further include: a seventh processing module for:

[0062] The third client is selected from the second client, wherein the third client is used to perform primary aggregation of the model parameters of other clients and the model parameters of the third client based on the proximity between the third client and other clients.

[0063] Optionally, the apparatus may further include: an eighth processing module for:

[0064] Send the final model parameter aggregation result to the first client.

[0065] Fourthly, a federated learning device is provided for use on a third client, comprising:

[0066] The first receiving module is used to receive model parameters sent by other clients;

[0067] The first processing module is used to perform primary aggregation on the model parameters of the other clients and the model parameters of the third client to obtain the first model parameters, and send the first model parameters to the central aggregation server.

[0068] Fifthly, a federated learning device is provided, comprising: a central aggregation server, third-party clients, and other clients;

[0069] The central aggregation server is configured to: predict the computing and / or network resources of at least one first client based on a time-series large model; determine the task efficiency factor of the first client based on the computing and / or network resources of the first client; select a second client from the first clients based on the task efficiency factor, wherein the second client is a client participating in federated learning; receive first model parameters sent by a third client, and aggregate according to the first model parameters, wherein the third client is a client selected from the second clients based on the task efficiency factor;

[0070] The third client is used to receive model parameters sent by other clients, perform primary aggregation on the model parameters of the other clients and the model parameters of the third client to obtain first model parameters, and send the first model parameters to the central aggregation server.

[0071] The other client is used to obtain the model parameters and send the model parameters to the third client.

[0072] Fifthly, embodiments of this application also provide a communication device, including: a memory, a processor, and a program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps in the federated learning method as described above.

[0073] Sixthly, embodiments of this application also provide a readable storage medium on which a program is stored, which, when executed by a processor, implements the steps in the federated learning method described above.

[0074] In a seventh aspect, embodiments of this application also provide a computer program product, including computer instructions that, when executed by a processor, implement the steps in the federated learning method described above.

[0075] In this embodiment, a third client is used as a client aggregator. The third client performs initial aggregation of model parameters and sends the aggregation results to the central aggregation server for final aggregation. This allows for asynchronous and parallel calculation and aggregation of model parameters, effectively improving the efficiency of federated learning and the resource utilization of the federated learning system. Attached Figure Description

[0076] Figure 1 This is one of the schematic diagrams of the federated learning system provided in the embodiments of this application;

[0077] Figure 2 This is a second schematic diagram of the federated learning system provided in the embodiments of this application;

[0078] Figure 3 This is one of the flowcharts of the federated learning method provided in the embodiments of this application;

[0079] Figure 4 This is the second flowchart of the federated learning method provided in the embodiments of this application;

[0080] Figure 5 This is the third flowchart of the federated learning method provided in the embodiments of this application;

[0081] Figure 6 This is one of the schematic diagrams of the federated learning device provided in the embodiments of this application;

[0082] Figure 7 This is a second schematic diagram of the federated learning device provided in the embodiments of this application. Detailed Implementation

[0083] The technical solutions of the embodiments of this application will be clearly described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this application. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0084] The terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and are not used to describe a specified order or sequence. It should be understood that such use of data can be interchanged where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first" and "second" are generally of the same class, not limited in number; for example, a first object can be one or more. Furthermore, in the specification and claims, "and" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.

[0085] It is worth noting that the technologies described in this application are not limited to Long Term Evolution (LTE) / LTE-Advanced (LTE-A) systems, but can also be used in other wireless communication systems, such as Code Division Multiple Access (CDMA), Time Division Multiple Access (TDMA), Frequency Division Multiple Access (FDMA), Orthogonal Frequency Division Multiple Access (OFDMA), Single-carrier Frequency-Division Multiple Access (SC-FDMA), and other systems. The terms "system" and "network" in this application are often used interchangeably, and the described technologies can be used with the systems and radio technologies mentioned above, as well as with other systems and radio technologies. However, the following description describes New Radio (NR) systems for illustrative purposes, and NR terminology is used in most of the following description. These technologies can also be applied to applications beyond NR systems, such as 6th generation (6G) radio systems. th Generation 6G communication system.

[0086] Existing federated learning architectures and methods suffer from two main drawbacks: First, with the surge in the number of clients, traditional methods require all intermediate gradients generated by clients to be transmitted to an aggregation server for processing. This not only incurs high bandwidth costs but also significantly increases the computational pressure on the aggregation server, leading to low federated learning efficiency. Second, with the development of computing network technology, client computing power has increased dramatically, and existing methods have not fully utilized this client computing power, resulting in a waste of computing resources.

[0087] Therefore, this application provides a federated learning method, apparatus, and system. In this application embodiment, based on a large text model, the average heartbeat response text between clients is vectorized to calculate the proximity between clients; secondly, in a federated learning scenario of client computing power and network resource prediction, the time-series large model is fine-tuned to predict the client's computing power and network resource status; thirdly, using the client's computing power and network resource status as input, a task efficiency factor model is trained. Based on the principle that clients with superior computing power and network capabilities have a higher probability of participating in federated learning gradient calculation, the model parameters (such as gradients) are obtained by selecting clients participating in federated learning through a compression ratio setting; finally, a preset number of clients with high task efficiency factors (such as...) are selected... (n is the total number of clients) Clients act as client aggregators, initially aggregating the intermediate gradients of clients with high proximity, and sending the aggregation results to the central aggregation server. Thus, by using text and temporal large model techniques to assist federated learning, the efficiency of federated learning can be improved.

[0088] See Figure 1 , Figure 1 This is a schematic diagram of a federated learning system provided in an embodiment of this application. The system may include: a central aggregation server 101, a third client 102, and other clients 103.

[0089] The central aggregation server 101 is configured to: predict the computing and / or network resources of at least one first client based on a time-series large model; determine the task efficiency factor of the first client based on its computing and / or network resources; select a second client from the first clients based on the task efficiency factor, wherein the second client is a client participating in federated learning; receive first model parameters sent by a third client and aggregate them according to the first model parameters, wherein the third client is a client selected from the second clients based on the task efficiency factor;

[0090] The third client 102 is used to receive model parameters sent by other clients, perform primary aggregation on the model parameters of the other clients and the model parameters of the third client to obtain first model parameters, and send the first model parameters to the central aggregation server.

[0091] The other client 103 is used to obtain the model parameters and send the model parameters to the third client.

[0092] Here, the other clients refer to clients whose proximity to the third client meets a certain preset condition, such as clients whose proximity is ranked first in order of closest to furthest.

[0093] In this embodiment, model parameters refer to the parameters of the model used for federated learning, such as gradients. The models that can be used for federated learning include, but are not limited to, deep learning models (such as convolutional neural networks) and machine learning models (such as support vector machines). Therefore, the first model parameter can be a gradient, and the model parameters of the other clients and the third client can also be gradients.

[0094] The central aggregation server can aggregate the first model parameters received from third clients according to a preset strategy, without limitation. For example, the central aggregation server can perform a weighted sum of multiple first model parameters according to the weights corresponding to each third client to obtain the final aggregation result.

[0095] Here, the specific method by which the third client performs primary aggregation is not limited. For example, the third client can perform a weighted sum of gradients from multiple clients according to the weights corresponding to each client, and use the sum as the first model parameter. The weights of each client can be set according to pre-agreed rules. Alternatively, the third client can also calculate the gradients of each client using a mathematical function to obtain the final gradient, which can also be used as the first model parameter.

[0096] See Figure 2 , Figure 2 This is yet another schematic diagram of the federated learning system according to an embodiment of this application. Figure 2 middle:

[0097] Client 201: A client refers to an IoT device or terminal server that calculates model gradients locally and uses privacy protection technologies such as homomorphic encryption and differential privacy to encrypt the gradients. In financial marketing scenarios, clients can be servers in various bank branches, credit card centers, servers of partner e-commerce platforms, or user mobile devices that have installed a compliant Software Development Kit (SDK). They hold sensitive data such as local user transactions, browsing history, and location (requiring user authorization).

[0098] Client Classifier 202 has two main functions: First, it calculates the proximity between any two clients and sorts them from closest to furthest. Second, it calculates the probability of all clients participating in the current federated learning process based on the compression ratio, thereby improving the efficiency of federated learning. Its goal is to intelligently select participating clients and their aggregation paths in each round of federated learning, based on predicted efficiency and network topology, thus optimizing aspects such as the update speed of marketing models or the synchronization efficiency of risk control models.

[0099] Gradient Message Queue 203: After calculating model parameters (such as gradients), the client sends the gradients to the gradient message queue. By converting synchronous to asynchronous processing, the pressure on the central aggregation server can be alleviated. The goal is to intelligently select participating clients and their aggregation paths in each round of federated learning based on prediction efficiency and network topology, optimizing aspects such as the update speed of marketing models or the synchronization efficiency of risk control models. In this embodiment, instead of each client directly sending gradients to the central aggregation server (which can be called synchronous processing), each client sends gradients to the gradient message queue. The gradients in the gradient message queue participate in subsequent processing in an orderly manner (which can be called asynchronous processing), allowing the client's gradient submission operation and the central aggregation server's operation to be executed independently, thereby reducing the real-time processing pressure on the central aggregation server.

[0100] The text and time-series large-scale model 204 consists of three parts: First, based on the text large-scale model, it transforms heartbeat response text samples between clients into vectors. Second, in the scenario of predicting client computing power and network resources (such as bandwidth) in federated learning, it fine-tunes the basic time-series large-scale model, predicting the computing power and network resource status of the next task based on the client's historical computing power and network resource status. Third, it trains a neural network model to evaluate the probability of a client becoming a client aggregation server based on computing power and network resource status. The heartbeat response text between clients processed by the text large-scale model implicitly contains information on network latency and stability between clients, which is crucial for cross-regional (such as between branches in different cities) federated learning collaboration. The client status (computing power, network) predicted by the time-series large-scale model is a key basis for evaluating whether it can efficiently participate in model training or aggregation during high-concurrency marketing activities such as shopping festivals. The task efficiency factor model directly serves to prioritize the participation of efficient nodes in training when resources are limited, ensuring the model update frequency of core businesses (such as real-time anti-fraud).

[0101] Client Aggregator 205: Essentially, it refers to clients with high computing power and network performance (such as core branch servers and regional data centers). Based on the computing power and network status predicted by the time-series large-scale model, it evaluates the capabilities of clients participating in federated learning, and selects the most capable clients as client aggregators. It aggregates intermediate gradients in a tiered manner, which alleviates the computing power pressure on aggregation servers and improves the utilization rate of client computing power. For example, in financial marketing scenarios, an aggregator can be responsible for aggregating gradients from multiple branches or partner merchants in its vicinity (such as within the same city), significantly reducing long-distance cross-regional transmission and accelerating the updating of regional marketing strategy models. In insurance risk control, it can aggregate gradients from different branches within the same medical group.

[0102] Gradient Coordinator 206: Based on the client classifier's results, the gradients calculated by the client are distributed to the corresponding client aggregators for aggregation according to proximity. By intelligently routing gradient calculation tasks to the optimal client aggregator based on the classifier results and predicted proximity, network traffic localization is achieved, improving the training efficiency of precision marketing models such as those based on location services.

[0103] Central Aggregator Server 207: After the client aggregator completes its aggregation, it sends the results to the central aggregation server. The central aggregation server aggregates the received results and sends the aggregated results back to the client to begin the next round of federated learning. For example, it receives results from aggregators at various regions / levels, performs final fusion, generates a global model (such as a unified customer value scoring model or anti-fraud model), and distributes it to all participants for use in the next round of business decisions.

[0104] Therefore, by using a client-side aggregator in this system to aggregate gradients at the client level, gradient calculation and aggregation can be performed asynchronously and in parallel, which can effectively improve the efficiency of federated learning and the resource utilization of the federated learning system.

[0105] The working principles of the central aggregation server and client in the above system are described below with reference to different method embodiments.

[0106] See Figure 3 , Figure 3 This is a flowchart of the federated learning method provided in the embodiments of this application, applied to a central aggregation server, such as... Figure 3 As shown, it includes the following steps:

[0107] Step 301: Predict the computing power resources and / or network resources of at least one first client based on the time series large model;

[0108] Step 302: Determine the task efficiency factor of the first client based on the computing power resources and / or network resources of the first client;

[0109] Step 303: Based on the task efficiency factor, select a second client from the first client, wherein the second client is a client participating in federated learning;

[0110] Step 304: Receive the first model parameters sent by the third client, and aggregate them according to the first model parameters. The third client is selected from the second client according to the task efficiency factor, and the first model parameters are obtained by the third client through primary aggregation of the model parameters of the third client and the model parameters of other clients.

[0111] In this embodiment, a third client is used as a client aggregator. The third client performs initial aggregation of model parameters and sends the aggregation results to the central aggregation server for final aggregation. This allows for asynchronous and parallel calculation and aggregation of model parameters, effectively improving the efficiency of federated learning and the resource utilization of the federated learning system.

[0112] The computing resources may include CPU resources, and network resources may include memory, disk, and network utilization. The first client may be one or more clients within a client group. In practical applications, the concept of a client group may not be distinguished; that is, in step 301, the computing resources and / or network resources of the first client among all clients are directly predicted based on a time-series large-scale model. Alternatively, all clients may be considered to form a terminal group. The model parameters include, but are not limited to, gradients.

[0113] Optionally, in this embodiment, the proximity between clients (e.g., two or more) can be calculated based on the large text model; further, the clients can be divided into at least one client group according to the proximity, where the first client is a client in any client group. The large text model includes, but is not limited to, a Generative Pre-trained Transformer (GPT) and Bidirectional Encoder Representations from Transformers (BERT).

[0114] Specifically, in the process of calculating the proximity between clients based on the large text model, the average heartbeat response text between the clients can be obtained; the average heartbeat response text between the clients is vectorized based on the large text model to obtain the average heartbeat response text vector between the clients; the cosine similarity between the average heartbeat response text vectors between any two clients is determined, and the cosine similarity is used as the proximity between the two clients.

[0115] For example, suppose a federated learning system has a total of During federated learning gradient calculation, historical average heartbeat response samples are collected from any two clients. Assume any two clients... , Client To the client The average heart rate response is expressed as Similarly, the client To the client The average heart rate response is expressed as The average heartbeat response between clients is shown in Table 1:

[0116] Table 1

[0117]

[0118] Calculating the proximity between clients can specifically include:

[0119] (1) Generate average heartbeat response text between clients:

[0120] Based on the average heartbeat response data between clients in Table 1, generate the average heartbeat response text for each client. For example, the format of the average heartbeat response text between clients is as follows:

[0121] =“Heartbeat to Client 1: Heartbeat to Client 2: ...;To the client's nth heartbeat: "......(Expression 1);

[0122] Optionally, synchronization can generate text in the above format for each of the other clients.

[0123] (2) Average heartbeat response text vectorization between clients:

[0124] In this embodiment, the BERT text model in Huggingface can be used to vectorize the text. Through the transformer library in Huggingface, the BERT text model is selected, and after processing steps such as word segmentation and text encoding, a vectorized result of the average heartbeat response text between clients is generated, such as the average heartbeat response text vector of client i. It can be calculated using expression 2:

[0125] ... (Expression 2);

[0126] (3) Calculate the proximity between clients:

[0127] Computing Client , The cosine similarity between the average heartbeat response text vectors of different clients is the proximity of the heartbeat vectors. A cosine similarity closer to 1 indicates a closer proximity between the clients. , The more similar the network conditions between the two clients, the better. Calculate the network conditions between any two clients. , proximity The method is as shown in Formula 1:

[0128] ....(Formula 1);

[0129] in, This represents the text vector representing the average heartbeat response between clients, i. This represents the average heartbeat response text vector between clients, represented by client j.

[0130] According to the above formula (1), the proximity between any two clients can be calculated sequentially. For any client, the proximity can be sorted from closest to furthest, so that client groups can be obtained based on the proximity (for example, the top M (M≥1) clients in the proximity ranking can be grouped into the same client group). This proximity is crucial for building client groups based on geographical region or network latency in financial marketing, such as grouping bank branches and partner merchants in the same business district so that their aggregators can efficiently aggregate gradients derived from local user behavior data and optimize localized marketing models.

[0131] The large-scale time series model includes, but is not limited to, GPT (TIMEGPT) from the field of time series forecasting. TIMEGPT is a time series forecasting model based on the Transformer architecture, which has the ability to capture long-term dependencies. Furthermore, in this embodiment, the large-scale time series model can be fine-tuned, including:

[0132] Obtain historical computing power resources and / or historical network resources of at least one client; based on the historical computing power resources and / or historical network resources, and using the training computing power resources and / or training network resources of the at least one client obtained through historical training of the time series large model, and the parameters currently used by the time series large model, obtain differential parameters; use the differential parameters to fine-tune the time series large model to obtain a fine-tuned time series large model.

[0133] Historical computing resources and / or historical network resources refer to statistical information on the client's computing and network resources collected over one or more time periods or points in time. Training computing resources and / or training network resources refer to the client's computing and / or network resources output by the time-series large model during the model's historical training process. The parameters currently used by the time-series large model can be understood as the parameters possessed by the time-series large model when executing the embodiments of this application, or the parameters possessed by the time-series large model before fine-tuning.

[0134] The differential parameter can be obtained in the following ways:

[0135] Based on the historical computing power resources and / or the historical network resources, and using the training computing power resources and / or training network resources of the at least one client obtained through historical training of the time-series large model, and the parameters currently used by the time-series large model, backpropagation iteration is performed to obtain a first low-rank matrix and a second low-rank matrix; based on the first low-rank matrix and the second low-rank matrix, the difference parameter is obtained. Wherein, the rank r of the first low-rank matrix and the second low-rank matrix is ​​8.

[0136] After obtaining the fine-tuned time series model, in step 301, the fine-tuned time series model can also be used to predict the computing resources and / or network resources of the first client in at least one client group.

[0137] Specifically, during federated learning, the client's computing power and network resources should exhibit temporal characteristics when training intermediate gradients. To improve the prediction accuracy of client computing power and network resources in federated learning scenarios, taking TIMEGPT as an example, the basic temporal series model of TIMEGPT can be fine-tuned. The specific fine-tuning steps are as follows:

[0138] (1) Dataset preparation:

[0139] The computing power, memory, disk, and network usage of each client in each round of federated learning in the federated learning system are collected, as shown in Table 2:

[0140] Table 2

[0141]

[0142] (2) Fine-tuning of the large time series model:

[0143] Assuming the parameters of TIMEGPT For one Matrix, used to improve fine-tuning efficiency while maintaining the original model parameters. While keeping the parameters unchanged, add differential parameters (differential parameter matrix). , making ,in These are the fine-tuned parameters. To further improve fine-tuning efficiency and save computing resources, the following will be implemented: Decompose into a low-rank matrix. The decomposition means that two smaller ones are needed. To replace larger matrices ,in, (The first low-rank matrix) is matrix, (The second address matrix) is Matrix, where, , k≥1.

[0144] Using the historical computing power resources and historical network resources of each client as samples (as shown in Table 2), the previous... The client computing power and network resources obtained from the previous training gradient are the independent variables, and the client computing power and network resources obtained from the next training gradient are the dependent variables. All parameters are known. Since it is an unknown, it can be obtained using backpropagation iteration. , Matrix, thus obtaining .

[0145] (3) Predicting computing power and network resources based on time series large-scale models:

[0146] This involves using the fine-tuned temporal large model to pre-calculate the computational and network resources available for the next round of training gradients for the client. For any federated learning client, this can be based on the previous... The computing and network resources of the client in the second federated learning process are used to predict the first... The computing power and network status of the secondary federated learning client are shown in Formula 2:

[0147] .............(Formula 2);

[0148] in, The client's CPU, memory, disk, and network usage at time t+m+1.

[0149] In step 302, the computing power resources and / or network resources of the first client can be used as input to the task efficiency factor model, and the output of the task efficiency factor model can be used as the task efficiency factor of the first client; wherein, the loss function of the task efficiency factor model is obtained based on the true value of the task efficiency factor of at least one client and the predicted value of the task efficiency factor.

[0150] In federated learning, task efficiency is primarily affected by two factors: first, computational efficiency, which refers to the time required to compute intermediate gradients in the federated learning process, mainly influenced by computing power, memory, and disk space; and second, the transmission time for calculating intermediate gradients and sending them from the client to the aggregation server, primarily influenced by network bandwidth. Based on these ideas, a task efficiency factor model is trained using a neural network framework, with the following specific steps:

[0151] (1) Input layer of task efficiency factor model:

[0152] For each client, the computing and network resources of each client are predicted based on the fine-tuned time-series model described above, as shown in Formula 2. Since the input layer of this model includes CPU, memory, disk, and network parameters, it incorporates both computational and network efficiency, thus ensuring the accuracy of the model.

[0153] (2) Hidden layer of task efficiency factor model:

[0154] The hidden layers consist of three layers, each containing eight fully connected neurons. The activation function for the hidden layers is selected as follows: function.

[0155] (3) Output layer of the task efficiency factor model:

[0156] The output layer is the task efficiency factor. The calculation method is shown in Formula 3:

[0157] ........................ (Formula 3);

[0158] in, The sum of the time to compute gradients for the client and the time to transmit gradients to the aggregation server.

[0159] (4) Construct the loss function and train the task efficiency factor model:

[0160] Construct a task efficiency factor loss function, assuming the system has For each sample, the loss function is... Represented as:

[0161] .............(Formula 4);

[0162] in, For loss function, This is the predicted value of the task efficiency factor. For the true value of the task efficiency factor, when The model is most accurate when the value is minimized. The task efficiency factor parameters are calculated using backpropagation, resulting in the task efficiency factor model. By accurately predicting client states and calculating the task efficiency factor, the system can dynamically select the most reliable and fastest-responding nodes (such as core data centers or subsets of high-performance user devices) for training during peak marketing periods (e.g., holiday promotions) or periods of high risk control pressure (e.g., periods of high fraud incidence), ensuring timely updates to critical business models.

[0163] Optionally, in step 303, the probability of the first client participating in federated learning is calculated based on the task efficiency factor; the first client whose probability of participating in federated learning is greater than or equal to a first preset value is designated as the second client. The first preset value can be set as needed.

[0164] Due to differences in computing power and storage capabilities among clients, as well as variations in heterogeneous network bandwidth, clients with high gradient calculation efficiency and fast transmission speeds may have to wait for slower clients before entering the next round of federated learning, thus affecting the efficiency of federated learning. To address this issue, in this embodiment, clients with superior computing and network capabilities have a higher probability of participating in federated learning computations, while clients with weaker computing and network capabilities have a relatively lower probability of participating. This ensures the accuracy of federated learning while improving its efficiency. The specific method for calculating the probability of a client participating in federated learning computations is shown in Formula 5.

[0165] Specifically, the task efficiency factor for each client calculated according to step 303 above is: The probability of each client participating in federated learning is:

[0166] ............(Formula 5);

[0167] in, For the first The probability of each client participating in federated learning; This refers to the compression ratio of federated learning. The compression ratio represents the percentage of clients participating in federated learning computations. When there are many clients, this value can be appropriately decreased; when there are few clients, this value can be appropriately increased. The value range is between 0 and 1. This represents the minimum value of the task efficiency factor among all clients. This represents the maximum value of the task efficiency factor for each client.

[0168] For example, each time a client is selected to participate in federated learning, a random number between 0 and 1 is generated. If the probability of participating in federated learning is greater than or equal to this random number, the client can participate in the current federated learning computation; if the probability is less than this random number, the client will not participate. By optimizing the clients participating in gradient calculation for federated learning, both gradient calculation time and gradient propagation time can be reduced, thereby improving the efficiency of federated learning.

[0169] When initiating a new round of federated learning sessions, the probability of each client participating in federated learning is obtained according to step 303 above. Based on the probability of each client participating in federated learning calculated in the previous steps, clients are selected to participate in this round of federated learning at the beginning of gradient calculation. The selected clients will participate in the federated learning gradient calculation. Through probability selection and compression ratio control, the system can significantly reduce the number of clients participating in each round of calculation while ensuring model accuracy. This is particularly effective in large-scale precision marketing model training involving a massive number of potential clients (such as hundreds of millions of user devices participating in a lightweight manner), and can greatly reduce the overall communication and computational load.

[0170] Furthermore, in this embodiment, the second client can be selected in other ways, such as random selection. The task efficiency factor can also be calculated using algorithms such as Naive Bayes or decision trees.

[0171] Optionally, in this embodiment, the third client may also be selected from the second client, wherein the third client is used to perform primary aggregation of the model parameters of other clients and the model parameters of the third client based on the proximity between the third client and other clients.

[0172] For example, a third client can receive model parameters sent by one or more other clients ranked higher in the proximity calculation, and perform initial aggregation of these model parameters with its own model parameters. In other words, each client selects a client with high proximity as its aggregator for initial aggregation; finally, each client aggregator sends its aggregation result to the central aggregation server, which completes the final summarization of intermediate gradients. Thus, by using client aggregators to perform initial gradient aggregation, and asynchronously and in parallel performing gradient calculation and aggregation, the efficiency of federated learning and the resource utilization of the federated learning system can be effectively improved.

[0173] Other clients send the calculated gradients to the gradient message queue. Since the computing power and network bandwidth of the clients vary, the time it takes for each client to transmit the gradients to the queue differs. This application's embodiment plans to fully utilize the client resources of the terminal, selecting some clients with high task efficiency factors, balancing the responsibilities of both the trainer and the aggregator, performing primary aggregation on the client side, and then sending the aggregation results to the aggregation node. This improves the utilization rate of terminal computing resources and enhances the efficiency of federated learning through parallel aggregation. The introduction of a message queue and a client aggregator enables asynchronous gradient calculation and hierarchical / parallel aggregation. This completely changes the traditional federated learning model of synchronously waiting for the slowest node, making it particularly suitable for scenarios with highly heterogeneous clients (such as those containing both high-performance servers and ordinary mobile phones). In financial marketing, this means that regional marketing models (generated by regional aggregators) can be deployed faster without waiting for the complete convergence of the global model, improving the responsiveness of marketing campaigns.

[0174] First, assume that among all the selected participants in the federated learning computation are One client, in Select from clients Each client simultaneously acts as both a trainer and an aggregator (i.e., a third client). Next, using Formula 1, the proximity between each of the other clients and the aggregator acting as both trainer and aggregator is calculated, and the aggregator with the highest proximity is selected as the primary aggregator. That is, other clients can also select a high-proximity aggregator as the primary aggregator based on their own proximity ranking and send gradients to this aggregator. Finally, each client aggregator sends its aggregation result to the central aggregation server, which then performs the final aggregation of intermediate gradients.

[0175] Optionally, in this embodiment of the application, the central aggregation server may also send the final model parameter aggregation result to the first client.

[0176] See Figure 4 , Figure 4 This is a flowchart of the federated learning method provided in the embodiments of this application, applied to a third client, such as... Figure 4 As shown, it includes the following steps:

[0177] Step 401: Receive model parameters sent by other clients;

[0178] Step 402: Perform primary aggregation on the model parameters of the other clients and the model parameters of the third client to obtain the first model parameters, and send the first model parameters to the central aggregation server.

[0179] For example, a third client can receive model parameters sent by one or more other clients ranked higher in the proximity calculation, and perform a preliminary aggregation of these model parameters with the model parameters obtained by the third client itself. Other clients can also select a higher-ranked client as the third client based on their own calculated proximity and send model parameters to that third client.

[0180] Optionally, the third client can also receive the final model parameter aggregation results sent by the central aggregation server.

[0181] In this embodiment, a third client is used as a client aggregator. The third client performs initial aggregation of model parameters and sends the aggregation results to the central aggregation server for final aggregation. This allows for asynchronous and parallel calculation and aggregation of model parameters, effectively improving the efficiency of federated learning and the resource utilization of the federated learning system.

[0182] See Figure 5 , Figure 5 This is a flowchart of the federated method provided in the embodiments of this application, which may include:

[0183] Step 501: Calculate client proximity based on the large text model;

[0184] Step 502: Use the fine-tuned time series model to predict client computing power and network resources;

[0185] Step 503: Calculate the client task efficiency factor;

[0186] Step 504: Calculate the probability of each client participating in federated learning;

[0187] Step 505: Federated learning session initialization;

[0188] When initiating a new round of federated learning sessions, the probability of each client participating in federated learning is obtained according to step 504 above. Based on the probability of each client participating in federated learning calculated in the previous steps, at the beginning of gradient calculation in each round of federated learning, clients participating in this round of federated learning are selected according to the above probabilities. The selected clients will participate in the federated learning gradient calculation. Through probability selection and compression ratio control, the system can significantly reduce the number of clients participating in each round of calculation while ensuring model accuracy. This is particularly effective in large-scale precision marketing model training involving a massive number of potential clients (such as hundreds of millions of user devices participating in a lightweight manner), and can greatly reduce the overall communication and computational load.

[0189] Step 506: The client sends the gradient results to the queue, and the central aggregation server completes the gradient aggregation.

[0190] The relevant descriptions of steps 501 to 506 can be found in the descriptions of the foregoing method embodiments.

[0191] Taking the financial marketing scenario as an example, the following details the specific application process of the solution in this application embodiment in the scenario of "real-time precision marketing based on multi-party data fusion", combining the advantages of operator big data:

[0192] 1. Scenario Objective: A bank (the initiator) wants to collaborate with an e-commerce platform (Partner A) and a telecom operator (Partner B, providing data) to build a real-time updated customer purchase intention prediction model for accurately pushing credit card coupons. Core Challenges: The bank possesses users' financial attributes (Assets Under Management (AUM), credit history), the e-commerce platform has purchase / browsing history, and the telecom operator has anonymized location data, app usage preferences, and online consumption behavior (requiring strict compliance and user authorization). All parties have high data privacy requirements and cannot directly share data. Traditional federated learning is inefficient enough to meet the real-time needs of marketing campaigns.

[0193] 2. System Initialization and Roles:

[0194] 21. Client-side components: Bank branch servers, e-commerce platform regional data centers, and provincial telecom operator data centers (acting as data holder clients). The central aggregation server is operated by the bank or a neutral third party.

[0195] 22. Carrier Data Applications: Carrier clients provide features that have undergone strict anonymization and aggregation processing, for example:

[0196] User's permanent residence / work area (business district, residential area code); frequently accessed APP categories (finance, shopping, entertainment); monthly data / call charges consumption level; (anonymized) regional population flow heat map (used to supplement regional trend analysis for banks / e-commerce); / / Emphasize the specific types and value of operator data.

[0197] 3. Single-round federated learning process (using the example provided in this application):

[0198] 31. Calculate Proximity: The large text model processes historical heartbeat response texts between data centers and calculates cosine similarity. For example, it finds that "Shanghai Branch", "Shanghai E-commerce Data Center", and "Shanghai Mobile Data Center" have high network proximity and automatically groups them together.

[0199] 32. Resource Status Prediction: Time-series large-scale model analysis of historical resource usage data for each client. It predicts that during peak evening hours, the network load of the "Shanghai E-commerce Data Center" will be high (during promotional periods), while the "Shanghai Mobile Data Center" will have ample computing power.

[0200] 33. Calculate Task Efficiency Factor & Participation Probability: The task efficiency factor model comprehensively predicts computing power and network status. It is assumed that the "Shanghai Mobile Data Center" (operator) has the highest predicted efficiency factor, followed by the Shanghai Branch (bank), while the "Shanghai E-commerce Data Center" (e-commerce) has a lower efficiency factor due to its high predicted network load. Combining the compression ratio (e.g., 0.7), the participation probability of each data center is calculated. In this round, the "Shanghai E-commerce Data Center" may temporarily not participate in the calculation due to a missed random number (its data characteristics can be compensated for through historical models or subsequent rounds).

[0201] 34. Select the client aggregator: Select the client with the highest task efficiency factor log2(n') (e.g., log2(2)=1) from the selected clients (e.g., Shanghai Branch, Shanghai Mobile), i.e., "Shanghai Mobile Data Center" as the client aggregator for this group.

[0202] 35. Local training and gradient upload:

[0203] Shanghai branch of the bank: Trains the model using local user data (financial attributes + some regional characteristics of operators), calculates the gradient, and sends it to the gradient message queue after encryption. Shanghai Mobile, the operator: Trains the model using local anonymized data (location, app, consumption characteristics), calculates the gradient (and also acts as an aggregator).

[0204] 36. Primary Aggregation: The gradient coordinator allocates the gradient of the Shanghai branch of the bank to the aggregator with the highest network proximity, namely "Shanghai Mobile Data Center", based on proximity. This aggregator performs local gradient calculation (step 35) and receives the gradient of the Shanghai branch of the bank in parallel, and securely aggregates the two gradients (representing bank and operator data in the Shanghai area) locally (on the client side).

[0205] 37. Global Aggregation: The "Shanghai Mobile Data Center" sends the aggregated (Shanghai region) gradient ciphertext to the central aggregation server. The central aggregation server simultaneously receives aggregated gradients from other regional groups (such as the Beijing group and the Guangzhou group, reported by their respective aggregators), performs final global secure aggregation, and generates a new global purchase intention prediction model.

[0206] 38. Model Distribution and Application: The new global model is distributed to all participating clients (banks, e-commerce platforms, and telecom operators). Banks immediately use the model to score the purchase intentions of target customers and push the most relevant credit card coupons in real time. E-commerce platforms can also use the model to optimize their in-site recommendations.

[0207] The beneficial effects of the above solution are as follows:

[0208] Efficiency Improvement: Through proximity grouping and a client-side aggregator (Shanghai Mobile), gradient calculation and aggregation in the Shanghai region are efficiently completed locally, avoiding the need for all raw gradients to be transmitted across cities to the central server. The efficiency factor model and probability selection avoid waiting on slow nodes (such as high-load e-commerce centers). Message queues enable asynchronous processing. Overall, the model update cycle is significantly shortened, meeting the real-time requirements of marketing.

[0209] Resource optimization: The excess computing power of the operator's regional data center (Shanghai Mobile) was fully utilized to undertake the aggregation task, which reduced the pressure on the central server and also avoided the need for banks / e-commerce companies to invest additional aggregation resources.

[0210] Privacy protection: Strictly adhering to the federated learning principle, raw user data (bank transactions, e-commerce browsing, carrier details) does not leave the local machine. The carrier provides processed features or aggregated statistical information.

[0211] Unlocking the value of data: Under the premise of legality and compliance, it effectively integrates complementary data from finance, e-commerce, and telecom operators (especially the location and behavioral data of telecom operators, which make up for the lack of spatiotemporal dimensions of financial / e-commerce data), significantly improving the accuracy of the purchase intention prediction model, thereby increasing marketing conversion rate and return on investment (ROI).

[0212] Scalability: The layered architecture and dynamic selection mechanism easily support the integration of more partners (such as offline merchant POS systems) or expansion to a larger scale (more cities / users).

[0213] Through laboratory testing and industry scenario simulation verification, the solution of this application embodiment significantly outperforms traditional federated learning solutions in terms of efficiency, resource utilization, and business effectiveness.

[0214] 1. Efficiency Improvement Data (Financial Marketing Scenario Test), as shown in Table 3, compares the data from this application's embodiment with the data from traditional federated learning:

[0215] Table 3

[0216]

[0217] The efficiency improvements are mainly due to: dynamic client selection reducing the participation of inefficient nodes by 30%; hierarchical aggregation reducing long-distance cross-regional transmission by 72%; and asynchronous mechanisms eliminating 90% of synchronous waiting time.

[0218] 2. Resource utilization optimization (carrier data center stress test): The data comparison between the central server in the traditional solution and the client aggregator and central aggregation server in this embodiment is shown in Table 4.

[0219] Table 4

[0220]

[0221] Among them, the resource consumption of the central aggregation server is reduced by 55%+, the utilization rate of idle computing power of the client aggregator (operator node) is increased to 70%+, and the overall computing resource utilization rate of the system is increased by 40%.

[0222] 3. Business effectiveness verification (bank-telecom operator joint marketing case), data comparison is shown in Table 5:

[0223] Table 5

[0224]

[0225] Among them, the core data value points are:

[0226] Operator location data improved the accuracy of regional consumption tendency prediction by 32%; APP usage behavior data optimized the coverage of user interest tags by 58%; and network consumption level data enhanced the accuracy of user purchasing power assessment by 41%.

[0227] 4. Risk control scenario effectiveness (joint modeling of insurance anti-fraud), data comparison is shown in Table 6:

[0228] Table 6

[0229]

[0230] Key improvements: Predict client load during high-risk periods using a large time-series model and dynamically allocate aggregation tasks; the client selection mechanism ensures that 98% of high-risk transactions are judged by the model within 200ms.

[0231] 5. Data Source Explanation:

[0232] Test environment:

[0233] Financial marketing scenarios: simulated bank (20 nodes), e-commerce platform (15 nodes), telecom operator (data center in 31 provinces);

[0234] Risk control scenario: Joint modeling by branches of 5 insurance companies;

[0235] Client types: 60% cloud servers, 30% edge nodes, 10% mobile terminals;

[0236] Implementation cost comparison:

[0237] Traditional solution: Annual maintenance cost of central server ≈ $1.2M.

[0238] Example of this application: Edge aggregation resource reuse + bandwidth saving → cost $0.4M (66.7%↓ (66.7% decrease)).

[0239] As can be seen from the above description, the efficient framework of this application's embodiments makes it feasible to integrate more types and sources of data (especially supplementary data such as spatiotemporal and behavioral data provided by operators) for federated learning. This enables the training of more accurate and powerful models in scenarios such as financial marketing (building a 360-degree user view) and insurance risk control (combining medical and behavioral data), thereby releasing greater business value while strictly protecting the data privacy and compliance of all parties.

[0240] See Figure 6 , Figure 6 This is a structural diagram of the federated learning device provided in the embodiments of this application, applied to a central aggregation server, including:

[0241] A first processing module 601 is used to predict the computing power resources and / or network resources of at least one first client based on a time-series large model; a second processing module 602 is used to determine the task efficiency factor of the first client based on the computing power resources and / or network resources of the first client; a third processing module 603 is used to select a second client from the first clients based on the task efficiency factor, wherein the second client is a client participating in federated learning; a fourth processing module 604 is used to receive first model parameters sent by a third client and aggregate them according to the first model parameters, wherein the third client is a client selected from the second client according to the task efficiency factor, and the first model parameters are obtained by the third client through a preliminary aggregation of the model parameters of the third client and the model parameters of other clients.

[0242] Optionally, the device may further include: a fifth processing module, configured to calculate the proximity between clients based on a large text model, and to divide the clients into at least one client group according to the proximity, wherein the first client is a client in any client group.

[0243] Optionally, the fifth processing module is further configured to:

[0244] Obtain the average heartbeat response text between the clients of the client;

[0245] Based on the aforementioned large text model, the average heartbeat response text between clients is vectorized to obtain the average heartbeat response text vector between clients.

[0246] Determine the cosine similarity between the average heartbeat response text vectors of any two clients, and use the cosine similarity as the proximity between the two clients.

[0247] Optionally, the device may further include: a sixth processing module for fine-tuning the large time-series model, including:

[0248] Obtain the historical computing power resources and / or historical network resources of at least one client;

[0249] Based on the historical computing power resources and / or the historical network resources, and using the training computing power resources and / or training network resources of the at least one client obtained through historical training of the time series large model, and the parameters currently used by the time series large model, the differential parameters are obtained.

[0250] The time series model is fine-tuned using the difference parameters to obtain the fine-tuned time series model.

[0251] Optionally, the difference parameter can be obtained in the following way:

[0252] Based on the historical computing power resources and / or the historical network resources, and using the training computing power resources and / or training network resources of the at least one client obtained through historical training of the time series large model, and the parameters currently used by the time series large model, backpropagation iteration is performed to obtain a first low-rank matrix and a second low-rank matrix.

[0253] The difference parameter is obtained based on the first low-rank matrix and the second low-rank matrix.

[0254] Optionally, the first processing module is further configured to predict the computing resources and / or network resources of the first client based on the fine-tuned time-series large model.

[0255] Optionally, the second processing module is further configured to:

[0256] The computing power resources and / or network resources of the first client are used as inputs to the task efficiency factor model, and the output of the task efficiency factor model is used as the task efficiency factor of the first client.

[0257] Optionally, the third processing module is further configured to:

[0258] Calculate the probability of the first client participating in federated learning based on the task efficiency factor;

[0259] The first client, whose probability of participating in federated learning is greater than or equal to a first preset value, is designated as the second client.

[0260] Optionally, the device may further include: a seventh processing module for:

[0261] The third client is selected from the second client, wherein the third client is used to perform primary aggregation of the model parameters of other clients and the model parameters of the third client based on the proximity between the third client and other clients.

[0262] Optionally, the apparatus may further include: an eighth processing module for:

[0263] Send the final model parameter aggregation result to the first client.

[0264] The apparatus provided in this application embodiment can execute the above method embodiment, and its implementation principle and technical effect are similar, so it will not be described again here.

[0265] See Figure 7 , Figure 7 This is a structural diagram of the federated learning device provided in the embodiments of this application, applied to a third client, including:

[0266] The first receiving module 701 is used to receive model parameters sent by other clients; the first processing module 702 is used to perform primary aggregation on the model parameters of the other clients and the model parameters of the third client to obtain the first model parameters, and send the first model parameters to the central aggregation server.

[0267] The apparatus provided in this application embodiment can execute the above method embodiment, and its implementation principle and technical effect are similar, so it will not be described again here.

[0268] It should be noted that the division of units in the embodiments of this application is illustrative and only represents one logical functional division. In actual implementation, other division methods may be used. Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated units described above can be implemented in hardware or as software functional units.

[0269] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a processor-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0270] This application provides a communication device, including: a memory, a processor, and a program stored in the memory and executable on the processor; the processor is configured to read the program from the memory to implement the steps in the federated learning method as described above.

[0271] This application also provides a readable storage medium storing a program. When executed by a processor, this program implements the various processes of the above-described federated learning method embodiments and achieves the same technical effects. To avoid repetition, it will not be described again here. The readable storage medium can be any available medium or data storage device that the processor can access, including but not limited to magnetic storage (e.g., floppy disks, hard disks, magnetic tapes, magneto-optical disks (MO), etc.), optical storage (e.g., CDs, DVDs, BDs, HVDs, etc.), and semiconductor storage (e.g., ROMs, EPROMs, EEPROMs, non-volatile memory (NAND flash), solid-state drives (SSDs)).

[0272] This application also provides a computer program product, including computer instructions. When executed by a processor, these computer instructions implement the various processes of the above-described federated learning method embodiments and achieve the same technical effects. To avoid repetition, they will not be described again here.

[0273] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0274] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a computer software product. This computer software product is stored in a storage medium (such as ROM / RAM, disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0275] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.

Claims

1. A federated learning method, characterized in that, Applications to central aggregation servers include: Based on the time series large model, predict the computing power resources and / or network resources of at least one first client; Based on the computing power resources and / or network resources of the first client, determine the task efficiency factor of the first client; Based on the task efficiency factor, a second client is selected from the first client, wherein the second client is a client participating in federated learning; The system receives first model parameters sent by a third client and aggregates them based on the first model parameters. The third client is selected from the second client based on a task efficiency factor, and the first model parameters are obtained by the third client through a primary aggregation of its own model parameters and those of other clients.

2. The method according to claim 1, characterized in that, The method further includes: The proximity between clients is calculated based on the large text model, and the clients are divided into at least one client group according to the proximity, wherein the first client is a client in any client group.

3. The method according to claim 2, characterized in that, The calculation of proximity between clients based on the large text model includes: Obtain the average heartbeat response text between the clients of the client; Based on the aforementioned large text model, the average heartbeat response text between clients is vectorized to obtain the average heartbeat response text vector between clients. Determine the cosine similarity between the average heartbeat response text vectors of any two clients, and use the cosine similarity as the proximity between the two clients.

4. The method according to claim 1, characterized in that, The method further includes: Fine-tuning the aforementioned large-scale time-series model includes: Obtain the historical computing power resources and / or historical network resources of at least one client; Based on the historical computing power resources and / or the historical network resources, and using the training computing power resources and / or training network resources of the at least one client obtained through historical training of the time series large model, and the parameters currently used by the time series large model, the differential parameters are obtained. The time series model is fine-tuned using the difference parameters to obtain the fine-tuned time series model.

5. The method according to claim 4, characterized in that, The difference parameter is obtained in the following way: Based on the historical computing power resources and / or the historical network resources, and using the training computing power resources and / or training network resources of the at least one client obtained through historical training of the time series large model, and the parameters currently used by the time series large model, backpropagation iteration is performed to obtain a first low-rank matrix and a second low-rank matrix. The difference parameter is obtained based on the first low-rank matrix and the second low-rank matrix.

6. The method according to claim 4 or 5, characterized in that, The method further includes: Based on the fine-tuned time-series large model, predict the computing resources and / or network resources of the first client.

7. The method according to claim 1, characterized in that, The step of determining the task efficiency factor of the first client based on the computing power resources and / or network resources of the first client includes: The computing power resources and / or network resources of the first client are used as inputs to the task efficiency factor model, and the output of the task efficiency factor model is used as the task efficiency factor of the first client.

8. The method according to claim 1, characterized in that, The step of selecting a second client from the first client based on the task efficiency factor includes: Calculate the probability of the first client participating in federated learning based on the task efficiency factor; The first client, whose probability of participating in federated learning is greater than or equal to a first preset value, is designated as the second client.

9. The method according to claim 1, characterized in that, The method further includes: The third client is selected from the second client, wherein the third client is used to perform primary aggregation of the model parameters of other clients and the model parameters of the third client based on the proximity between the third client and other clients.

10. The method according to claim 1, characterized in that, The method further includes: The final model parameter aggregation result is sent to the first client.

11. A federated learning method, characterized in that, Applied to third-party clients, including: Receive model parameters sent by other clients; The model parameters of the other clients and the model parameters of the third client are initially aggregated to obtain the first model parameters, and the first model parameters are sent to the central aggregation server.

12. A federated learning device, characterized in that, include: Applications to central aggregation servers include: The first processing module is used to predict the computing power resources and / or network resources of at least one first client based on a time series large model. The second processing module is used to determine the task efficiency factor of the first client based on the computing power resources and / or network resources of the first client. The third processing module is used to select a second client from the first client based on the task efficiency factor, wherein the second client is a client participating in federated learning; The fourth processing module is used to receive the first model parameters sent by the third client and aggregate them according to the first model parameters. The third client is a client selected from the second client according to the task efficiency factor, and the first model parameters are obtained by the third client through primary aggregation of the model parameters of the third client and the model parameters of other clients.

13. A federated learning device, characterized in that, Applied to third-party clients, including: The first receiving module is used to receive model parameters sent by other clients; The first processing module is used to perform primary aggregation on the model parameters of the other clients and the model parameters of the third client to obtain the first model parameters, and send the first model parameters to the central aggregation server.

14. A federated learning device, characterized in that, include: Central aggregation server, third-party clients, and other clients; The central aggregation server is configured to: predict the computing and / or network resources of at least one first client based on a time-series large model; determine the task efficiency factor of the first client based on the computing and / or network resources of the first client; select a second client from the first clients based on the task efficiency factor, wherein the second client is a client participating in federated learning; receive first model parameters sent by a third client, and aggregate according to the first model parameters, wherein the third client is a client selected from the second clients based on the task efficiency factor; The third client is used to receive model parameters sent by other clients, perform primary aggregation on the model parameters of the other clients and the model parameters of the third client to obtain first model parameters, and send the first model parameters to the central aggregation server. The other client is used to obtain the model parameters and send the model parameters to the third client.

15. A communication device, comprising: A memory, a processor, and a program stored in the memory and executable on the processor; characterized in that the processor is configured to read the program from the memory to implement the steps of the federated learning method as described in any one of claims 1 to 11.

16. A computer-readable storage medium for storing a program, characterized in that, When the program is executed by the processor, it implements the steps in the federated learning method as described in any one of claims 1 to 11.

17. A computer program product, characterized in that, It includes computer instructions that, when executed by a processor, implement the steps in the federated learning method as described in any one of claims 1 to 11.

Images