Model training method and device

Abstract

The invention relates to a model training method and device. The method comprises the following steps: under the condition that a cloud server obtains global model parameters of a to-be-trained model, optimizing a local model parameter updating frequency and a local model parameter updating frequency according to local model parameter transmission time of the to-be-trained model and local model parameter transmission time of the to-be-trained model under the constraint of training time; acquiring updated local model parameters based on the optimized local model parameter updating frequency, the optimized local model parameter updating frequency and the global model parameters; and updating the to-be-trained model according to the updated local model parameters until the to-be-trained model converges to obtain a trained model. By adopting the method, the communication overhead in the model training process can be reduced, and meanwhile, the model training precision is improved.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a model training method and device. Background Art

[0002] Federated learning is an emerging distributed machine learning method that has received extensive attention and research. The core of federated learning is to build a global model by exchanging model parameters or intermediate results without exchanging local data, so as to improve the generalization ability of the model and data privacy.

[0003] In related technologies, during model training, federated learning is usually combined with edge-cloud collaboration scenarios. After each terminal server obtains the global model from the cloud server, it uses the local data set to train the global model to obtain local model parameters. Multiple edge servers then perform preliminary aggregation of the local model parameters to obtain local model parameters. The cloud server then aggregates the local model parameters to obtain global model parameters. The cloud server then sends the global model parameters to the terminal server through the edge server for the next round of iterative training.

[0004] However, in the process of model training in related technologies, there is a technical problem of high communication overhead. Summary of the invention

[0005] Based on this, it is necessary to provide a model training method and device to address the above technical problems, which can reduce the communication overhead in the model training process and at the same time improve the training accuracy of the model.

[0006] In a first aspect, an embodiment of the present application provides a model training method, which is applied to a cloud server, comprising:

[0007] When the cloud server obtains the global model parameters of the model to be trained, under the constraint of training time, the local model parameter update frequency and the local model parameter update frequency are optimized according to the local model parameter transmission time of the model to be trained and the local model parameter transmission time of the model to be trained;

[0008] Based on the adjusted local model parameter update frequency and the optimized local model parameter update frequency, and the global model parameters, obtaining updated local model parameters;

[0009] The model to be trained is updated according to the updated local model parameters until the model to be trained converges to obtain a trained model.

[0010] In one embodiment, the cloud server is connected to a plurality of edge servers, each edge server is connected to a plurality of terminal servers; the method further comprises:

[0011] Determine the transmission model parameter size of all terminal servers under each edge server according to the number of samples of the local data set of each terminal server, the parameter dimension of each data sample in each local data set, and the data volume of each parameter dimension;

[0012] Determine the uplink communication rate between the terminal server and the edge server according to the communication bandwidth, power spectrum density, transmission power and channel gain of each terminal server and each edge server;

[0013] The local model parameter transmission time is determined according to the transmission model parameter size and the uplink communication rate between the terminal server and each edge server.

[0014] In one embodiment, the method further comprises:

[0015] Obtain the transmission model parameter sizes of all terminal servers under each edge server;

[0016] Determine the uplink communication rate between the edge server and the cloud server based on the communication bandwidth, power spectrum density, transmit power and channel gain of each edge server and the cloud server;

[0017] The local model parameter transmission time is determined according to the transmission model parameter size and the uplink communication rate between the edge server and the cloud server.

[0018] In one embodiment, the local model parameter update frequency and the local model parameter update frequency are optimized according to the local model parameter transmission time of the model to be trained and the local model parameter transmission time of the model to be trained, including:

[0019] Determine the local model training time based on the number of machine cycles required to calculate the local model parameters for a single data sample and the computing resources of each terminal server;

[0020] Optimize the local model parameter update frequency and the local model parameter update frequency according to the local model training time, the local model parameter transmission time and the local model parameter transmission time.

[0021] In one embodiment, optimizing the local model parameter update frequency and the local model parameter update frequency according to the local model training time, the local model parameter transmission time and the local model parameter transmission time includes:

[0022] Obtain the training time constraints and update frequency objective function of the model to be trained;

[0023] Under the constraint of training time and taking the minimum calculation result of the update frequency objective function as the goal, the local model parameter update frequency and the local model parameter update frequency are determined according to the local model training time, the local model parameter transmission time and the local model parameter transmission time.

[0024] In one embodiment, a cloud server is connected to a plurality of edge servers, each edge server is connected to a plurality of terminal servers; based on the optimized local model parameter update frequency and the optimized local model parameter update frequency, and the global model parameter, obtaining the updated local model parameter includes:

[0025] Sending the local model parameter update frequency, the local model parameter update frequency and the global model parameter to each edge server to instruct each edge server to transmit the updated local model parameter based on the local model parameter update frequency, and each terminal server to transmit the updated local model parameter based on the local model parameter update frequency and the global model parameter;

[0026] Receive updated local model parameters transmitted by each edge server.

[0027] In one of the embodiments, upon receiving the local model parameter update frequency and the global model parameters, each edge server transmits the local model parameter update frequency and the global model parameters to each terminal server to instruct each terminal server to perform model training based on the global model parameters to obtain updated local model parameters, and transmit the updated local model parameters to each edge server according to the local model parameter update frequency.

[0028] In one of the embodiments, upon receiving each updated local model parameter, each edge server aggregates each updated local model parameter to obtain an updated local model parameter, and transmits the updated local model parameter to the cloud server according to the local model parameter update frequency.

[0029] In one embodiment, the cloud server is connected to a plurality of edge servers, each edge server is connected to a plurality of terminal servers; the method further comprises:

[0030] Obtaining local ciphertext model parameters sent by each edge server; the local ciphertext model parameters are obtained by aggregating multiple local ciphertext model parameters; the multiple local ciphertext model parameters are obtained by encrypting the corresponding local model parameters by each terminal server;

[0031] Aggregate the local ciphertext model parameters to obtain the global model parameters.

[0032] In a second aspect, the embodiment of the present application further provides a model training device, including:

[0033] A frequency optimization module, for optimizing the local model parameter update frequency and the local model parameter update frequency according to the local model parameter transmission time of the model to be trained and the local model parameter transmission time of the model to be trained under the constraint of training time when the cloud server obtains the global model parameters of the model to be trained;

[0034] A parameter updating module, used for obtaining updated local model parameters based on the optimized local model parameter update frequency and the optimized local model parameter update frequency, and the global model parameters;

[0035] The model updating module is used to update the model to be trained according to the updated local model parameters until the model to be trained converges to obtain a trained model.

[0036] In a third aspect, an embodiment of the present application further provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the steps of the method in any one of the embodiments of the first aspect above are implemented.

[0037] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method in any one of the embodiments of the first aspect above.

[0038] In a fifth aspect, an embodiment of the present application further provides a computer program product, including a computer program, which, when executed by a processor, implements the steps of the method in any one of the embodiments of the first aspect above.

[0039] The model training method and device provided in the embodiments of the present application, when the cloud server obtains the global model parameters of the model to be trained, optimizes the local model parameter update frequency and the local model parameter update frequency according to the local model parameter transmission time of the model to be trained and the local model parameter transmission time of the model to be trained under the constraint of training time, and then obtains the updated local model parameters based on the optimized local model parameter update frequency and the optimized local model parameter update frequency, as well as the global model parameters, and then updates the model to be trained according to the updated local model parameters until the model to be trained converges to obtain a trained model. In this method, when training the model to be trained, after the cloud server obtains the global model parameters of the model to be trained, it dynamically optimizes the local model parameter update frequency and the local model parameter update frequency according to the local model parameter transmission time and the local model parameter transmission time within the limited training time. This is equivalent to optimizing the communication frequency between the terminal server and the edge server and between the edge server and the cloud server under time constraints, maximizing resource utilization, avoiding unnecessary communication, and thus reducing communication overhead; at the same time, by dynamically adjusting the communication frequency, the convergence performance of the model is optimized within the limited training time, thereby improving the accuracy and efficiency of model training. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the related technologies, the drawings required for use in the embodiments or the related technical descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0041] FIG1 is an application environment diagram of a model training method in one embodiment;

[0042] FIG2 is a schematic diagram of a flow chart of a model training method in one embodiment;

[0043] FIG3 is a schematic diagram of a process for obtaining a local model parameter transmission time in one embodiment;

[0044] FIG4 is a schematic diagram of a process for obtaining a local model parameter transmission time in one embodiment;

[0045] FIG5 is a schematic diagram of a process for optimizing the local model parameter update frequency and the partial model parameter update frequency in one embodiment;

[0046] FIG6 is a schematic diagram of a flow chart of optimizing the local model parameter update frequency and the partial model parameter update frequency in another embodiment;

[0047] FIG7 is a schematic diagram of a process for obtaining updated local model parameters in one embodiment;

[0048] FIG8 is a schematic diagram of a flow chart of a model training method in another embodiment;

[0049] FIG9 is a schematic diagram of the structure of a model training device in one embodiment;

[0050] FIG. 10 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0051] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0052] The technical background of the embodiments of the present application is first described below.

[0053] In the current field of distributed computing and machine learning, federated learning, as an emerging distributed machine learning method, has received extensive attention and research. The main idea of ​​federated learning is to improve the generalization ability of the model and data privacy by coordinating multiple distributed participants to train the model, while avoiding centralized storage and processing of data. The traditional federated learning training process consists of a central server and multiple distributed participant terminals. The participants first download the latest global model from the central server, then train the model on the local data set, and upload the trained model parameters to the central server, which then aggregates and distributes the model parameters. This method achieves data collaboration across devices and organizations without sharing the original data, which not only improves the generalization ability of the model, but also improves the privacy and security of the data. It has been widely used in finance, medical care, and personal device data processing.

[0054] However, with the rapid development of the Internet of Things and edge computing, the number of devices and the amount of data have increased dramatically, and the traditional cloud computing model faces bottlenecks in processing this data. The cloud-based collaborative training method adopted by traditional federated learning mainly relies on the central server to aggregate and distribute the model. In the scenario of federated learning, cloud servers and terminals are mostly distributed across geographies, and the frequent communication between the two is limited in low-latency, high-effectiveness application scenarios. Edge computing distributes computing tasks to edge devices close to the data source, significantly reducing data transmission delays and improving system response speed and efficiency. Therefore, the cloud-edge collaborative architecture came into being. By flexibly allocating computing tasks between the cloud and the edge, the system's response speed and computing efficiency can be improved, providing an efficient and reliable solution for fields such as the Internet of Things and distributed computing.

[0055] In the federated learning architecture for cloud-edge collaborative scenarios, edge devices are responsible for the initial processing, storage, and local model training of data, and share some computing tasks with edge servers, while cloud servers handle more complex model training and large-scale data analysis. This architecture can effectively balance the computing load, reduce the amount of data transmission, and improve the overall performance and response speed of the system by reasonably allocating tasks between edge devices and cloud servers. Edge devices usually have lower computing power and storage capacity, but they are close to data sources and can respond and process data quickly, making them suitable for processing latency-sensitive tasks. Cloud servers have powerful computing power and storage resources, making them suitable for processing complex model training and large-scale data analysis tasks.

[0056] Although cloud-edge collaboration brings many advantages, it still faces several technical challenges in practical applications, especially in terms of optimizing communication overhead and model update frequency. Frequent transmission of model parameters will bring high communication overhead, especially when there are a large number of edge devices and they are widely distributed geographically. In addition, although federated learning avoids centralized storage and processing of data, there is still a risk of data leakage and attack during the transmission and aggregation of model parameters. Therefore, ensuring the security of data during transmission has become an important issue in federated learning. Common encryption methods include secure multi-party computing technology and differential privacy technology. The former allows multiple participants to collaborate on computing without leaking their respective inputs, but it has high computing and communication overhead; the latter protects privacy by adding noise to data or model parameters, but differential privacy will affect model accuracy to a certain extent.

[0057] Based on this, the present application provides a model training method. When training the model to be trained, after the cloud server obtains the global model parameters of the model to be trained, within the limited training time, the local model parameter update frequency and the local model parameter update frequency are dynamically optimized according to the local model parameter transmission time and the local model parameter transmission time. This is equivalent to optimizing the communication frequency between the terminal server and the edge server and between the edge server and the cloud server under time constraints, maximizing resource utilization, avoiding unnecessary communication, and thus reducing communication overhead; at the same time, by dynamically adjusting the communication frequency, the convergence performance of the model is optimized within the limited training time, thereby improving the accuracy and efficiency of model training.

[0058] It should be noted that the beneficial effects brought about by the embodiments of the present application or the technical problems solved are not limited to this one, but may also include other implicit or related problems. For details, please refer to the description of the following embodiments.

[0059] The application environment of the model training method provided in the embodiment of the present application is described below. The model training method provided in the embodiment of the present application can be applied in the application environment shown in Figure 1. Among them, the cloud server 106 is connected to multiple edge servers 104, and each edge server 104 is connected to multiple terminal servers 102. When performing model training, the cloud server 106 sends the model to be trained to the edge server 104, and the edge server 104 sends the model to be trained to the terminal server 102. The terminal server 102 uses the local data set to train the model to be trained, obtains local model parameters, and sends the local model parameters to the corresponding edge server 104. The edge server 104 aggregates the local model parameters to obtain local model parameters, and sends the local model parameters to the cloud server 106. The cloud server 106 aggregates the local model parameters to obtain global model parameters, and optimizes the local model parameter update frequency and the local model parameter update frequency according to the local model parameter transmission time and the local model parameter transmission time, so that in the subsequent iterative training process, the terminal server 102 and the edge server 104 transmit the local model parameters and the local model parameters according to the optimized local model parameter update frequency and the local model parameter update frequency, respectively. The cloud server 106 continuously updates the model to be trained based on the transmitted local model parameters and local model parameters until the model to be trained converges to obtain a trained model.

[0060] The technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems are described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0061] In an exemplary embodiment, as shown in FIG2 , a model training method is provided, which is described by taking the method applied to the cloud server 106 in FIG1 as an example, and includes the following steps 201 to 203. Among them:

[0062] S201, when the cloud server obtains the global model parameters of the model to be trained, under the constraint of training time, the local model parameter update frequency and the local model parameter update frequency are optimized according to the local model parameter transmission time of the model to be trained and the local model parameter transmission time of the model to be trained.

[0063] In the embodiment of the present application, the cloud server is connected to multiple edge servers, and each edge server is connected to multiple terminal servers. For example, there are M edge nodes, each edge server is connected to terminal servers, then the number of terminal servers is ,and The cloud server is responsible for receiving and aggregating local model parameters from all edge servers, calculating the latest global model parameters and sending them to each edge server; the edge server is responsible for receiving local model parameters from its associated terminal server, calculating local model parameters and uploading the results to the cloud server; the terminal server is responsible for downloading the latest global model parameters from its associated edge server, and then performing local updates according to the gradient descent algorithm, and uploading the local model parameters to its associated edge server.

[0064] In order to enhance the security of model parameters, this application uses homomorphic encryption algorithm to protect model parameters and sets the public key to , the private key is During the local model parameter update process, the terminal server uses the trained local model parameters to Encryption is performed to obtain a local ciphertext model, and then the local ciphertext model is transmitted to the edge server. Both the edge server and the cloud server use the properties of homomorphic encryption to complete the model update process in the ciphertext state.

[0065] The process of the cloud server obtaining the global model parameters of the model to be trained is the model parameter update process. When the model parameters are updated, the above-mentioned homomorphic encryption algorithm can be used to complete the model update process.

[0066] In one embodiment, local ciphertext model parameters sent by each edge server are obtained; each local ciphertext model parameter is aggregated to obtain a global model parameter; the local ciphertext model parameter is obtained by aggregating multiple local ciphertext model parameters; and multiple local ciphertext model parameters are obtained by encrypting the corresponding local model parameters by each terminal server.

[0067] In the embodiment of the present application, each terminal server encrypts the local model parameters to obtain the local ciphertext model parameters, and then each terminal server transmits the local ciphertext model parameters to the edge server connected thereto. Each edge server directly aggregates the received local ciphertext model parameters to obtain the local ciphertext model parameters, and then each edge server transmits the local ciphertext model parameters to the cloud server.

[0068] After the cloud server obtains the local ciphertext model parameters sent by each edge server, it aggregates the local ciphertext model parameters to obtain the global model parameters. Among them, the global model parameters at this time are encrypted global model parameters, that is, global ciphertext model parameters. It should be noted that because the model training is iterative, each time the local model parameters and local model parameters are updated, the terminal server needs to transmit the local model parameters to the edge server, and the edge server transmits the local model parameters to the cloud server. In these processes, the transmitted model parameters are protected by the homomorphic encryption algorithm. Taking the first model parameter update as an example, the cloud server sends the model to be trained to the edge server, and the edge server sends the model to be trained to the terminal server. The terminal server uses local data to train the model to be trained, obtains the local model parameters, and encrypts the local model parameters to obtain the local ciphertext model parameters, and sends the local ciphertext model parameters to the edge server. The edge server aggregates the local ciphertext model parameters to obtain the local ciphertext model parameters, and sends the local ciphertext model parameters to the cloud server. The cloud server aggregates the local ciphertext model parameters to obtain the global ciphertext model parameters, and sends the global ciphertext model parameters to the edge server. The edge server sends the global ciphertext model parameters to the terminal server, and then performs a new round of model parameter updates.

[0069] For example, using Indicates The edge server associated with Terminal servers, each terminal server Have a local dataset , For the dataset A single data sample, For this data set Input sample, is the corresponding output. Set is the parameter dimension of a single data sample, and the amount of data in each dimension is Bits are represented using Represents the number of samples in the dataset.

[0070] During the local model parameter update process, In the first iteration, the terminal server First, download the latest global ciphertext model parameters from the edge server , after decryption, the initial local model parameters Set to , Terminal Server Leveraging the Dataset Update the local model parameters based on the gradient descent algorithm, as shown in formula (1) below:

[0071] (1)

[0072] in, is the learning rate of the model to be trained, is the local loss function The gradient value of is the weighted average of the loss function of a single data sample, and the calculation formula is shown in (2).

[0073] (2)

[0074] Terminal Server The local model parameters in the latest iteration round are encrypted as , and transmit it to the edge server. Each edge server receives all local ciphertext model parameters transmitted from its associated end server. ,in The edge server calculates the local ciphertext model parameters based on the homomorphic encryption property and transmits them to the cloud server. The calculation method of is shown in the following formula (3).

[0075] (3)

[0076] in, , indicating the The sum of all data of the terminal servers associated with each edge server.

[0077] The edge server transmits the calculated local ciphertext model parameters to the cloud server. The cloud server first receives all the local ciphertext model parameters transmitted from the edge server. ,in Then, according to the homomorphic encryption property, the global ciphertext model parameters are calculated and transmitted to each edge server, where the global ciphertext model parameters The calculation method of is shown in the following formula (4).

[0078] (4)

[0079] in, , represents the sum of all data of the entire system model.

[0080] After obtaining the global ciphertext model parameters based on the above method, in order to optimize the energy loss caused by the interaction between the terminal server and the edge server, as well as the edge server and the cloud server, the embodiment of the present application takes into account the transmission energy consumption generated in the system, and dynamically optimizes the local model parameter update frequency and the local model parameter update frequency in the scenario where time resources are limited, that is, under the constraints of training time, so that the convergence performance of the model reaches the optimal level.

[0081] Each time the terminal server sends local model parameters to the edge server, the local model parameters are updated once and the local model parameters are updated to local model parameters. The local model parameter update frequency can also be understood as the frequency at which the terminal server transmits local model parameters to the edge server. Each time the edge server sends local model parameters to the cloud server, the local model parameters are updated once and updated to global model parameters. The local model parameter update frequency can also be understood as the frequency at which the edge server transmits local model parameters to the cloud server.

[0082] In the embodiment of the present application, based on the local model parameter transmission time and the local model parameter transmission time, the local model parameter update frequency and the local model parameter update frequency are optimized. Wherein, the local model parameter transmission time refers to the time required for the terminal server to transmit the local model parameters to the edge server each time. The local model parameter transmission time refers to the time required for the edge server to transmit the local model parameters to the cloud server each time.

[0083] Exemplarily, an objective function can be constructed based on the local model parameter transmission time and the local model parameter transmission time, as well as the local model parameter update frequency and the local model parameter update frequency, so as to obtain the optimized local model parameter update frequency and the optimized local model parameter update frequency by solving the objective function.

[0084] It is understandable that in practical applications, model parameters are prone to the risk of data leakage during transmission. Based on this, the embodiments of the present application introduce homomorphic encryption technology to ensure the security of data during transmission and processing. Even if model aggregation and processing are performed in edge servers and cloud servers, there is no need to decrypt the data, thereby preventing malicious nodes from inferring local data through model parameters.

[0085] S202, based on the optimized local model parameter update frequency and the optimized local model parameter update frequency, and the global model parameters, obtaining updated local model parameters.

[0086] After obtaining the optimized local model parameter update frequency and the optimized local model parameter update frequency, the terminal server transmits the local model parameters according to the optimized local model parameter update frequency, and the edge server transmits the local model parameters according to the optimized local model parameter update frequency.

[0087] The cloud server sends the optimized local model parameter update frequency, the optimized local model parameter update frequency, and the global model parameters to the edge server, so that the edge server sends the global model parameters and the optimized local model parameter update frequency to the terminal server. The terminal server sends the updated local model parameters to the edge server based on the global model parameters and the optimized local model parameter update frequency. The edge server then obtains the updated local model parameters based on the updated local model parameters and sends the updated local model parameters to the cloud server. In this way, the cloud server can obtain the updated local model parameters.

[0088] S203, updating the model to be trained according to the updated local model parameters until the model to be trained converges to obtain a trained model.

[0089] After obtaining the updated local model parameters, the cloud server aggregates the updated local model parameters to obtain updated global model parameters. After obtaining the updated global model parameters, the aforementioned steps are still used to obtain new local model parameter update frequencies and new local model parameter update frequencies, and the terminal server and the edge server transmit the local model parameters and the local model parameters according to the new local model parameter update frequencies and the new local model parameter update frequencies, respectively. Then, the global model parameters are updated based on the transmitted parameters, and the iteration is repeated until the training time is exhausted to obtain the optimal global model parameters. At this time, the convergence performance of the model to be trained reaches the optimal level, and a trained model is obtained.

[0090] In this embodiment, by introducing edge-cloud collaborative technology, the edge server is used to preliminarily aggregate the local model parameters transmitted by the terminal server, thereby reducing the size of data transmitted to the cloud, thereby reducing communication costs and improving the training efficiency of the model.

[0091] In the model training method provided in the embodiment of the present application, when the cloud server obtains the global model parameters of the model to be trained, under the constraint of training time, the local model parameter update frequency and the local model parameter update frequency are optimized according to the local model parameter transmission time of the model to be trained and the local model parameter transmission time of the model to be trained, and then based on the optimized local model parameter update frequency and the optimized local model parameter update frequency, as well as the global model parameters, the updated local model parameters are obtained, and then the model to be trained is updated according to the updated local model parameters until the model to be trained converges to obtain a trained model. In this method, when training the model to be trained, after the cloud server obtains the global model parameters of the model to be trained, it dynamically optimizes the local model parameter update frequency and the local model parameter update frequency according to the local model parameter transmission time and the local model parameter transmission time within the limited training time. This is equivalent to optimizing the communication frequency between the terminal server and the edge server and between the edge server and the cloud server under time constraints, maximizing resource utilization, avoiding unnecessary communication, and thus reducing communication overhead; at the same time, by dynamically adjusting the communication frequency, the convergence performance of the model is optimized within the limited training time, thereby improving the accuracy and efficiency of model training.

[0092] Based on the above embodiment, an embodiment is provided to illustrate the process of obtaining the transmission time of the above local model parameters.

[0093] In an exemplary embodiment, as shown in FIG3 , the cloud server is connected to a plurality of edge servers, and each edge server is connected to a plurality of terminal servers; the method further includes:

[0094] S301, determining the transmission model parameter size of all terminal servers under each edge server according to the sample quantity of the local data set of each terminal server, the parameter dimension of each data sample in each local data set, and the data volume of each parameter dimension.

[0095] Exemplarily, the transmission model parameter sizes of all terminal servers under each edge server may be calculated according to the following formula (5).

[0096] (5)

[0097] in, is the transmission model size of all terminal servers under the i-th edge server; is the parameter dimension for each data sample; The amount of data for each parameter dimension; is the number of samples of the local data set of the jth terminal server under the i-th edge server.

[0098] S302: Determine an uplink communication rate between the terminal server and the edge server according to the communication bandwidth, power spectrum density, transmission power and channel gain of each terminal server and each edge server.

[0099] In the embodiment of the present application, the uplink communication rate between the terminal server and the edge server can be determined based on Shannon's theorem, specifically using the following formula (6).

[0100] (6)

[0101] in, is the uplink communication rate between the terminal server and the edge server; The communication bandwidth between the terminal server and the edge server; is the transmitting power; is the channel gain; is the power spectral density.

[0102] S303: Determine the local model parameter transmission time according to the transmission model parameter size and the uplink communication rate between the terminal server and the edge server.

[0103] In the embodiment of the present application, the ratio of the transmission model parameter size to the uplink communication rate between the terminal server and the edge server is determined as the local model parameter transmission time. For example, the following formula (7) can be used.

[0104] (7)

[0105] in, is the local model parameter transmission time.

[0106] In the model training method provided in the embodiment of the present application, the transmission model parameter size of all terminal servers under each edge server is determined according to the sample number of the local data set of each terminal server, the parameter dimension of each data sample in each local data set and the data amount of each parameter dimension, and then the uplink communication rate between the terminal server and the edge server is determined according to the communication bandwidth, power spectrum density, transmission power and channel gain of each terminal server and each edge server, and then the local model parameter transmission time is determined according to the transmission model parameter size and the uplink communication rate between the terminal server and the edge server. In this method, the transmission model parameter size of all terminal servers under each edge server is first determined through the sample information of the local data set in the terminal server, and then the uplink communication rate of the terminal server and the edge server is determined based on the data transmission communication information between the terminal server and the edge server. On this basis, the local model parameter transmission time is quickly and accurately determined based on the ratio between the transmission model parameter size and the uplink communication rate, thereby quickly optimizing the parameter update frequency and improving the model training efficiency.

[0107] Based on the above embodiment, an embodiment is provided to illustrate the process of obtaining the transmission time of the above local model parameters.

[0108] In an exemplary embodiment, as shown in FIG4 , the method further includes:

[0109] S401, obtaining the transmission model parameter sizes of all terminal servers under each edge server.

[0110] The transmission model parameter size of all terminal servers under each edge server is determined based on the number of samples in the local data set, the parameter dimension of each data sample in each local data set, and the amount of data in each parameter dimension. The specific method can be found in the above formula (5), which will not be repeated here.

[0111] S402, determining an uplink communication rate between the edge server and the cloud server according to the communication bandwidth, power spectrum density, transmission power and channel gain of each edge server and the cloud server.

[0112] In the embodiment of the present application, the uplink communication rate between the edge server and the cloud server can still use Shannon's theorem, and specifically can use the following formula (8).

[0113] (8)

[0114] in, is the uplink communication rate between the edge server and the cloud server; is the communication bandwidth between edge servers and cloud servers.

[0115] S403, determining the local model parameter transmission time according to the transmission model parameter size and the uplink communication rate between the edge server and the cloud server.

[0116] In an embodiment of the present application, the ratio of the transmission model parameter size to the uplink communication rate between the edge server and the cloud server is determined as the local model parameter transmission time. For example, the following formula (9) can be used.

[0117] (9)

[0118] in, is the local model parameter transmission time.

[0119] In the model training method provided in the embodiment of the present application, the transmission model parameter size of all terminal servers under each edge server is obtained, and then the uplink communication rate between the edge server and the cloud server is determined according to the communication bandwidth, power spectrum density, transmission power and channel gain of each edge server and the cloud server, and then the local model parameter transmission time is determined according to the transmission model parameter size and the uplink communication rate between the edge server and the cloud server. In this method, on the basis of obtaining the transmission model parameter size, the uplink communication rate between the edge server and the cloud server is determined based on the data transmission communication information between the edge server and the cloud server, and then the local model parameter transmission time is quickly determined based on the ratio between the transmission model parameter size and the uplink communication rate, which provides data support for the optimization of the local model parameter update frequency and the local model parameter update frequency, improves the accuracy and efficiency of the update frequency optimization, and thus improves the model training efficiency and training accuracy.

[0120] Based on the above embodiment, an embodiment is provided to illustrate the process of optimizing the local model parameter update frequency and the partial model parameter update frequency.

[0121] In an exemplary embodiment, as shown in FIG5 , the local model parameter update frequency and the local model parameter update frequency are optimized according to the local model parameter transmission time of the model to be trained and the local model parameter transmission time of the model to be trained, including:

[0122] S501, determining the local model training time according to the number of machine cycles required for calculating the local model parameters for a single data sample and the computing resources of each terminal server.

[0123] In the embodiment of the present application, the number of machine cycles required for calculating the local model parameters is first calculated based on the computing resources of each terminal server and a single data sample to determine the training time required for a local model. Then, the local model training time is determined based on the training time required for a local model. For example, the local model training time is determined using the following formulas (10) and (11).

[0124] (10)

[0125] (11)

[0126] in, The number of machine cycles required to compute the local model parameters for a single data sample; The computing resources (CPU cycle frequency) of each terminal server; The time required to train the local model parameters for the jth terminal server under the i-th edge server, that is, the training time required for a local model; is the local model training time of all terminal servers under the i-th edge server.

[0127] S502, optimizing the local model parameter update frequency and the local model parameter update frequency according to the local model training time, the local model parameter transmission time and the local model parameter transmission time.

[0128] In the embodiment of the present application, when optimizing the local model parameter update frequency and the partial model parameter update frequency, an objective function may be established and the optimized local model parameter update frequency and the optimized partial model parameter update frequency may be obtained by solving the objective function.

[0129] In one embodiment, as shown in FIG6 , the process of optimizing the local model parameter update frequency and the partial model parameter update frequency may include the following steps:

[0130] S601, obtaining the training time constraint and update frequency objective function of the model to be trained.

[0131] In the embodiment of the present application, the terminal server is set to The local model parameters are uploaded in the iteration process, and the edge server is The local model parameters are uploaded in each iteration process. Therefore, during the training process, each interval The iteration process completes a global aggregation. Assume that the total number of iterations is ,in is the number of global aggregations, then The total time for edge servers to complete model training is expressed as . Set the total time of model training to , , then the objective function can be established as , the constraints are .

[0132] For the global loss function ,in , is the local loss function of the i-th edge server, and the global loss function is a convex function. Make the following assumptions: For any and , all satisfy: ; ;in, and is a parameter to measure the effect of the model. According to the above formula, it can be estimated as: ; ; For any and , ,in , .

[0133] In order to solve the objective function, the embodiment of the present application introduces the ideal loss function , then the update frequency objective function can be established as the following formula (12), and the constraints are as follows formula (13).

[0134] (12)

[0135] (13)

[0136] S602, under the training time constraint and with the minimum calculation result of the update frequency objective function as the goal, determine the optimized local model parameter update frequency and the optimized local model parameter update frequency according to the local model training time, the local model parameter transmission time and the local model parameter transmission time.

[0137] The update frequency objective function will be minimized under the constraints of C1-C3. Constraint C1 constrains the total time of model training. After each round of global aggregation is completed, the remaining total time To update, ; Constraints C2 and C3 are assumptions about the loss function, where represents the control parameters of the system model, which remain unchanged during the training process. , then the denominator of the objective function is always a positive number, then when L takes the maximum value, that is , the objective function reaches the minimum, and is solved according to the convex optimization theory to obtain the optimized local model parameter update frequency and the optimized local model parameter update frequency, until the time resources are consumed and the optimization ends.

[0138] In this embodiment, an update frequency objective function is constructed based on the global loss function and the ideal loss function, and constraints on the loss function and the training time are established. Under the constraints, the update frequency objective function is solved with the goal of minimizing the update frequency objective function to obtain the optimized local model parameter update frequency and the optimized local model parameter update frequency, thereby performing parameter update based on the optimized parameter update frequency, maximizing resource utilization efficiency, and avoiding waste of communication resources and model training delays.

[0139] In the model training method provided in the embodiment of the present application, the local model training time is determined based on the number of machine cycles required to calculate the local model parameters for a single data sample and the computing resources of each terminal server, and then the local model parameter update frequency and the local model parameter update frequency are optimized based on the local model training time, the local model parameter transmission time and the local model parameter transmission time. In this method, the local model training time is first calculated, and then the local model parameter update frequency and the local model parameter update frequency are optimized in combination with the local model parameter transmission time and the local model parameter transmission time, so as to adjust the communication frequency, avoid resource waste, reduce communication overhead, and improve the model training efficiency.

[0140] Based on the above embodiment, an embodiment is provided to illustrate the process of obtaining the updated local model parameters.

[0141] In an exemplary embodiment, as shown in FIG7 , a cloud server is connected to a plurality of edge servers, each edge server is connected to a plurality of terminal servers; based on the optimized local model parameter update frequency and the optimized local model parameter update frequency, and the global model parameter, the updated local model parameter is obtained, including:

[0142] S701, sending the local model parameter update frequency, the local model parameter update frequency and the global model parameters to each edge server to instruct each edge server to transmit the updated local model parameters based on the local model parameter update frequency, and each terminal server to transmit the updated local model parameters based on the local model parameter update frequency and the global model parameters.

[0143] After optimizing the local model parameter update frequency and the local model parameter update frequency, the cloud server sends the optimized local model parameter update frequency, the optimized local model parameter update frequency, and the currently aggregated global model parameters to each edge server, so that each edge server and each terminal server can perform the next round of training.

[0144] Exemplarily, upon receiving the local model parameter update frequency and the global model parameters, each edge server transmits the local model parameter update frequency and the global model parameters to each terminal server to instruct each terminal server to perform model training based on the global model parameters to obtain updated local model parameters, and transmit the updated local model parameters to each edge server according to the local model parameter update frequency.

[0145] After receiving the optimized local model parameter update frequency and global model parameters, each edge server sends the local model parameter update frequency and global model parameters to each terminal server. Each terminal server first performs a new round of training based on the global model parameters to obtain updated local model parameters, and then sends the updated local model parameters to the corresponding edge server. At this time, when sending, it is necessary to send according to the local model parameter update frequency. If the local model parameter update frequency is not reached at present, it is necessary to continue training and continue to obtain new local model parameters until the local model parameter update frequency is reached, and the latest local model parameters are sent to the corresponding edge server.

[0146] Exemplarily, upon receiving each updated local model parameter, each edge server aggregates each updated local model parameter to obtain an updated local model parameter, and transmits the updated local model parameter to the cloud server according to the local model parameter update frequency.

[0147] After each edge server receives the updated local model parameters sent by the associated terminal server, each edge server aggregates the updated local model parameters to obtain updated local model parameters. If the local model parameter update frequency is currently reached, the updated local model parameters can be sent to the cloud server. If the local model parameter update frequency is not currently reached, each edge server sends the updated local model parameters to the corresponding terminal server so that the terminal server performs model training based on the updated local model parameters. Only when the local model parameter update frequency is reached will the edge server transmit the latest local model parameters to the cloud server for aggregation.

[0148] S702: Receive updated local model parameters transmitted by each edge server.

[0149] After each edge server transmits the updated local model parameters, the cloud server receives the updated local model parameters transmitted by each edge server, and aggregates the updated local model parameters to obtain updated global model parameters.

[0150] In the model training method provided in the embodiment of the present application, the local model parameter update frequency, the local model parameter update frequency and the global model parameter are sent to each edge server to instruct each edge server to transmit the updated local model parameters based on the local model parameter update frequency, and each terminal server to transmit the updated local model parameters based on the local model parameter update frequency and the global model parameter; the updated local model parameters transmitted by each edge server are received. In this method, the cloud server sends the optimized local model parameter update frequency and the local model parameter update frequency, as well as the global model parameters to each edge server, so that the edge server updates the local model parameters according to the optimized local model parameter update frequency, and the terminal server updates the local model parameters according to the optimized local model parameter update frequency, optimizes and adjusts the communication frequency between the edge server and the cloud server and between the edge server and the terminal server, reduces the communication overhead, and improves the model training accuracy and training efficiency.

[0151] In addition, in an exemplary embodiment, the specific process of model training in the embodiment of the present application is described.

[0152] As shown in Figure 8, each terminal server first updates the local model parameters, then each edge server updates the local model parameters, and then the cloud server performs global aggregation and optimizes the local model parameter update frequency and the local model parameter update frequency through a dynamic optimization method until the training time is exhausted and a global model is obtained.

[0153] In this embodiment, through cloud-edge collaboration technology, the edge server is used to perform preliminary aggregation of the model update data transmitted by the terminal server, reducing the size of data transmitted to the cloud, thereby reducing the communication cost; through a dynamic optimization method, the communication frequency between the terminal server and the edge server and between the edge server and the cloud server is dynamically adjusted according to time constraints to maximize resource utilization efficiency; by balancing the communication time and model training accuracy, it is ensured that the accuracy and efficiency of model training are optimized under limited resource conditions.

[0154] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0155] Based on the same inventive concept, the embodiment of the present application also provides a model training device for implementing the model training method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more model training device embodiments provided below can refer to the limitations on the model training method above, and will not be repeated here.

[0156] In an exemplary embodiment, as shown in FIG9 , a model training device 1 is provided, comprising: a frequency optimization module 10, a parameter updating module 20 and a model updating module 30, wherein:

[0157] A frequency optimization module 10 is used to optimize the local model parameter update frequency and the local model parameter update frequency according to the local model parameter transmission time of the model to be trained and the local model parameter transmission time of the model to be trained under the constraint of training time when the cloud server obtains the global model parameters of the model to be trained;

[0158] A parameter updating module 20, configured to obtain updated local model parameters based on the optimized local model parameter update frequency and the optimized local model parameter update frequency, and the global model parameters;

[0159] The model updating module 30 is used to update the model to be trained according to the updated local model parameters until the model to be trained converges to obtain a trained model.

[0160] In one embodiment, the model training device 1 further includes:

[0161] A parameter size determination module, used to determine the transmission model parameter size of all terminal servers under each edge server according to the number of samples of the local data set of each terminal server, the parameter dimension of each data sample in each local data set, and the data volume of each parameter dimension;

[0162] A first rate determination module, used to determine the uplink communication rate between the terminal server and the edge server according to the communication bandwidth, power spectrum density, transmission power and channel gain of each terminal server and each edge server;

[0163] The local time determination module is used to determine the local model parameter transmission time according to the transmission model parameter size and the uplink communication rate between the terminal server and the edge server.

[0164] In one embodiment, the model training device 1 further includes:

[0165] A parameter size acquisition module is used to obtain the transmission model parameter sizes of all terminal servers under each edge server;

[0166] A second rate determination module, used to determine the uplink communication rate between the edge server and the cloud server according to the communication bandwidth, power spectrum density, transmission power and channel gain of each edge server and the cloud server;

[0167] The local time determination module is used to determine the local model parameter transmission time according to the transmission model parameter size and the uplink communication rate between the edge server and the cloud server.

[0168] In one embodiment, the frequency optimization module 10 is further used for:

[0169] Determine the local model training time based on the number of machine cycles required to calculate the local model parameters for a single data sample and the computing resources of each terminal server; optimize the local model parameter update frequency and local model parameter update frequency based on the local model training time, local model parameter transmission time and local model parameter transmission time.

[0170] In one embodiment, the frequency optimization module 10 is further used for:

[0171] Obtain the training time constraints and update frequency objective function of the model to be trained; under the training time constraints and with the minimum calculation result of the update frequency objective function as the goal, determine the optimized local model parameter update frequency and the optimized local model parameter update frequency according to the local model training time, the local model parameter transmission time and the local model parameter transmission time.

[0172] In one embodiment, the parameter updating module 20 is further used for:

[0173] The local model parameter update frequency, the local model parameter update frequency and the global model parameters are sent to each edge server to instruct each edge server to transmit the updated local model parameters based on the local model parameter update frequency, and each terminal server to transmit the updated local model parameters based on the local model parameter update frequency and the global model parameters; and the updated local model parameters transmitted by each edge server are received.

[0174] In one embodiment, the model training device 1 further includes:

[0175] A parameter acquisition module is used to acquire local ciphertext model parameters sent by each edge server; the local ciphertext model parameters are obtained by aggregating multiple local ciphertext model parameters; the multiple local ciphertext model parameters are obtained by encrypting the corresponding local model parameters by each terminal server;

[0176] The parameter aggregation module is used to aggregate the local ciphertext model parameters to obtain the global model parameters.

[0177] Each module in the above-mentioned model training device can be implemented in whole or in part by software, hardware and a combination thereof. Each of the above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory in the computer device in the form of software, so that the processor can call and execute the operations corresponding to each of the above modules.

[0178] In an exemplary embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be shown in FIG10. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. Among them, the processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store model training data. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a model training method is implemented.

[0179] Those skilled in the art will understand that the structure shown in FIG. 10 is merely a block diagram of a partial structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different arrangement of components.

[0180] In an exemplary embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the following steps are implemented:

[0181] When the cloud server obtains the global model parameters of the model to be trained, under the constraint of training time, the local model parameter update frequency and the local model parameter update frequency are optimized according to the local model parameter transmission time of the model to be trained and the local model parameter transmission time of the model to be trained;

[0182] Based on the optimized local model parameter update frequency and the optimized local model parameter update frequency, and the global model parameters, obtaining updated local model parameters;

[0183] The model to be trained is updated according to the updated local model parameters until the model to be trained converges to obtain a trained model.

[0184] The implementation principles and technical effects of each step implemented by the processor in the embodiment of the present application are similar to the principles of the above-mentioned model training method and will not be repeated here.

[0185] In one embodiment, a computer readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented:

[0186] When the cloud server obtains the global model parameters of the model to be trained, under the constraint of training time, the local model parameter update frequency and the local model parameter update frequency are optimized according to the local model parameter transmission time of the model to be trained and the local model parameter transmission time of the model to be trained;

[0187] Based on the optimized local model parameter update frequency and the optimized local model parameter update frequency, and the global model parameters, obtaining updated local model parameters;

[0188] The model to be trained is updated according to the updated local model parameters until the model to be trained converges to obtain a trained model.

[0189] The implementation principles and technical effects of each step implemented when the computer program in the embodiment of the present application is executed by the processor are similar to the principles of the above-mentioned model training method and will not be repeated here.

[0190] In one embodiment, a computer program product is provided, comprising a computer program, which, when executed by a processor, implements the following steps:

[0191] When the cloud server obtains the global model parameters of the model to be trained, under the constraint of training time, the local model parameter update frequency and the local model parameter update frequency are optimized according to the local model parameter transmission time of the model to be trained and the local model parameter transmission time of the model to be trained;

[0192] Based on the optimized local model parameter update frequency and the optimized local model parameter update frequency, and the global model parameters, obtaining updated local model parameters;

[0193] The model to be trained is updated according to the updated local model parameters until the model to be trained converges to obtain a trained model.

[0194] The implementation principles and technical effects of each step implemented when the computer program in the embodiment of the present application is executed by the processor are similar to the principles of the above-mentioned model training method and will not be repeated here.

[0195] It should be noted that the personnel information (including but not limited to personnel equipment information, personnel personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the personnel or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0196] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.

[0197] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0198] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A model training method, characterized in that: Applied to a cloud server, the method comprises: When the cloud server obtains the global model parameters of the model to be trained, under the constraint of training time, the local model parameter update frequency and the local model parameter update frequency are optimized according to the local model parameter transmission time of the model to be trained and the local model parameter transmission time of the model to be trained; Based on the optimized local model parameter update frequency and the optimized local model parameter update frequency, and the global model parameters, obtaining updated local model parameters; The model to be trained is updated according to the updated local model parameters until the model to be trained converges to obtain a trained model.

2. The method according to claim 1, characterized in that The cloud server is connected to a plurality of edge servers, each edge server is connected to a plurality of terminal servers; the method further comprises: Determine the transmission model parameter size of all terminal servers under each edge server according to the number of samples of the local data set of each terminal server, the parameter dimension of each data sample in each local data set, and the data volume of each parameter dimension; Determine the uplink communication rate between the terminal server and the edge server according to the communication bandwidth, power spectrum density, transmit power and channel gain of each terminal server and each edge server; The local model parameter transmission time is determined according to the transmission model parameter size and the uplink communication rate between the terminal server and the edge server.

3. The method according to claim 1, characterized in that The method further comprises: Obtain the transmission model parameter sizes of all terminal servers under each edge server; Determine the uplink communication rate between the edge server and the cloud server according to the communication bandwidth, power spectrum density, transmit power and channel gain of each edge server and the cloud server; The local model parameter transmission time is determined according to the transmission model parameter size and the uplink communication rate between the edge server and the cloud server.

4. The method according to any one of claims 1 to 3, characterized in that: The optimizing the local model parameter update frequency and the local model parameter update frequency according to the local model parameter transmission time of the model to be trained and the local model parameter transmission time of the model to be trained includes: Determine the local model training time based on the number of machine cycles required to calculate the local model parameters for a single data sample and the computing resources of each terminal server; The local model parameter update frequency and the local model parameter update frequency are optimized according to the local model training time, the local model parameter transmission time and the local model parameter transmission time.

5. The method according to claim 4, characterized in that Optimizing the local model parameter update frequency and the local model parameter update frequency according to the local model training time, the local model parameter transmission time and the local model parameter transmission time comprises: Obtaining training time constraints and update frequency objective function of the model to be trained; Under the training time constraint and with the minimum calculation result of the update frequency objective function as the goal, the optimized local model parameter update frequency and the optimized local model parameter update frequency are determined according to the local model training time, the local model parameter transmission time and the local model parameter transmission time.

6. The method according to any one of claims 1 to 3, characterized in that: The cloud server is connected to a plurality of edge servers, each edge server is connected to a plurality of terminal servers; the updated local model parameters are obtained based on the optimized local model parameter update frequency and the optimized local model parameter update frequency, and the global model parameters, including: Sending the local model parameter update frequency, the local model parameter update frequency and the global model parameter to each edge server to instruct each edge server to transmit the updated local model parameter based on the local model parameter update frequency, and each terminal server to transmit the updated local model parameter based on the local model parameter update frequency and the global model parameter; Receive updated local model parameters transmitted by each edge server.

7. The method according to claim 6, characterized in that Upon receiving the local model parameter update frequency and the global model parameters, each edge server transmits the local model parameter update frequency and the global model parameters to each terminal server to instruct each terminal server to perform model training based on the global model parameters to obtain updated local model parameters, and transmit the updated local model parameters to each edge server according to the local model parameter update frequency.

8. The method according to claim 7, characterized in that Upon receiving the updated local model parameters, each edge server aggregates the updated local model parameters to obtain the updated local model parameters, and transmits the updated local model parameters to the cloud server according to the local model parameter update frequency.

9. The method according to any one of claims 1 to 3, characterized in that: The cloud server is connected to a plurality of edge servers, each edge server is connected to a plurality of terminal servers; the method further comprises: Acquire local ciphertext model parameters sent by each edge server; the local ciphertext model parameters are obtained by aggregating multiple local ciphertext model parameters; the multiple local ciphertext model parameters are obtained by encrypting the corresponding local model parameters by each terminal server; Aggregate the local ciphertext model parameters to obtain the global model parameters.

10. A model training device, characterized in that: The device comprises: A frequency optimization module, for optimizing the local model parameter update frequency and the local model parameter update frequency according to the local model parameter transmission time of the model to be trained and the local model parameter transmission time of the model to be trained under the constraint of training time when the cloud server obtains the global model parameters of the model to be trained; A parameter updating module, configured to obtain updated local model parameters based on the optimized local model parameter update frequency and the optimized local model parameter update frequency, and the global model parameters; The model updating module is used to update the model to be trained according to the updated local model parameters until the model to be trained converges to obtain a trained model.

Images