Global model parameter determination method and device, storage medium and electronic equipment
By directly aggregating local model parameters in the cloud server network, the problem of low global parameter aggregation efficiency in the distributed machine learning framework is solved, and efficient model construction and data robustness are achieved.
Patent Information
- Application Number
- CN202411178332.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-26
- Publication Date
- 2024-12-06
AI Technical Summary
The existing Peer-to-Peer cloud server network distributed machine learning framework requires multiple rounds of time iterations when global parameter aggregation, resulting in large communication overhead and low model construction efficiency.
By obtaining the filtering measurement data of each user end in the cloud server network, determining the local model parameters, and using a complete graph model to directly aggregate the local model parameters of each server to obtain global model parameters and avoiding the iterative process.
It effectively reduces the determination time of global model parameters, improves model construction efficiency, and improves the robustness of training data, prevents the impact of poisoning attacks.
Smart Images

Figure CN119094366A_ABST
Abstract
Description
Technical Field
[0001] The embodiments of the present application relate to the field of computer technology, and in particular, to a method and device for determining global model parameters, a storage medium, and an electronic device. Background Art
[0002] With the development of computer technology, we have entered the digital age. In the digital age, massive amounts of data from edge digital devices, such as mobile phones and sensors connected to industrial equipment, are usually sent to central processing units (such as cloud servers), which perform data processing, model learning, packaging, aggregation, etc.
[0003] In order to improve the efficiency of data processing, computing and storage, the Peer-t0-Peer (P2P) cloud server network distributed machine learning framework came into being. This learning framework forms multiple clusters by splitting the user end, and each cluster is configured with a cloud server to form a local centralized learning module, and then obtain local parameters through the local learning module. The cloud servers of multiple clusters aggregate local parameters to obtain global parameters, so that model construction can be carried out based on global parameters.
[0004] However, this learning framework still faces many problems. After each cluster’s cloud server obtains the local parameters of the cluster, it often needs to go through multiple rounds of time iterations when aggregating the global parameters. This not only makes the communication overhead of the entire network very large, but also causes the time to determine the global parameters to be too long, thus affecting the efficiency of model construction. Summary of the invention
[0005] The embodiments of the present application provide a method and device for determining global model parameters, a storage medium and an electronic device, so as to at least solve the problem that when global parameter aggregation is carried out in the related technology, multiple rounds of time iterations are often required, which not only makes the communication overhead of the entire network very large, but also causes the time spent on determining the global parameters to be too long, thereby affecting the efficiency of model building.
[0006] According to an embodiment of the present application, a method for determining global model parameters is provided, including: obtaining screening measurement data corresponding to each user terminal in a cluster where a first server in a cloud server network is located; obtaining first local model parameters of the first server based on each of the screening measurement data; obtaining second local model parameters of a second server other than the first server in the cloud server network; and aggregating the first local model parameters and the second local model parameters to obtain global model parameters.
[0007] In an exemplary embodiment, the obtaining of filtered measurement data corresponding to each user terminal in the cluster where the first server in the cloud server network is located includes: receiving online measurement data corresponding to each user terminal in the cluster where the first server in the cloud server network is located; filtering each of the online measurement data to determine the filtered measurement data corresponding to each of the online measurement data.
[0008] In an exemplary embodiment, the online measurement data of each user terminal includes multiple input parameters; the screening of each of the online measurement data to determine the screening measurement data corresponding to each of the online measurement data includes: for any online measurement data, the following steps are used to carry out the screening: determine the output column vector of each of the input parameters included in the online measurement data; select any two input parameters from each of the input parameters to determine as target input parameters; obtain verification constants based on the target input parameters and the output column vectors corresponding to the target input parameters; determine the screening results of the target input parameters based on the verification constants, and re-select the target parameters from each of the input parameters, return to the step of obtaining the verification constant based on the target input parameters and the output column vectors corresponding to the target input parameters, until the screening of each of the input parameters is completed and the screening measurement data corresponding to the targeted online measurement data is obtained.
[0009] In an exemplary embodiment, determining the screening result of the target input parameter based on the verification constant includes: comparing the verification constant with a preset constant threshold to obtain a comparison result; if the comparison result is that the verification constant is less than the constant threshold, then the screening result is to retain the target input parameter.
[0010] In an exemplary embodiment, the target input parameters include a first target input parameter and a second target input parameter; the output column vector includes a first output column vector corresponding to the first target input parameter and a second output column vector corresponding to the second target input parameter; the obtaining of the verification constant based on the target input parameters and the output column vector corresponding to the target input parameters includes: determining an input difference between the first target input parameter and the second target input parameter; determining an output difference between the first output column vector and the second output column vector; and obtaining the verification constant based on the output difference and the input difference. In an exemplary embodiment, the obtaining of the first local model parameters of the first server based on each of the screened measurement data includes: performing local prediction based on each of the screened measurement data to obtain the model parameters of each user terminal; and aggregating according to the model parameters of each user terminal to obtain the first local model parameters of the first server.
[0011] In an exemplary embodiment, the local prediction is performed based on each of the filtered measurement data to obtain the model parameters of each user terminal, including: determining the output column vector of each of the filtered measurement data based on each of the filtered measurement data; obtaining the prediction expectation and prediction variance of each user terminal based on each of the filtered measurement data, each of the output column vectors and a pre-set prior variance; and obtaining the model parameters of each user terminal based on the prediction expectation and prediction variance of each user terminal.
[0012] In an exemplary embodiment, the model parameters include prediction expectations and prediction variances of each user terminal; the aggregating the model parameters of each user terminal to obtain the first local model parameters of the first server includes: obtaining the number of user terminals in the cloud server network and the number of user terminal groups in the cloud server network; determining the sum of prediction expectations based on the prediction expectations of each user terminal; and determining the sum of prediction variances based on the prediction variances of each user terminal; obtaining the first local model parameters of the first server based on the number of user terminals, the number of user terminal groups, the sum of prediction expectations and the sum of prediction variances.
[0013] In an exemplary embodiment, the first local model parameters of the first server are obtained based on the number of user terminals, the number of user terminal groups, the sum of predicted expectations and the sum of predicted variances, including: obtaining an average predicted expectation based on the number of user terminals, the number of user terminal groups and the sum of predicted expectations; obtaining an average predicted variance based on the number of user terminals, the number of user terminal groups and the sum of predicted variances; and obtaining the first local model parameters of the first server based on the average predicted expectation and the average predicted variance.
[0014] In an exemplary embodiment, obtaining the second local model parameters of a second server other than the first server in the cloud server network includes: obtaining a complete graph model of the first server and the second server; and obtaining the second local model parameters of the second server other than the first server in the cloud server network based on the complete graph model.
[0015] In an exemplary embodiment, the aggregating the first local model parameters and the second local model parameters to obtain global model parameters includes: obtaining a first coefficient of the first server, and obtaining a second coefficient of the second server; obtaining a first global model parameter of the first server based on the first coefficient and the first local model parameters; obtaining a second global model parameter of the second server based on the second coefficient and the two local model parameters; and obtaining global model parameters based on the first global model parameters and the second global model parameters.
[0016] In an exemplary embodiment, the first local model parameters include a first local prediction expectation and a first local prediction variance; obtaining the first global model parameters of the first server based on the first coefficient and the first local model parameters includes: obtaining a predefined prior variance; determining a first product of the first local prediction variance, the first local prediction expectation and the first coefficient; determining a second product of the first coefficient and the first local prediction expectation, and determining a third product of the first coefficient and the prior variance; obtaining the first global model parameters of the first server according to the first product, the second product and the third product.
[0017] In an exemplary embodiment, the first global model parameters include a first global prediction expectation and a first global prediction variance; obtaining the first global model parameters of the first server according to the first product, the second product and the third product includes: determining the first global prediction expectation based on the first product; determining the first global prediction variance based on the second product and the third product.
[0018] In an exemplary embodiment, the second local model parameters include a second local prediction expectation and a second local prediction variance; obtaining the second global model parameters of the second server based on the second coefficient and the two local model parameters includes: obtaining a predefined prior variance; determining the fourth product of the second local prediction variance, the second local prediction expectation and the second coefficient; determining the fifth product of the second coefficient and the second local prediction expectation, and determining the sixth product of the second coefficient and the prior variance; obtaining the second global model parameters of the second server according to the fourth product, the fifth product and the sixth product.
[0019] In an exemplary embodiment, the second global model parameters include a second global prediction expectation and a second global prediction variance; obtaining the second global model parameters of the second server according to the fourth product, the fifth product and the sixth product includes: determining the second global prediction expectation based on the fourth product; and determining the second global prediction variance based on the fifth product and the sixth product.
[0020] In an exemplary embodiment, the global model parameters include a global prediction expectation and a global prediction variance; the first global model parameters include a first global prediction expectation and a first global prediction variance; the second global model parameters include a second global prediction expectation and a second global prediction variance; obtaining the global model parameters based on the first global model parameters and the second global model parameters includes: determining the global prediction expectation as the product of the sum of the first global prediction expectation and the second global prediction expectation and the sum of the average prediction variances; determining the global prediction variance as the sum of the first global prediction variance and the two global prediction variances.
[0021] In an exemplary embodiment, the method further includes: feeding back the global model parameters to each of the user terminals to instruct each of the user terminals to carry out model construction according to the global model parameters.
[0022] According to another embodiment of the present application, a device for determining global model parameters is provided, including: a first acquisition module, used to acquire screening measurement data corresponding to each user terminal of a cluster where a first server in a cloud server network is located; a data processing module, used to obtain a first local model parameter of the first server based on each of the screening measurement data; a second acquisition module, used to acquire a second local model parameter of a second server other than the first server in the cloud server network; the data processing module is also used to aggregate the first local model parameter and the second local model parameter to obtain a global model parameter.
[0023] According to another embodiment of the present application, a computer-readable storage medium is provided, in which a computer program is stored, wherein the computer program is configured to execute the steps of any of the above method embodiments when run.
[0024] According to another embodiment of the present application, an electronic device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.
[0025] According to another embodiment of the present application, a computer program product is provided, including a computer program, and when the computer program is executed by a processor, the steps in any of the above method embodiments are implemented.
[0026] Through the present application, online measurement data sent by each user terminal of the cluster where the first service is located is obtained, and based on each online measurement data, the first local model parameter of the first server is obtained, and the second local model parameter of the second server other than the first server in the cloud server network is further obtained. Therefore, there is no need for iteration, and the global model parameter can be obtained directly by aggregating the first local model parameter and the second local model parameter, which effectively reduces the time for determining the global model parameter and improves the efficiency of subsequent model construction. BRIEF DESCRIPTION OF THE DRAWINGS
[0027] Figure 1 It is a hardware structure block diagram of a computer terminal of a method for determining global model parameters according to an embodiment of the present application;
[0028] Figure 2 is a flow chart of a method for determining global model parameters according to an embodiment of the present application;
[0029] Figure 3 It is a distributed cloud shared learning framework in a cloud server network according to an embodiment of the present application;
[0030] Figure 4 is a flowchart of a method for determining global model parameters according to an embodiment of the present application;
[0031] Figure 5 It is a structural block diagram of a device for determining global model parameters according to an embodiment of the present application. DETAILED DESCRIPTION
[0032] The embodiments of the present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.
[0033] It should be noted that the terms "first", "second", etc. in the specification and claims of this application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.
[0034] The method embodiments provided in the embodiments of the present application can be executed in a mobile terminal, a computer terminal or a similar computing device. Taking running on a computer terminal as an example, Figure 1 1 is a hardware structure block diagram of a computer terminal of a method for determining global model parameters in an embodiment of the present application. Figure 1 As shown, the computer terminal may include one or more ( Figure 1 Only one is shown in the figure) a processor 102 (the processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA) and a memory 104 for storing data, wherein the above-mentioned computer terminal may also include a transmission device 106 and an input / output device 108 for communication functions. It can be understood by those skilled in the art that Figure 1The structure shown is only for illustration and does not limit the structure of the above-mentioned computer terminal. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations shown.
[0035] The memory 104 can be used to store computer programs, for example, software programs and modules of application software, such as the computer program corresponding to the method for allocating load resources in the embodiment of the present application. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, that is, to implement the above method. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include a memory remotely arranged relative to the processor 102, and these remote memories may be connected to the server via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.
[0036] The transmission device 106 is used to receive or send data via a network. The specific example of the above network may include a wireless network provided by a communication provider of a computer terminal. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, referred to as NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 can be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.
[0037] Figure 2 is a flow chart of a method for determining global model parameters according to an embodiment of the present application, and the method is described by taking the application of the method to a computer terminal as an example. The computer terminal may specifically be a first server in a cloud server network, such as Figure 2 As shown, the process includes the following steps:
[0038] Step S202, obtaining screening measurement data corresponding to each client of the cluster where the first server in the cloud server network is located;
[0039] It should be noted that the first server is any server in the cloud server network, and specifically can be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, and big data and artificial intelligence platforms. In the cloud server network, other servers except the first server can be the second server.
[0040] The first server and multiple clients may form a federated learning subsystem, and the client directly connected to the first server may be a client of the cluster in which the first server is located. The filtered measurement data is data corresponding to each client of the cluster in which the first server is located.
[0041] It is understandable that the screened measurement data can be obtained by filtering the online measurement data after the user terminal collects the online measurement data online, and then sends it to the first server. Of course, the user terminal can also directly send the online measurement data to the first server, and the first server filters the online measurement data.
[0042] Step S204: obtaining a first local model parameter of the first server based on each of the screened measurement data.
[0043] It should be noted that model parameters are numerical quantities used to describe model behavior in machine learning or statistical modeling. Model parameters can be various parameters such as weights, biases, slopes, etc. In an embodiment of the present application, model parameters can specifically be expectations and variances. The first local model parameters are model parameters corresponding to the first server.
[0044] Step S206: obtaining a second local model parameter of a second server other than the first server in the cloud server network.
[0045] It should be noted that the second server is a server other than the first server in the cloud server network. The number of second servers can be one or more, and the embodiment of the present application does not limit this. The local model parameters corresponding to the second server are the second local model parameters. It is understandable that the second server and multiple user terminals can also form a federated learning subsystem, and the user terminal directly connected to the second server can be the user terminal of the cluster where the second server is located.
[0046] In some embodiments, obtaining the second local model parameters of the second server other than the first server in the cloud server network includes: obtaining a complete graph model of the first server and the second server; and based on the complete graph model, obtaining the second local model parameters of the second server other than the first server in the cloud server network.
[0047] Among them, the complete graph model is a model used to characterize the connection relationship between the first server and the second server in the cloud server network. Since each cloud server in the complete graph model communicates with other cloud servers, this cloud server can be regarded as a central control unit relative to other cloud servers, that is, a centralized network structure. Then, in such a sparse cloud server centralized network, the first server can obtain the second local model parameters of all second servers in the cloud server network except the first server at one time.
[0048] The embodiment of the present application establishes a complete connection between the first server and the second server through a complete graph model between the first server and the second server, so that when the first local model parameters and the second local model parameters are subsequently aggregated, there is no need for iteration, and aggregation processing is directly performed based on the first local model parameters and each second local model parameter at one time to obtain the global model parameters, which effectively reduces the time for determining the global model parameters and improves the efficiency of subsequent model construction.
[0049] Step S208: Aggregate the first local model parameters and the second local model parameters to obtain global model parameters.
[0050] Among them, the first local model parameter and the second local model parameter are aggregated, that is, the first local model parameter and the second local model parameter are fused, and the fusion method can be averaging, summing, weighted summing, etc., which is not limited in the embodiment of the present application.
[0051] It can be understood that in addition to determining its own first local model parameters, the first server also obtains the second local model parameters of the second server in the cloud server network, that is, all local model parameters required to determine the global model parameters are obtained at one time. Therefore, there is no need for iteration, and the global model parameters can be obtained by directly aggregating the first local model parameters and the second local model parameters, which effectively reduces the time for determining the global model parameters and improves the efficiency of subsequent model construction.
[0052] Through the above steps, the online measurement data sent by each user terminal of the cluster where the first service is located is obtained, and the first local model parameters of the first server are obtained according to each online measurement data, and the second local model parameters of the second server other than the first server in the cloud server network are further obtained. Therefore, the global model parameters can be obtained directly by aggregating the first local model parameters and the second local model parameters without iteration, which effectively reduces the time for determining the global model parameters and improves the efficiency of subsequent model construction.
[0053] In an exemplary embodiment, the obtaining of filtered measurement data corresponding to each user terminal in the cluster where the first server in the cloud server network is located includes: receiving online measurement data corresponding to each user terminal in the cluster where the first server in the cloud server network is located; filtering each of the online measurement data to determine the filtered measurement data corresponding to each of the online measurement data.
[0054] It can be understood that the screened measurement data refers to the data obtained after the online measurement data is screened, and the screened measurement data is obtained by screening the online measurement data to filter out the online measurement data that does not meet the requirements. Among them, the online measurement data that does not meet the requirements can be unstable data that has been contaminated by poisoning attacks. Further, on the basis of obtaining the screened measurement data, the first local model parameter is determined based on the screened measurement data.
[0055] In the embodiment of the present application, the online measurement data is screened to obtain the screened measurement data. Further, on the basis of obtaining the screened measurement data, the first local model parameter is determined based on the screened measurement data, so as to ensure the robustness of the local prediction.
[0056] In an exemplary embodiment, the online measurement data of each user terminal includes multiple input parameters; the screening of each of the online measurement data to obtain the screened measurement data corresponding to each of the online measurement data includes: for any online measurement data, the following steps are used to carry out the screening: determine the output column vector of each of the input parameters included in the online measurement data; select any two input parameters from each of the input parameters to determine as target input parameters; obtain a verification constant based on the target input parameters and the output column vector corresponding to the target input parameters; determine the screening result of the target input parameter based on the verification constant, and re-select the target parameter from each of the input parameters, return to the step of obtaining the verification constant based on the target input parameter and the output column vector corresponding to the target input parameter, until the screening of each of the input parameters is completed and the screened measurement data corresponding to the targeted online measurement data is obtained.
[0057] It should be noted that the input parameters are the input parameters of the user terminal included in the online measurement data. For any user terminal in the cluster where the first server is located, the online measurement data can be sent to the first server. For the online measurement data of any user terminal, screening can be carried out to obtain the screened measurement data of the user terminal. The verification parameter is a calculated parameter used to determine whether the target input parameter and the output column vector corresponding to the target input parameter meet the requirements.
[0058] Specifically, when screening any online measurement data, the first server can determine the output column vector of each input parameter. Specifically, the output column vector can be determined by a set functional relationship, that is, each input parameter is input into the preset functional relationship, and the output parameter corresponding to each input parameter can be obtained. Further, the first server can select any two input parameters from each input parameter to determine as the target input parameters. During the selection process, the first server can adopt a traversal method to sequentially expand and select from the input parameters to obtain two continuous input parameters to determine as the target input parameters; of course, the first server can also randomly expand and select from the input parameters, and the embodiment of the present application is not limited here.
[0059] The first server obtains a verification constant based on the selected target input parameter and the output column vector corresponding to the target input parameter, and determines the screening result of the target input parameter according to the verification constant. The first server re-selects from the input parameters based on the traversal method, obtains two new continuous input parameters as the target input parameters, and returns the step of obtaining the verification constant based on the target input parameter and the output column vector corresponding to the target input parameter, and continues to execute until all input parameters are screened and the screened measurement data corresponding to the targeted online measurement data is obtained.
[0060] The embodiment of the present application can ensure that the subsequent model parameter determination process is not affected by virus attacks by screening the input parameters included in each online measurement data in turn, thereby improving the accuracy of model parameter determination.
[0061] In an exemplary embodiment, determining the screening result of the target input parameter based on the verification constant includes: comparing the verification constant with a preset constant threshold to obtain a comparison result; if the comparison result is that the verification constant is less than the constant threshold, then the screening result is to retain the target input parameter.
[0062] It is understandable that the constant threshold is a pre-set parameter used to determine whether the verification constant meets the requirements. The specific value of the constant threshold can be set according to the actual model application scenario, model accuracy requirements, etc.
[0063] In some embodiments, the first server compares the verification constant with the constant threshold. When the verification constant is less than the constant threshold, it indicates that the target input parameter meets the requirements, that is, the target input parameter is a parameter that has not been attacked by a virus, and the target input parameter can be retained, and the output column vector corresponding to the target input parameter can be retained. When the verification constant is greater than or equal to the constant threshold, it indicates that the target input parameter does not meet the requirements, that is, the target input parameter is a parameter attacked by a virus, and the target input parameter can be removed, and the output column vector corresponding to the target input parameter can be removed.
[0064] The embodiment of the present application sets a constant threshold and then compares the verification constant with the constant verification threshold. Through the comparison result, it can accurately determine whether to retain the target input parameter, thereby avoiding the subsequent model parameter prediction process from being affected by virus attacks and improving the accuracy of model parameter determination.
[0065] In an exemplary embodiment, the target input parameters include a first target input parameter and a second target input parameter; the output column vector includes a first output column vector corresponding to the first target input parameter and a second output column vector corresponding to the second target input parameter; obtaining a verification constant based on the target input parameters and the output column vector corresponding to the target input parameters includes: determining an input difference between the first target input parameter and the second target input parameter; determining an output difference between the first output column vector and the second output column vector; and obtaining a verification constant based on the output difference and the input difference.
[0066] It should be noted that when the first server determines the verification constant, it can be determined based on the principle of the Lipschitz constant. The principle of the Lipschitz constant is as follows:
[0067] The Lipschitz continuity of the objective function f means that for any two inputs x1 and x2, there exists a constant l f , so that
[0068] |f(x1)-f(x2)|≤l f ||x1-x2||, where any x1 and
[0069] Therefore, for each user-side screening measurement data, the threshold of the Lipschitz constant, i.e., the constant threshold, can be set. Then, by traversing all the input parameters of the screening measurement data, the target input parameter and the output column vector corresponding to the target input parameter can be determined. For any two target input parameters, and and the output column vector and calculate Then determine whether it is less than a constant threshold Right now Less than a constant threshold , the target input parameter and the output column vector of the target input parameter can be retained.
[0070] The embodiment of the present application determines the verification constant based on the principle of the Lipschitz constant, and then compares the verification constant with the constant threshold, which can accurately determine whether to retain the target input parameter, thereby avoiding the subsequent model parameter prediction process from being affected by virus attacks and improving the accuracy of model parameter determination.
[0071] In an exemplary embodiment, obtaining the first local model parameters of the first server based on each of the filtered measurement data includes: performing local prediction based on each of the filtered measurement data to obtain the model parameters of each user terminal; and aggregating the model parameters of each user terminal to obtain the first local model parameters of the first server.
[0072] Specifically, the first server can carry out local prediction for the screening measurement data of each user terminal to determine the model parameters of each user terminal. The first server obtains the first local model parameters of the first server by aggregating the model parameters of each user terminal in the cluster to which it belongs.
[0073] It should be understood that when the second server determines the second local model parameters, the second server can also obtain the corresponding screening measurement data for each user terminal in the cluster where the second server is located, and then carry out local prediction for the screening measurement data corresponding to each user terminal to determine the model parameters of each user terminal. The second server obtains the second local model parameters of the second server by aggregating the model parameters of each user terminal in the cluster where the second server is located.
[0074] In the embodiment of the present application, the model parameters of each user terminal in the cluster where the first server is located are first determined, and then the model parameters of each user terminal are aggregated to obtain the first local model parameters of the first server.
[0075] In an exemplary embodiment, the local prediction is performed based on each of the filtered measurement data to obtain the model parameters of each user terminal, including: determining the output column vector of each of the filtered measurement data based on each of the filtered measurement data; obtaining the prediction expectation and prediction variance of each user terminal based on each of the filtered measurement data, each of the output column vectors and a pre-set prior variance; and obtaining the model parameters of each user terminal based on the prediction expectation and prediction variance of each user terminal.
[0076] Specifically, when the first server determines the model parameters of the user terminal, it is essentially a process of determining the prediction expectation and prediction variance of the user terminal. The prediction expectation is also called the posterior expectation, and the prediction variance is also called the posterior variance. When determining the prediction expectation and the prediction variance, it is necessary to determine the screening measurement data, each output column vector, and the pre-set prior variance.
[0077]
[0078]
[0079] Among them, k is the set kernel function, y is the output column vector, × is the screened measurement data, and the prior variance is is the user's prediction expectation, Refers to the prediction variance of the user end, and i refers to any user end.
[0080] The embodiment of the present application can carry out local prediction by screening the measurement data, each output column vector and a pre-set prior variance, and accurately obtain the model parameters of each user terminal.
[0081] In an exemplary embodiment, the model parameters include the prediction expectation and prediction variance of each user terminal; the aggregation based on each of the model parameters to obtain the first local model parameters of the first server includes: obtaining the number of user terminals of the user terminals in the cloud server network, and the number of user terminal groups in the cloud server network; determining the sum of the prediction expectations based on the prediction expectations of each user terminal; and determining the sum of the prediction variances based on the prediction variances of each user terminal; obtaining the first local model parameters of the first server based on the number of user terminals, the number of user terminal groups, the sum of the prediction expectations and the sum of the prediction variances.
[0082] It can be understood that the number of user terminals is the actual number of user terminals connected to the first server and the second server in the cloud server network, and the number of user terminals can be adaptively adjusted according to server resources, network status, etc. The number of user terminal groups is oriented to the cloud server network, that is, the number of user terminal groups is the same as the number of federated learning subsystems in the cloud server network. The total predicted expectation is the sum of the predicted expectations of each user terminal in the cluster where the first server is located, and the total predicted variance is the sum of the predicted variances of each user terminal in the cluster where the first server is located.
[0083] Specifically, the first server can obtain the sum of predicted expectations based on the predicted expectations of each user terminal in its cluster, and obtain the sum of predicted variances based on the predicted variances of each user terminal in its cluster. Then, the first server can obtain the first local model parameters of the first server based on the number of user terminals, the number of user terminal groups, the sum of predicted expectations and the sum of predicted variances.
[0084] In some embodiments, for the second server, the sum of predicted expectations can be obtained based on the predicted expectations of each user terminal in its cluster, and the sum of predicted variances can be obtained based on the predicted variances of each user terminal in its cluster. Then, the second server can obtain the second local model parameters of the second server based on the number of user terminals, the number of user terminal groups, the sum of predicted expectations and the sum of predicted variances.
[0085] The embodiment of the present application aggregates the model parameters of the user end of the cluster where the first server is located to accurately obtain the first local model parameters.
[0086] In an exemplary embodiment, the first local model parameters of the first server are obtained based on the number of user terminals, the number of user terminal groups, the sum of predicted expectations and the sum of predicted variances, including: obtaining an average predicted expectation based on the number of user terminals, the number of user terminal groups and the sum of predicted expectations; obtaining an average predicted variance based on the number of user terminals, the number of user terminal groups and the sum of predicted variances; and obtaining the first local model parameters of the first server based on the average predicted expectation and the average predicted variance.
[0087] Specifically, after the first server determines the prediction expectation and prediction variance of all the users in the cluster, the first server can use the average method to calculate the first local model parameters, that is, calculate the first local prediction expectation and the first local prediction variance. The specific calculation formula is as follows:
[0088]
[0089]
[0090] in, refers to the first local prediction expectation, Refers to the first local prediction variance. N refers to the number of user terminals, and p refers to the number of user terminal groups. Refers to the predicted expectations of the user side, Refers to the prediction variance on the user side.
[0091] It is understandable that the second local model parameters of the second server may be developed in the same manner as the first server calculates the first local model parameters, which is not elaborated in detail in the embodiment of the present application.
[0092] The embodiment of the present application uses the average value method to quickly and accurately obtain the first local prediction expectation and the first local prediction variance.
[0093] In an exemplary embodiment, the aggregating the first local model parameters and the second local model parameters to obtain global model parameters includes: obtaining a first coefficient of the first server, and obtaining a second coefficient of the second server; obtaining a first global model parameter of the first server based on the first coefficient and the first local model parameters; obtaining a second global model parameter of the second server based on the second coefficient and the two local model parameters; and obtaining global model parameters based on the first global model parameters and the second global model parameters.
[0094] It can be understood that a server coefficient is set for each server in the cloud server network. The server coefficients can be the same or different. For the first server, its server coefficient is the first coefficient, and for the second server, its server coefficient is the second coefficient.
[0095] Specifically, in addition to its own first local model parameters and first coefficients, the first server will also obtain the second local model parameters and second coefficients of the second server other than itself in the cloud server network. For the first server, its first global model parameters can be determined based on its own first local model parameters and first coefficients, and for the second server, its second global model parameters can be determined based on its own second local model parameters and second coefficients. Finally, the global model parameters are obtained by aggregating the first global model parameters and the second global model parameters.
[0096] In an embodiment of the present application, for the first server, its corresponding first global model parameter is calculated, and for the second server, its corresponding second global model parameter is calculated, so that the global model parameter can be obtained efficiently and accurately by aggregating the first global model parameter and the second global model parameter.
[0097] In an exemplary embodiment, the global model parameters include a global prediction expectation and a global prediction variance; the first global model parameters include a first global prediction expectation and a first global prediction variance; the second global model parameters include a second global prediction expectation and a second global prediction variance; obtaining the global model parameters based on the first global model parameters and the second global model parameters includes: determining the global prediction expectation as the product of the sum of the first global prediction expectation and the second global prediction expectation and the sum of the average prediction variances; determining the global prediction variance as the sum of the first global prediction variance and the two global prediction variances.
[0098] Specifically, after each cloud server obtains the prediction values of other cloud servers, it can use the RBCM (representation based classification method, an aggregation algorithm) algorithm with a smaller prediction variance to calculate the global prediction expectation and prediction variance. The prior variance is defined as Right now In addition, the coefficient of the cloud server of each cluster is defined as β [i] .
[0099] When obtaining the global model parameters, the following global model parameter calculation formula can be used:
[0100]
[0101]
[0102] in, is the global prediction expectation, is the global prediction variance. is the global prediction variance sum, To calculate the expression of the sum of the first global prediction expectation and the second global prediction expectation, is an expression for calculating the sum of the first global prediction variance and the second global prediction variance.
[0103] The embodiment of the present application obtains the global model parameters quickly and accurately by aggregating the first global model parameters and the second global model parameters.
[0104] In an exemplary embodiment, the first local model parameters include a first local prediction expectation and a first local prediction variance; obtaining the first global model parameters of the first server based on the first coefficient and the first local model parameters includes: obtaining a predefined prior variance; determining a first product of the first local prediction variance, the first local prediction expectation and the first coefficient; determining a second product of the first coefficient and the first local prediction expectation, and determining a third product of the first coefficient and the prior variance; obtaining the first global model parameters of the first server according to the first product, the second product and the third product.
[0105] It can be understood that for the first server, a multiplication operation can be performed based on the first local prediction variance, the first local prediction expectation and the first coefficient to obtain a first product; further, the first coefficient and the first local prediction expectation are multiplied to obtain a second product, and the first coefficient and the prior variance are multiplied to obtain a third product; finally, the first global model parameter of the first server is obtained based on the first product, the second product and the third product. Combined with the above global model calculation formula, for the first server, represents the first local prediction variance, represents the first local prediction expectation, β [j] Represents the first coefficient. This is the first product. This is the second product, β [j] and The product obtained by multiplication is the third product.
[0106] The embodiment of the present application calculates the first global model parameter for the first server, and then determines the global model parameter based on the first global model parameter.
[0107] In an exemplary embodiment, the first global model parameters include a first global prediction expectation and a first global prediction variance; obtaining the first global model parameters of the first server according to the first product, the second product and the third product includes: determining the first global prediction expectation based on the first product; determining the first global prediction variance based on the second product and the third product.
[0108] Specifically, the first server may determine the first global prediction expectation according to the first product, and determine the first global prediction variance according to the second product and the third product. When determining the first global prediction variance based on the second product and the third product, the third product may be subtracted from the prior variance to obtain the difference between the two, and then the second product and the difference between the two are added to obtain the first global prediction variance.
[0109] In the embodiment of the present application, the first global model parameter can be quickly determined based on the calculated products.
[0110] In an exemplary embodiment, the second local model parameters include a second local prediction expectation and a second local prediction variance; obtaining the second global model parameters of the second server based on the second coefficient and the two local model parameters includes: obtaining a predefined prior variance; determining the fourth product of the second local prediction variance, the second local prediction expectation and the second coefficient; determining the fifth product of the second coefficient and the second local prediction expectation, and determining the sixth product of the second coefficient and the prior variance; obtaining the second global model parameters of the second server according to the fourth product, the fifth product and the sixth product.
[0111] It can be understood that for the second server, a multiplication operation can be performed based on the second local prediction variance, the second local prediction expectation, and the second coefficient to obtain a fourth product; further, the second coefficient and the second local prediction expectation are multiplied to obtain a fifth product, and the second coefficient and the prior variance are further multiplied to obtain a sixth product; finally, the second global model parameters of the second server are obtained according to the fourth product, the fifth product, and the sixth product. The second server calculates the second global model parameters in the same way as the first server calculates the first global model parameters.
[0112] The embodiment of the present application calculates the second global model parameters for the second server, and then determines the global model parameters based on the second global model parameters.
[0113] In an exemplary embodiment, the second global model parameters include a second global prediction expectation and a second global prediction variance; obtaining the second global model parameters of the second server according to the fourth product, the fifth product and the sixth product includes: determining the second global prediction expectation based on the fourth product; and determining the second global prediction variance based on the fifth product and the sixth product.
[0114] Specifically, the second server may determine the second global prediction expectation according to the fourth product, and determine the second global prediction variance according to the fifth product and the sixth product. When determining the second global prediction variance based on the fourth product and the fifth product, the sixth product may be subtracted from the a priori variance to obtain the difference between the two, and then the fifth product and the difference between the two are added to obtain the second global prediction variance.
[0115] In the embodiment of the present application, the second global model parameters can be quickly determined based on the calculated products.
[0116] In an exemplary embodiment, the global model parameters are fed back to each of the user terminals to instruct each of the user terminals to carry out model construction according to the global model parameters.
[0117] Specifically, for the global model parameters calculated by the first server, the first server may feed them back to each user terminal in the cluster where it is located, and each user terminal may start model construction based on the obtained global model parameters.
[0118] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, disk, CD), and includes a number of instructions for a terminal device (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods described in each embodiment of the present application.
[0119] In order to better understand the process of the above-mentioned method for determining global model parameters, the implementation process of the above-mentioned method for determining global model parameters is described below in combination with an optional embodiment, but it is not used to limit the technical solution of the embodiment of the present application.
[0120] In the digital age, data is an important asset. The source of data usually includes edge digital devices, such as mobile phones, sensors connected to industrial equipment, etc. However, the collection, sharing and release of this data often lead to many privacy issues. As companies and institutions collect more and more user data, they need to ensure that the collected user data is not violated, and they need to strengthen data protection. It should be noted that the data collected by sensors in daily collection, smart watches, and the Internet of Things will be sent to the central processing unit (cloud server) for data processing, model learning, packaging, aggregation, etc. For example, the sensors of a modern car can collect hundreds of Gb of measurement data in a few hours, and the data collected by thousands of cars in a city over a longer period of time will undoubtedly bring a huge burden to the cloud server during transmission and storage. In order to improve the efficiency of data processing, computing and storage, the emerging edge computing provides a powerful and promising learning framework. The current federated machine learning allows each edge device to perform local training and send the obtained local model to the cloud server for model aggregation. Generally, distributed machine learning uses deep neural networks as machine learning models, which have achieved unprecedented success in many applications, such as model classification and pattern recognition. However, the structure of federated learning will be limited in large-scale networks: the scalability of end users is poor and when a large number of users are connected to the cloud server, the cloud server itself will become a performance bottleneck.
[0121] On the one hand, for each round of communication, the existing distributed machine learning algorithms mainly use stochastic gradient descent (SGD). Using such an algorithm, in each iteration, the cloud server will receive the gradient calculated by the user for a data, and then apply this gradient to perform a gradient descent step. The convergence rate of SGD is much slower than the exponential convergence rate of standard gradient descent. Therefore, SGD requires a large number of iterations, which is extremely time-consuming. On the other hand, when the number of user terminals connected to the cloud server increases dramatically, the gradient obtained by each user terminal through SGD in each round is sent to the cloud server, and the storage capacity and computing power of the cloud server will be put to the test.
[0122] To solve the above thorny problems, the Peer-to-Peer (P2P) cloud server network distributed machine learning framework came into being. This learning framework forms multiple clusters by splitting the user end, and each cluster is configured with a cloud server to form a local centralized learning module, and then obtains local parameters through the local learning module. The cloud servers of multiple clusters aggregate local parameters to obtain global parameters, so that model construction can be carried out based on global parameters.
[0123] However, this learning framework still faces many problems. After each cluster's cloud server obtains the local parameters of the cluster, it often needs to go through multiple rounds of time iterations when expanding the global parameter aggregation. This not only makes the communication overhead of the entire network very large, but also causes the global parameter determination time to be too long, thus affecting the efficiency of model building. And such a centralized-distributed hybrid online learning platform still faces security issues. First, when the user end of each cluster uses the local training set for training localization, the samples of the local training set may be poisoned and contaminated by the attacker, so that the model obtained by the user using the contaminated data for training is not accurate enough.
[0124] This application proposes a method for determining global model parameters, and applies this method to a distributed cloud shared learning framework. Specifically, in the process of determining global model parameters, each user terminal uses the Lipschitz constant limit method to screen the online measurement data, obtain the screened measurement data, and send the screened measurement data to the server (i.e., the first server and the second server in the cloud server network). Of course, each user terminal can also directly send the online measurement data to the server, and the server will screen the online measurement data to obtain the screened measurement data. By screening the measurement data, it can be ensured that the screened training data is insensitive to poisoning attacks and the robustness of local predictions is ensured; in the cloud-cloud information sharing network, sparse cloud servers form a complete graph, and communicate using the connections between cloud servers. Due to the complete graph, each cloud server can be regarded as a central control unit, so each cloud server uses the RBCM aggregation method to calculate the global prediction expectation and variance, making the centralized calculation results more efficient.
[0125] refer to Figure 3 As shown, it is a distributed cloud shared learning framework in the cloud server network in the embodiment of the present application: the distributed shared learning framework includes cloud server 1, cloud server 2, cloud server 3 and cloud server 4, each cloud server is fully connected, and each cloud server is connected to multiple user terminals. Specifically, there are N users in the cloud server network, and the N users are evenly divided into groups. Here, the project does not need to evenly distribute users. For the convenience of description, this embodiment assumes that the users in the cloud server network are evenly distributed, so each group contains users. We have Each user is equipped with a cloud server, so a cloud server and The users form a federated learning subsystem; different cloud servers form a sparse cloud server network, and each cloud server transmits information with other different clouds. The cloud server network is defined as G = (V, E(k), A(k)), where V is a set of p cloud servers, and E(k) is a set of directed edges that connect each cloud server. is a weight matrix whose elements are For any i∈V, define the user set U [i] Make In this embodiment, 4 cloud servers and 400 users are used, that is, p=4, N=400.
[0126] In order to describe the method for determining the global model parameters of the present application in detail, the present embodiment can be expanded from the following two modules: (1) a federated learning module based on sample screening of the training data set; (2) a cloud-based global prediction module based on RBCM. Note that the data in this application is collected online and needs to be distinguished from the offline learning algorithm.
[0127] (1) Federated learning module based on training dataset sample screening
[0128] The goal of the federated learning module based on training data set sample screening is to design an algorithm so that in the cluster, multiple local user terminals can use the Lipschitz constant to screen the required online measurement data, and give the local prediction expectation and prediction variance through full-GPR, so as to achieve the purpose of learning the objective function. Specifically, the objective function is defined as in Yes x dimensional input space. Without loss of generality, assume that the output is one-dimensional, that is, At time t, given The corresponding output is y = f(x) + ε, where ε has a mean of 0 and a variance of Gaussian noise with a Gaussian probability distribution, that is For each user Define the training set as follows in is the input data set of user i, y [i] =[y [i] (1), y [i] (2), ..., y [i] (n s )] T is the output column vector of user i. The Gaussian process regression goal is to use the training set For test input Obtain the test output f(x * ).
[0129] For each user terminal, define the following training set, where: is the input data set of user i, y [i] =[y [i] (1), y [i] (2), ..., y [i] (n s )] T is the output column vector of user i. The Gaussian process regression goal is to use the training set Obtain the test output f(x) for the test input * ).
[0130] Define a symmetric positive semidefinite kernel function That is, ∫k(x, x′)f(x)f(x′)dv(x)dv(x′)≥0; where, ν is the measure. Let Return a column vector whose i-th element is equal to f(x(i)). Assume that function f is a sample from a Gaussian process prior probability distribution, the mean function of this prior distribution is μ, and the kernel function is k. Then the training output and test output Obey the joint probability distribution:
[0131]
[0132] in, and Returns the value of μ(x(i)) and μ(x * (i)) composed of a vector, Return a matrix such that the element in row i and column j is k(x(i), x * (j)).
[0133] According to the properties of Gaussian process, the Gaussian process regression of user i uses the training set Predict the test dataset x * The output of this output f(x * ) still obeys the normal distribution, that is, Here the posterior expectation and posterior variance of user i are:
[0134]
[0135]
[0136] Specifically, refer to Figure 4As shown, the federated learning module based on training data set sample screening can include three submodules: 1) a training sample screening module based on the Lipschitz constant; 2) a user local prediction module based on full-GPR; 3) a cluster prediction aggregation module based on the average method.
[0137] Among them, 1) Training sample screening module based on Lipschitz constant:
[0138] The Lipschitz continuity of the objective function f means that for any two inputs x1 and x2, there exists a constant l f Make
[0139] |f(x1)-f(x2)|≤l f ||x1-x2||For any x1 and
[0140] Therefore, for each user terminal i, this scheme sets the threshold of the Lipschitz constant, and then sets the Lipschitz constant threshold by traversing all the input and output data in the user's training set Then consider the training set For any two inputs and and output and calculate Then determine whether it is less than the threshold Right now
[0141] Finally, put all the input and output data that meet this condition into the collection When there is input and output data that does not satisfy this Lipschitz inequality, we automatically determine that the data has been contaminated by a poisoning attack.
[0142] For 2) User local prediction module based on full-GPR
[0143] Using the new filtered training set For the test input x * , this scheme gives the posterior expectation and posterior variance as
[0144]
[0145]
[0146] Then, this posterior expectation and variance Sent to the cloud servers in the cluster.
[0147] For 3) Cluster prediction aggregation module based on average method
[0148] When the cloud server in the cluster collects the posterior expectation obtained by local training of all users in the cluster and the posterior variance Afterwards, the cloud server uses the average method to calculate the global prediction expectation and global prediction variance in the cluster Here the superscript (k) represents the kth cluster. The calculation formula is as follows:
[0149]
[0150]
[0151] (2) Cloud-sharing global prediction module based on RBCM
[0152] The information exchange in the sparse cloud server network is designed as a complete graph model. If at time k (j, i) ∈ E(k), then (i, j) ∈ E(k). First, we can assume that there is a constant α>0 such that and When k ≥ 0, Secondly, construct weights So that for all i∈V, and for all j∈V,
[0153] Since each cloud server i∈V in the complete graph communicates with other cloud servers, this cloud server can be regarded as a central control unit relative to other cloud servers, that is, a centralized network structure. Then, in such a sparse cloud server centralized network, after each cloud server obtains the predicted value of other cloud servers, the RBCM algorithm with a smaller prediction variance can be used to calculate the global prediction expectation and prediction variance. The prior variance is defined as That is, k(x * (i), x * (i)) = 0. In addition, the coefficient of the cloud server of each cluster is defined as β [i] , the specific calculation formula is as follows:
[0154]
[0155]
[0156] The beneficial effects of the present application can be reflected in the following aspects: First, the Lipschitz constant is used to limit the training data in the training set, that is, to screen the online measurement data, to ensure that the prediction expectation and prediction variance of the user end in each cluster are robust to the poisoning attack of the training set, and to avoid the deviation of the prediction results of subsequent calculations due to the poisoning attack on the training data; second, for the server complete graph network, a centralized RBCM algorithm is used to calculate the global prediction expectation and prediction variance, which avoids distributed iterative calculation, greatly improves the calculation efficiency, and reduces the communication overhead.
[0157] In this embodiment, a device for determining global model parameters is also provided, which is used to implement the above-mentioned embodiments and preferred implementation modes, and the descriptions that have been made will not be repeated. As used below, the term "module" can implement a combination of software and / or hardware of a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.
[0158] Figure 5 is a structural block diagram of a device for determining global model parameters according to an embodiment of the present application, such as Figure 5 As shown, the device comprises:
[0159] A first acquisition module 52, used to acquire screening measurement data corresponding to each client of the cluster where the first server in the cloud server network is located;
[0160] A data processing module 54, configured to obtain a first local model parameter of the first server based on each of the screened measurement data;
[0161] A second acquisition module 56, configured to acquire a second local model parameter of a second server other than the first server in the cloud server network;
[0162] The data processing module 54 is further configured to aggregate the first local model parameters and the second local model parameters to obtain global model parameters.
[0163] Through the above-mentioned device, the online measurement data sent by each user terminal of the cluster where the first service is located is obtained, and the first local model parameters of the first server are obtained according to each online measurement data, and the second local model parameters of the second server other than the first server in the cloud server network are further obtained. Therefore, the global model parameters can be obtained directly by aggregating the first local model parameters and the second local model parameters without iteration, which effectively reduces the time for determining the global model parameters and improves the efficiency of subsequent model construction.
[0164] In an exemplary embodiment, the device also includes a screening module; the screening module is used to receive online measurement data corresponding to each user terminal in the cluster where the first server in the cloud server network is located; and screen each of the online measurement data to determine the screening measurement data corresponding to each of the online measurement data.
[0165] In an exemplary embodiment, the online measurement data of each user terminal includes multiple input parameters; the screening module is also used to screen any online measurement data using the following steps: determine the output column vector of each input parameter included in the online measurement data; select any two input parameters from each input parameter to determine as target input parameters; obtain a verification constant based on the target input parameters and the output column vector corresponding to the target input parameters; determine the screening result of the target input parameter based on the verification constant, and re-select the target parameter from each input parameter, return to the step of obtaining the verification constant based on the target input parameter and the output column vector corresponding to the target input parameter, until the screening of each input parameter is completed and the screened measurement data corresponding to the targeted online measurement data is obtained.
[0166] In an exemplary embodiment, the screening module is further used to compare the verification constant with a preset constant threshold to obtain a comparison result; if the comparison result is that the verification constant is less than the constant threshold, the screening result is to retain the target input parameter.
[0167] In an exemplary embodiment, the target input parameters include a first target input parameter and a second target input parameter; the output column vector includes a first output column vector corresponding to the first target input parameter and a second output column vector corresponding to the second target input parameter; the screening module is also used to determine the input difference between the first target input parameter and the second target input parameter; determine the output difference between the first output column vector and the second output column vector; and obtain a verification constant based on the output difference and the input difference.
[0168] In an exemplary embodiment, the data processing module 54 is further used to perform local prediction based on each of the filtered measurement data to obtain model parameters of each user terminal; and to aggregate the model parameters of each user terminal to obtain the first local model parameters of the first server.
[0169] In an exemplary embodiment, the data processing module 54 is also used to determine the output column vector of each of the filtered measurement data based on each of the filtered measurement data; obtain the prediction expectation and prediction variance of each user terminal based on each of the filtered measurement data, each of the output column vectors and a pre-set prior variance; and obtain the model parameters of each user terminal based on the prediction expectation and prediction variance of each user terminal.
[0170] In an exemplary embodiment, the model parameters include the prediction expectation and prediction variance of each user terminal; the data processing module 54 is also used to obtain the number of user terminals in the cloud server network and the number of user terminal groups in the cloud server network; based on the prediction expectation of each user terminal, the sum of the prediction expectations is determined; and based on the prediction variance of each user terminal, the sum of the prediction variances is determined; based on the number of user terminals, the number of user terminal groups, the sum of the prediction expectations and the sum of the prediction variances, the first local model parameters of the first server are obtained.
[0171] In an exemplary embodiment, the data processing module 54 is also used to obtain an average prediction expectation based on the number of user terminals, the number of user terminal groups and the sum of the prediction expectations; to obtain an average prediction variance based on the number of user terminals, the number of user terminal groups and the sum of the prediction variances; and to obtain the first local model parameters of the first server based on the average prediction expectation and the average prediction variance.
[0172] In an exemplary embodiment, the second data acquisition module 56 is further used to acquire a complete graph model of the first server and the second server;
[0173] Based on the complete graph model, second local model parameters of a second server other than the first server in the cloud server network are obtained.
[0174] In an exemplary embodiment, the data processing module 54 is also used to obtain the first coefficient of the first server and the second coefficient of the second server; based on the first coefficient and the first local model parameter, obtain the first global model parameter of the first server; based on the second coefficient and the two local model parameters, obtain the second global model parameter of the second server; according to the first global model parameter and the second global model parameter, obtain the global model parameter.
[0175] In an exemplary embodiment, the first local model parameters include a first local prediction expectation and a first local prediction variance; the data processing module 54 is also used to obtain a predefined prior variance; determine a first product of the first local prediction variance, the first local prediction expectation and the first coefficient; determine a second product of the first coefficient and the first local prediction expectation, and determine a third product of the first coefficient and the prior variance; and obtain the first global model parameters of the first server based on the first product, the second product and the third product.
[0176] In an exemplary embodiment, the first global model parameters include a first global prediction expectation and a first global prediction variance; the data processing module 54 is further used to determine the first global prediction expectation based on the first product; and to determine the first global prediction variance based on the second product and the third product.
[0177] In an exemplary embodiment, the second local model parameters include a second local prediction expectation and a second local prediction variance; the data processing module 54 is also used to obtain a predefined prior variance; determine the fourth product of the second local prediction variance, the second local prediction expectation and the second coefficient; determine the fifth product of the second coefficient and the second local prediction expectation, and determine the sixth product of the second coefficient and the prior variance; and obtain the second global model parameters of the second server based on the fourth product, the fifth product and the sixth product.
[0178] In an exemplary embodiment, the second global model parameters include a second global prediction expectation and a second global prediction variance; the data processing module 54 is further used to determine the second global prediction expectation based on the fourth product; and to determine the second global prediction variance based on the fifth product and the sixth product.
[0179] In an exemplary embodiment, the global model parameters include a global prediction expectation and a global prediction variance; the first global model parameters include a first global prediction expectation and a first global prediction variance; the second global model parameters include a second global prediction expectation and a second global prediction variance; the data processing module 54 is also used to determine the global prediction expectation by multiplying the sum of the first global prediction expectation and the second global prediction expectation by the sum of the average prediction variances; and determine the sum of the first global prediction variance and the two global prediction variances as the global prediction variance.
[0180] In an exemplary embodiment, the apparatus further comprises an application module, which is used to feed back the global model parameters to each of the user terminals, so as to instruct each of the user terminals to carry out model construction according to the global model parameters.
[0181] An embodiment of the present application further provides a computer-readable storage medium, in which a computer program is stored, wherein the computer program is configured to execute the steps of any of the above method embodiments when running.
[0182] In an exemplary embodiment, the computer-readable storage medium may include, but is not limited to, various media that can store computer programs, such as a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk.
[0183] An embodiment of the present application further provides an electronic device, including a memory and a processor, wherein a computer program is stored in the memory, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.
[0184] In an exemplary embodiment, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor, and the input / output device is connected to the processor.
[0185] An embodiment of the present application further provides a computer program product, which includes a computer program. When the computer program is executed by a processor, the steps in any one of the above method embodiments are implemented.
[0186] An embodiment of the present application further provides another computer program product, including a non-volatile computer-readable storage medium, wherein the non-volatile computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps in any of the above method embodiments are implemented.
[0187] An embodiment of the present application also provides a computer program, which includes computer instructions stored in a computer-readable storage medium; a processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device performs the steps in any one of the above method embodiments.
[0188] For specific examples in this embodiment, reference may be made to the examples described in the above embodiments and exemplary implementation modes, and this embodiment will not be described in detail herein.
[0189] Obviously, those skilled in the art should understand that the above modules or steps of the present application can be implemented by a general computing device, they can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, and in some cases, the steps shown or described can be executed in a different order from that herein, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. Thus, the present application is not limited to any specific combination of hardware and software.
[0190] The above description is only the preferred embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the principles of the present application shall be included in the protection scope of the present application.
Claims
1. A method for determining global model parameters, characterized in that: include: Obtaining screening measurement data corresponding to each client of the cluster where the first server in the cloud server network is located; Based on each of the screened measurement data, obtaining a first local model parameter of the first server; Acquire a second local model parameter of a second server in the cloud server network other than the first server; The first local model parameter and the second local model parameter are aggregated to obtain a global model parameter.
2. The method according to claim 1, characterized in that The obtaining of screening measurement data corresponding to each user terminal in the cluster where the first server in the cloud server network is located includes: Receiving online measurement data corresponding to each client of the cluster where the first server in the cloud server network is located; Screening is performed on each of the online measurement data to determine the screened measurement data corresponding to each of the online measurement data.
3. The method according to claim 1, characterized in that The online measurement data of each user end includes multiple input parameters; The step of screening each of the online measurement data to determine the screening measurement data corresponding to each of the online measurement data includes: For any online measurement data, the following steps are used to filter: Determine an output column vector of each of the input parameters included in the online measurement data; Select any two input parameters from the input parameters to be determined as target input parameters; Obtaining a verification constant based on the target input parameter and an output column vector corresponding to the target input parameter; According to the verification constant, the screening result of the target input parameter is determined, and the target parameter is re-selected from each of the input parameters, and the output column vector based on the target input parameter and the target input parameter is returned to obtain the verification constant step until each of the input parameters has completed the screening and the screened measurement data corresponding to the targeted online measurement data is obtained.
4. The method according to claim 3, characterized in that Determining the screening result of the target input parameter according to the verification constant includes: Comparing the verification constant with a preset constant threshold to obtain a comparison result; If the comparison result is that the verification constant is less than the constant threshold, the screening result is to retain the target input parameter.
5. The method according to claim 3, characterized in that: The target input parameter includes a first target input parameter and a second target input parameter; the output column vector includes a first output column vector corresponding to the first target input parameter and a second output column vector corresponding to the second target input parameter; The obtaining of a verification constant based on the target input parameter and an output column vector corresponding to the target input parameter comprises: determining an input difference between the first target input parameter and the second target input parameter; determining an output difference between the first output column vector and the second output column vector; Based on the output difference and the input difference, a verification constant is obtained.
6. The method according to claim 1, characterized in that The obtaining of the first local model parameter of the first server based on each of the screening measurement data includes: Performing local prediction based on each of the screened measurement data to obtain model parameters for each user terminal; The model parameters of each client are aggregated to obtain the first local model parameters of the first server.
7. The method according to claim 6, characterized in that The local prediction is performed based on each of the screened measurement data to obtain the model parameters of each user terminal, including: Based on each of the screening measurement data, determining an output column vector of each of the screening measurement data; Based on each of the screened measurement data, each of the output column vectors and a preset prior variance, a prediction expectation and a prediction variance of each user terminal are obtained; Based on the prediction expectation and prediction variance of each user terminal, the model parameters of each user terminal are obtained.
8. The method according to claim 6, characterized in that The model parameters include the prediction expectation and prediction variance of each user terminal; The step of aggregating the model parameters of each user terminal to obtain the first local model parameters of the first server includes: Get the number of client terminals in the cloud server network and the number of client terminal groups in the cloud server network; Determining a sum of predicted expectations based on the predicted expectations of each user terminal; and determining a sum of predicted variances based on the predicted variances of each user terminal; A first local model parameter of the first server is obtained according to the number of user terminals, the number of user terminal groups, the predicted expected sum, and the predicted variance sum.
9. The method according to claim 8, characterized in that The obtaining, according to the number of the user terminals, the number of the user terminal groups, the predicted expected sum, and the predicted variance sum, of the first local model parameters of the first server includes: Obtaining an average predicted expectation based on the number of user terminals, the number of user terminal groups, and the sum of the predicted expectations; Obtaining an average prediction variance based on the number of user terminals, the number of user terminal groups, and the sum of the prediction variances; A first local model parameter of the first server is obtained according to the average prediction expectation and the average prediction variance.
10. The method according to claim 1, characterized in that The obtaining of a second local model parameter of a second server other than the first server in the cloud server network includes: Obtaining a complete graph model of the first server and the second server; Based on the complete graph model, second local model parameters of a second server other than the first server in the cloud server network are obtained.
11. The method according to claim 1, characterized in that: The aggregating the first local model parameter and the second local model parameter to obtain a global model parameter includes: Obtaining a first coefficient of the first server, and obtaining a second coefficient of the second server; Based on the first coefficient and the first local model parameter, obtaining a first global model parameter of the first server; Based on the second coefficient and the two local model parameters, obtaining a second global model parameter of the second server; A global model parameter is obtained according to the first global model parameter and the second global model parameter.
12. The method according to claim 11, characterized in that The first local model parameters include a first local prediction expectation and a first local prediction variance; and obtaining the first global model parameters of the first server based on the first coefficient and the first local model parameters includes: Get the predefined prior variance; determining a first product of the first local prediction variance, the first local prediction expectation, and the first coefficient; determining a second product of the first coefficient and the first local prediction expectation, and determining a third product of the first coefficient and the prior variance; A first global model parameter of the first server is obtained according to the first product, the second product and the third product.
13. The method according to claim 12, characterized in that The first global model parameter includes a first global prediction expectation and a first global prediction variance; and obtaining the first global model parameter of the first server according to the first product, the second product, and the third product includes: Determining a first global prediction expectation based on the first product; A first global prediction variance is determined based on the second product and the third product.
14. The method according to claim 11, characterized in that The second local model parameters include a second local prediction expectation and a second local prediction variance; and obtaining the second global model parameters of the second server based on the second coefficient and the two local model parameters includes: Get the predefined prior variance; determining a fourth product of the second local prediction variance, the second local prediction expectation, and the second coefficient; determining a fifth product of the second coefficient and the second local prediction expectation, and determining a sixth product of the second coefficient and the prior variance; A second global model parameter of the second server is obtained according to the fourth product, the fifth product and the sixth product.
15. The method according to claim 14, characterized in that The second global model parameters include a second global prediction expectation and a second global prediction variance; and obtaining the second global model parameters of the second server according to the fourth product, the fifth product and the sixth product includes: Based on the fourth product, a second global prediction expectation is determined; and based on the fifth product and the sixth product, a second global prediction variance is determined.
16. The method according to claim 11, characterized in that The global model parameters include a global prediction expectation and a global prediction variance; the first global model parameters include a first global prediction expectation and a first global prediction variance; the second global model parameters include a second global prediction expectation and a second global prediction variance; The step of obtaining a global model parameter according to the first global model parameter and the second global model parameter comprises: Determine the product of the sum of the first global prediction expectation and the second global prediction expectation and the sum of the average prediction variances as the global prediction expectation; The sum of the first global prediction variance and the second global prediction variance is determined as the global prediction variance.
17. The method according to claim 1, characterized in that The method further comprises: The global model parameters are fed back to each of the user terminals to instruct each of the user terminals to carry out model construction according to the global model parameters.
18. A device for determining global model parameters, characterized in that: include: A first acquisition module, used to acquire screening measurement data corresponding to each user terminal of the cluster where the first server in the cloud server network is located; A data processing module, configured to obtain a first local model parameter of the first server based on each of the screened measurement data; A second acquisition module, used to acquire a second local model parameter of a second server other than the first server in the cloud server network; The data processing module is further used to aggregate the first local model parameters and the second local model parameters to obtain global model parameters.
19. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, wherein the computer program implements the steps of the method described in any one of claims 1 to 17 when executed by a processor.
20. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the steps of the method described in any one of claims 1 to 17 are implemented.
Citation Information
Cited By
Distributed machine learning optimization method, device, equipment, system and medium
CN119886287A
