A method, system and device for updating model parameters based on federated learning
A model parameter and federation technology, applied in the field of model parameter update based on federated learning, can solve problems such as large data differences, difficult model convergence, and low model training efficiency.
Active Publication Date: 2021-05-11
ALIPAY (HANGZHOU) INFORMATION TECH CO LTD
View PDF4 Cites 0 Cited by
- Summary
- Abstract
- Description
- Claims
- Application Information
AI Technical Summary
Problems solved by technology
However, the data held by multiple data owners is quite different, which makes it difficult for the trained model to converge and the efficiency of model training is very low
Method used
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View moreImage
Smart Image Click on the blue labels to locate them in the text.
Smart ImageViewing Examples
Examples
Experimental program
Comparison scheme
Effect test
Embodiment Construction
[0016] In order to more clearly illustrate the technical solutions of the embodiments of the present specification, the following briefly introduces the drawings that need to be used in the description of the embodiments. Apparently, the accompanying drawings in the following description are only some examples or embodiments of this specification, and those skilled in the art can also apply this specification to other similar scenarios. Unless otherwise apparent from context or otherwise indicated, like reference numerals in the figures represent like structures or operations.
[0017] It should be understood that "system", "device", "unit" and / or "module" used in this specification is a method for distinguishing different components, elements, parts, parts or assemblies of different levels. However, the words may be replaced by other expressions if other words can achieve the same purpose.
[0018] As indicated in the specification and claims, the terms "a", "an", "an" and / ...
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More PUM
Login to View More Abstract
This specification relates to a method, system and device for updating model parameters based on federated learning, which can be used for data privacy protection. The method is implemented by any training member of the participating parties, including multiple rounds of updating model parameters, wherein one round of iterative updating includes: obtaining a model to be trained that has the same structure as other participating parties, and the model to be trained includes multiple network layers ;Train based on the held training samples and sample labels to obtain the gradient matrix; update the first-order moment matrix and the second-order moment matrix based on the gradient matrix, where the first-order moment matrix and the second-order moment matrix respectively include the network layer of the model to be trained One-to-one correspondence of multiple layer sub-matrices; interacting with the server, transmitting at least some elements in at least part of the layer sub-matrices to the server layer by layer, so that the server updates the model parameters layer by layer; obtaining updated model parameters from the server Use it as the model to be trained for the next training, or determine the final model based on this.
Description
technical field [0001] One or more embodiments of this specification relate to multi-party data joint processing, and in particular to a method, system and device for updating model parameters based on federated learning. Background technique [0002] In data analysis, data mining, economic forecasting and other fields, machine learning models can be used to analyze and discover potential data value. Since the data held by a single data owner may be incomplete, it is difficult to accurately describe the target. In order to obtain better model prediction results, the federated learning mode can be used for model training on different data held by different data owners. . However, the data held by multiple data owners are quite different, which makes it difficult for the trained model to converge and the efficiency of model training is very low. [0003] Therefore, it is necessary to propose a scheme to improve the training efficiency of the model in the federated learning p...
Claims
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More Application Information
Patent Timeline
Login to View More Patent Type & Authority Patents(China)
IPC IPC(8): G06N20/20G06F17/16
CPCG06F17/16G06N20/20
Inventor 郑龙飞陈超超王力张本宇
Owner ALIPAY (HANGZHOU) INFORMATION TECH CO LTD