Method for determining super parameters, method for training federal learning model and electronic equipment
A technology for learning models and electronic devices, applied in the field of artificial intelligence, which can solve the problems of long training time for federated learning models
Pending Publication Date: 2021-09-10
BEIJING WODONG TIANJUN INFORMATION TECH CO LTD +1
View PDF0 Cites 0 Cited by
- Summary
- Abstract
- Description
- Claims
- Application Information
AI Technical Summary
Problems solved by technology
[0003] In view of this, the embodiment of the present application provides a method for determining hyperparameters, a method for training a federated learning model, and electronic equipment to solve the problem in the related art that the training time of the federated learning model is too long due to inappropriate artificially set hyperparameters question
Method used
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View moreImage
Smart Image Click on the blue labels to locate them in the text.
Smart ImageViewing Examples
Examples
Experimental program
Comparison scheme
Effect test
Embodiment Construction
[0044] In related technologies, federated learning is divided into three categories according to the difference in data distribution among multiple training nodes: horizontal federated learning, vertical federated learning, and federated transfer learning. Among them, horizontal federated learning, also known as federated learning by sample, is applied to scenarios where the data sets of each training node have the same feature space and different sample spaces (different users). For example, the training nodes correspond to banks in different regions. Vertical federated learning has the same sample space and different feature spaces on the data set, and the essence of vertical federated learning is the combination of features. Federated transfer learning is applied to scenarios where both feature space and sample space overlap less.
[0045] figure 1 It shows a system architecture diagram of horizontal federated learning provided by related technologies, such as figure 1 As...
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More PUM
Login to View More Abstract
The invention discloses a method for determining super parameters, a method for training a federated learning model and electronic equipment, and the method for determining the super parameters comprises the steps: determining first time consumption and second time consumption through training the federated learning model; and determining a first super parameter and / or a second super parameter based on the first time consumption, the second time consumption, set gradient precision corresponding to the set loss function and the first parameter, wherein the first time consumption represents the duration consumed by the training node for calculating the gradient of one sample, the second consumed time represents the duration from the first moment to the second moment, the first moment represents a moment when the training node uploads a second parameter to an aggregation node, the second moment represents the moment when the training node successfully downloads the corresponding aggregated third parameter from the aggregation node, the first parameter represents a parameter used for calculating the convergence upper bound of the federated learning model, the first hyper-parameter represents the number of samples used for one-time iterative training, and the second hyper-parameter represents the interval of the number of iterations for uploading the second parameter.
Description
technical field [0001] The present application relates to the technical field of artificial intelligence, and in particular to a method for determining hyperparameters, a method for training a federated learning model, and electronic equipment. Background technique [0002] Federated learning aims to establish a federated learning model based on distributed data sets without sharing data samples. In related technologies, in the case of horizontal federated learning based on the federated averaging (FedAvg, Federated Averaging) algorithm, it is necessary to manually adjust the hyperparameters to adjust the training time, but the hyperparameters manually set are not suitable, resulting in too long training time for the federated learning model . Among them, the training nodes of horizontal federated learning have different data samples. Hyperparameters refer to parameters that are artificially set before model training. Contents of the invention [0003] In view of this, ...
Claims
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More Application Information
Patent Timeline
Login to View More Patent Type & Authority Applications(China)
IPC IPC(8): G06F16/2458G06F16/27G06N20/20
CPCG06F16/2465G06F16/27G06N20/20
Inventor 王哲杨恺王虎黄志翔
Owner BEIJING WODONG TIANJUN INFORMATION TECH CO LTD