Federated learning model training methods, systems, computer equipment, and storage media

Abstract

This invention belongs to the field of computer technology and discloses a method, system, computer device, and storage medium for training a federated learning model. The method includes: acquiring encrypted data submitted by each participant and combining it with a pre-set federated learning pre-trained model to obtain data quality evaluation results for each participant; correcting the loss function of the federated learning pre-trained model based on the data quality evaluation results of each participant to obtain a corrected loss function; and updating the model parameters of the federated learning pre-trained model based on the corrected loss function. This allows for differentiated evaluation and utilization of the data quality itself, increasing the contribution of high-quality data to the model training results and reducing the impact of low-quality data on the model training results, thereby effectively improving the training effect of the federated learning model and ensuring the effectiveness of the trained federated learning model. Furthermore, it can guide each participant to submit encrypted data based on their real data, further improving the quality of training data and model training.

Description

Technical Field

[0001] This invention belongs to the field of computer technology and relates to a federated learning model training method, system, computer device, and storage medium. Background Technology

[0002] With the development of smart grids, there is a tendency to deepen the research and application of big data technologies in various fields, build data sharing platforms for society, construct power data sharing models, provide diversified data services for enterprises, governments and other demanders, enhance the value of power data assets, and accelerate the construction of a smart and interactive energy ecosystem. Meanwhile, machine learning and deep learning technologies require large amounts of data for model training, making the need for data sharing—both horizontal collaboration between internal units and vertical integration between departments—increasingly urgent.

[0003] Federated learning is a distributed machine learning technique that trains models across multiple data sources with local data. It constructs a global model based on virtual fused data by exchanging model parameters or intermediate results, without needing to exchange local individual or sample data. This achieves a balance between data privacy and shared computation. However, due to data sensitivity and privacy requirements, sharing data between organizations is challenging, easily leading to "data silos." This hinders model training, makes it difficult to fully extract the intrinsic value of the data, and results in wasted data. While addressing this issue, federated learning also exposes data quality problems during multi-party data sharing, directly impacting the accuracy of the final model training results. Therefore, ensuring data and user privacy security during multi-party data sharing, and on this basis, enabling collaborative data training to improve the effectiveness and accuracy of data models and achieve secure data sharing, is a pressing problem to be solved.

[0004] Scholars have proposed a VCG-based incentive mechanism for federated learning to incentivize data owners to contribute all their data and truthfully report their costs. Furthermore, the effectiveness and economic rationality of this VCG-based incentive mechanism have been demonstrated. However, current federated learning model training methods utilizing the VCG mechanism fail to consider the inherent quality differences between different data types during training. They do not evaluate the quality of different data types and therefore cannot account for the impact of data quality and its causes during training, thus limiting the effectiveness of the federated learning model training results. Summary of the Invention

[0005] The purpose of this invention is to overcome the shortcomings of the prior art and provide a federated learning model training method, system, computer device and storage medium.

[0006] To achieve the above objectives, the present invention employs the following technical solution:

[0007] In a first aspect, the present invention provides a method for training a federated learning model, comprising:

[0008] The encrypted data submitted by each participant is obtained and combined with the pre-trained federated learning model to obtain the data quality evaluation results of each participant.

[0009] The loss function of the federated learning pre-trained model is adjusted based on the data quality evaluation results of each participant, resulting in the adjusted loss function. The model parameters of the federated learning pre-trained model are then updated based on the adjusted loss function.

[0010] Optionally, the data quality evaluation results obtained by combining the preset federated learning pre-trained model include:

[0011] The absolute error rate of each participant can be obtained using the following formula:

[0012]

[0013] Among them, e ki It is the absolute error rate of the i-th group of encrypted data submitted by participant k. It is the tag value of the i-th group of encrypted data submitted by participant k. It is the calculated value of the i-th set of encrypted data submitted by participant k using a federated learning pre-trained model;

[0014] The data quality evaluation results for each participant are obtained using the following formula:

[0015]

[0016] Where, q k This is the data quality evaluation result of participant k. It concerns the absolute value error rate e ki The function.

[0017] Optionally, the federated learning model is an electricity consumption credit federated learning model;

[0018] The encrypted data types include credit data, basic power data, and power analysis data;

[0019] When the encrypted data submitted by the participants is credit data, the data quality evaluation result of the participants is obtained by the following formula:

[0020]

[0021] Where, N k It is the total number of groups of encrypted data submitted by participant k;

[0022] When the encrypted data submitted by the participants is basic power data, the data quality evaluation result of the participants is obtained by the following formula:

[0023]

[0024] When the encrypted data submitted by the participants is power analysis data, the data quality evaluation result of the participants is obtained by the following formula:

[0025]

[0026] Optionally, the loss function for adjusting the federated learning pre-trained model based on the data quality evaluation results of each participant includes:

[0027] The loss function L of the federated learning pre-trained model is modified by the following formula:

[0028] L′=q' k L

[0029] Where L′ is the modified loss function, q' k =relu(q k -0.6), q k It is the data quality evaluation result of participant k, and relu() is the relu function used to remove encrypted data with a data quality evaluation result lower than 0.6.

[0030] Optionally, the loss function L of the federated learning pre-trained model is specifically:

[0031]

[0032] Where f(·) represents the activation function, b represents the bias term of the federated learning pre-trained model, and y label The label value represents the encrypted data, and w represents the weight parameter vector of the federated learning pre-trained model. i Let x represent the i-th weight parameter in the weight parameter vector of the federated learning pre-trained model, and let x represent the encrypted data vector. i Let v represent the i-th encrypted data and v represent the total number of weight parameters.

[0033] Optionally, updating the model parameters of the federated learning pre-trained model according to the modified loss function includes:

[0034] Based on the corrected loss function, the model parameters of the federated learning pre-trained model are updated using the gradient descent algorithm; wherein, the model parameters include a weight parameter vector and a bias term; when updating the weight parameter vector using the gradient descent algorithm, the gradient calculation formula for the weight parameter vector is:

[0035]

[0036] Among them, g k w represents the k-th gradient of the weight parameter vector. k h represents the k-th weight parameter in the weight parameter vector. w,b (x) represents the output corresponding to input x, weight w, and bias b, where δ represents the sign of the partial derivative. k This represents the k-th encrypted data;

[0037] When updating the bias term using the gradient descent algorithm, the gradient calculation formula for the bias term is:

[0038]

[0039] Among them, g' k This represents the k-th gradient of the bias term.

[0040] Optionally, it also includes: adjusting the updated model parameters of the federated learning pre-trained model based on the data quality evaluation results of each participant, obtaining the model parameter adjustment results of each participant, and distributing them to each participant.

[0041] Optionally, the step of correcting the updated model parameters of the federated learning pre-trained model based on the data quality evaluation results of each participant, obtaining the model parameter correction results for each participant, and distributing them to each participant includes:

[0042] Based on the data quality evaluation results of each participant, the updated model parameters of the federated learning pre-trained model are corrected using the following formula to obtain the corrected model parameters for each participant:

[0043]

[0044] Among them, S k This represents the model parameter correction result for participant k; q k This represents the data quality evaluation result of participant k; This represents the updated model parameters after the federated learning pre-trained model;

[0045] The model parameter correction results for each participant will be distributed to each participant respectively.

[0046] A second aspect of the present invention provides a federated learning model training system, comprising:

[0047] The quality evaluation module is used to obtain the encrypted data submitted by each participant and, in combination with the pre-trained federated learning model, to obtain the data quality evaluation results of each participant.

[0048] The parameter update module is used to correct the loss function of the federated learning pre-trained model based on the data quality evaluation results of each participant, obtain the corrected loss function, and update the model parameters of the federated learning pre-trained model according to the corrected loss function.

[0049] Optionally, the federated learning model is an electricity consumption credit federated learning model;

[0050] The encrypted data types include credit data, basic power data, and power analysis data;

[0051] When the encrypted data submitted by the participants is credit data, the data quality evaluation result of the participants is obtained by the following formula:

[0052]

[0053] Where, N k It is the total number of groups of encrypted data submitted by participant k;

[0054] When the encrypted data submitted by the participants is basic power data, the data quality evaluation result of the participants is obtained by the following formula:

[0055]

[0056] When the encrypted data submitted by the participants is power analysis data, the data quality evaluation result of the participants is obtained by the following formula:

[0057]

[0058] Optionally, it also includes a parameter distribution module; the parameter distribution module is used to correct the updated model parameters of the federated learning pre-trained model based on the data quality evaluation results of each participant, obtain the model parameter correction results of each participant, and distribute them to each participant.

[0059] In a third aspect, the present invention provides a computer device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the above-described federated learning model training method.

[0060] In a fourth aspect, the present invention provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the above-described federated learning model training method.

[0061] Compared with the prior art, the present invention has the following beneficial effects:

[0062] This invention discloses a federated learning model training method. Based on encrypted data submitted by each participant and combined with a pre-set federated learning pre-trained model, the method evaluates the data quality of each participant, obtaining data quality evaluation results. Then, based on these evaluation results, the loss function of the federated learning pre-trained model is adjusted, and the model parameters are updated according to the adjusted loss function. This achieves differentiated evaluation and utilization of data quality, increasing the contribution of high-quality data to the model training results and reducing the impact of low-quality data, thereby effectively improving the training effect of the federated learning model and ensuring its effectiveness after training. Furthermore, it can guide participants to submit encrypted data based on their actual data, further improving the quality of training data and model training. Attached Figure Description

[0063] Figure 1 This is a flowchart of the federated learning model training method according to an embodiment of the present invention.

[0064] Figure 2 This is a block diagram of the federated learning model training system according to an embodiment of the present invention. Detailed Implementation

[0065] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0066] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0067] First, let's introduce the relevant terminology used in the embodiments of this invention:

[0068] The VCG mechanism consists of the Vickrey auction, the Clarke tax, and the Groves mechanism. The Vickrey auction, also known as a second-price sealed-bid auction, guides market participants to bid according to their true prices. The Clarke tax is a mechanism that taxes key entities affecting the alliance's profits; the tax amount equals the net loss incurred by other entities when that key entity participates in the alliance. This tax system addresses the sustainability and free-rider issues in cooperative game theory within alliances, while also ensuring that market participants bid according to their true prices. The Groves mechanism aims to maximize the overall profits of the alliance.

[0069] The present invention will now be described in further detail with reference to the accompanying drawings:

[0070] See Figure 1 In one embodiment of the present invention, a federated learning model training method is provided to achieve differentiated evaluation and utilization of the quality of the data itself, thereby increasing the contribution of high-quality data to the training results and reducing the impact of low-quality data on the training results. Specifically, the federated learning model training method includes the following steps:

[0071] S1: Obtain the encrypted data submitted by each participant and combine it with the pre-trained federated learning model to obtain the data quality evaluation results of each participant.

[0072] S2: Adjust the loss function of the federated learning pre-trained model based on the data quality evaluation results of each participant, obtain the adjusted loss function, and update the model parameters of the federated learning pre-trained model based on the adjusted loss function.

[0073] Optionally, the preset federated learning pre-trained model is obtained by pre-training under the VCG mechanism using conventional federated learning model training methods.

[0074] In the training or inference process of federated learning models, there are usually many participants. Due to the special nature of federated learning, the federated learning agent cannot access the collection process of the participants' private data; it can only obtain the encrypted data from each participant. However, due to issues with the quality of the data itself or data tampering by participants for privacy protection, the quality of the data submitted by each participant is often inconsistent. If the federated learning agent does not perform data quality evaluation and corresponding processing, it can easily lead to global model contamination, severely affecting the accuracy of model training.

[0075] This invention discloses a federated learning model training method. Based on encrypted data submitted by each participant and combined with a pre-set federated learning pre-trained model, the method evaluates the data quality of each participant, obtaining data quality evaluation results. Then, based on these evaluation results, the loss function of the federated learning pre-trained model is adjusted, and the model parameters are updated according to the adjusted loss function. This achieves differentiated evaluation and utilization of data quality, increasing the contribution of high-quality data to the model training results and reducing the impact of low-quality data, thereby effectively improving the training effect of the federated learning model and ensuring its effectiveness after training. Furthermore, it can guide participants to submit encrypted data based on their actual data, further improving the quality of training data and model training.

[0076] In one possible implementation, in order to incentivize data management departments to submit authentic and valid data and to identify and eliminate distorted data, the absolute value error rate is used to evaluate the data quality of different participants.

[0077] Specifically, the data quality evaluation results for each participant, obtained by combining the pre-trained federated learning model, include: obtaining the absolute error rate of each participant using the following formula:

[0078]

[0079] Among them, e ki It is the absolute error rate of the i-th group of encrypted data submitted by participant k. It is the tag value of the i-th group of encrypted data submitted by participant k. It is the calculated value of the i-th set of encrypted data submitted by participant k using a federated learning pre-trained model.

[0080] The data quality evaluation results for each participant are obtained using the following formula:

[0081]

[0082] Where, q k This is the data quality evaluation result of participant k. It concerns the absolute value error rate e ki The function.

[0083] In one possible implementation, the federated learning model for electricity consumption credit involves data such as enterprise registration information, tax information, bidding information, electricity consumption capacity, electricity consumption potential, payment ability, default status, and historical credit scores. The enterprise registration information, tax information, and bidding information are owned by the Administration for Industry and Commerce, the Tax Bureau, and the bidding agency, respectively, and their data quality is generally reliable; however, data distortion can occur due to system defects or data management loopholes. The information on enterprise electricity consumption capacity, electricity consumption potential, payment ability, default status, and historical credit scores is owned by the power company or can be calculated based on the power company's electricity consumption, electricity charges, and payment records. Due to differences in data management and analysis capabilities among the power company's departments or subsidiaries, the quality of this data often varies significantly.

[0084] Based on the aforementioned analysis, the federated learning model for electricity consumption credit involves data from the Administration for Industry and Commerce, the Tax Bureau, bidding agencies, and multiple departments / subsidiaries of power companies. Therefore, the encrypted data submitted by each participant may include credit data, basic electricity data, and electricity analysis data. Credit data includes enterprise registration information, tax information, and bidding information; basic electricity data includes electricity consumption, electricity bills, payment status, and default records; and electricity analysis data includes electricity consumption potential, payment ability, and historical credit scores. Therefore, different data quality evaluation methods should be designed for different types of encrypted data.

[0085] Specifically, when the encrypted data type is credit data, such as enterprise registration information, tax information, and bidding information, the data quality evaluation score can be directly calculated based on the absolute value error rate, since this data is generally reliable and has a small error rate. That is, when the encrypted data submitted by the participant is credit data, the participant's data quality evaluation result can be obtained using the following formula:

[0086]

[0087] Where, N k This represents the total number of encrypted data sets submitted by participant k. It should be noted that this total number of encrypted data sets submitted by participant k refers to the total number of sets of data of type credit data. When the same participant provides encrypted data of different types, separate data quality evaluations are required.

[0088] When the encrypted data submitted by the participants is basic electricity data—that is, data on electricity consumption, billing, payment status, and default information held by the power company—is generated from actual business operations and is generally considered reliable. However, due to factors such as business system applications, business management, or performance evaluations by various departments or subsidiaries, there is a risk of data quality distortion. Therefore, the data quality evaluation score is calculated directly based on the absolute value error rate and then processed using a sigmoid-like function to highlight the differences in data quality and to strengthen the penalty for subjective data tampering. Specifically, when the encrypted data submitted by the participants is basic electricity data, the data quality evaluation result of the participants is obtained using the following formula:

[0089]

[0090] When the encrypted data submitted by the participants is power analysis data—that is, data held by the power company regarding electricity consumption potential, payment ability, and historical credit scores—it needs to be analyzed and calculated based on actual business data, which can result in significant discrepancies. However, these discrepancies are often not due to subjective actions but rather to objective rules. Therefore, the quality evaluation differences of these data types can be downplayed. The data quality evaluation score is calculated directly based on the absolute value error rate, and then processed using the ln function to smooth out data differences and reduce the emphasis on evaluating these data quality discrepancies. Specifically, when the encrypted data submitted by the participants is power analysis data, the data quality evaluation result of the participants is obtained using the following formula:

[0091]

[0092] In the federated learning model training method of this invention, the loss function of the federated learning pre-trained model is modified according to the data quality evaluation results of each participant. This can be understood as weighting the error propagation based on the federated learning pre-trained model and using the weighted error to update the model parameters.

[0093] In one possible implementation, the loss function for adjusting the federated learning pre-trained model based on the data quality evaluation results of each participant includes:

[0094] The loss function of the federated learning pre-trained model is modified using the following formula:

[0095]

[0096] Where L′ is the modified loss function, q' k =relu(q k -0.6), q kIt is the data quality evaluation result of participant k, and relu() is the relu function used to remove encrypted data with a data quality evaluation result lower than 0.6.

[0097] Specifically, most current federated learning model training algorithms use neural network algorithms. The model parameters of the neural network algorithm are updated in each iteration by calculating the gradient of the loss function on the model parameters until the algorithm converges or reaches the maximum iteration value.

[0098] In current federated learning model training algorithms, the commonly used loss function L is:

[0099]

[0100] Where f(·) represents the activation function, b represents the bias term of the federated learning pre-trained model, and y label The label value represents the encrypted data, and w represents the weight parameter vector of the federated learning pre-trained model. i Let x represent the i-th weight parameter in the weight parameter vector of the federated learning pre-trained model, and let x represent the encrypted data vector. i Let v represent the i-th encrypted data and v represent the total number of weight parameters.

[0101] In this embodiment, the error propagation is weighted, and the weighted error is used to update the parameters of the federated learning model, that is, the loss function is corrected to: L′=q' k L.

[0102] In order to distinguish the role of data quality evaluation results in model parameter updates and to eliminate the impact of severely poor data on model training, the data quality evaluation results were processed using the ReLU function, and datasets with data quality evaluation results below 0.6 (judged as unqualified data quality) were removed.

[0103] After correcting the loss function of the federated learning pre-trained model, a mini-batch gradient descent algorithm is performed on the corrected loss function to find the value w that minimizes the loss function. A smaller loss function value indicates a better training value h. w,b (x) and label value y label The smaller the difference between them, the higher the accuracy of the federated learning model.

[0104] Specifically, updating the model parameters of the federated learning pre-trained model according to the modified loss function includes: updating the model parameters of the federated learning pre-trained model using the gradient descent algorithm according to the modified loss function; wherein, the model parameters include a weight parameter vector and a bias term.

[0105] When updating the weight parameter vector using the gradient descent algorithm, the gradient calculation formula for the weight parameter vector is:

[0106]

[0107] Among them, g k w represents the k-th gradient of the weight parameter vector. k h represents the k-th weight parameter in the weight parameter vector. w,b (x) represents the output corresponding to input x, weight w, and bias b, where δ represents the sign of the partial derivative. k This represents the k-th encrypted data;

[0108] When updating the bias term using the gradient descent algorithm, the gradient calculation formula for the bias term is:

[0109]

[0110] Among them, g' k This represents the k-th gradient of the bias term.

[0111] In one possible implementation, the federated learning model training method further includes: correcting the updated model parameters of the federated learning pre-trained model based on the data quality evaluation results of each participant, obtaining the model parameter correction results of each participant, and distributing them to each participant.

[0112] The federated learning model training method of this invention uses a federated learning model distribution strategy based on data quality evaluation results. Following the VCG mechanism, and combining the data quality evaluation results of each participant, a federated learning model distribution strategy based on data quality evaluation results is designed.

[0113] Specifically, the step of correcting the updated model parameters of the federated learning pre-trained model based on the data quality evaluation results of each participant, and obtaining the corrected model parameters for each participant and distributing them to each participant, includes: correcting the updated model parameters of the federated learning pre-trained model according to the data quality evaluation results of each participant using the following formula, thereby obtaining the corrected model parameters for each participant:

[0114]

[0115] Among them, S k This represents the model parameter correction result for participant k; q k This represents the data quality evaluation result of participant k; This represents the updated model parameters after the federated learning pre-trained model.

[0116] Then, the model parameter correction results for each participant are distributed to each participant respectively.

[0117] In order to strengthen the impact of data quality evaluation results on the federated learning model updates of the participants and to further highlight the impact of data quality of each participant, the data quality evaluation results were squared before being distributed to each participant in the distribution strategy.

[0118] The following are embodiments of the apparatus of the present invention, which can be used to execute embodiments of the method of the present invention. For details not disclosed in the apparatus embodiments, please refer to the embodiments of the method of the present invention.

[0119] See Figure 2 In another embodiment of the present invention, a federated learning model training system is provided, which can be used to implement the above-mentioned federated learning model training method. Specifically, the federated learning model training system includes a quality evaluation module and a parameter update module.

[0120] The quality evaluation module is used to obtain the encrypted data submitted by each participant and, in combination with the pre-set federated learning pre-trained model, obtain the data quality evaluation results of each participant; the parameter update module is used to correct the loss function of the federated learning pre-trained model according to the data quality evaluation results of each participant, obtain the corrected loss function, and update the model parameters of the federated learning pre-trained model according to the corrected loss function.

[0121] In one possible implementation, obtaining the data quality evaluation results for each participant by combining a pre-set federated learning pre-trained model includes: obtaining the absolute error rate of each participant using the following formula:

[0122]

[0123] Among them, e ki It is the absolute error rate of the i-th group of encrypted data submitted by participant k. It is the tag value of the i-th group of encrypted data submitted by participant k. It is the calculated value of the i-th set of encrypted data submitted by participant k using a federated learning pre-trained model.

[0124] The data quality evaluation results for each participant are obtained using the following formula:

[0125]

[0126] Where, q k This is the data quality evaluation result of participant k. It concerns the absolute value error rate e ki The function.

[0127] In one possible implementation, the federated learning model is an electricity consumption credit federated learning model; the encrypted data types include credit data, electricity basic data, and electricity analysis data.

[0128] When the encrypted data submitted by the participants is credit data, the data quality evaluation result of the participants is obtained by the following formula:

[0129]

[0130] Where, N k It is the total number of encrypted data sets submitted by participant k.

[0131] When the encrypted data submitted by the participants is basic power data, the data quality evaluation result of the participants is obtained by the following formula:

[0132]

[0133] When the encrypted data submitted by the participants is power analysis data, the data quality evaluation result of the participants is obtained by the following formula:

[0134]

[0135] In one possible implementation, the loss function for adjusting the federated learning pre-trained model based on the data quality evaluation results of each participant includes:

[0136] The loss function L of the federated learning pre-trained model is modified by the following formula:

[0137] L′=q' k L

[0138] Where L′ is the modified loss function, q' k =relu(q k -0.6), q k It is the data quality evaluation result of participant k, and relu() is the relu function used to remove encrypted data with a data quality evaluation result lower than 0.6.

[0139] In one possible implementation, the loss function L of the federated learning pre-trained model is specifically:

[0140]

[0141] Where f(·) represents the activation function, b represents the bias term of the federated learning pre-trained model, and y label The label value represents the encrypted data, and w represents the weight parameter vector of the federated learning pre-trained model. i Let x represent the i-th weight parameter in the weight parameter vector of the federated learning pre-trained model, and let x represent the encrypted data vector. i Let v represent the i-th encrypted data and v represent the total number of weight parameters.

[0142] In one possible implementation, updating the model parameters of the federated learning pre-trained model according to the modified loss function includes: updating the model parameters of the federated learning pre-trained model using a gradient descent algorithm according to the modified loss function; wherein, the model parameters include a weight parameter vector and a bias term; when updating the weight parameter vector using the gradient descent algorithm, the gradient calculation formula for the weight parameter vector is:

[0143]

[0144] Among them, g k w represents the k-th gradient of the weight parameter vector. k h represents the k-th weight parameter in the weight parameter vector. w,b (x) represents the output corresponding to input x, weight w, and bias b, where δ represents the sign of the partial derivative. k This represents the k-th encrypted data.

[0145] When updating the bias term using the gradient descent algorithm, the gradient calculation formula for the bias term is:

[0146]

[0147] Among them, g' k This represents the k-th gradient of the bias term.

[0148] In one possible implementation, the federated learning model training system further includes a parameter distribution module; the parameter distribution module is used to correct the updated model parameters of the federated learning pre-trained model based on the data quality evaluation results of each participant, obtain the model parameter correction results of each participant, and distribute them to each participant.

[0149] In one possible implementation, the parameter distribution module is specifically used to: based on the data quality evaluation results of each participant, correct the updated model parameters of the federated learning pre-trained model using the following formula, to obtain the corrected model parameters for each participant:

[0150]

[0151] Among them, S k This represents the model parameter correction result for participant k; q k This represents the data quality evaluation result of participant k; This represents the updated model parameters of the federated learning pre-trained model. The model parameter correction results for each participant are then distributed to each participant.

[0152] All relevant content of each step involved in the aforementioned embodiments of the federated learning model training method can be referenced to the functional description of the corresponding functional module of the federated learning model training system in the embodiments of the present invention, and will not be repeated here.

[0153] The module division in this embodiment of the invention is illustrative and represents only one logical functional division. In actual implementation, other division methods may be used. Furthermore, the functional modules in the various embodiments of the invention can be integrated into a single processor, exist as separate physical entities, or be integrated into a single module. The integrated modules described above can be implemented in hardware or as software functional modules.

[0154] In another embodiment of the present invention, a computer device is provided, comprising a processor and a memory. The memory stores a computer program, which includes program instructions. The processor executes the program instructions stored in the computer storage medium. The processor may be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. It is the computing and control core of the terminal, suitable for implementing one or more instructions, specifically suitable for loading and executing one or more instructions from the computer storage medium to achieve a corresponding method flow or corresponding function. The processor described in this embodiment of the present invention can be used for the operation of federated learning model training methods.

[0155] In another embodiment of the present invention, a storage medium is provided, specifically a computer-readable storage medium (Memory), which is a memory device in a computer device used to store programs and data. It is understood that the computer-readable storage medium here can include both the built-in storage medium in the computer device and extended storage media supported by the computer device. The computer-readable storage medium provides storage space that stores the terminal's operating system. Furthermore, the storage space also stores one or more instructions suitable for loading and execution by a processor. These instructions can be one or more computer programs (including program code). It should be noted that the computer-readable storage medium here can be high-speed RAM or non-volatile memory, such as at least one disk storage device. The processor can load and execute one or more instructions stored in the computer-readable storage medium to implement the corresponding steps of the federated learning model training method in the above embodiments.

[0156] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0157] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0158] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1The function specified in one or more boxes.

[0159] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0160] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the specific implementation of the present invention. Any modifications or equivalent substitutions that do not depart from the spirit and scope of the present invention should be covered within the scope of protection of the claims of the present invention.

Claims

1. A federated learning model training method, characterized in that, include: The encrypted data submitted by each participant is obtained and combined with the pre-trained federated learning model to obtain the data quality evaluation results of each participant. The loss function of the federated learning pre-trained model is adjusted based on the data quality evaluation results of each participant, and the model parameters of the federated learning pre-trained model are updated based on the adjusted loss function. The federated learning model is a federated learning model for electricity consumption credit. The encrypted data types include credit data, basic power data, and power analysis data; When the encrypted data submitted by the participants is credit data, the data quality evaluation result of the participants is obtained by the following formula: in, N k Participants k The total number of byte groups of encrypted data submitted; Participants k The submitted number i The absolute error rate of group encrypted data; When the encrypted data submitted by the participants is basic power data, the data quality evaluation result of the participants is obtained by the following formula: When the encrypted data submitted by the participants is power analysis data, the data quality evaluation result of the participants is obtained by the following formula: The loss function for adjusting the federated learning pre-trained model based on the data quality evaluation results of each participant includes: The loss function of the federated learning pre-trained model is modified using the following formula. L : in, It is a modified loss function. , Participants k Data quality evaluation results It is a ReLU function used to remove encrypted data with a data quality rating below 0.

6.

2. The federated learning model training method according to claim 1, characterized in that, The data quality evaluation results obtained by combining the pre-trained federated learning model include: The absolute error rate of each participant can be obtained using the following formula: in, Participants k The submitted number i Tag values ​​of group encrypted data, The participants are obtained using a federated learning pre-trained model. k The submitted number i The calculated value of the group encrypted data; The data quality evaluation results for each participant are obtained using the following formula: in, Participants k Data quality evaluation results It's about the absolute value error rate. The function.

3. The federated learning model training method according to claim 1, characterized in that, The loss function of the federated learning pre-trained model L Specifically: in, This represents the activation function. This represents the bias term of the federated learning pre-trained model. The tag value representing encrypted data, This represents the weight parameter vector of the federated learning pre-trained model. The first parameter in the weight parameter vector of the federated learning pre-trained model represents the... i One weight parameter, Represents an encrypted data vector. Indicates the first i Encrypted data, v This indicates the total number of weight parameters.

4. The federated learning model training method according to claim 3, characterized in that, The process of updating the model parameters of the federated learning pre-trained model according to the modified loss function includes: Based on the corrected loss function, the model parameters of the federated learning pre-trained model are updated using the gradient descent algorithm; wherein, the model parameters include a weight parameter vector and a bias term; when updating the weight parameter vector using the gradient descent algorithm, the gradient calculation formula for the weight parameter vector is: in, The weight parameter vector represents the first... k Gradient, Represents the first element in the weight parameter vector. k One weight parameter, Indicates input x Weight w and bias b The corresponding output, Indicates the partial derivative sign. Indicates the first k Encrypted data; When updating the bias term using the gradient descent algorithm, the gradient calculation formula for the bias term is: in, The first term represents the bias term. k There are several gradients.

5. The federated learning model training method according to claim 1, characterized in that, Also includes: Based on the data quality evaluation results of each participant, the updated model parameters of the federated learning pre-trained model are corrected, and the corrected model parameters of each participant are obtained and distributed to each participant.

6. The federated learning model training method according to claim 5, characterized in that, The step of correcting the updated model parameters of the federated learning pre-trained model based on the data quality evaluation results of each participant, obtaining the corrected model parameters for each participant, and distributing them to each participant includes: Based on the data quality evaluation results of each participant, the updated model parameters of the federated learning pre-trained model are corrected using the following formula to obtain the corrected model parameters for each participant: in, Indicates the participating parties k The model parameter correction results; Indicates the participating parties k Data quality evaluation results; This represents the updated model parameters after the federated learning pre-trained model; The model parameter correction results for each participant will be distributed to each participant respectively.

7. A federated learning model training system, characterized in that, include: The quality evaluation module is used to obtain the encrypted data submitted by each participant and, in combination with the pre-trained federated learning model, to obtain the data quality evaluation results of each participant. The parameter update module is used to correct the loss function of the federated learning pre-trained model based on the data quality evaluation results of each participant, obtain the corrected loss function, and update the model parameters of the federated learning pre-trained model according to the corrected loss function. The federated learning model is a federated learning model for electricity consumption credit. The encrypted data types include credit data, basic power data, and power analysis data; When the encrypted data submitted by the participants is credit data, the data quality evaluation result of the participants is obtained by the following formula: in, N k Participants k The total number of byte groups of encrypted data submitted; Participants k The submitted number i The absolute error rate of group encrypted data; When the encrypted data submitted by the participants is basic power data, the data quality evaluation result of the participants is obtained by the following formula: When the encrypted data submitted by the participants is power analysis data, the data quality evaluation result of the participants is obtained by the following formula: The loss function for adjusting the federated learning pre-trained model based on the data quality evaluation results of each participant includes: The loss function of the federated learning pre-trained model is modified using the following formula. L : in, It is a modified loss function. , Participants k Data quality evaluation results It is a ReLU function used to remove encrypted data with a data quality rating below 0.

6.

8. The federated learning model training system according to claim 7, characterized in that, It also includes a parameter distribution module; the parameter distribution module is used to correct the updated model parameters of the federated learning pre-trained model based on the data quality evaluation results of each participant, obtain the model parameter correction results of each participant, and distribute them to each participant.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the federated learning model training method as described in any one of claims 1 to 6.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the federated learning model training method as described in any one of claims 1 to 6.

Images