Federal model adaptation learning system

Abstract

The application provides a federal large model adaptive learning system. On the basis of the combination of multi-objective optimization and incremental learning, multiple optimization indicators are constructed, and adaptive small model incremental learning is designed; in view of the data privacy protection under the federal learning, a small model gradient scaling method is proposed, so that the gradient information is fully utilized; by revealing the correlation between the model generalization ability and the sampled data, a generalization ability evaluation function is proposed; in view of the real problems such as performance degradation and fault faced by industrial equipment during operation, combined with the generalization ability evaluation function, multiple optimization objectives are designed, and the model is adaptively updated and repaired through multi-objective evolutionary learning, so that the usability of the large model in the real industrial scene is improved. Finally, the adaptive and accurate updating of the large and small models is realized, and the model generalization ability is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to federated learning, evolutionary algorithm, adaptive intelligent algorithm, in particular to a federated large model adaptive learning system. BACKGROUND

[0002] Many enterprises are difficult to directly share big data due to industry data secrecy and other reasons. Under this demand, federated learning with privacy protection ability has been highly valued by scholars at home and abroad. Qiang Yang proposed secure federated learning in "Federated Learning with Privacy-preserving and Model IP-right-protection", which uses encryption technology to protect data to ensure that data is safely protected during transmission and storage. Guoyi Shi detailedly summarized the privacy protection research in the federated learning framework in "Privacy preservation in federated learning: An insightful survey from the GDPR perspective", and discussed the advantages and disadvantages of existing privacy protection technologies and potential solutions. Bin Cao proposed multiple federated learning algorithms and their evolution strategies in "Federated neural architecture search for medical data security". Weifeng Lv designed a cross-platform order scheduling federated learning framework in "Fed-LTD: Towards Cross-Platform Ride Hailing via Federated Learning to Dispatch", that is, multiple platforms make scheduling decisions without sharing their local data, and explore solutions to privacy and efficiency challenges.

[0003] On the other hand, the degradation of device performance such as aging, failure, may lead to the failure of AI model, which requires the design of adaptive intelligent algorithm. The research of adaptive intelligent algorithm has emerged. In the field of evolutionary computation, Yun Li et al. proposed an adaptive evolutionary algorithm in "Adaptive particle swarm optimization", which automatically controls the algorithm parameters during the evolution process. In the field of adaptive deep learning, Yining Dong proposed a deep dynamic adaptive transfer network in "Deep dynamic adaptive transfer network for rolling bearing fault diagnosis with considering cross-machine instance", which realizes the dynamic adaptive update of the model. Islam proposed an automatic machine learning co-exploration framework EVE in "EVE: Environmental Adaptive Neural Network Models for Low-power Energy Harvesting System", which adaptively selects candidate models.

[0004] There are few works that combine federated learning, adaptation and evolution. Bin Cao combined federated learning and evolutionary ideas in "Large-Scale Multiobjective Federated Neuroevolution for Privacy and Security in the Internet of Things", starting from a new idea, and proposed a heterogeneous large-scale multi-objective federated neural evolution strategy. Chunhua Xiao proposed a sparse network model evolution algorithm for federated learning in "CBFL: A Communication-Efficient Federated Learning Framework From Data Redundancy Perspective". Zhang Zehui proposed an adaptive model aggregation scheme for federated learning in "Adaptive Federated Deep Learning Algorithm for Non-Independent and Identically Distributed Data".

[0005] To address the challenge of combining federated learning algorithms, evolutionary algorithms, and adaptive intelligent algorithms, this invention clarifies the relationships between them and organically integrates them into a unified system known as the Federated Large Model Adaptive Learning System. This invention analyzes the task characteristics of large and small models, designs various optimization objectives such as generalization ability and model accuracy, and combines evolutionary thinking with adaptive intelligent algorithms. Furthermore, by designing a gradient scaling method, it further unifies federated learning, evolutionary thinking, and adaptive intelligent algorithms, thus forming the Federated Large Model Adaptive Learning System. Summary of the Invention

[0006] The purpose of this invention is to construct a federated large-model adaptive learning system that generates AI models while reducing the risk of data privacy leakage, and achieves accurate adaptive updates of the AI ​​models using a small amount of the latest data. The invention includes: constructing adaptive small-model incremental learning; proposing a gradient scaling method to protect data privacy in federated learning; revealing the correlation between model generalization ability and training data, and proposing a generalization ability evaluation function; designing multiple optimization objectives, and adaptively updating and repairing the model through multi-objective evolutionary learning to improve the usability of the large model.

[0007] The technical solution of the present invention:

[0008] A federated large-model adaptive learning system is disclosed. This system can be widely applied to numerous pre-trained large models, such as BERT and ChatGPT, and effectively improves the versatility, functionality, and efficiency of pre-trained large models. The federated adaptive learning system mainly consists of a small model adaptive update module, a BERT large model and small model normalization module, a BERT large model adaptive update module, and a system privacy protection module. The process of the federated adaptive learning system applying to pre-trained large models will be described sequentially according to the order of these modules (using BERT as an example, but the system's application is not limited to BERT).

[0009] (1) Small model adaptive update module

[0010] To address the challenge of interaction between the large and small models in BERT, an adaptive update module for the small model is designed. Through adaptive updates of the small model, the performance of the large BERT model is improved, such as increasing model accuracy and reducing computational overhead. Considering three optimization directions—small model accuracy, small model forgetting rate, and small model error—the adaptive update module establishes its adaptive criteria based on these optimization directions.

[0011] 1) Small model accuracy:

[0012] From a generality perspective, in the adaptive update of the small model, the accuracy of the small model will determine the generality of the BERT large model. Therefore, a small model accuracy submodule is proposed, as follows:

[0013]

[0014] Among them, C m t represents the average accuracy of the small model after the m-th increment stage. i This represents the precision value corresponding to the i-th stage;

[0015] 2) Forgetting rate of small models:

[0016] From a functional perspective, in the adaptive update of the small model, the forgetting rate of the small model determines the convergence property of the small model, and further determines the convergence of the BERT large model. Therefore, a small model forgetting rate submodule is designed, as follows:

[0017] min F(q)=ud+(1-u)F(q-1)

[0018] Where F(q) represents the small model forgetting rate at time q; F(q) represents the small model forgetting rate at time q-1; u is a coefficient ranging from [0,1], used to control the decay rate; d is the data at the current time.

[0019] 3) Small model error gradient:

[0020] From an efficiency perspective, in small-model adaptive updates, the gradient error directly determines the efficiency of the BERT model. Therefore, a small-model error gradient submodule is designed, as follows:

[0021]

[0022] Where w(v) represents the weight at time v, w(v-1) represents the weight at time v-1, and η is the learning rate. This represents the gradient of the error at time v;

[0023] (2) Normalization module for BERT large and small models

[0024] During the adaptive update of the small model, gradient information of the small model is continuously generated. Therefore, the BERT large model and small model normalization module will normalize the gradient information of the BERT large model and the small model, laying the foundation for the implementation of the BERT large model adaptive update module. Gradient information has both magnitude and direction attributes. The privacy protection principle of federated learning allows the small model to pass gradient information to the BERT large model, using the gradient information generated by the small model to feed back into the BERT large model's learning. However, the number of parameters in the BERT large model and the small model differs greatly, and the gradient of the small model cannot be directly used by the BERT large model. To address this, a gradient scaling method is proposed to evaluate the difference in the number of parameters between the small model and the BERT large model, as well as prior knowledge, to establish the correspondence between the gradient values ​​of the small model and the gradient values ​​of the BERT large model. The gradient scaling method is expressed as follows:

[0025]

[0026] Among them, T g ′ rad t′ represents the gradient value of the smaller model on the larger model. grad T represents the gradient value of the small model. grad For the prior gradient values ​​of the large model, t grad T represents the prior gradient value of the small model. n For large model parameter count, t n For small model parameter count;

[0027] (3) BERT large model adaptive update module

[0028] In the BERT large model and small model normalization module, the gradient information of the small model and the gradient information of the BERT large model are normalized; the BERT large model adaptive update module determines the gradient information normalization from the aspects of generalization ability and gradient fitting to help the BERT large model adaptive update.

[0029] 1) Generalization ability

[0030] To monitor the learning direction gap between the large and small BERT models and maintain consistency in their collaborative learning direction, a distribution-aware approach is used. First, local data is used to initially measure the bias of the large model, followed by a secondary measurement using online data from the edge. To this end, a generalization evaluation method is proposed to assist the adaptive learning function of the federated adaptive learning system, as follows:

[0031]

[0032]

[0033] Where f(x,y) is called the generalization evaluation function, and g(x,y) is called the distribution-aware similarity function; x and y are the evaluation results of the large model and the small model, respectively, and x and y are one-dimensional vectors, μ x μ y These are the average values ​​of the two, σ x ,σ y These are the variances of the two, σ. x,y δ1 and δ2 are the covariance of the two; δ1 and δ2 are two minimum constants to prevent the denominator from being 0; α is the scaling factor, which takes values ​​in the range [10,20] to ensure that the range of f(x,y) is between (0,1); λ is the normalization constant to limit the domain range.

[0034] The generalization ability is expressed as follows using the generalization evaluation method:

[0035]

[0036]

[0037] Among them, f i (x,y) represents the generalization ability value in different tasks; n is the number of tasks; C is a constant between [0,1], which constrains the difference in generalization ability evaluation between different tasks;

[0038] 2) Gradient fitting

[0039] Since the large BERT model cannot access the latest data, we start with gradient information to make the gradient of the large BERT model fit the small model as closely as possible, thereby indirectly learning the features of the latest data. The gradient fitting is represented as follows:

[0040]

[0041] Among them, T i For the gradient values ​​of the large model, t i Here, represents the gradient value of the small model, and S represents the maximum number of training iterations.

[0042] (4) System privacy protection module

[0043] The federated adaptive learning system is implemented on the BERT large model in the small model adaptive update module, the BERT large model and small model normalization module, and the BERT large model adaptive update module. The system privacy protection module is integrated into the federated adaptive learning system, placing it within a privacy protection mechanism. This module implements privacy protection for the federated adaptive learning system, including noise addition mechanisms and mechanisms to approximate the average weight matrix.

[0044] 1) Noise Addition Mechanism

[0045] To address the opaque nature of the interaction learning process between the large and small models in BERT, a noise addition mechanism is proposed. This mechanism adds noise after subsampling, hiding the contribution of individual clients within the aggregation, thus concealing it from the entire distributed learning process. The specific implementation is as follows:

[0046] In random subsampling, let the total number of clients be K; in each round of communication, a random subset Z of size T is drawn. t The subscript t represents the current round number; then the administrator distributes the central model of the current round to each client, and the central model of the current round is denoted as W. t The central model is optimized by the client using their data, Z. t Each independent client in the system has its own different client model. Add Gaussian noise to each client:

[0047]

[0048] in, The client model with Gaussian noise added, the noise follows the pattern N(0,σ). 2 The noise distribution and standard deviation σ determine the noise scale; the optimized client model With the central model W t The difference between them is called the update of the client model, denoted as .

[0049] At the end of each round of communication t, the client updates... The administrator whose data was sent back to the central model;

[0050] 2) Approximate weight matrix averaging mechanism

[0051] To address the complexity of the interaction between the large and small models in BERT, a method is proposed that uses a Gaussian mechanism to distort the sum of all updates to approximate the average value of the weight matrix. This method uses a Gaussian mechanism to distort the sum of all updates, enhancing operational sensitivity by using a scaled version instead of the actual update. The specific implementation is as follows:

[0052]

[0053] in, Scaling is applied to the scaled updated model, where S represents the scaling sensitivity;

[0054] Add S to the sum of all scaling updates, then divide the output of the mixed noise mechanism by T to obtain the true average of all client updates. To prevent the leakage of critical personal information, this approximation is again added to the Laplace noise and then added to the current central model W. tAssign a new central model:

[0055]

[0056] Where Laplace represents the Laplace noise calculation, and ε represents the privacy budget.

[0057] The beneficial effects of this invention are as follows: This invention uses model accuracy, learning forgetting rate, and error gradient as optimization objectives to form a multi-objective incremental learning optimization method. Utilizing the privacy protection principle of federated learning, a gradient scaling method is used to establish the correspondence between the gradient values ​​of the large and small models. The gradient information generated by the small model is fed back to the large model for learning, maintaining collaborative learning between the large and small models. A distribution-aware method is introduced to monitor the difference in learning directions between the large and small models. Combining gradient fitting optimization, generalization ability optimization, and model accuracy optimization objectives, a multi-objective evolutionary algorithm is formed, enabling the large model to adaptively update and significantly improving its generalization ability. Attached Figure Description

[0058] Figure 1 This is a schematic diagram of the method of the present invention.

[0059] Figure 2 This is an adaptive update module for small models.

[0060] Figure 3 This is the normalization module for BERT's large and small models.

[0061] Figure 4 This is a privacy protection module.

[0062] Figure 5 This is the adaptive update module for BERT large models. Detailed Implementation

[0063] With reference to the accompanying drawings and specific technical solutions, the embodiments of the present invention will be further described below (taking BERT as an example, but the learning system is not limited to BERT).

[0064] See Figure 1 The present invention provides a federated large model adaptive learning system, which first uses online data to make the incremental learning small model adaptively update, then performs gradient scaling and privacy protection on the gradient information obtained from the small model, and finally uses the gradient information to make the large model adaptively update.

[0065] 1. Adaptive update of small model

[0066] See Figure 2In the small model adaptive update module, the adaptive direction is jointly determined by three factors: small model accuracy, small model forgetting rate, and error gradient. Whenever new data is added, incremental learning only adaptively updates the gradient parameters resulting from the new data, thus learning the latest data at a relatively low cost.

[0067] 2. Gradient scaling and privacy protection of gradient information

[0068] like Figure 3 As shown, the small model generates gradient information during the incremental learning process based on multi-objective evolution. Therefore, by using gradient scaling, the gradient information of the large and small BERT models is normalized, so that the large BERT model can use the gradient information generated by the small model to feed back into the learning process.

[0069] See Figure 4 For privacy protection purposes, the gradient information generated by the small model also needs to be encrypted. This invention proposes a differential privacy algorithm based on a hybrid noise mechanism. This algorithm involves two steps: adding noise after random subsampling parameter updates and distorting the sum of all updates using a Gaussian mechanism. This allows the privacy protection process to be implicitly embedded into the distributed learning process.

[0070] 3. Adaptive Update of Large BERT Model Based on Gradient Scaling

[0071] See Figure 5 From the perspectives of gradient information and generalization ability, gradient fitting optimization objectives and generalization ability optimization objectives are designed respectively.

[0072] The optimization objective of gradient fitting:

[0073]

[0074] Among them, T i t represents the gradient value of the large BERT model. i Here, represents the gradient value of the small model, and S represents the maximum number of training iterations.

[0075] A series of tasks, including classification and regression, are created for the small model, and the loss function is varied to obtain dozens of tasks. The generalization ability of the small model under each different task is independently evaluated, yielding corresponding generalization ability values. Based on the proposed adaptive collaborative control function, these dozens of index values ​​are used as collaborative variables to obtain the generalization ability optimization objective.

[0076] The optimization objective for generalization ability:

[0077]

[0078]

[0079] Among them, f i (x,y) represents the generalization ability value in different tasks, n is the number of tasks, and C is a constant between [0,1]. C constrains the difference in generalization ability evaluation between different tasks in order to obtain more accurate optimization results.

[0080] Table 1. Classification accuracy of BERT and Federated Adaptive BERT on different datasets.

[0081]

[0082] Higher classification accuracy indicates better performance of the large model in classification tasks, and it helps to evaluate the large model's ability to classify samples of different classes. As shown in Table 1, thanks to the optimization objectives of gradient fitting and generalization ability, the classification accuracy of BERT under the federated large model adaptive learning system is significantly improved compared to the original BERT model on both datasets.

[0083] Table 2. Classification recall rates of BERT and Federated Adaptive BERT on different datasets.

[0084]

[0085] Considering the potential class imbalance in the datasets, it is necessary to measure the recall of the model. As shown in Table 2, the BERT model under the Federated Large Model Adaptive Learning System also showed improved classification recall compared to the original model on both datasets. Based on the classification precision and recall in Table 2, the Federated Large Model Adaptive Learning System is effective on both datasets.

Claims

1. A federated large model adaptive learning system, which mainly consists of a small model adaptive update module, a BERT large model and small model normalization module, a BERT large model adaptive update module, and a system privacy protection module; characterized in that, (1) Small model adaptive update module; (2) Normalization module for BERT large and small models: Gradient information possesses both magnitude and direction attributes. The privacy protection principle of federated learning allows small models to pass gradient information to the large BERT model, and the gradient information generated by the small model can be used to feed back into the large BERT model for learning. A gradient scaling method is proposed to evaluate the difference in parameter sizes between the small model and the large BERT model, as well as prior knowledge, to establish the correspondence between the gradient values ​​of the small model and the large BERT model. The gradient scaling method is expressed as follows: ； in, This represents the gradient value of the smaller model on the larger model. This represents the gradient value of the small model. For the prior gradient values ​​of the large model, For the prior gradient values ​​of the small model, For large model parameter count, For small model parameter count; (3) BERT large model adaptive update module; (4) System privacy protection module: In the small model adaptive update module, the BERT large model and small model normalization module, and the BERT large model adaptive update module, the federated adaptive learning system is implemented to operate on the BERT large model; the system privacy protection module is integrated into the federated adaptive learning system, placing the federated adaptive learning system in a privacy protection mechanism; the system privacy protection module implements the privacy protection of the federated adaptive learning system, including a noise addition mechanism and a mechanism to average the approximate weight matrix. 1) Noise addition mechanism: A noise addition mechanism is proposed, which hides the contribution of individual clients in the aggregation by adding noise after subsampling, thereby hiding it in the entire decentralized learning process. The specific implementation is as follows: In random subsampling, let the total number of clients be K; in each round of communication, a sample of size K is drawn. random subset subscript This indicates the current round number; then the administrator distributes the central model for the current round to each client. The central model for the current round is denoted as... The central model is optimized by the client using their data. Each independent client in the system has its own different client model. Add Gaussian noise to each client: ； in, The client model after adding Gaussian noise, the noise follows... Distribution, standard deviation The noise scale is determined; the optimized client model With center model The difference between them is called the update of the client model, denoted as . ; At the end of each round of communication t, the client updates... The administrator whose data was sent back to the central model; 2) Approximate weight matrix averaging mechanism: A method is proposed that uses a Gaussian mechanism to distort the sum of all updates to approximate the average value of the weight matrix. This method uses a Gaussian mechanism to distort the sum of all updates, thereby enhancing operational sensitivity by using a scaled version instead of the actual update. The specific implementation is as follows: ； in, For the scaled client model, Represents scaling sensitivity; Will Add to the sum of all scaling updates, then divide the output of the noise mixing mechanism by . To obtain the true average of all client updates, and to prevent the leakage of critical personal information, this approximation is further supplemented with Laplace noise and added to the current central model. Assign a new central model: ； in, Represents Laplace noise calculation, This refers to a privacy budget.

2. The federated large model adaptive learning system according to claim 1, characterized in that, The small model adaptive update module is designed as follows: Considering three optimization directions—small model accuracy, small model forgetting rate, and small model error—the small model adaptive update module establishes adaptive criteria based on these optimization directions: 1) Small model precision submodule: From a generalization perspective, a small model accuracy submodule is proposed in the small model adaptive update, as shown below: ； in, This indicates that the small model has gone through the first... The average precision value after the incremental phase. Indicates the first The precision value corresponding to each stage; 2) Small model forgetting rate submodule: The small model forgetting rate submodule is designed as follows: ； in, Indicates the small model at time... The small model forgetting rate; Indicates the small model at time... The small model forgetting rate; It is a coefficient with a value between [0,1], used to control the attenuation rate; This is the data at the current moment; 3) Small model error gradient submodule: The small model error gradient submodule is designed as follows: ； in, Indicates at time The weight, Indicates at time The weight, It's the learning rate. Indicates at time The gradient of the error.

3. The federated large model adaptive learning system according to claim 1, characterized in that, The BERT large model adaptive update module is as follows: 1) Generalization ability: Using a distributed sensing approach, the prediction bias of the large model is first measured using local data, followed by a secondary measurement using online data from the edge computing environment. To this end, a generalization evaluation method is proposed to assist the adaptive learning function of the federated adaptive learning system, as follows: ； in, This is called the generalization evaluation function. This is called the distribution-aware similarity function; x and y are the evaluation results of the large model and the small model, respectively, and x and y are one-dimensional vectors. These are the average values ​​of the two, , Let these be the variances of the two variables. Let be the covariance of the two. and These are two extremely small constants to prevent the denominator from being 0; This is the scaling factor, and its value ranges from [10, 20], ensuring... The range of its value is between (0, 1); This is a normalization constant, which restricts the range of the domain. The generalization ability is expressed as follows using the generalization evaluation method: ； in, denoted as the generalization ability value for different tasks; n is the number of tasks; C is a constant between [0,1], which constrains the difference in generalization ability evaluation between different tasks; 2) Gradient fitting: Starting with gradient information, the gradient of the large BERT model is made to fit the small model as closely as possible, thereby indirectly learning the features of the latest data. The gradient fitting is represented as follows: ； in, For the gradient values ​​of the large model, Here, represents the gradient value of the small model, and S represents the maximum number of training iterations.

Images