Model-based segmentation robust learning method, system, medium, and device

Abstract

The present application relates to the fields of artificial intelligence, distributed systems and information security technology, and provides a Byzantine robust learning method, system, medium and equipment based on model segmentation. The Byzantine robust learning method based on model segmentation comprises: sending a current business model and partial weight parameters divided by all weight parameters of the current business model to each first device, so that the first device uses the partial weight parameters to train the current business model using local data; receiving gradients of respective current business models after training from each first device, sorting all the gradients, calculating a weighted average of all the gradients after removing extreme values, and using the weighted average to update the current business model; and using the updated business model as the current business model in the next round of model training process. The method can reduce the heavy computing burden of the first device caused by uneven weight distribution, thereby improving the model training efficiency of the Byzantine robust learning system.

Description

Technical Field

[0001] This invention relates to the fields of artificial intelligence, distributed systems and information security, and in particular to a Byzantine robust learning method, system, medium and device based on model segmentation. Background Technology

[0002] The statements in this section are merely background information related to the present invention and do not necessarily constitute prior art.

[0003] Model segmentation techniques can be applied based on the hierarchical structure of the model. Some studies propose dividing the neural network model into different layers and assigning each layer to different client devices for computation, with each client device responsible for computing a portion of the model. Other studies propose segmenting the model by module. This method requires partitioning based on the different model structures. For example, the ResNet-50 model has 16 residual modules, which are assigned to different client devices. The gradient parameters calculated by each client are passed to the next client until the final layer outputs. Still other studies propose segmenting by channel. This type of segmentation is mainly applicable to convolutional neural networks, especially convolutional layers. Based on the different channels of the convolutional layer, these channels are assigned to different client nodes. Each client only computes the channels it is responsible for, and after each layer's computation, it shares data with other nodes to complete the model's forward and backward propagation.

[0004] In distributed learning, because multiple client devices are involved, some client nodes may malfunction, be maliciously attacked, or otherwise cause errors, false or even maliciously tampered calculation results, which will affect the update of the global model and lead to a decline in the overall system performance. This type of problem is called "Byzantine fault".

[0005] There are many studies dedicated to the Byzantine problem, including: (1) Secure aggregation algorithms based on encryption technology: Encryption and decryption operations usually require high computing resources, which may bring significant performance burden to resource-constrained devices (such as sensors and mobile terminals). At the same time, the transmission of ciphertext may be more verbose than plaintext, thus increasing communication overhead. Moreover, such algorithms require pre-allocation and periodic updating of keys. (2) Node selection strategies based on trust models: Trust models usually rely on historical behavior data or interaction records between nodes to calculate trust values. Malicious nodes can manipulate trust models by forging multiple identities, making them appear more reliable than they actually are, thereby infiltrating the network and destroying the aggregation results. Building and maintaining trust models requires real-time monitoring and data statistics of node behavior, which increases the computational complexity of the system. Summary of the Invention

[0006] To address the technical problems existing in the background art, the present invention provides a Byzantine robust learning method, system, medium, and device based on model segmentation. The method involves each first device using partial weight parameters equally distributed among all weight parameters of the current business model to train the current business model using local data. A second device receives the gradients from each first device after the training of its respective current business model, sorts all gradients, calculates the weighted average of all gradients after removing extreme values, and updates the current business model. This reduces the excessive computational burden on the first devices caused by uneven weight distribution, thereby improving the model training efficiency of the Byzantine robust learning system and enhancing its adaptability.

[0007] To achieve the above objectives, the present invention adopts the following technical solution:

[0008] The first aspect of the present invention provides a Byzantine robust learning method based on model segmentation.

[0009] A Byzantine robust learning method based on model segmentation includes: a second device and at least two first devices training a business model, each first device having local data, and the second device maintaining the trained business model. The method is applied to the second device and includes: sending the current business model and a portion of weight parameters equally distributed among all weight parameters of the current business model to each first device, so that the first devices can use the portion of weight parameters to train the current business model using local data; receiving the gradients after the training of the current business model from each first device, sorting all gradients, calculating the weighted average of all gradients after removing extreme values, and using this average to update the current business model; the updated business model serving as the current business model for the next round of model training.

[0010] Optionally, in one example of the above aspects, the partial weight parameters equally divided by all weight parameters of the current business model include: converting all weight parameters of the current business model into one-dimensional vectors, merging them according to the training arrangement to obtain a total weight vector; dividing the total weight vector into several equal parts, restoring each part of the weight vector to obtain partial weight parameters.

[0011] Alternatively, in one example of the above aspects, the calculation of the average of all gradients after removing extreme values ​​is performed using the following formula:

[0012]

[0013] in, This represents the average of all gradients after removing extreme values, where n represents the number of first-order devices and f represents the number of Byzantine first-order devices. Let represent the model gradient of the i-th first device.

[0014] A second aspect of the present invention provides a Byzantine robust learning system based on model segmentation.

[0015] A Byzantine robust learning system based on model segmentation includes: a second device and at least two first devices training a business model, each first device having local data, and the second device maintaining the trained business model. The system is applied to the second device. The system includes: a sending unit that sends the current business model and a portion of weight parameters equally distributed among all weight parameters of the current business model to each first device, so that the first devices can use the portion of weight parameters to train the current business model using local data; a model updating unit that receives the gradients after the training of the current business model from each first device, sorts all gradients, calculates the weighted average of all gradients after removing extreme values, and uses this average to update the current business model; the updated business model serves as the current business model for the next round of model training.

[0016] Optionally, in one example of the above aspects, the sending unit includes: converting all weight parameters of the current business model into one-dimensional vectors, merging them according to the training arrangement to obtain a total weight vector; dividing the total weight vector into several equal parts, restoring each part of the weight vector to obtain partial weight parameters.

[0017] Alternatively, in one example of the above aspects, the calculation of the average of all gradients after removing extreme values ​​is performed using the following formula:

[0018]

[0019] in, This represents the average of all gradients after removing extreme values, where n represents the number of first-order devices and f represents the number of Byzantine first-order devices. Let represent the model gradient of the i-th first device.

[0020] A third aspect of the present invention provides a Byzantine robust learning method based on model segmentation.

[0021] A Byzantine robust learning method based on model segmentation includes: a second device and at least two first devices training a business model, each first device having local data, and the second device maintaining the trained business model. The method is applied to the first devices, and includes: receiving a current business model and partial weight parameters equally distributed by all weight parameters of the current business model from the second device; training the received current business model using the received partial weight parameters and local data; providing the gradients of the model parameters in the locally trained business model to the second device, so that the second device can sort all gradients and calculate the weighted average of all gradients after removing extreme values, which is used to update the business model.

[0022] A fourth aspect of the present invention provides a Byzantine robust learning system based on model segmentation.

[0023] A Byzantine robust learning system based on model segmentation includes: at least two first devices and a second device training a business model, each first device having local data, and the second device maintaining the trained business model. The apparatus is applied to the first devices. The system includes: a receiving unit that receives a current business model and partial weight parameters equally distributed among all weight parameters of the current business model from the second device; a model training unit that uses the received partial weight parameters and local data to train the received current business model; and a model parameter providing unit that provides the gradients of the model parameters in the locally trained business model to the second device, so that the second device can sort all gradients and calculate the weighted average of all gradients after removing extreme values, for updating the business model.

[0024] A fifth aspect of the present invention provides a computer-readable storage medium.

[0025] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the model segmentation-based Byzantine robust learning method as described in the first or third aspect above.

[0026] A sixth aspect of the present invention provides a computer device.

[0027] A computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the program, implements the steps of the model segmentation-based Byzantine robust learning method as described in the first or third aspect above.

[0028] Compared with the prior art, the beneficial effects of the present invention are:

[0029] This invention provides a Byzantine robust learning method, system, medium, and device based on model segmentation. It involves each first device using partial weight parameters equally distributed across all weight parameters of the current business model to train the current business model using local data. A second device receives the gradients from each first device after the training of its respective current business model, sorts all gradients, calculates the weighted average of all gradients after removing extreme values, and updates the current business model. This reduces the excessive computational burden on the first devices caused by uneven weight distribution, thereby improving the model training efficiency of the Byzantine robust learning system and enhancing its adaptability.

[0030] This invention is based on weight partitioning and is not dependent on specific network structures, making it more adaptable and applicable to different types of networks. Furthermore, it does not require concern about the input and output dimensions of each layer; it simply flattens and partitions the data, greatly reducing the complexity of the cutting process. Attached Figure Description

[0031] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an improper limitation of the invention.

[0032] Figure 1 A schematic diagram of the architecture of a Byzantine robust learning system based on model segmentation is shown.

[0033] Figure 2 A flowchart of a Byzantine robust learning method based on model segmentation is shown on the second device side;

[0034] Figure 3 A block diagram of a Byzantine robust learning system based on model segmentation is shown on the second device side.

[0035] Figure 4 A flowchart of a Byzantine robust learning method based on model segmentation is shown on the first device side;

[0036] Figure 5 A block diagram of a Byzantine robust learning system based on model segmentation is shown on the first device side. Detailed Implementation

[0037] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0038] It should be noted that the following detailed description is illustrative and intended to provide further explanation of the invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0039] It should be noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the scope of exemplary embodiments according to the invention. As used herein, the singular form is intended to include the plural form as well, unless the context clearly indicates otherwise. Furthermore, it should be understood that when the terms "comprising" and / or "including" are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof.

[0040] It should be noted that the flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of methods and systems according to various embodiments of this disclosure. It should be noted that each block in a flowchart or block diagram may represent a module, segment, or portion of code, which may include one or more executable instructions for implementing the logical functions specified in the various embodiments. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than that shown in the drawings. For example, two consecutively represented blocks may actually be executed substantially in parallel, or they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the flowcharts and / or block diagrams, and combinations of blocks in the flowcharts and / or block diagrams, may be implemented using a dedicated hardware-based system that performs the specified functions or operations, or using a combination of dedicated hardware and computer instructions.

[0041] The subject matter described herein will now be discussed with reference to exemplary embodiments. It should be understood that these embodiments are discussed merely to enable those skilled in the art to better understand and implement the subject matter described herein, and are not intended to limit the scope, applicability, or examples set forth in the claims. The function and arrangement of the elements discussed may be changed without departing from the scope of this specification. Various processes or components may be omitted, substituted, or added as needed in the various examples. For example, the described methods may be performed in a different order than described, and steps may be added, omitted, or combined. Furthermore, features described in some examples may be combined in other examples.

[0042] As used herein, the term "comprising" and its variations are open terms meaning "including but not limited to". The term "based on" means "at least partially based on". The terms "one embodiment" and "an embodiment" mean "at least one embodiment". The term "another embodiment" means "at least one other embodiment". The terms "first", "second", etc., may refer to different or the same objects. Other definitions, whether explicit or implicit, may be included below. Unless explicitly indicated by the context, the definition of a term shall remain consistent throughout the specification.

[0043] In this specification, the term "business model" refers to a machine learning model applied in a business scenario to provide business prediction services, such as machine learning models used for classification prediction, business risk prediction, etc. Examples of machine learning models may include, but are not limited to: linear regression models, logistic regression models, neural network models, decision tree models, support vector machines, etc. Examples of neural network models may include, but are not limited to: deep neural network (DNN) models, convolutional neural network (CNN) models, ResNet-50 models, backpropagation (BP) neural networks, etc.

[0044] The specific implementation of a business model depends on the applicable business scenario. For example, if the business model is used to classify users, it is implemented as a user classification model. Accordingly, user classification prediction can be performed based on the user characteristic data of the users to be classified. If the business model is used to predict business risks associated with business transactions occurring on a business system, it is implemented as a business risk prediction model. Accordingly, business risk prediction can be performed based on the business transaction characteristic data of that transaction.

[0045] With the development of artificial intelligence technology, machine learning technology is widely used as a business model in various business application scenarios to provide various business prediction services, such as classification prediction and business risk prediction. For example, business models have wide applications in fields such as financial fraud, recommendation systems, and image recognition. To achieve better model performance, more training data is needed to train the business model. In application areas such as healthcare and finance, different enterprises or institutions have different data samples. Once these data are jointly trained, the accuracy of the business model will be greatly improved, thereby bringing huge economic benefits to enterprises.

[0046] In view of the above, embodiments of this specification provide a Byzantine robust learning method, system, medium, and device based on model segmentation. Each first device trains a current business model using local data, utilizing partial weight parameters evenly distributed among all weight parameters of the current business model. A second device receives gradients from each first device after the training of its respective current business model, sorts all gradients, calculates the weighted average of all gradients after removing extreme values, and updates the current business model. This reduces the excessive computational burden on the first devices caused by uneven weight distribution, thereby improving the model training efficiency of the Byzantine robust learning system and enhancing its adaptability.

[0047] The following will describe in detail, with reference to the accompanying drawings, a model segmentation-based Byzantine robust learning method, system, medium, and device for training business models according to embodiments of this specification.

[0048] Figure 1 A schematic diagram of the architecture of a model segmentation-based Byzantine robust learning system for training business models, according to an embodiment of this specification, is shown.

[0049] like Figure 1 As shown, the Byzantine robust learning system based on model segmentation includes at least two first devices and a second device. Figure 1 The diagram shows N first devices. At least two first devices and a second device can communicate with each other via a network, such as, but not limited to, the Internet or a local area network.

[0050] In the embodiments of this specification, the first device may be a device or device provider for locally collecting data samples, such as a smart terminal device, server device, edge device, etc. The second device maintains a business model to be trained. For example, the second device may be a terminal device at the model provider's location. The business model maintained on the second device may also be referred to as a global business model.

[0051] In embodiments of this specification, the local data of the first device may include business data collected locally by each member device. Business data may include characteristic data of business objects. Examples of business objects may include, but are not limited to, users, products, events, or relationships. Accordingly, business data may include, for example, locally collected user characteristic data, product characteristic data, event characteristic data, or relationship characteristic data, such as user characteristic data, business processing data, financial transaction data, product transaction data, medical and health data, etc. Business data may be applied to business models for model prediction, model training, and other suitable multi-party data joint processing.

[0052] In this specification, business data may include business data based on text data, image data, and / or voice data. Accordingly, business models can be applied to business risk identification, business classification, or business decision-making based on text data, image data, and / or voice data. For example, local data may be medical data collected by a hospital, and the business model can be used for disease examination or diagnosis. Alternatively, the collected local data may include user characteristic data. Accordingly, business models can be applied to business risk identification, business classification, business recommendation, or business decision-making based on user characteristic data. Examples of business models may include, but are not limited to, facial recognition models, disease diagnosis models, business risk prediction models, service recommendation models, and so on.

[0053] In this specification, the local data possessed by each first device constitutes the training sample data of the business model, and the local data possessed by each first device is the secret of the second device and cannot be known or fully known by other first devices.

[0054] In a practical application example, the first devices could be, for example, smart terminal devices belonging to the business application provider or its affiliates, such as smart terminal devices from different financial institutions or medical institutions. The second devices could be, for example, servers belonging to the business provider or business operator.

[0055] Figure 2 A flowchart illustrating a model segmentation-based Byzantine robust learning method for training a business model according to an embodiment of this specification, applied to a second device, is shown. Figure 2 The flowchart shown is the processing flow for one cycle of training in the Byzantine robust learning process.

[0056] like Figure 2 As shown, the second device sends the current business model and a portion of the weight parameters, which are equally distributed among all the weight parameters of the current business model, to each of the first devices. The first device uses the portion of the weight parameters to train the current business model using local data. The second device receives the gradients from each of the first devices after the training of their respective current business models, sorts all the gradients, calculates the weighted average of all gradients after removing extreme values, and updates the current business model. Here, the current business model is the updated business model obtained in the previous loop processing executed at the second device.

[0057] In one example, the Byzantine robust learning method based on model segmentation used to train the business model performs the following steps:

[0058] Step (1) Initialize and segment the model:

[0059] Assume the weights of the initial business model are ω 0, the second device divides it into p Each part has a weight of [number]. =( + +...+ ) / p That is, the initial weights of each sub-model are the average values ​​of the initial models.

[0060] Step (2) Assign the model to the first device:

[0061] Based on the computing power and historical performance of the first devices, different parts of the model are dynamically allocated. First devices with higher computing power will receive more complex model parts, thus accelerating the learning process. All first devices receive the model's partitioned parts via a random number generator and begin training.

[0062] Assumption Representing the k Local data held by the first device, among which During training, the second device selects a subset of users from all customers to participate in each round of training, thereby obtaining model parameters that minimize the loss function. Finally, the second device collects the parameters from each participating customer and performs a weighted average, as shown in the following formula:

[0063]

[0064] in, It is the first k The parameters of the model trained on the first device. w These are the model parameters after aggregation of the second device. , D It is the total data sample size of all the first devices. Indicates the first k The weight of the first device among all the first devices, measuring the weight of the first device. k The proportion of each primary device in global aggregation. If the amount of local data on each primary device varies significantly, the differences in the trained and updated models will also be large. Therefore, to balance this difference, a [specific approach / mechanism] is needed. The parameters are used for weighting. The optimization process of the loss function can be expressed by the following formula:

[0065]

[0066] in, This represents the optimal business model parameters. Indicates the first i The weight factor of the first device in the global optimization. L (·) is the first i The loss function of the first device, where w represents the weighted average of all first devices. Indicates the first i The data sample size of the first device.

[0067] Step (3) Local training and update:

[0068] Each first device performs a local update process, training using local data and model parameters. During local training, to improve stability and accelerate convergence, the first device employs the Adam adaptive learning rate algorithm to adjust the learning rate, ensuring smoother updates in each training round. Then, the first device calculates the gradient according to the current gradient formula. And update the local model parameters based on the adaptive learning rate: .in, Indicates the first device k exist t Updated during the training round p Model parameters for each business sub-model. Indicates the first device k exist t Updated during -1 round of training p Model parameters for each business sub-model. Indicates the first device k In the t -1 training round for sub-models p The loss gradient, This represents the adaptive learning rate, used to adjust the step size for updating model parameters in each training round.

[0069] Step (4) Upload the gradient to the second device:

[0070] Each first device calculates the update gradient of its local model. These gradients are then uploaded to a second device.

[0071] Step (5) Update the current business model:

[0072] The second device receives gradients from all the first devices. The received gradients are first sorted. To ensure the stability of the current business model, a confidence-based weighted averaging strategy is used to process the gradients.

[0073] In the t In round-robin communication, the second device will send parameters Distributed to the first device. Each first device. i Perform multiple local training operations on local data to obtain updated local parameters. Then, the first device i Calculate the local gradient As shown below, it is then sent to the second device.

[0074]

[0075] in, Indicates the first t Parameters of the business model.

[0076] Finally, the second device collects local gradients and uses the average gradient to update the business model.

[0077]

[0078]

[0079] in, Indicates the first t The gradient of the training round does not refer to any specific device.

[0080] Throughout the training process of federated learning, n first devices collaborate to learn the machine learning model, and with the help of second devices, finally optimize equation (2) to converge to the global optimal solution of the model.

[0081] The received gradients are weighted, and each gradient from a first device is assigned a confidence weight. The historical performance of the first device and the confidence level of its current gradient determine the weight of its uploaded gradient in the business model update. In this way, gradients from first devices with higher confidence levels will receive more attention.

[0082] Byzantine Threat Model: Some malicious first-devices may launch Byzantine attacks, using their private data or sending arbitrary local model parameters to attempt to tamper with the local training phase, thereby disrupting the current business model or stealing the privacy information of other first-devices. In this scheme, it is assumed that the number of malicious first-devices is finite, totaling [number missing]. n Among the first group of equipment, there are f One was identified as the first Byzantine device. This embodiment uses the set B⊆[ n To identify these first Byzantine devices, and the collection M = [ n ]\ B This is used to refer to the remaining honest first device. In the context of the existence of the Byzantine first device, the... t During round communication, the first device i The submitted gradient information can be expressed as follows:

[0083]

[0084] in, For Byzantine equipment, M For the remaining non-Byzantine equipment, Indicated as the first device i In the t Gradient information updated during each training round. Indicates the first t Parameters of the round business model, Indicates the first device iexist t The parameters generated after the first round of local training (used for the next round of updates).

[0085] Exclude outlier gradients (such as extremely unreasonable gradients), and then calculate a weighted average of the remaining gradients:

[0086]

[0087] in, This represents the average of all gradients after removing extreme values, where n represents the number of first-order devices and f represents the number of Byzantine first-order devices. This represents the model parameters of the i-th first device.

[0088] The FedAvg aggregation algorithm is commonly used to eliminate outlier gradients, but a few extreme outliers may prevent the global model from achieving optimal results. This invention addresses this by removing outlier gradients before... f One and after f We average the extreme gradients with the intermediate gradients to eliminate the influence of the largest outlier. This aggregates high-quality gradients, reduces the interference of invalid gradients on model updates, and also makes the global model update more balanced, resulting in faster convergence and more stable performance. The choice of f needs to be determined through multiple trials.

[0089] Step (6) Update the current business model:

[0090] The updated gradient is used to update the current business model. Based on the calculated global gradient, the second device adjusts the model parameters. Update:

[0091]

[0092] in, α This represents the learning rate.

[0093] The above scheme can also be explained as follows: This invention can use the ResNet-50 model. The Byzantine robust learning method based on model segmentation is mainly reflected in the second device. Before the model is sent from the second device to the first device, the model is first segmented. Because the ResNet-50 model is relatively complex, a weight-based segmentation method is used. First, all weight parameters of the ResNet-50 model are obtained. The weights of the ResNet-50 model include the parameters of all layers such as convolutional layers, fully connected layers, and batch normalization layers. The weights of each layer of the ResNet-50 model are usually high-dimensional. In order to perform average segmentation, all weights need to be flattened into a one-dimensional vector. The flattened weight vectors of all layers are merged to obtain a large long vector. This vector contains all the parameters of all layers of ResNet-50, arranged in layer order. Then, it is divided into P parts according to the number of first devices. Each part is still a one-dimensional vector. In order for each part to be usable in the model, this invention needs to restore the weights of each part to their original shape. The weights of each part are restored to the weight shape of the corresponding convolutional layer, fully connected layer, etc. (This process is performed on each first device, which trains with its own weight part and uploads the gradients). Each split weight part is distributed to different first devices. Each first device receives only a portion of the weights of the ResNet-50 model and trains locally using these portions. After each first device completes training and uploads its gradients, the second device aggregates the gradients uploaded by all first devices. To prevent Byzantine attacks, the second device sorts the gradients, removes extreme values, and calculates the average of the remaining portions to update the weights of the business model. Then, the second device redistributes the updated weights to the first devices. In each iteration, the first device continues local training with the updated partial weights. After each update, the second device re-merges and redistributes the weights until the model converges.

[0094] The process of restoring the weights of each part to their original shape can be achieved using existing methods such as the `reshape` function and the `view` function. Specifically, during the first flattening, the start position, end position, length, offset, and other information for each layer are recorded. During reconstruction, based on the previously recorded offsets and shape information, the `reshape` or `view` function is used to reshape the vector, restoring the one-dimensional vector to the multi-dimensional tensor of the corresponding layer.

[0095] This invention utilizes the aforementioned model-segmentation-based Byzantine robust learning method. By equally dividing the weight vector, each first device receives a nearly equal weight, thereby avoiding excessive computational burden on some first devices due to uneven weight distribution. Moreover, unlike layer-based or module-based partitioning, this invention is based on weight partitioning and does not depend on the specific network structure, making it more adaptable and applicable to different types of networks. Furthermore, it does not require concern about the input and output dimensions of each layer; it simply flattens and partitions the data, greatly reducing the complexity of the segmentation.

[0096] Figure 3 A block diagram of a second-device-side model-segmentation-based Byzantine robust learning system according to an embodiment of this specification is shown. Figure 3 As shown, the Byzantine robust learning system based on model segmentation includes a sending unit and a model update unit.

[0097] The sending unit is configured to send the current business model and a portion of the weight parameters equally distributed among all the weight parameters of the current business model to each first device, so that the first device can use the portion of the weight parameters to train the current business model using local data.

[0098] The model update unit is configured to receive the gradients after the training of the current business model from each first device, sort all gradients, calculate the weighted average of all gradients after removing extreme values, and use it to update the current business model; the updated business model is used as the current business model in the next round of model training.

[0099] In some embodiments, the sending unit includes: converting all weight parameters of the current business model into one-dimensional vectors, merging them according to the training arrangement to obtain a total weight vector; dividing the total weight vector into several equal parts, restoring each part of the weight vector to obtain partial weight parameters.

[0100] In some embodiments, the average of all gradients after removing extreme values ​​is calculated using the following formula:

[0101]

[0102] in, This represents the average of all gradients after removing extreme values, where n represents the number of first-order devices and f represents the number of Byzantine first-order devices. Let represent the model gradient of the i-th first device.

[0103] Figure 4 A flowchart illustrating a model segmentation-based Byzantine robust learning method for training a business model according to an embodiment of this specification, applied to a first device, is shown. Figure 4The flowchart shown is the processing flow for one cycle of training in the Byzantine robust learning process.

[0104] like Figure 4 As shown, the system receives the current business model and a partial weight parameter that is equally distributed among all the weight parameters of the current business model from the second device; it uses the received partial weight parameter to train the received current business model using local data; and it provides the gradient of the model parameters in the locally trained business model to the second device so that the second device can sort all gradients and calculate the weighted average of all gradients after removing extreme values, which is used to update the business model.

[0105] In one example, the Byzantine robust learning method based on model segmentation used to train the business model performs the following steps:

[0106] Step (1) Receive sub-model:

[0107] The first device receives the model sub-part it is assigned from the second device. And began local training.

[0108] Step (2) Local training process:

[0109] The first device trains on local data and performs multiple rounds of optimization using gradient descent and adaptive learning rate algorithms (such as Adam) to gradually reduce model loss. The first device updates the model after each training round.

[0110]

[0111] in, Indicates the first device k exist t Updated during the training round p Model parameters for each business sub-model. Indicates the first device k exist t Updated during -1 round of training p Model parameters for each business sub-model. Indicates the first device k In the t -1 training round for sub-models p The loss gradient, This represents the adaptive learning rate, used to adjust the step size for updating model parameters in each training round.

[0112] Step (3) Upload local gradients:

[0113] The first device will calculate the gradient after each round of training. Uploaded to a second device for use in updating the current business model.

[0114] Step (4) Collect all gradients uploaded by the first device:

[0115] The second device receives the uploaded gradients from all the first devices and calculates a weighted gradient based on the historical performance and credibility of the first devices. This stage not only eliminates malicious first devices but also considers the validity of the first devices and their contribution to the current business model.

[0116] Step (5) Weighted aggregation gradient:

[0117] By using a weighted average method combined with the gradients of each first device, the stability of the current business model can be further improved, avoiding interference caused by malicious or abnormal first devices.

[0118] Step (6) Update the business model:

[0119] The second device updates the current business model based on the weighted aggregated gradient, thereby ensuring that the training process of all first devices is optimized toward a consistent goal and is not affected by Byzantine attacks.

[0120] Figure 5 A block diagram of a model segmentation-based Byzantine robust learning system on the first device side according to an embodiment of this specification is shown. Figure 3 As shown, the Byzantine robust learning system based on model segmentation includes: a receiving unit, a model training unit, and a model parameter providing unit.

[0121] like Figure 5 As shown, the receiving unit is configured to receive the current service model and a portion of the weight parameters equally distributed by all the weight parameters of the current service model from the second device;

[0122] The model training unit is configured to train the received current business model using local data with the received partial weight parameters;

[0123] The model parameter providing unit is configured to provide the gradients of the model parameters in the locally trained business model to the second device, so that the second device can sort all gradients and calculate the weighted average of all gradients after removing extreme values, which is used to update the business model.

[0124] According to one embodiment, a computer-readable storage medium is provided having a computer program stored thereon that, when executed by a processor, implements the steps in the above-described model segmentation-based Byzantine robust learning method.

[0125] According to one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps in the above-described model segmentation-based Byzantine robust learning method.

[0126] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of hardware embodiments, software embodiments, or embodiments combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage and optical storage) containing computer-usable program code.

[0127] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0128] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0129] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0130] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. The storage medium can be a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM), etc.

[0131] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. Byzantine robust learning methods based on model segmentation, including: A second device and at least two first devices train a business model, each first device having local data, and the second device maintains the trained business model. The method is applied to the second device, characterized in that the method includes: The current business model and the partial weight parameters that are equally distributed among all the weight parameters of the current business model are sent to each first device, so that the first device can use the partial weight parameters to train the current business model using local data. The system receives gradients from each first device after the training of its current business model. After sorting all gradients, it calculates the weighted average of all gradients after removing extreme values, which is used to update the current business model. The updated business model serves as the current business model for the next round of model training. By dividing the weight vector evenly, the system avoids excessive computational burden on some first devices due to uneven weight distribution. Wherein, the business model is applied to the application scenario of predicting business risks of business transactions occurring on the business system, and the business model is implemented as a business risk prediction model; based on the business model, business risk prediction is performed on the business transaction feature data of the business transaction; the local data of the first device includes business data collected locally by each member device, the business data includes feature data of business objects, the business objects include users; the business data includes user feature data collected locally; the business data includes text data; The weight parameters that are equally distributed among all weight parameters of the current business model include: Convert all weight parameters of the current business model into one-dimensional vectors, arrange them according to the training order, and then merge them to obtain the total weight vector. The total weight vector is divided into several equal parts, and the weight vector of each part is restored to obtain partial weight parameters. During the first leveling, record the start position, end position, length, and offset of each layer. During the reconstruction, use the reshape function or view function to reshape the vector based on the previously recorded offset and shape information, and restore the one-dimensional vector to the multi-dimensional tensor of the corresponding layer. Assume the weights of the initial business model are ω 0, the second device divides it into p The weight of each part is... =( + +...+ ) / p That is, the initial weights of each sub-model are the average values ​​of the initial models; Based on the computing power and historical performance of the first device, different parts of the model are dynamically allocated; the first device with high computing power will receive more complex parts of the model, and all first devices will receive the segmented parts of the model through a random number generator and begin training. Assumption Representing the k Local data held by the first device, among which During training, the second device selects a subset of users from all customers to participate in each round of training, thereby obtaining model parameters to minimize the loss function. Finally, the second device collects the parameters from each participating customer and performs a weighted average, as shown in the following formula: in, It is the first k The parameters of the model trained on the first device. w These are the model parameters after aggregation of the second device. , D It is the total data sample size of all the first devices. Indicates the first k The weight of the first device among all the first devices, measuring the weight of the first device. k The proportion of the first device in global aggregation; the optimization process of the loss function is expressed by the following formula: in, This represents the optimal business model parameters. Indicates the first i The weight factor of the first device in the global optimization. L (·) is the first i The loss function of the first device This represents the weighted average of all first-level devices. Indicates the first i Data sample size of the first device; Each first device performs a local update process, training using local data and model parameters. During local training, the first device uses the Adam adaptive learning rate algorithm to adjust the learning rate, ensuring smoother updates in each training round. Then, the first device calculates the gradient according to the current gradient formula. And update the local model parameters based on the adaptive learning rate: ;in, Indicates the first device k exist t Updated during the training round p Model parameters for each business sub-model. Indicates the first device k exist t Updated during -1 round of training p Model parameters for each business sub-model. Indicates the first device k In the t -1 training round for sub-models p The loss gradient, This represents the adaptive learning rate, used to adjust the step size for updating model parameters in each training round; Each first device calculates the update gradient of its local model. And upload these gradients to the second device; The second device receives gradients from all the first devices. The received gradients are first sorted, and a confidence-based weighted average strategy is used to process the gradients to ensure the stability of the current business model. In the t In round-robin communication, the second device will send parameters Distributed to the first device, each first device i Perform multiple local training operations on local data to obtain updated local parameters. Then, the first device i Calculate the local gradient And send it to the second device; in, Indicates the first t Parameters of the business model; Finally, the second device collects local gradients and updates the business model using the average gradient; in, Indicates the first t The gradient for each training round, without specifying any particular device; Throughout the training process of federated learning, n first devices collaborate to learn the machine learning model, and with the help of second devices, obtain the global optimal solution of the model; The received gradients are weighted, and each gradient of the first device is assigned a confidence weight. The historical performance of the first device and the confidence of the current gradient will determine the weight of its uploaded gradient in the business model update. In this way, the gradient of the first device with high confidence will receive more attention. Assuming the number of malicious primary devices is finite, the total number is... n Among the first group of equipment, there are f One was identified as the first Byzantine device; employing a set B⊆[ n To identify these first Byzantine devices, and the collection M = [ n ]\ B Then it is used to refer to the remaining honest first device; in the context of the existence of the Byzantine first device, the first... t During round communication, the first device i The submitted gradient information can be expressed as follows: in, For Byzantine equipment, M For the remaining non-Byzantine equipment, Indicated as the first device i In the t Gradient information updated during each training round. Indicates the first t Parameters of the round business model, Indicates the first device i exist t Parameters generated after local training; The average of all gradients after removing extreme values ​​is calculated using the following formula: in, This represents the average of all gradients after removing extreme values, where n represents the number of first-order devices and f represents the number of Byzantine first-order devices. Let represent the model gradient of the i-th first device.

2. A Byzantine robust learning system based on model segmentation, including: A second device and at least two first devices train a business model, each first device having local data, and the second device maintains the trained business model. The system is applied to the second device, characterized in that the system comprises: The sending unit sends the current business model and a portion of the weight parameters equally distributed among all the weight parameters of the current business model to each first device, so that the first device can use the portion of the weight parameters to train the current business model using local data. The model update unit receives the gradients from each first device after the training of its current business model. After sorting all gradients, it calculates the weighted average of all gradients after removing extreme values, which is used to update the current business model. The updated business model serves as the current business model for the next round of model training. The transmitting unit includes: Convert all weight parameters of the current business model into one-dimensional vectors, arrange them according to the training order, and then merge them to obtain the total weight vector. The total weight vector is divided into several equal parts, and the weight vector of each part is restored to obtain partial weight parameters. During the first leveling, record the start position, end position, length, and offset of each layer. During the reconstruction, use the reshape function or view function to reshape the vector based on the previously recorded offset and shape information, and restore the one-dimensional vector to the multi-dimensional tensor of the corresponding layer. Assume the weights of the initial business model are ω 0, the second device divides it into p The weight of each part is... =( + +...+ ) / p That is, the initial weights of each sub-model are the average values ​​of the initial models; Based on the computing power and historical performance of the first device, different parts of the model are dynamically allocated; the first device with high computing power will receive more complex parts of the model, and all first devices will receive the segmented parts of the model through a random number generator and begin training. Assumption Representing the k Local data held by the first device, among which During training, the second device selects a subset of users from all customers to participate in each round of training, thereby obtaining model parameters to minimize the loss function. Finally, the second device collects the parameters from each participating customer and performs a weighted average, as shown in the following formula: in, It is the first k The parameters of the model trained on the first device. w These are the model parameters after aggregation of the second device. , D It is the total data sample size of all the first devices. Indicates the first k The weight of the first device among all the first devices, measuring the weight of the first device. k The proportion of the first device in global aggregation; the optimization process of the loss function is expressed by the following formula: in, This represents the optimal business model parameters. Indicates the first i The weight factor of the first device in the global optimization. L (·) is the first i The loss function of the first device This represents the weighted average of all first-level devices. Indicates the first i Data sample size of the first device; Each first device performs a local update process, training using local data and model parameters. During local training, the first device uses the Adam adaptive learning rate algorithm to adjust the learning rate, ensuring smoother updates in each training round. Then, the first device calculates the gradient according to the current gradient formula. And update the local model parameters based on the adaptive learning rate: ;in, Indicates the first device k exist t Updated during the training round p Model parameters for each business sub-model. Indicates the first device k exist t Updated during -1 round of training p Model parameters for each business sub-model. Indicates the first device k In the t -1 training round for sub-models p The loss gradient, This represents the adaptive learning rate, used to adjust the step size for updating model parameters in each training round; Each first device calculates the update gradient of its local model. And upload these gradients to the second device; The second device receives gradients from all the first devices. The received gradients are first sorted, and a confidence-based weighted average strategy is used to process the gradients to ensure the stability of the current business model. In the t In round-robin communication, the second device will send parameters Distributed to the first device, each first device i Perform multiple local training operations on local data to obtain updated local parameters. Then, the first device i Calculate the local gradient And send it to the second device; in, Indicates the first t Parameters of the business model; Finally, the second device collects local gradients and updates the business model using the average gradient; in, Indicates the first t The gradient for each training round, without specifying any particular device; Throughout the training process of federated learning, n first devices collaborate to learn the machine learning model, and with the help of second devices, obtain the global optimal solution of the model; The received gradients are weighted, and each gradient of the first device is assigned a confidence weight. The historical performance of the first device and the confidence of the current gradient will determine the weight of its uploaded gradient in the business model update. In this way, the gradient of the first device with high confidence will receive more attention. Assuming the number of malicious primary devices is finite, the total number is... n Among the first group of equipment, there are f One was identified as the first Byzantine device; employing a set B⊆[ n To identify these first Byzantine devices, and the collection M = [ n ]\ B Then it is used to refer to the remaining honest first device; in the context of the existence of the Byzantine first device, the first... t During round communication, the first device i The submitted gradient information can be expressed as follows: in, For Byzantine equipment, M For the remaining non-Byzantine equipment, Indicated as the first device i In the t Gradient information updated during each training round. Indicates the first t Parameters of the round business model, Indicates the first device i exist t Parameters generated after local training; The average of all gradients after removing extreme values ​​is calculated using the following formula: in, This represents the average of all gradients after removing extreme values, where n represents the number of first-order devices and f represents the number of Byzantine first-order devices. Let represent the model gradient of the i-th first device.

3. Byzantine robust learning methods based on model segmentation, including: A second device and at least two first devices train a business model, each first device having local data, and the second device maintains the trained business model. The method is applied to the first devices, characterized in that the method includes: Receive the current business model and the partial weight parameters that are equally distributed among all the weight parameters of the current business model from the second device; Using the received partial weight parameters, train the received current business model with local data; The gradients of the model parameters in the locally trained business model are provided to the second device, which sorts all gradients and calculates the weighted average of all gradients after removing extreme values, which is then used to update the business model. By dividing the weight vector evenly, the computational burden on some first devices due to uneven weight distribution is avoided. Wherein, the business model is applied to the application scenario of predicting business risks of business transactions occurring on the business system, and the business model is implemented as a business risk prediction model; based on the business model, business risk prediction is performed on the business transaction feature data of the business transaction; the local data of the first device includes business data collected locally by each member device, the business data includes feature data of business objects, the business objects include users; the business data includes user feature data collected locally; the business data includes text data; The weight parameters that are equally distributed among all weight parameters of the current business model include: Convert all weight parameters of the current business model into one-dimensional vectors, arrange them according to the training order, and then merge them to obtain the total weight vector. The total weight vector is divided into several equal parts, and the weight vector of each part is restored to obtain partial weight parameters. During the first leveling, record the start position, end position, length, and offset of each layer. During the reconstruction, use the reshape function or view function to reshape the vector based on the previously recorded offset and shape information, and restore the one-dimensional vector to the multi-dimensional tensor of the corresponding layer. Assume the weights of the initial business model are ω 0, the second device divides it into p The weight of each part is... =( + +...+ ) / p That is, the initial weights of each sub-model are the average values ​​of the initial models; Based on the computing power and historical performance of the first device, different parts of the model are dynamically allocated; the first device with high computing power will receive more complex parts of the model, and all first devices will receive the segmented parts of the model through a random number generator and begin training. Assumption Representing the k Local data held by the first device, among which During training, the second device selects a subset of users from all customers to participate in each round of training, thereby obtaining model parameters to minimize the loss function. Finally, the second device collects the parameters from each participating customer and performs a weighted average, as shown in the following formula: in, It is the first k The parameters of the model trained on the first device. w These are the model parameters after aggregation of the second device. , D It is the total data sample size of all the first devices. Indicates the first k The weight of the first device among all the first devices, measuring the weight of the first device. k The proportion of the first device in global aggregation; the optimization process of the loss function is expressed by the following formula: in, This represents the optimal business model parameters. Indicates the first i The weight factor of the first device in the global optimization. L (·) is the first i The loss function of the first device This represents the weighted average of all first-level devices. Indicates the first i Data sample size of the first device; Each first device performs a local update process, training using local data and model parameters. During local training, the first device uses the Adam adaptive learning rate algorithm to adjust the learning rate, ensuring smoother updates in each training round. Then, the first device calculates the gradient according to the current gradient formula. And update the local model parameters based on the adaptive learning rate: ;in, Indicates the first device k exist t Updated during the training round p Model parameters for each business sub-model. Indicates the first device k exist t Updated during -1 round of training p Model parameters for each business sub-model. Indicates the first device k In the t -1 training round for sub-models p The loss gradient, This represents the adaptive learning rate, used to adjust the step size for updating model parameters in each training round; Each first device calculates the update gradient of its local model. And upload these gradients to the second device; The second device receives gradients from all the first devices. The received gradients are first sorted, and a confidence-based weighted average strategy is used to process the gradients to ensure the stability of the current business model. In the t In round-robin communication, the second device will send parameters Distributed to the first device, each first device i Perform multiple local training operations on local data to obtain updated local parameters. Then, the first device i Calculate the local gradient And send it to the second device; in, Indicates the first t Parameters of the business model; Finally, the second device collects local gradients and updates the business model using the average gradient; in, Indicates the first t The gradient for each training round, without specifying any particular device; Throughout the training process of federated learning, n first devices collaborate to learn the machine learning model, and with the help of second devices, obtain the global optimal solution of the model; The received gradients are weighted, and each gradient of the first device is assigned a confidence weight. The historical performance of the first device and the confidence of the current gradient will determine the weight of its uploaded gradient in the business model update. In this way, the gradient of the first device with high confidence will receive more attention. Assuming the number of malicious primary devices is finite, the total number is... n Among the first group of equipment, there are f One was identified as the first Byzantine device; employing a set B⊆[ n To identify these first Byzantine devices, and the collection M = [ n ]\ B Then it is used to refer to the remaining honest first device; in the context of the existence of the Byzantine first device, the first... t During round communication, the first device i The submitted gradient information can be expressed as follows: in, For Byzantine equipment, M For the remaining non-Byzantine equipment, Indicated as the first device i In the t Gradient information updated during each training round. Indicates the first t Parameters of the round business model, Indicates the first device i exist t Parameters generated after local training; The average of all gradients after removing extreme values ​​is calculated using the following formula: in, This represents the average of all gradients after removing extreme values, where n represents the number of first-order devices and f represents the number of Byzantine first-order devices. Let represent the model gradient of the i-th first device.

4. A Byzantine robust learning system based on model segmentation, including: At least two first devices and second devices train a business model, each first device having local data, and the second device maintaining the trained business model. The system is applied to the first devices, characterized in that the system includes: The receiving unit receives the current business model and a portion of the weight parameters equally distributed among all the weight parameters of the current business model from the second device. The model training unit uses the received partial weight parameters and local data to train the received current business model. The model parameter providing unit provides the gradients of the model parameters in the locally trained business model to the second device, so that the second device can sort all gradients and calculate the weighted average of all gradients after removing extreme values, which is used to update the business model; by dividing the weight vector equally, the computational burden on some first devices due to uneven weight distribution is avoided. Wherein, the business model is applied to the application scenario of predicting business risks of business transactions occurring on the business system, and the business model is implemented as a business risk prediction model; based on the business model, business risk prediction is performed on the business transaction feature data of the business transaction; the local data of the first device includes business data collected locally by each member device, the business data includes feature data of business objects, the business objects include users; the business data includes user feature data collected locally; the business data includes text data; The weight parameters that are equally distributed among all weight parameters of the current business model include: Convert all weight parameters of the current business model into one-dimensional vectors, arrange them according to the training order, and then merge them to obtain the total weight vector. The total weight vector is divided into several equal parts, and the weight vector of each part is restored to obtain partial weight parameters. During the first leveling, record the start position, end position, length, and offset of each layer. During the reconstruction, use the reshape function or view function to reshape the vector based on the previously recorded offset and shape information, and restore the one-dimensional vector to the multi-dimensional tensor of the corresponding layer. Assume the weights of the initial business model are ω 0, the second device divides it into p The weight of each part is... =( + +...+ ) / p That is, the initial weights of each sub-model are the average values ​​of the initial models; Based on the computing power and historical performance of the first device, different parts of the model are dynamically allocated; the first device with high computing power will receive more complex parts of the model, and all first devices will receive the segmented parts of the model through a random number generator and begin training. Assumption Representing the k Local data held by the first device, among which During training, the second device selects a subset of users from all customers to participate in each round of training, thereby obtaining model parameters to minimize the loss function. Finally, the second device collects the parameters from each participating customer and performs a weighted average, as shown in the following formula: in, It is the first k The parameters of the model trained on the first device. w These are the model parameters after aggregation of the second device. , D It is the total data sample size of all the first devices. Indicates the first k The weight of the first device among all the first devices, measuring the weight of the first device. k The proportion of the first device in global aggregation; the optimization process of the loss function is expressed by the following formula: in, This represents the optimal business model parameters. Indicates the first i The weight factor of the first device in the global optimization. L (·) is the first i The loss function of the first device This represents the weighted average of all first-level devices. Indicates the first i Data sample size of the first device; Each first device performs a local update process, training using local data and model parameters. During local training, the first device uses the Adam adaptive learning rate algorithm to adjust the learning rate, ensuring smoother updates in each training round. Then, the first device calculates the gradient according to the current gradient formula. And update the local model parameters based on the adaptive learning rate: ;in, Indicates the first device k exist t Updated during the training round p Model parameters for each business sub-model. Indicates the first device k exist t Updated during -1 round of training p Model parameters for each business sub-model. Indicates the first device k In the t -1 training round for sub-models p The loss gradient, This represents the adaptive learning rate, used to adjust the step size for updating model parameters in each training round; Each first device calculates the update gradient of its local model. And upload these gradients to the second device; The second device receives gradients from all the first devices. The received gradients are first sorted, and a confidence-based weighted average strategy is used to process the gradients to ensure the stability of the current business model. In the t In round-robin communication, the second device will send parameters Distributed to the first device, each first device i Perform multiple local training operations on local data to obtain updated local parameters. Then, the first device i Calculate the local gradient And send it to the second device; in, Indicates the first t Parameters of the business model; Finally, the second device collects local gradients and updates the business model using the average gradient; in, Indicates the first t The gradient for each training round, without specifying any particular device; Throughout the training process of federated learning, n first devices collaborate to learn the machine learning model, and with the help of second devices, obtain the global optimal solution of the model; The received gradients are weighted, and each gradient of the first device is assigned a confidence weight. The historical performance of the first device and the confidence of the current gradient will determine the weight of its uploaded gradient in the business model update. In this way, the gradient of the first device with high confidence will receive more attention. Assuming the number of malicious primary devices is finite, the total number is... n Among the first group of equipment, there are f One was identified as the first Byzantine device; employing a set B⊆[ n To identify these first Byzantine devices, and the collection M = [ n ]\ B Then it is used to refer to the remaining honest first device; in the context of the existence of the Byzantine first device, the first... t During round communication, the first device i The submitted gradient information can be expressed as follows: in, For Byzantine equipment, M For the remaining non-Byzantine equipment, Indicated as the first device i In the t Gradient information updated during each training round. Indicates the first t Parameters of the round business model, Indicates the first device i exist t Parameters generated after local training; The average of all gradients after removing extreme values ​​is calculated using the following formula: in, This represents the average of all gradients after removing extreme values, where n represents the number of first-order devices and f represents the number of Byzantine first-order devices. Let represent the model gradient of the i-th first device.

5. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the program implements the steps of the model segmentation-based Byzantine robust learning method as described in any one of claims 1 or 3.

6. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps of the Byzantine robust learning method based on model segmentation as described in any one of claims 1 or the Byzantine robust learning method based on model segmentation as described in claim 3.

Images