Noise data identification method, model training method and related equipment

Abstract

The invention provides a noise data identification method, a model training method and related equipment, and relates to the technical field of data processing, the method comprises the following steps: receiving a first model parameter sent by each training device in N training devices, the first model parameter being a parameter obtained after each training device performs a t-th round of model training; performing feature extraction on the first model parameter based on a first model to obtain a target time domain feature corresponding to each training device; performing feature extraction on the first model parameter based on a second model to obtain a target spatial domain feature corresponding to each training device; aggregating the target time domain feature and the target spatial domain feature to obtain a target feature corresponding to each training device after the t-th round of model training; and analyzing the target features to obtain an analysis result corresponding to each training device. According to the method, the noise data is identified, so that the prediction accuracy of the model is improved.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to a noise data recognition method, a model training method and related equipment. Background Art

[0002] Model training through wide-area computing tasks is a key technology in model training. In related technologies, training data is collected from different devices, and the models are trained separately. The trained model parameters are then merged to enable model training across multiple devices. However, because the training data is distributed across different devices, this technology is susceptible to Progressive Byzantine Attacks (PBA). PBA tampering with the training data through noise data can cause deviations in the model parameters obtained through device training, resulting in lower prediction accuracy for the final trained model.

[0003] It can be seen that in related technologies, there is a problem that the training data is tampered with by noise data, resulting in low model prediction accuracy. Summary of the Invention

[0004] The embodiments of the present invention provide a noise data identification method, a model training method and related equipment to solve the problem in related technologies that training data is tampered with by noise data, resulting in low model prediction accuracy.

[0005] To solve the above problems, the present invention is achieved as follows:

[0006] In a first aspect, an embodiment of the present invention provides a noise data identification method, comprising:

[0007] Receive a first model parameter sent by each of N training devices, where the first model parameter is a parameter obtained by each training device after performing a t-th round of model training, where N is a positive integer greater than 1 and t is a positive integer greater than 1;

[0008] Performing feature extraction on the first model parameters based on the first model to obtain target time-domain features corresponding to each training device, wherein the target time-domain features are used to characterize changes between model parameters obtained from each round of model training for each training device, wherein the first model is a model trained to extract time-domain features;

[0009] Performing feature extraction on the first model parameters based on the second model to obtain target spatial domain features corresponding to each training device, wherein the target spatial domain features are used to characterize the association between the first model parameters of the N training devices after the tth round of model training, wherein the second model is a model trained to extract spatial domain features;

[0010] Aggregating the target time domain features and the target spatial domain features to obtain target features corresponding to each training device after the tth round of model training;

[0011] The target features are analyzed to obtain analysis results corresponding to each training device, where the analysis results are used to characterize whether there is noise data in the first model parameters of each training device.

[0012] In a second aspect, an embodiment of the present invention provides a model training method, comprising:

[0013] Receive a first model parameter sent by each of N training devices, where the first model parameter is a parameter obtained by each training device after performing a t-th round of model training, where N is a positive integer greater than 1 and t is a positive integer greater than 1;

[0014] performing noise data identification on the first model parameters to obtain an analysis result for each training device, wherein the analysis result is used to indicate whether noise data exists in the first model parameters;

[0015] The global model parameters are updated based on the analysis results and the first model parameters, where the global model parameters are the parameters of the model after the t-1th round of model training is completed.

[0016] In a third aspect, an embodiment of the present invention further provides a noise data identification device, comprising:

[0017] A receiving module, configured to receive a first model parameter sent by each of N training devices, where the first model parameter is a parameter obtained by each training device after performing a t-th round of model training, where N is a positive integer greater than 1, and t is a positive integer greater than 1;

[0018] a first extraction module, configured to perform feature extraction on the first model parameters based on a first model to obtain target time-domain features corresponding to each training device, wherein the target time-domain features are used to characterize changes between model parameters obtained in each round of model training for each training device, wherein the first model is a model trained to extract time-domain features;

[0019] a second extraction module, configured to perform feature extraction on the first model parameters based on a second model to obtain target spatial domain features corresponding to each training device, wherein the target spatial domain features are used to characterize the association between the first model parameters of the N training devices after the tth round of model training, wherein the second model is a model trained to extract spatial domain features;

[0020] an aggregation module, configured to aggregate the target time domain features and the target spatial domain features to obtain target features corresponding to each training device after the tth round of model training;

[0021] The analysis module is used to analyze the target features to obtain analysis results corresponding to each training device, and the analysis results are used to characterize whether there is noise data in the first model parameters of each training device.

[0022] In a fourth aspect, an embodiment of the present invention further provides a model training device, comprising:

[0023] A receiving module, configured to receive a first model parameter sent by each of N training devices, where the first model parameter is a parameter obtained by each training device after performing a t-th round of model training, where N is a positive integer greater than 1, and t is a positive integer greater than 1;

[0024] an identification module, configured to perform noise data identification on the first model parameters to obtain an analysis result for each training device, wherein the analysis result is used to indicate whether noise data exists in the first model parameters;

[0025] An updating module is used to update global model parameters based on the analysis results and the first model parameters, where the global model parameters are the parameters of the model after the t-1th round of model training is completed.

[0026] In a fifth aspect, an embodiment of the present invention further provides an electronic device, comprising a transceiver and a processor,

[0027] The transceiver is configured to receive a first model parameter sent by each of N training devices, where the first model parameter is a parameter obtained by each training device after performing a t-th round of model training, where N is a positive integer greater than 1 and t is a positive integer greater than 1;

[0028] The processor is configured to perform feature extraction on the first model parameters based on the first model to obtain target time domain features corresponding to each training device, wherein the target time domain features are used to characterize changes between model parameters obtained by each round of model training for each training device, wherein the first model is a model trained to extract time domain features;

[0029] The processor is further configured to perform feature extraction on the first model parameters based on a second model to obtain a target spatial domain feature corresponding to each training device, wherein the target spatial domain feature is used to characterize the association between the first model parameters of the N training devices after the tth round of model training, wherein the second model is a model trained to extract spatial domain features;

[0030] The processor is further configured to aggregate the target time domain features and the target spatial domain features to obtain target features corresponding to each training device after the tth round of model training;

[0031] The processor is further configured to analyze the target feature to obtain an analysis result corresponding to each training device, wherein the analysis result is used to characterize whether there is noise data in the first model parameter of each training device.

[0032] In a sixth aspect, an embodiment of the present invention further provides an electronic device, including a transceiver and a processor,

[0033] The transceiver is configured to receive a first model parameter sent by each of N training devices, where the first model parameter is a parameter obtained by each training device after performing a t-th round of model training, where N is a positive integer greater than 1 and t is a positive integer greater than 1;

[0034] The processor is configured to perform noise data identification on the first model parameters to obtain an analysis result for each training device, wherein the analysis result is used to indicate whether noise data exists in the first model parameters;

[0035] The processor is further configured to update global model parameters based on the analysis results and the first model parameters, where the global model parameters are parameters of the model after the t-1th round of model training is completed.

[0036] In a seventh aspect, an embodiment of the present invention provides an electronic device, comprising: a processor, a memory, and a program stored in the memory and executable on the processor, wherein when the program is executed by the processor, the steps of the noise data recognition method described in the first aspect are implemented, or the steps of the model training method described in the second aspect are implemented.

[0037] In an eighth aspect, an embodiment of the present invention provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the noise data recognition method described in the first aspect are implemented, or the steps of the model training method described in the second aspect are implemented.

[0038] In a ninth aspect, the present invention further provides a computer program product comprising computer instructions, which, when executed by a processor, implement the steps of the noise data recognition method described in the first aspect, or implement the steps of the model training method described in the second aspect.

[0039] In an embodiment of the present invention, a first model parameter sent by each of N training devices is received, where the first model parameter is a parameter obtained after each training device performs the t-th round of model training, N is a positive integer greater than 1, and t is a positive integer greater than 1; feature extraction is performed on the first model parameter based on the first model to obtain a target time domain feature corresponding to each training device, and the target time domain feature is used to characterize the changes between the model parameters obtained by each round of model training of each training device, and the first model is a model trained to extract time domain features; feature extraction is performed on the first model parameter based on the second model to obtain a target spatial domain feature corresponding to each training device, and the target spatial domain feature is used to characterize the correlation between the first model parameters of the N training devices after the t-th round of model training, and the second model is a model trained to extract spatial domain features; the target time domain feature and the target spatial domain feature are aggregated to obtain the target feature corresponding to each training device after the t-th round of model training; the target feature is analyzed to obtain an analysis result corresponding to each training device, and the analysis result is used to characterize whether there is noise data in the first model parameter of each training device. In this way, the target time domain features are extracted through the LSTM model, the target spatial domain features are extracted through the attention algorithm model, and the target time domain features and the target spatial domain features are aggregated to obtain the target features corresponding to each training device after the tth round of model training. This allows the target features to simultaneously analyze whether the training device is affected by PBA in time and space (that is, whether there is a noise signal in the first parameter model), thereby improving the accuracy of identifying progressive Byzantine attacks and thus improving the prediction accuracy of the trained model. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments of the present invention. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0041] Figure 1 is a flow chart of a noise data identification method provided by an embodiment of the present invention;

[0042] Figure 2 This is a flow chart of a model training method provided by an embodiment of the present invention;

[0043] Figure 3 is a structural diagram of a noise data identification device provided by an embodiment of the present invention;

[0044] Figure 4 is a structural diagram of an electronic device provided by an embodiment of the present invention;

[0045] Figure 5 is a structural diagram of a model training device provided by an embodiment of the present invention;

[0046] Figure 6 This is a structural diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0047] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of them. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0048] It should be noted that for wide-area computing tasks, data is collected from different locations in a wide area, decentralized, cleaned, aggregated, and stored in multiple data centers. Cross-domain collaborative computing, as a distributed computing framework for processing wide-area large-scale data tasks, allows multiple participants (such as data centers, devices, or organizations) to process global transactions with minimal data interaction between participants. For global model training, each participant (such as a training device) can train the model locally, and the server only collects the model parameters (such as gradient updates) of each party, rather than the original data, thereby avoiding the transmission overhead and privacy protection problems caused by data interaction.

[0049] However, distributing the training process across multiple machines inevitably leads to failures and risks, including process crashes, computational errors, and network delays. It also increases the exposed attack surface. In addition to common single-node attacks, Byzantine attacks can simultaneously control multiple nodes, disrupting model training by tampering with local data and modifying uploaded gradients, affecting the model's correctness, robustness, and security.

[0050] Among them, progressive Byzantine attacks (PBAs) are the most insidious of these Byzantine attacks. Specifically, in a progressive Byzantine attack, the attacker, after controlling certain nodes on the training equipment, intermittently and gradually adds noise data to the trained model parameters during each round of training. Traditional Byzantine attack detection methods struggle to identify these changes due to the small magnitude of data changes. However, as global model training progresses and errors accumulate, the noise data misleads the model's training direction, thereby affecting the accuracy of the trained model's predictions.

[0051] Therefore, the embodiments of the present invention provide a noise data identification method, a model training method and related equipment to solve the problem in related technologies that training data is tampered with by noise data, resulting in low model prediction accuracy.

[0052] See Figure 1 , Figure 1 is a flow chart of a noise data identification method provided by an embodiment of the present invention, such as Figure 1 As shown, the following steps are included:

[0053] Step 101: Receive a first model parameter sent by each of N training devices, where the first model parameter is a parameter obtained after each training device performs the tth round (epoch) of model training, where N is a positive integer greater than 1, and t is a positive integer greater than 1.

[0054] The N training devices mentioned above are devices for model training. Each training device trains the model based on the data local to the device. Since multiple rounds of training are required during the model training process, after each round of training, the N training devices will send the model parameters obtained from this round of model training to the parameter server. The parameter server will then identify whether the training device has been PBA-ed. The first model parameters mentioned above are the parameters obtained by each training device after the tth round of model training. The first model parameters of the N training devices can be expressed as: in, The first model parameters of the kth training device in the tth round.

[0055] It should be noted that the parameter server is the device body that executes the noise data identification method. The parameter server is communicated with N training devices. The parameter server can send the current global model parameters to the N training devices, and the N training devices can send the model parameters obtained after each round of training to the parameter server.

[0056] Step 102: Perform feature extraction on the first model parameters based on the first model to obtain target time domain features corresponding to each training device. The target time domain features are used to characterize the changes between the model parameters obtained in each round of model training for each training device. The first model is a model trained to extract time domain features.

[0057] The first model can identify and memorize long-term dependencies between model parameter changes. By extracting temporal variations in model parameters, it can identify PBA over time. In some embodiments, the first model can be a Long Short Term Memory (LSTM) model.

[0058] Among them, the target time domain feature is the feature exhibited by a training device in the model training parameters of different rounds. It should be noted that if the training device is subjected to PBA, the time domain features corresponding to the round of model training after PBA will be different from the time domain features corresponding to the previous round. By extracting the target time domain features of each training device, it is possible to analyze in time whether the training device is subjected to PBA, that is, whether the first model parameter is noise data.

[0059] Step 103: perform feature extraction on the first model parameters to obtain target spatial domain features corresponding to each training device, where the target spatial domain features are used to characterize the correlation between the first model parameters of the N training devices after the tth round of model training.

[0060] The second model can extract the correlation between different training devices (represented by target spatial domain features) and determine whether there is a training device receiving the PDA among the N training devices through the correlation. In some embodiments, the second model can be an attention algorithm model.

[0061] It should be noted that when the training devices are not subject to PBA, the model parameters obtained from training on different training devices should change in the same optimization direction, and the first model parameters of different training devices can show a high correlation. If a training device is subject to PBA, the correlation between the first model parameters of this training device and the first model parameters of other training devices will change significantly. In the present invention, the target spatial domain features of different training devices are extracted through an attention algorithm, thereby determining whether a training device is subject to PBA from the distribution space of N training devices, that is, whether the first model parameters are noise data.

[0062] Step 104: Aggregate the target time domain features and the target spatial domain features to obtain the target features corresponding to each training device after the tth round of model training.

[0063] The above-mentioned target time domain features can be used to analyze in time whether the training device is affected by PBA, and the above-mentioned target spatial domain features can be used to determine whether a training device is affected by PBA from the distribution space of N training devices. By aggregating the target time domain features and the target spatial domain features, the target features corresponding to each training device after the tth round of model training are obtained, so that the target features can analyze in time and space whether the training device is affected by PBA at the same time, thereby improving the recognition accuracy.

[0064] Aggregating the target time domain features and the target spatial domain features may be performed by concatenating the vector representation of the target time domain features and the vector representation of the target spatial domain features to obtain the target features.

[0065] Step 105: Analyze the target features to obtain analysis results corresponding to each training device, where the analysis results are used to characterize whether there is noise data in the first model parameters of each training device.

[0066] The above analysis of the target features to obtain the analysis results corresponding to each training device can be performed by analyzing the target features through a pre-trained recognition model to obtain the analysis results corresponding to each training device.

[0067] The recognition model may be a model constructed by a multilayer perceptron (MLP) model, and the model training is performed by constructing a loss function. Finally, the optimized MLP model (i.e., the recognition model) is used to analyze the target features to obtain the analysis results.

[0068] In an embodiment of the present invention, a first model parameter sent by each of N training devices is received, where the first model parameter is a parameter obtained after each training device performs the t-th round of model training, N is a positive integer greater than 1, and t is a positive integer greater than 1; feature extraction is performed on the first model parameter based on a long short-term memory network (LSTM) model to obtain a target time domain feature corresponding to each training device, and the target time domain feature is used to characterize the changes between the model parameters obtained in each round of model training for each training device; feature extraction is performed on the first model parameter based on an attention algorithm model to obtain a target spatial domain feature corresponding to each training device, and the target spatial domain feature is used to characterize the correlation between the first model parameters of the N training devices after the t-th round of model training; the target time domain feature and the target spatial domain feature are aggregated to obtain the target feature corresponding to each training device after the t-th round of model training; the target feature is analyzed to obtain an analysis result corresponding to each training device, and the analysis result is used to characterize whether there is noise data in the first model parameter of each training device. In this way, the target time domain features are extracted through the LSTM model, the target spatial domain features are extracted through the attention algorithm model, and the target time domain features and the target spatial domain features are aggregated to obtain the target features corresponding to each training device after the tth round of model training. This allows the target features to simultaneously analyze whether the training device is affected by PBA in time and space (that is, whether there is a noise signal in the first parameter model), thereby improving the accuracy of identifying progressive Byzantine attacks and thus improving the prediction accuracy of the trained model.

[0069] In one embodiment, the first model is a long short-term memory network (LSTM) model, and the feature extraction of the first model parameters based on the first model to obtain the target time domain features corresponding to each training device includes:

[0070] Obtaining historical model parameters of each training device, where the historical model parameters are model parameters obtained by each training device after the previous t-1 rounds of model training;

[0071] Predicting the historical model parameters of each training device based on the LSTM model to obtain time domain features corresponding to each round of model training, wherein the time domain features are used to temporally analyze whether each training device is subject to a progressive Byzantine attack;

[0072] The first feature and the first model parameter are predicted based on the LSTM model to obtain the target time domain feature corresponding to each training device, where the first feature is the time domain feature corresponding to the t-1 round of model training of each training device.

[0073] The above historical model parameters are the model parameters obtained by the training device in the previous round of training, which can be expressed as: X k is the historical model parameter of the kth training device, including the parameters obtained from T rounds of model training, Represents the model parameters obtained by the k-th training device in the t-th round of model training.

[0074] It should be noted that LSTM can extract adjustments that change over time. Using the LSTM model, the historical model parameters of each training device are predicted to obtain the time domain features corresponding to each round of model training. In other words, given the historical model parameters, the time domain features corresponding to different rounds of model training can be obtained. The calculation process is expressed as follows:

[0075]

[0076] In the formula, h k t-1 is the time domain feature (i.e., the first feature) corresponding to the model parameters of the kth training device after the t-1th round of training, h k t is the time domain feature (i.e., target time domain feature) corresponding to the model parameters of the kth training device after the tth round of training, W f 、W i 、W c 、W o 、b f 、b i 、b c 、b o The above formula can be used to calculate the target time domain features of the tth round based on the first model parameters of the tth round and the time domain features of the t-1th round.

[0077] It should be noted that if the first model parameter obtained in each round of training is a two-dimensional matrix with a shape of (m,n), it needs to be flattened into a one-dimensional vector, so that the first parameter obtained in each round of model training becomes a one-dimensional vector with a shape of m × n. The first model parameter uploaded by each training device is (t,m × n), so that the parameter server can extract features from the one-dimensional vector using the LSTM model to obtain the target time domain features.

[0078] In an embodiment of the present invention, historical model parameters of each training device are obtained, where the historical model parameters are model parameters obtained after the first t-1 rounds of model training for each training device; the historical model parameters of each training device are predicted based on the LSTM model to obtain time domain features corresponding to each round of model training; the time domain features are used to temporally analyze whether each training device is subject to a progressive Byzantine attack; the first feature and the first model parameter are predicted based on the LSTM model to obtain target time domain features corresponding to each training device, where the first feature is the time domain feature corresponding to the t-1 round of model training for each training device. In this way, the target time domain features corresponding to the historical model parameters and the first model parameters are calculated using the LSTM model.

[0079] In one embodiment, the second model is an attention algorithm model, and the feature extraction of the first model parameters based on the second model to obtain the target spatial domain features corresponding to each training device includes:

[0080] Converting the first model parameters into a query vector, a key vector, and a value vector corresponding to each training device;

[0081] Processing the query vector and the key vector based on an attention algorithm model to obtain an association matrix corresponding to each training device, wherein the association matrix is ​​used to represent the association between each training device;

[0082] The sum of the target products corresponding to other devices among the N training devices is set as the target spatial domain feature corresponding to the target training device, the other devices are training devices other than the target training device among the N training devices, the target product is the product of the association matrix and the value vector corresponding to the other devices, and the target spatial domain feature is used to analyze whether each training device is subject to a progressive Byzantine attack in the distribution space of the N training devices.

[0083] The query vector Q, key vector K, and value vector V are obtained by transforming the first model parameters based on a preset coefficient matrix, which can be expressed by the following formula:

[0084]

[0085] Q in the formula k is the query vector of the kth training device, K k is the key vector of the kth training device, V k is the value vector of the kth training device, is the first model parameter obtained by the kth training device after t rounds of model training, W Q 、W K 、W V is the preset coefficient matrix. The preset coefficient matrix is ​​a trainable linear transformation matrix used to map high-dimensional model parameters to a lower-dimensional vector space.

[0086] The above-mentioned attention algorithm model can be a multi-head attention network model, which is used to calculate the association matrix between different training devices based on the query vector Q, key vector K and value vector V. The calculation process can be expressed by the following formula:

[0087]

[0088] The dot product calculation of the query vector Q and the key vector K reflects the similarity between the first model parameters uploaded by different training devices, and the scaling factor This is to maintain dot product similarity within an appropriate range, preventing excessively high dot product values ​​from affecting model training when the input dimension is too large. After calculating score(Q,K), score(Q,K) is normalized (for example, using the Softmax function) to obtain the correlation matrix α corresponding to each training device. Each column in the correlation matrix α represents the degree of influence of other training devices on the current training.

[0089] After calculating the correlation matrix corresponding to each training device, the sum of the target products corresponding to the other devices in the N training devices is set as the target spatial domain feature corresponding to the target training device. The specific calculation process can be expressed by the following formula:

[0090]

[0091] Output in the formula k is the target spatial feature of the kth training device, α k,j is the correlation matrix of the jth training device among N training devices, V j is the value vector of the jth training device.

[0092] In an embodiment of the present invention, the first model parameters are converted into a query vector, a key vector, and a value vector corresponding to each training device; the query vector and the key vector are processed based on the attention algorithm model to obtain an association matrix corresponding to each training device, and the association matrix is ​​used to characterize the association between each training device; the sum of the target products corresponding to the other devices in the N training devices is set as the target spatial domain feature corresponding to the target training device, the other devices are training devices other than the target training device in the N training devices, the target product is the product of the association matrix and the value vector corresponding to the other devices, and the target spatial domain feature is used to analyze whether each training device is subject to a progressive Byzantine attack in the distribution space of the N training devices. In this way, the target spatial domain feature of each training device is extracted and obtained through the attention algorithm model, and then it is possible to analyze whether each training device is subject to a progressive Byzantine attack in the distribution space of the N training devices.

[0093] See Figure 2 , Figure 2 This is a flow chart of a model training method provided by an embodiment of the present invention. Figure 2 As shown, the following steps are included:

[0094] Step 201: Receive a first model parameter sent by each of N training devices, where the first model parameter is a parameter obtained by each training device after the t-th round of model training, where N is a positive integer greater than 1, and t is a positive integer greater than 1;

[0095] Step 202: Perform noise data identification on the first model parameters to obtain analysis results for each training device, wherein the analysis results are used to indicate whether the first model parameters contain noise data.

[0096] Step 203: Update global model parameters based on the analysis results and the first model parameters, where the global model parameters are the parameters of the model after the t-1th round of model training is completed.

[0097] In an embodiment of the present invention, a first model parameter sent by each of N training devices is received, where the first model parameter is a parameter obtained after each training device performs the t-th round of model training, where N is a positive integer greater than 1, and t is a positive integer greater than 1; noise data identification is performed on the first model parameter to obtain an analysis result of each training device, where the analysis result is used to characterize whether noise data exists in the first model parameter; and global model parameters are updated based on the analysis result and the first model parameter, where the global model parameter is a parameter of the model after the t-1-th round of model training is completed. In this way, by performing noise data identification on the first model parameter to obtain an analysis result to determine whether noise data exists in the first model parameter; and then updating the global model parameter based on the analysis result and the first model parameter, the interference of the first model parameter with noise data on the global model parameter can be effectively eliminated, thereby improving the prediction accuracy of the trained model.

[0098] The analysis results include two situations: the presence of noise data and the absence of noise data. These two situations need to be processed differently.

[0099] In some embodiments, updating the global model parameters based on the analysis result and the first model parameters includes: updating the global model parameters based on the first model parameters when the analysis result of the target training device indicates that the first model parameters do not contain noise data. That is, when no noise data is present, the global model parameters can be directly updated using the first model parameters and are not affected by the noise data.

[0100] In some embodiments, updating the global model parameters based on the analysis results and the first model parameters includes: when the analysis results of the target training device indicate that the first model parameters contain noise data, updating the global model parameters using the first model parameters of other training devices. That is, when noise data exists, the first model parameters containing noise data may not be directly used, and the global model parameters may be updated using the first model parameters of other training devices without noise data, thereby effectively eliminating the interference of the first model parameters containing noise data on the global model parameters, thereby improving the prediction accuracy of the trained model.

[0101] It should be noted that directly not using the first model parameters with noisy data will result in the loss of all potentially valuable data in the training device. In order to maximize data utilization, when a training device is subjected to a progressive Byzantine attack, since its model parameters are still trained based on local data and still have great value for global model training, variational mode decomposition (VMD) is used to retain the model parameters as much as possible and eliminate the influence of noise data. Among them, VMD, as an adaptive signal decomposition method based on variational optimization, can decompose the input signal into a set of intrinsic mode functions with different frequency characteristics, thereby more accurately identifying and isolating abnormal signals.

[0102] Specifically, in one embodiment, updating the global model parameters based on the analysis results and the first model parameters includes:

[0103] When the analysis result of the target training device indicates that the first model parameters have noise data, performing modal decomposition on the first model parameters of the target training device to obtain k modal parameters, the target training device being a training device among the N training devices;

[0104] Calculating a center frequency parameter corresponding to each of the k modal parameters based on a Lagrangian function model;

[0105] Identifying the k modal parameters based on the center frequency to obtain target modal parameters, where the target modal parameters are modal parameters corresponding to the noise data;

[0106] Merging parameters other than the target modal parameters among the k modal parameters to obtain intermediate model parameters;

[0107] The global model parameters are updated based on the intermediate model parameters.

[0108] In the embodiment of the present invention, the first model parameter is directly As the input signal x(t), the first model parameter is decomposed into k modal parameters u k (t). Different modal parameters have different center frequencies w k , the variational problem can be defined as follows by VMD:

[0109]

[0110] in the formula is the time derivative, * represents the convolution operation, δ(t) is the Dirac function, t represents the round number, K represents the number of modal parameters, and j is a constant. The variational problem can be used to minimize the bandwidth of each modal parameter around its respective center frequency.

[0111] In order to solve the above variational problem, the present invention introduces the Lagrange multiplier λ(t) to constrain the mode decomposition, which can be specifically expressed by the following formula:

[0112]

[0113] In the formula, λ(t) is the introduced Lagrange multiplier. Through the Lagrange function, the Lagrange multiplier method and VMD are used to decompose the original signal (first model parameter) into multiple modes, ensuring that the center frequency and noise characteristics are accurately extracted.

[0114] Among them, the Lagrange multiplier method can use the alternating direction multiplier method (ADMM) to iteratively calculate the modal parameter u in the above formula multiple times. k (t) and the center frequency parameter w k In each iteration, the center frequency parameters of each mode are adjusted to ensure that each part of the first model parameter is decomposed into the corresponding mode, and finally k modal parameters and corresponding center frequency parameters are obtained.

[0115] It should be noted that if a target modal parameter exists among the k modal parameters, the frequency of the target modal parameter is significantly greater or less than the other modal parameters. In some embodiments, a difference threshold can be set to subtract the center frequency parameters of the k modal parameters from each other to obtain k-1 differences between each modal parameter and the other modal parameters. Among the k-1 differences in the noise data, multiple or all differences are greater than the difference threshold. In this way, the target modal parameter is identified through the difference threshold time.

[0116] In other embodiments, the center frequencies of k modal parameters may be aggregated to obtain an aggregated frequency parameter; the difference between the center frequency parameter and the aggregated frequency parameter of each modal parameter is calculated, and the modal parameter with the largest difference is set as the target modal parameter.

[0117] In an embodiment of the present invention, when the analysis result of the target training device indicates that there is noise data in the first model parameter, the first model parameter of the target training device is modally decomposed to obtain k modal parameters, and the target training device is a training device among the N training devices; the center frequency parameter corresponding to each modal parameter of the k modal parameters is calculated based on the Lagrangian function model; the k modal parameters are identified based on the center frequency to obtain target modal parameters, and the target modal parameters are modal parameters corresponding to the noise data; the parameters other than the target modal parameters in the k modal parameters are merged to obtain intermediate model parameters; and the global model parameters are updated based on the intermediate model parameters. In this way, by performing modal decomposition on the first parameter model to obtain k modal parameters and identifying the modal parameters corresponding to the noise data, the modal parameters corresponding to the noise data can be filtered out to obtain intermediate model parameters, and then the global model parameters are updated based on the intermediate model parameters, which will not be affected by the noise data, retain valuable part of the data, and improve data utilization.

[0118] In some implementations, the updating of the global model parameters based on the intermediate model parameters may directly set the intermediate model parameters as global model parameters.

[0119] In some other embodiments, updating the global model parameters based on the intermediate model parameters includes:

[0120] Calculating the similarity between the intermediate model parameters and the global model parameters;

[0121] The product of the similarity and the intermediate model parameter is set as the updated global model parameter.

[0122] In this embodiment of the present invention, similarity is calculated between the intermediate model parameters and the global model parameters; the product of the similarity and the intermediate model parameters is set as the updated global model parameters. This dynamically adjusts the intermediate model parameters based on similarity, improving the accuracy and robustness of the updated global model parameters.

[0123] Specifically, the first model parameter uploaded by the kth training device in the tth round is After using VMD to separate the noise of these parameters, the intermediate model parameters corresponding to the training device are obtained. In order to ensure that the model aggregation fully considers the contribution of each training device, the dynamic weight of each training is calculated by similarity, and then the global model parameters are updated. Let the current global model parameter be θ t , then the similarity of the kth training device It can be calculated by the following formula:

[0124]

[0125] The similarity of each training device is calculated by the above formula, and the similarity is used as a dynamic weight. The similarity of each training device in different rounds is different, and the intermediate model parameters of each round can be adjusted by using the similarity as a dynamic weight.

[0126] In some embodiments, when the first model parameters of multiple training devices are subject to PBA, the first model parameters of the multiple training devices can be converted into intermediate model parameters using the method of the present invention, and the similarity corresponding to the intermediate model parameters of each training device is calculated to obtain multiple products, which are then summed to update the global model parameters. This is specifically expressed as follows:

[0127]

[0128] In the formula, θ t+1 is the updated global parameter.

[0129] In one embodiment, before receiving the first model parameters sent by each of the N training devices, the method further includes:

[0130] Sending initial global model parameters to the N training devices, where the N training devices are used to perform model training on the initial model including the initial global model parameters;

[0131] The initial global model parameters include initial values ​​of multiple sub-parameters, and the initial values ​​of the multiple sub-parameters are randomly generated based on Gaussian distribution within a preset range.

[0132] It should be noted that before model training, the parameter server needs to generate the initial global model parameters. The initial global model parameters are sent to each training device so that each training recognition can perform model training based on the initial global model parameters.

[0133] The initial global model parameters are a vector or tensor that represents the set of all sub-parameters of the model. For example, in a deep learning model, the initial global model parameters include sub-parameters such as weights and biases, filter weights of convolutional layers, and weight matrices of fully connected layers. The initial global model parameters can be expressed as: Where d represents the total number of sub-parameters.

[0134] In some embodiments, the initial global model parameters are typically randomly assigned weights to ensure that each training device has the same starting point when training the model in the initial stage. Specifically, each sub-parameter is randomly generated according to a Gaussian distribution with a mean of 0 and a variance of 0.01 during initialization, which can be expressed as follows:

[0135]

[0136] In one embodiment, after updating the global model parameters based on the analysis results and the first model parameters, the method further includes:

[0137] Sending updated global model parameters to the N training devices.

[0138] In an embodiment of the present invention, the updated global model parameters are sent to the N training devices, so that each training device can perform the next round of model training according to the updated global model parameters.

[0139] Furthermore, after sending the updated global model parameters to N training devices, each training device performs model training based on the local dataset. k Including k training data, which can be expressed as: D k =(x1,y1),(x2,y2),…,(x nk ,y nk ), x i is the input feature, y i The training device constructs a cross entropy loss function and uses its local dataset D through gradient descent or its variants. k Perform training and update model parameters. Specifically expressed by the following formula:

[0140]

[0141] The first model parameters of the kth training device in the tth round, is the global model parameter of the t-1 round, η is the learning rate, is the gradient of the loss function.

[0142] In this way, multiple rounds of model training and global model parameter updates are achieved through the above formula and global model parameters.

[0143] See Figure 3 , Figure 3 is a structural diagram of a noise data identification device provided by an embodiment of the present invention, such as Figure 3 As shown, the noise data identification device 300 includes:

[0144] A receiving module 301 is configured to receive a first model parameter sent by each of N training devices, where the first model parameter is a parameter obtained by each training device after performing a t-th round of model training, where N is a positive integer greater than 1 and t is a positive integer greater than 1;

[0145] A first extraction module 302 is configured to perform feature extraction on the first model parameters based on the first model to obtain target time-domain features corresponding to each training device, wherein the target time-domain features are used to characterize changes between model parameters obtained in each round of model training for each training device, wherein the first model is a model trained to extract time-domain features;

[0146] A second extraction module 303 is configured to perform feature extraction on the first model parameters based on a second model to obtain target spatial features corresponding to each training device, wherein the target spatial features are used to characterize the association between the first model parameters of the N training devices after the tth round of model training, wherein the second model is a model trained to extract spatial features;

[0147] Aggregation module 304, configured to aggregate the target time domain features and the target spatial domain features to obtain target features corresponding to each training device after the tth round of model training;

[0148] The analysis module 305 is used to analyze the target features to obtain analysis results corresponding to each training device, where the analysis results are used to characterize whether there is noise data in the first model parameters of each training device.

[0149] In one embodiment, the first model is a long short-term memory network (LSTM) model, and the first extraction module 302 includes:

[0150] An acquisition unit, configured to acquire historical model parameters of each training device, wherein the historical model parameters are model parameters obtained by each training device after the previous t-1 rounds of model training;

[0151] a first prediction unit, configured to predict the historical model parameters of each training device based on the LSTM model to obtain time domain features corresponding to each round of model training, wherein the time domain features are used to temporally analyze whether each training device is subject to a progressive Byzantine attack;

[0152] The second prediction unit is used to predict the first feature and the first model parameter based on the LSTM model to obtain the target time domain feature corresponding to each training device, where the first feature is the time domain feature corresponding to each training device in the t-1th round of model training.

[0153] In one embodiment, the second model is an attention algorithm model, and the second extraction module 303 includes:

[0154] a conversion unit, configured to convert the first model parameters into a query vector, a key vector, and a value vector corresponding to each training device;

[0155] a processing unit, configured to process the query vector and the key vector based on an attention algorithm model to obtain an association matrix corresponding to each training device, wherein the association matrix is ​​used to represent an association between each training device;

[0156] A computing unit is used to set the sum of target products corresponding to other devices among the N training devices as the target spatial domain feature corresponding to the target training device, wherein the other devices are training devices other than the target training device among the N training devices, and the target product is the product of the association matrix and the value vector corresponding to the other devices. The target spatial domain feature is used to analyze whether each training device is subject to a progressive Byzantine attack in the distribution space of the N training devices.

[0157] The noise data identification device provided in the embodiment of the present invention is capable of implementing the various processes of the various embodiments of the above-mentioned noise data identification method. The technical features correspond one to one and can achieve the same technical effects. To avoid repetition, they will not be described here.

[0158] It should be noted that the noise data identification device in the embodiment of the present invention may be a device, or a component, integrated circuit, or chip in an electronic device.

[0159] An embodiment of the present invention further provides an electronic device, comprising: a processor, a memory, and a program stored in the memory and executable on the processor. When the program is executed by the processor, the various processes of the above-mentioned noise data recognition method embodiment are implemented, and the same technical effects can be achieved. To avoid repetition, they are not described here.

[0160] For details, see Figure 4 As shown, an embodiment of the present invention further provides an electronic device, including a bus 401 , a transceiver 402 , an antenna 403 , a bus interface 404 , a processor 405 and a memory 406 .

[0161] The transceiver 402 is configured to receive a first model parameter sent by each of the N training devices, where the first model parameter is a parameter obtained by each training device after the t-th round of model training, where N is a positive integer greater than 1 and t is a positive integer greater than 1;

[0162] The processor 405 is configured to perform feature extraction on the first model parameters based on the first model to obtain target time domain features corresponding to each training device, wherein the target time domain features are used to characterize changes between model parameters obtained in each round of model training for each training device, wherein the first model is a model trained to extract time domain features;

[0163] The processor 405 is further configured to perform feature extraction on the first model parameters based on the second model to obtain target spatial domain features corresponding to each training device, wherein the target spatial domain features are used to characterize the association between the first model parameters of the N training devices after the tth round of model training, and the second model is a model trained to extract spatial domain features;

[0164] The processor 405 is further configured to aggregate the target time domain features and the target spatial domain features to obtain target features corresponding to each training device after the tth round of model training;

[0165] The processor 405 is further configured to analyze the target feature to obtain an analysis result corresponding to each training device, where the analysis result is used to characterize whether there is noise data in the first model parameter of each training device.

[0166] In one embodiment, the feature extraction of the first model parameters based on the long short-term memory network (LSTM) model to obtain the target time domain features corresponding to each training device includes:

[0167] Obtaining historical model parameters of each training device, where the historical model parameters are model parameters obtained by each training device after the previous t-1 rounds of model training;

[0168] Predicting the historical model parameters of each training device based on the LSTM model to obtain time domain features corresponding to each round of model training, wherein the time domain features are used to temporally analyze whether each training device is subject to a progressive Byzantine attack;

[0169] The first feature and the first model parameter are predicted based on the LSTM model to obtain the target time domain feature corresponding to each training device, where the first feature is the time domain feature corresponding to the t-1 round of model training of each training device.

[0170] In one embodiment, the feature extraction of the first model parameters based on the attention algorithm model to obtain the target spatial domain features corresponding to each training device includes:

[0171] Converting the first model parameters into a query vector, a key vector, and a value vector corresponding to each training device;

[0172] Processing the query vector and the key vector based on an attention algorithm model to obtain an association matrix corresponding to each training device, wherein the association matrix is ​​used to represent the association between each training device;

[0173] The sum of the target products corresponding to other devices among the N training devices is set as the target spatial domain feature corresponding to the target training device, the other devices are training devices other than the target training device among the N training devices, the target product is the product of the association matrix and the value vector corresponding to the other devices, and the target spatial domain feature is used to analyze whether each training device is subject to a progressive Byzantine attack in the distribution space of the N training devices.

[0174] exist Figure 4 In the embodiment, the bus architecture (represented by bus 401) is shown. Bus 401 may include any number of interconnected buses and bridges. Bus 401 links together various circuits including one or more processors represented by processor 405 and memory represented by memory 406. Bus 401 may also link together various other circuits such as peripheral devices, voltage regulators, and power management circuits, which are well known in the art and are therefore not described further herein. Bus interface 404 provides an interface between bus 401 and transceiver 402. Transceiver 402 may be one element or multiple elements, such as multiple receivers and transmitters, providing a unit for communicating with various other devices on a transmission medium. Data processed by processor 405 is transmitted on a wireless medium via antenna 403. Furthermore, antenna 403 receives data and transmits the data to processor 405.

[0175] Processor 405 is responsible for managing bus 401 and general processing, and may also provide various functions, including timing, peripheral interfaces, voltage regulation, power management, and other control functions. Memory 406 may be used to store data used by processor 405 when performing operations.

[0176] Optionally, the processor 405 may be a CPU, an ASIC, an FPGA, or a CPLD.

[0177] An embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon. When executed by a processor, the computer program implements the various processes of the aforementioned noise data identification method embodiment and achieves the same technical effects. To avoid repetition, the details are not described here. The computer-readable storage medium may be, for example, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0178] The present invention also provides a computer program product, comprising computer instructions, which, when executed by a processor, implement the above Figure 1 The various processes corresponding to the embodiments of the noise data identification method can achieve the same technical effects, and will not be described again here to avoid repetition.

[0179] See Figure 5 , Figure 5 is a structural diagram of a noise data identification device provided by an embodiment of the present invention, such as Figure 5 As shown, the noise data identification device 500 includes:

[0180] A receiving module 501 is configured to receive a first model parameter sent by each of N training devices, where the first model parameter is a parameter obtained by each training device after performing a t-th round of model training, where N is a positive integer greater than 1 and t is a positive integer greater than 1;

[0181] an identification module 502 for performing noise data identification on the first model parameters to obtain an analysis result for each training device, wherein the analysis result is used to indicate whether the first model parameters have noise data;

[0182] The updating module 503 is used to update the global model parameters based on the analysis results and the first model parameters, where the global model parameters are the parameters of the model after the t-1th round of model training is completed.

[0183] In one embodiment, the updating module 503 includes:

[0184] a decomposition unit, configured to perform modal decomposition on the first model parameters of the target training device to obtain k modal parameters when the analysis result of the target training device indicates that the first model parameters have noise data, and the target training device is a training device among the N training devices;

[0185] A calculation unit, configured to calculate a center frequency parameter corresponding to each of the k modal parameters based on a Lagrangian function model;

[0186] an identification unit, configured to identify the k modal parameters based on the center frequency to obtain target modal parameters, where the target modal parameters are modal parameters corresponding to the noise data;

[0187] a merging unit, configured to merge parameters other than the target modal parameters among the k modal parameters to obtain intermediate model parameters;

[0188] A first updating unit is configured to update the global model parameters based on the intermediate model parameters.

[0189] In one embodiment, the first updating unit includes:

[0190] a calculation subunit, configured to calculate the similarity between the intermediate model parameters and the global model parameters;

[0191] The updating subunit is configured to set the product of the similarity and the intermediate model parameter as the updated global model parameter.

[0192] In one embodiment, the updating module 503 includes:

[0193] The second updating unit is configured to update the global model parameters based on the first model parameters when the analysis result of the target training device indicates that the first model parameters do not have noise data.

[0194] In one embodiment, the noise data identification device 500 further includes:

[0195] A second sending module is used to send the initial global model parameters to the N training devices, and the N training devices are used to perform model training on the initial model including the initial global model parameters;

[0196] The initial global model parameters include initial values ​​of multiple sub-parameters, and the initial values ​​of the multiple sub-parameters are randomly generated based on Gaussian distribution within a preset range.

[0197] In one embodiment, the noise data identification device 500 further includes:

[0198] The third sending module is used to send the updated global model parameters to the N training devices.

[0199] The model training device provided in the embodiment of the present invention is capable of implementing each process of each embodiment of the above-mentioned model training method. The technical features correspond one to one and can achieve the same technical effects. To avoid repetition, they will not be described here.

[0200] It should be noted that the model training device in the embodiment of the present invention can be a device, or a component, integrated circuit, or chip in an electronic device.

[0201] An embodiment of the present invention also provides an electronic device, comprising: a processor, a memory, and a program stored in the memory and runnable on the processor. When the program is executed by the processor, the various processes of the above-mentioned model training method embodiment are implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0202] For details, see Figure 6As shown, an embodiment of the present invention further provides an electronic device, including a bus 601 , a transceiver 602 , an antenna 603 , a bus interface 604 , a processor 605 and a memory 606 .

[0203] The transceiver 602 is configured to receive a first model parameter sent by each of the N training devices, where the first model parameter is a parameter obtained by each training device after the t-th round of model training, where N is a positive integer greater than 1 and t is a positive integer greater than 1;

[0204] The processor 605 is configured to perform noise data identification on the first model parameters to obtain an analysis result for each training device, wherein the analysis result is used to indicate whether the first model parameters have noise data;

[0205] The processor 605 is further configured to update global model parameters based on the analysis results and the first model parameters, where the global model parameters are parameters of the model after the t-1th round of model training is completed.

[0206] In one embodiment, updating the global model parameters based on the analysis results and the first model parameters includes:

[0207] When the analysis result of the target training device indicates that the first model parameters have noise data, performing modal decomposition on the first model parameters of the target training device to obtain k modal parameters, the target training device being a training device among the N training devices;

[0208] Calculating a center frequency parameter corresponding to each of the k modal parameters based on a Lagrangian function model;

[0209] Identifying the k modal parameters based on the center frequency to obtain target modal parameters, where the target modal parameters are modal parameters corresponding to the noise data;

[0210] Merging parameters other than the target modal parameters among the k modal parameters to obtain intermediate model parameters;

[0211] The global model parameters are updated based on the intermediate model parameters.

[0212] In one embodiment, updating the global model parameters based on the intermediate model parameters includes:

[0213] Calculating the similarity between the intermediate model parameters and the global model parameters;

[0214] The product of the similarity and the intermediate model parameter is set as the updated global model parameter.

[0215] In one embodiment, updating the global model parameters based on the analysis results and the first model parameters includes:

[0216] In a case where the analysis result of the target training device indicates that the first model parameters do not have noise data, the global model parameters are updated based on the first model parameters.

[0217] In one embodiment, the transceiver 602 is further configured to send initial global model parameters to the N training devices, and the N training devices are configured to perform model training on the initial model including the initial global model parameters;

[0218] The initial global model parameters include initial values ​​of multiple sub-parameters, and the initial values ​​of the multiple sub-parameters are randomly generated based on Gaussian distribution within a preset range.

[0219] In one embodiment, the transceiver 602 is further configured to send updated global model parameters to the N training devices.

[0220] exist Figure 6 In the embodiment, the bus architecture (represented by bus 601) is shown. Bus 601 may include any number of interconnected buses and bridges. Bus 601 links together various circuits including one or more processors represented by processor 605 and memory represented by memory 606. Bus 601 may also link together various other circuits such as peripheral devices, voltage regulators, and power management circuits, which are well known in the art and are therefore not described further herein. Bus interface 604 provides an interface between bus 601 and transceiver 602. Transceiver 602 may be one element or multiple elements, such as multiple receivers and transmitters, providing a unit for communicating with various other devices on a transmission medium. Data processed by processor 605 is transmitted on a wireless medium via antenna 603. Furthermore, antenna 603 receives data and transmits the data to processor 605.

[0221] The processor 605 is responsible for managing the bus 601 and general processing, and may also provide various functions, including timing, peripheral interfaces, voltage regulation, power management, and other control functions. The memory 606 may be used to store data used by the processor 605 when performing operations.

[0222] Optionally, the processor 605 may be a CPU, an ASIC, an FPGA, or a CPLD.

[0223] The embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the various processes of the above-mentioned model training method embodiment are implemented, and the same technical effects are achieved. To avoid repetition, the details are not described here. The computer-readable storage medium is, for example, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0224] The present invention also provides a computer program product, comprising computer instructions, which, when executed by a processor, implement the above Figure 2 The various processes of the corresponding model training method embodiments can achieve the same technical effects, so they will not be repeated here to avoid repetition.

[0225] It should be noted that, in this document, the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or apparatus comprising a series of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or apparatus comprising the element.

[0226] Through the description of the above embodiments, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus the necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is a better embodiment. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for enabling a terminal (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in each embodiment of the present invention.

[0227] The embodiments of the present invention are described above in conjunction with the accompanying drawings, but the present invention is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present invention, ordinary technicians in this field can also make many forms without departing from the scope of protection of the present invention and the claims, all of which are protected by the present invention.

Claims

1. A noise data identification method, characterized in that: include: Receive a first model parameter sent by each of N training devices, where the first model parameter is a parameter obtained by each training device after performing a t-th round of model training, where N is a positive integer greater than 1 and t is a positive integer greater than 1; Performing feature extraction on the first model parameters based on the first model to obtain target time-domain features corresponding to each training device, wherein the target time-domain features are used to characterize changes between model parameters obtained from each round of model training for each training device, wherein the first model is a model trained to extract time-domain features; Performing feature extraction on the first model parameters based on the second model to obtain target spatial domain features corresponding to each training device, wherein the target spatial domain features are used to characterize the association between the first model parameters of the N training devices after the tth round of model training, wherein the second model is a model trained to extract spatial domain features; Aggregating the target time domain features and the target spatial domain features to obtain target features corresponding to each training device after the tth round of model training; The target features are analyzed to obtain analysis results corresponding to each training device, where the analysis results are used to characterize whether there is noise data in the first model parameters of each training device.

2. The method according to claim 1, wherein The first model is a long short-term memory network (LSTM) model, and the feature extraction of the first model parameters based on the first model to obtain the target time domain features corresponding to each training device includes: Obtaining historical model parameters of each training device, where the historical model parameters are model parameters obtained by each training device after the previous t-1 rounds of model training; Predicting the historical model parameters of each training device based on the LSTM model to obtain time domain features corresponding to each round of model training, wherein the time domain features are used to temporally analyze whether each training device is subject to a progressive Byzantine attack; The first feature and the first model parameter are predicted based on the LSTM model to obtain the target time domain feature corresponding to each training device, where the first feature is the time domain feature corresponding to the t-1 round of model training of each training device.

3. The method according to claim 1, wherein The second model is an attention algorithm model, and the feature extraction of the first model parameters based on the second model to obtain the target spatial domain features corresponding to each training device includes: Converting the first model parameters into a query vector, a key vector, and a value vector corresponding to each training device; Processing the query vector and the key vector based on an attention algorithm model to obtain an association matrix corresponding to each training device, wherein the association matrix is ​​used to represent the association between each training device; The sum of the target products corresponding to other devices among the N training devices is set as the target spatial domain feature corresponding to the target training device, the other devices are training devices other than the target training device among the N training devices, the target product is the product of the association matrix and the value vector corresponding to the other devices, and the target spatial domain feature is used to analyze whether each training device is subject to a progressive Byzantine attack in the distribution space of the N training devices.

4. A model training method, characterized in that: include: Receive a first model parameter sent by each of N training devices, where the first model parameter is a parameter obtained by each training device after performing a t-th round of model training, where N is a positive integer greater than 1 and t is a positive integer greater than 1; performing noise data identification on the first model parameters to obtain an analysis result for each training device, wherein the analysis result is used to indicate whether noise data exists in the first model parameters; The global model parameters are updated based on the analysis results and the first model parameters, where the global model parameters are the parameters of the model after the t-1th round of model training is completed.

5. The method according to claim 4, wherein The updating of global model parameters based on the analysis result and the first model parameters includes: When the analysis result of the target training device indicates that the first model parameters have noise data, performing modal decomposition on the first model parameters of the target training device to obtain k modal parameters, the target training device being a training device among the N training devices; Calculating a center frequency parameter corresponding to each of the k modal parameters based on a Lagrangian function model; Identifying the k modal parameters based on the center frequency to obtain target modal parameters, where the target modal parameters are modal parameters corresponding to the noise data; Merging parameters other than the target modal parameters among the k modal parameters to obtain intermediate model parameters; The global model parameters are updated based on the intermediate model parameters.

6. The method according to claim 5, wherein The updating of the global model parameters based on the intermediate model parameters includes: Calculating the similarity between the intermediate model parameters and the global model parameters; The product of the similarity and the intermediate model parameter is set as the updated global model parameter.

7. The method according to claim 4, wherein The updating of global model parameters based on the analysis result and the first model parameters includes: In a case where the analysis result of the target training device indicates that the first model parameters do not have noise data, the global model parameters are updated based on the first model parameters.

8. The method according to claim 4, wherein Before receiving the first model parameter sent by each of the N training devices, the method further includes: Sending initial global model parameters to the N training devices, where the N training devices are used to perform model training on the initial model including the initial global model parameters; The initial global model parameters include initial values ​​of multiple sub-parameters, and the initial values ​​of the multiple sub-parameters are randomly generated based on Gaussian distribution within a preset range.

9. The method according to claim 4, wherein After updating the global model parameters based on the analysis results and the first model parameters, the method further includes: Sending updated global model parameters to the N training devices.

10. A noise data identification device, characterized in that: include: A receiving module, configured to receive a first model parameter sent by each of N training devices, where the first model parameter is a parameter obtained by each training device after performing a t-th round of model training, where N is a positive integer greater than 1, and t is a positive integer greater than 1; a first extraction module, configured to perform feature extraction on the first model parameters based on a first model to obtain target time-domain features corresponding to each training device, wherein the target time-domain features are used to characterize changes between model parameters obtained in each round of model training for each training device, wherein the first model is a model trained to extract time-domain features; a second extraction module, configured to perform feature extraction on the first model parameters based on a second model to obtain target spatial domain features corresponding to each training device, wherein the target spatial domain features are used to characterize the association between the first model parameters of the N training devices after the tth round of model training, wherein the second model is a model trained to extract spatial domain features; an aggregation module, configured to aggregate the target time domain features and the target spatial domain features to obtain target features corresponding to each training device after the tth round of model training; The analysis module is used to analyze the target features to obtain analysis results corresponding to each training device, and the analysis results are used to characterize whether there is noise data in the first model parameters of each training device.

11. A model training device, characterized in that: include: A receiving module, configured to receive a first model parameter sent by each of N training devices, where the first model parameter is a parameter obtained by each training device after performing a t-th round of model training, where N is a positive integer greater than 1, and t is a positive integer greater than 1; an identification module, configured to perform noise data identification on the first model parameters to obtain an analysis result for each training device, wherein the analysis result is used to indicate whether noise data exists in the first model parameters; An updating module is used to update global model parameters based on the analysis results and the first model parameters, where the global model parameters are the parameters of the model after the t-1th round of model training is completed.

12. An electronic device, characterized in that: Including transceiver and processor, The transceiver is configured to receive a first model parameter sent by each of N training devices, where the first model parameter is a parameter obtained by each training device after performing a t-th round of model training, where N is a positive integer greater than 1 and t is a positive integer greater than 1; The processor is configured to perform feature extraction on the first model parameters based on the first model to obtain target time domain features corresponding to each training device, wherein the target time domain features are used to characterize changes between model parameters obtained by each round of model training for each training device, wherein the first model is a model trained to extract time domain features; The processor is further configured to perform feature extraction on the first model parameters based on a second model to obtain a target spatial domain feature corresponding to each training device, wherein the target spatial domain feature is used to characterize the association between the first model parameters of the N training devices after the tth round of model training, wherein the second model is a model trained to extract spatial domain features; The processor is further configured to aggregate the target time domain features and the target spatial domain features to obtain target features corresponding to each training device after the tth round of model training; The processor is further configured to analyze the target feature to obtain an analysis result corresponding to each training device, wherein the analysis result is used to characterize whether there is noise data in the first model parameter of each training device.

13. An electronic device, characterized in that: Including transceiver and processor, The transceiver is configured to receive a first model parameter sent by each of N training devices, where the first model parameter is a parameter obtained by each training device after performing a t-th round of model training, where N is a positive integer greater than 1 and t is a positive integer greater than 1; The processor is configured to perform noise data identification on the first model parameters to obtain an analysis result for each training device, wherein the analysis result is used to indicate whether noise data exists in the first model parameters; The processor is further configured to update global model parameters based on the analysis results and the first model parameters, where the global model parameters are parameters of the model after the t-1th round of model training is completed.

14. An electronic device, characterized in that: include: A processor, a memory, and a program stored on the memory and executable on the processor, wherein when the program is executed by the processor, the program implements the steps of the noise data recognition method according to any one of claims 1 to 3, or implements the steps of the model training method according to any one of claims 4 to 9.

15. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the noise data identification method according to any one of claims 1 to 3, or implements the steps of the model training method according to any one of claims 4 to 9.

16. A computer program product, characterized in that The method comprises computer instructions, which, when executed by a processor, implement the steps of the noise data recognition method according to any one of claims 1 to 3, or implement the steps of the model training method according to any one of claims 4 to 9.

Images