A manifold reconstruction based multi-source gradient data processing system

Abstract

The present application relates to computer bottom layer data processing and distributed feature operation technical field, specifically for a kind of multi-source gradient data processing system based on manifold reconstruction;Including: data and knowledge acquisition module, for obtaining the local model gradient update vector uploaded by multi-source data end, knowledge base contains gradient deviation typical mode feature;Ideal manifold reconstruction and simulation module, for generating gradient drift template and gradient anomaly template;Double-track difference feature extraction module, for calculating the theoretical drift deviation and theoretical anomaly deviation between gradient drift template, gradient anomaly template and quantization feature matrix respectively;Situation coupling decision module, for executing the trusted writing of multi-source gradient data or the release or isolated storage instruction of abnormal data rejection;The present application effectively improves the robustness of bottom layer multi-source feature data processing and the reliability of system memory update.

Description

Technical Field

[0001] This invention relates to the field of computer low-level data processing and distributed feature computation technology, specifically a multi-source gradient data processing system based on manifold reconstruction. Background Technology

[0002] In the current context of federated learning security defense, the server cannot directly audit the original training data of the participants, which makes the global model highly susceptible to various malicious attacks or interference from benign data distribution differences during the aggregation process.

[0003] Existing defense solutions are generally based on the statistical outlier detection assumption, which is to identify gradients that deviate significantly from the group mean as attacks by calculating the Euclidean distance or cosine similarity between the model updates of each participant. However, in complex environments such as wireless network security situation awareness, participants often have serious non-independent and identically distributed characteristics. The gradient shift caused by this normal business logic is highly similar to the attack perturbations injected by malicious attacks in terms of statistical characteristics. Traditional threshold determination methods lack a deep understanding of the model evolution mechanism and attack mode knowledge, making it difficult to effectively distinguish between benign drift and malicious attacks in black-box scenarios, resulting in a high false alarm rate and poor robustness in security detection.

[0004] Therefore, how to accurately identify the nature of model updates and improve the system's proactive defense capabilities in complex network environments by reconstructing mathematical mechanisms and comparing multidimensional patterns under the constraint of invisible data has become an urgent technical problem to be solved. Summary of the Invention

[0005] To address the aforementioned technical problems, this invention provides a multi-source gradient data processing system based on manifold reconstruction. Specifically, the technical solution of this invention includes:

[0006] The data and knowledge acquisition module is used to receive high-dimensional local model gradient update vectors from multiple sources, and to retrieve the global model parameters after the previous round of aggregation, as well as the preset model anomaly pattern knowledge base, which contains typical pattern features of gradient deviation.

[0007] The ideal manifold reconstruction and simulation module is used to perform forward inference and loss calculation on a preset reliable verification set based on global model parameters, generate the floating-point theoretical optimal gradient direction vector for the current training stage, and perform adaptive data quantization and compression processing on it through the processor to extract a low-bit-width quantization feature matrix; and generate gradient drift templates representing reliable data distribution shifts and gradient anomaly templates representing abnormal perturbations based on the model anomaly pattern knowledge base.

[0008] The dual-track differential feature extraction module is used to perform low-level tensor difference operations, calculate the actual deviation features between the local model gradient update vector and the quantization feature matrix, and calculate the theoretical drift deviation and theoretical anomaly deviation between the gradient drift template, the gradient anomaly template and the quantization feature matrix respectively.

[0009] The situation coupling decision module is used to calculate the first structural similarity between the actual deviation features and the theoretical drift deviation, and the second structural similarity with the theoretical abnormal deviation in a high-dimensional vector space. Based on the similarity matching results, it executes instructions for reliable writing of multi-source gradient data or removal, release or isolation of abnormal data, thereby completing the filtering of low-level feature data and updating of system memory.

[0010] Preferably, the ideal manifold reconstruction and simulation module generates the theoretically optimal gradient direction, including:

[0011] Input the trusted verification set into the neural network model constructed from the global model parameters;

[0012] Calculate the loss function between the model output and the true label;

[0013] The gradient of the loss function with respect to each parameter of the model is obtained by backpropagation algorithm, which constitutes the theoretically optimal gradient direction.

[0014] Preferably, the ideal manifold reconstruction and simulation module generates gradient drift templates and gradient anomaly templates, including:

[0015] The non-independent and identically distributed offset features of the data in the model's abnormal pattern knowledge base are called to generate a drift factor vector that conforms to a normal distribution offset.

[0016] The drift factor vector is vector-synthesized with the theoretical optimal gradient direction to generate a gradient drift template.

[0017] Call the parameter backdoor perturbation feature or gradient magnitude anomaly feature in the model anomaly pattern knowledge base to generate anomaly factor vector with direction reversal or nonlinear amplification characteristics.

[0018] The anomaly factor vector is synthesized with the theoretically optimal gradient direction to generate a gradient anomaly template.

[0019] Preferably, the dual-track differential feature extraction module calculates the actual deviation features, including:

[0020] Perform element-wise tensor difference operations on the local model gradient update vector and the quantized feature matrix;

[0021] The result vector of the difference operation is used as a feature to reflect the actual deviation of the local calculation process at the multi-source data terminal from the expected optimization path.

[0022] Preferably, the dual-track differential feature extraction module calculates the theoretical drift bias and theoretical anomaly bias, including:

[0023] The tensor difference vector between the gradient drift template and the quantized feature matrix is ​​calculated and defined as the theoretical drift bias.

[0024] The tensor difference vector between the gradient anomaly template and the quantized feature matrix is ​​calculated and defined as the theoretical anomaly deviation.

[0025] Preferably, the situational coupling decision module executes a response based on the similarity matching result, including:

[0026] If the similarity of the first structure is higher than the preset safety threshold and the similarity of the second structure is lower than the safety threshold, then the model update source is determined to be a trusted data offset.

[0027] The corresponding local model gradient update vector is incorporated into the global model aggregation update process.

[0028] Preferably, the situation coupling decision module executes a response based on the similarity matching result, and further includes:

[0029] If the similarity of the second structure is higher than the safety threshold, it is determined that there is a malicious perturbation at the parameter level in the source of the model update;

[0030] The corresponding local model gradient update vector is rejected for use in global model aggregation, and its source participant is marked as a low-reliability node.

[0031] Preferably, the situation coupling decision module executes a response based on the similarity matching result, and further includes:

[0032] If both the first structural similarity and the second structural similarity are below the safety threshold, the model update source is determined to be an undefined abnormal pattern.

[0033] The corresponding local model gradient update vectors are then transferred to an isolated analysis environment for further pattern recognition and attribution analysis.

[0034] Compared with the prior art, the present invention has the following beneficial effects:

[0035] 1. This invention breaks through the limitations of traditional defenses that rely on outlier detection by reconstructing an ideal manifold and simulating a twin template on the server side; without accessing the original data of the participants, it establishes benchmark indicators through mathematical mechanism reconstruction, realizes deep discrimination of the model update properties, and effectively solves the security protection problem under the constraint of data invisibility.

[0036] 2. This invention utilizes a knowledge-based simulation mechanism, enabling the system to simulate non-independent identically distributed data shifts caused by normal business logic and various malicious attack modes. Through dual-track differential comparison, it can accurately identify benign drifts and malicious disturbances with highly similar statistical characteristics, greatly reducing the false alarm rate of security detection and ensuring the accuracy of situational awareness.

[0037] 3. This invention introduces a hierarchical weighted structure similarity matching and dynamic security threshold mechanism to enforce consistency detection covering all network layers. This design prevents attackers from injecting malicious code using hidden layers, ensuring that the defense system still has strong robustness and proactive defense capabilities in complex and ever-changing environments such as wireless network security awareness.

[0038] 4. For abnormal updates of undefined patterns, the system can automatically identify specific attack types such as tag tampering or backdoor implantation by isolating the analysis environment and performing hierarchical energy distribution analysis. This hierarchical response and deep tracing mechanism provides a refined control method for the secure aggregation of the global model, and comprehensively improves the intrinsic security of the artificial intelligence system. Attached Figure Description

[0039] The present invention will be further explained below with reference to the accompanying drawings and embodiments:

[0040] Figure 1 This is a structural diagram of the system of the present invention. Detailed Implementation

[0041] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to specific embodiments.

[0042] Example 1:

[0043] Please see Figure 1 A multi-source gradient data processing system based on manifold reconstruction includes: a data and knowledge acquisition module, used to receive high-dimensional local model gradient update vectors from multiple sources, and retrieve global model parameters after aggregation in the previous round and a preset model anomaly pattern knowledge base, the knowledge base containing typical pattern features of gradient deviation;

[0044] The ideal manifold reconstruction and simulation module is used to perform forward inference and loss calculation on a preset reliable verification set based on global model parameters, generate the floating-point theoretical optimal gradient direction vector for the current training stage, and perform adaptive data quantization and compression processing on it through the processor to extract a low-bit-width quantization feature matrix; and generate gradient drift templates representing reliable data distribution shifts and gradient anomaly templates representing abnormal perturbations based on the model anomaly pattern knowledge base.

[0045] The dual-track differential feature extraction module is used to perform low-level tensor difference operations, calculate the actual deviation features between the local model gradient update vector and the quantization feature matrix, and calculate the theoretical drift deviation and theoretical anomaly deviation between the gradient drift template, the gradient anomaly template and the quantization feature matrix respectively.

[0046] The situation coupling decision module is used to calculate the first structural similarity between the actual deviation features and the theoretical drift deviation, and the second structural similarity with the theoretical abnormal deviation in a high-dimensional vector space. Based on the similarity matching results, it executes instructions for reliable writing of multi-source gradient data or removal, release or isolation of abnormal data, thereby completing the filtering of low-level feature data and updating of system memory.

[0047] This embodiment details the overall architecture and operational logic of the system; the data and knowledge acquisition module, as the system's perception front end, acquires data from various participants through an encrypted communication channel. Local model gradient update vector During this period, the module synchronously retrieves the global model parameters from the previous round of aggregation from the secure storage medium. and a pre-defined model anomaly pattern knowledge base ;

[0048] This knowledge base is derived from historical network attack feature databases and common data distribution statistical feature databases. Physically, it stores a set of parameterized features containing known attack patterns and compliant data offsets.

[0049] The ideal manifold reconstruction and simulation module, acting as the inference engine, does not rely on the original data from the federated learning participants, but rather on the global model parameters held by the server. In trusted verification sets The system performs forward inference to calculate the theoretically optimal gradient direction for the current training phase. The ideal manifold reconstruction mentioned here refers to the use of As a tangent vector, it linearly approximates and simulates the geometric structure of the model evolving along the optimal path within a local neighborhood of the parameter space; utilizing a knowledge base Feature generation gradient drift template With gradient anomaly template ;

[0050] The dual-track differential feature extraction module performs comparative analysis to calculate the features uploaded from the federated learning multi-source data source. With quantized feature matrix Reality deviation characteristics And simultaneously calculate the generated template and Theoretical drift deviation between and theoretical abnormal deviations The situational coupling decision module performs pattern matching in a high-dimensional vector space and calculates... respectively with and The structural similarity is used to determine the nature of the participant's update in response to the degree of matching, and gradient acceptance, rejection or isolation operations are performed accordingly;

[0051] This embodiment constructs an active defense system by introducing ideal manifold reconstruction and knowledge-driven simulation mechanisms. In the black-box scenario of federated learning where data is not visible, the system breaks the statistical assumption that outliers are attacks in traditional defenses. It effectively distinguishes between normal model deviations caused by business data and human-induced model deviations caused by malicious attacks from a mathematical perspective. This deep discrimination capability based on manifold geometry not only solves the problem of high false alarm rate in security detection in non-independent and identically distributed scenarios, but also effectively ensures the robustness of the global model in complex network environments.

[0052] Example 2:

[0053] The ideal manifold reconstruction and simulation module generates the theoretically optimal gradient direction, which specifically includes: inputting a reliable verification set into a neural network model constructed from global model parameters; calculating the loss function between the model output and the true label; and obtaining the gradient of the loss function with respect to each parameter of the model through the backpropagation algorithm to form the theoretically optimal gradient direction.

[0054] This embodiment further defines the theoretically optimal gradient direction. The generation process aims to construct a clean benchmark for comparison; the system will use a pre-set trusted verification set. Input to the current global model parameters In the constructed neural network model, the trusted validation set originates from gold standard samples held by the server that have been manually cleaned and confirmed to be uncontaminated. The system calculates the loss function value between the model's output on the validation set and the true label, using the following formula:

[0055]

[0056] in, The loss function value is derived from the forward propagation calculation results. Its physical meaning is the difference between the model's current prediction and the ideal target, and it is dimensionless.

[0057] The number of samples is derived from the trusted verification set configuration. Its physical meaning is the size of the verification set, expressed in units of samples.

[0058] The input feature vector is derived from the validation set. Each sample, in its physical sense, refers to the data characteristics of the sample.

[0059] The labels are real and come from the validation set. Each sample, in physical terms, represents the correct classification result of the sample;

[0060] The loss metric function is derived from the preset algorithm configuration; specifically, in this network security situation awareness classification scenario, the cross-entropy loss function is selected; for example, cross-entropy. The loss function is calculated using the backpropagation algorithm. For each parameter of the model The partial derivatives of constitute the theoretically optimal gradient direction, and its calculation formula is:

[0061]

[0062] in, This is the theoretically optimal gradient direction vector, derived from backpropagation calculations. Its physical meaning is the steepest descent direction for improving model performance under the current parameters. For gradient operators, subscript Explicitly specifying the global model parameters Performing differential operations, in physical terms, means taking partial derivatives with respect to the parameter space;

[0063] It should be noted that, in order to ensure the theoretically optimal gradient direction is generated Local model gradient update vectors uploaded from multiple data sources It is physically comparable on a numerical scale to avoid the influence of the learner's learning rate on the part of the participant. The resulting order-of-magnitude difference, in this embodiment, in obtaining the basic gradient Subsequently, an adaptive scale alignment step was introduced;

[0064] In federated learning practices, the local learning rates of participating parties may differ and be inconsistent with the global learning rate on the server. Simply using a pre-set learning rate on the server would not suffice. Scaling may introduce new scale biases; therefore, this embodiment uses a dynamic norm anchoring method to determine the effective scaling factor: the system statistically analyzes all local model gradient update vectors received in the current round. The mean of the L2 norm, where, This represents the total number of participants in the aggregation process during the current training round. The participant index is calculated using the following formula:

[0065]

[0066] Calculate the fundamental gradient The L2 norm is calculated using the following formula:

[0067]

[0068] The formula for calculating the alignment scalar is as follows:

[0069]

[0070] in, This is a numerical stability constant; its value is set based on the lower bound of the machine precision of single-precision floating-point numbers to prevent numerical instability caused by denominator underflow; the calculated aligned scalar is used to update the theoretical gradient: This process forces the energy level of the theoretical gradient to align with the average level of the observed gradient, effectively eliminating orders-of-magnitude differences caused by variations in local learning rate settings and optimizer steps among participants, thus ensuring realistic bias characteristics in subsequent calculations. It can accurately reflect the directional deviation of the vector space, rather than simply the difference in magnitude scaling. To further adapt to the tensor operation logic of the underlying computing device and reduce the register memory overhead when extracting high-dimensional vector difference features, after generating the aligned theoretically optimal gradient direction vector, the system performs adaptive data quantization compression processing on it through the processor to extract a low-bit-width quantized feature matrix. The specific data quantization operation formula is as follows:

[0071]

[0072] in, The quantization feature matrix, in physical terms, represents the extracted low-bit-width fixed-point data result. This is a sign function, and its physical meaning is to extract the positive and negative polarity characteristics of the current floating-point value; The quantization step size constant, in physical terms, determines the granularity of the intervals in which the hardware discretizes and divides the data.

[0073] In this hardware data processing formula, the dynamic calculation formula for the quantization step size constant is as follows:

[0074]

[0075] The above-described underlying computation steps map the theoretically optimal gradient vector with floating-point precision to a low-bit-width fixed-point data matrix; the letters in the formula This represents the target quantization bit width parameter of the hardware processor. Through this underlying data bit width conversion and memory compression mechanism, the system significantly reduces the memory bus bandwidth occupied by the dual-track differential feature extraction module when performing massive element-wise differential operations, effectively improving the data parallel throughput efficiency of the central processing unit.

[0076] This embodiment calculates in real time on a server-side trusted verification set. This establishes a dynamically changing North Star index for each round of model updates. This vector not only represents the optimal path for improving model performance, but also serves as the origin for subsequent difference calculations. This ensures that any deviation, whether it is a benign difference in data distribution or a malicious attack disturbance, can be quantified as a vector offset relative to this origin, thus providing a solid mathematical foundation for high-precision situational awareness.

[0077] Example 3:

[0078] The ideal manifold reconstruction and simulation module generates gradient drift templates and gradient anomaly templates. Specifically, this includes: calling the data non-independent and identically distributed offset features in the model anomaly pattern knowledge base to generate a drift factor vector that conforms to a normal distribution offset; combining the drift factor vector with the theoretical optimal gradient direction to generate a gradient drift template; calling the parameter backdoor perturbation features or gradient magnitude anomaly features in the model anomaly pattern knowledge base to generate anomaly factor vectors with direction reversal or nonlinear amplification characteristics; and combining the anomaly factor vector with the theoretical optimal gradient direction to generate a gradient anomaly template.

[0079] This embodiment is based on the idea of ​​synthetic analysis, which transforms abstract knowledge into specific gradient vector templates; it addresses the gradient drift template. The generation process involves the system calling upon the non-independent and identically distributed offset features of the data in the model's abnormal pattern knowledge base, such as Dirichlet distribution parameters, to establish hyperparameters for the distribution variance of the control projection coefficients conforming to a normal distribution offset. ;

[0080] To eliminate the direct use of scalar formulas The directional deviation generated in high-dimensional space, i.e., Gaussian white noise tends to be orthogonal to the main signal, is addressed in this embodiment by explicitly employing the subspace projection method. As a hyperparameter controlling the variance of the projection coefficient distribution, its physical meaning is the relative standard deviation coefficient within the projection subspace, i.e., dimensionless. It controls the range of the magnitude of the generated drift vector relative to the original gradient, and its specific value range is as follows: ;in, For the first scalar shift in each feature dimension Indicates a normal distribution. Let be the variance of the distribution;

[0081] The specific value of this coefficient is negatively correlated with the training phase of the global model. That is, a larger value is taken in the early stage of training when the model has not yet converged in order to tolerate larger benign fluctuations. As the number of training rounds increases, the coefficient is gradually reduced to 0.05 in order to tighten the drift judgment boundary.

[0082] To implement the drift subspace projection mechanism, the knowledge base stores historical benign drift gradients from the system's trusted initialization phase. eigenvector matrix of principal components ,in, The number of main components, This represents the set of real numbers. The trusted initialization phase refers to the initial stage of system deployment where the server utilizes its locally held trusted verification set to apply various preset data augmentation transformations, such as random rotation, brightness adjustment, and cropping, to simulate a non-independent, identically distributed data environment that participants might encounter. Based on this, the backpropagation gradient is calculated, thereby accumulating data in a pollution-free environment. A benign gradient sample is used as a baseline;

[0083] Here This was determined based on the elbow rule analysis of the eigenvalue decay curve of the historical gradient covariance matrix, and this number is sufficient to cover more than 95% of the cumulative variance contribution rate; given that Explicitly defined as the total dimension after flattening the global model parameters, for example Directly build The covariance matrix can cause memory overflow, therefore the matrix in this embodiment... It was constructed using the incremental principal component analysis algorithm;

[0084] System settings for batch size The orthogonal basis of the feature space is updated iteratively by reading pre-stored historical benign gradient samples from the aforementioned knowledge base using QR decomposition. The specific update steps are as follows: Let the current orthogonal basis be... The corresponding singular value diagonal matrix is The new batch of data is The formula for calculating the orthogonal projection residual is as follows:

[0085]

[0086] The residual matrix is ​​decomposed using the QR decomposition method, and the calculation formula is as follows:

[0087]

[0088] in, The orthogonal matrix obtained by decomposition, The upper triangular matrix obtained by decomposition;

[0089] The extended core matrix is ​​constructed using the following formula:

[0090]

[0091] And perform SVD decomposition. ;in for The left singular vector matrix, It is a singular value diagonal matrix. Given a right singular vector matrix; update the basis vectors. And cut off the front List; the process in The system extracts the projection basis under time complexity and limited memory constraints; the system is based on hyperparameters that control the variance of the projection coefficient distribution. Generating low-dimensional projection coefficients Through mapping The final drift factor vector is obtained, where, for An identity matrix of 3D;

[0092] The vector is synthesized with the theoretically optimal gradient direction, and its calculation formula is as follows:

[0093]

[0094] Based on this, for gradient anomaly templates The system generates anomaly features and executes anomaly feature selection logic: calculating the sparsity of the theoretically optimal gradient direction. Sparsity The calculation method is as follows:

[0095]

[0096] The range of values ​​for this indicator is: The closer the value is to the lower bound, the more concentrated the energy is on a few parameters; therefore, if This threshold The selected value is the statistical lower bound of the sparsity coefficients during normal training of ResNet and VGG series models. Values ​​below this indicate excessively concentrated gradient updates, potentially malicious. Therefore, the parameter backdoor perturbation feature is invoked to set the flip coefficient. Flip coefficient The physical meaning of this term is the strength factor of a malicious attacker's inversion of the gradient direction; the range of this value is an empirical interval derived from statistical data of attack effectiveness experiments of common gradient inversion attacks under different learning rate configurations; and an anomaly factor vector is generated. ;like If the gradient update is globally diffuse, then the gradient magnitude anomaly feature is invoked.

[0097] To ensure that the generated outlier vectors are significantly different from the normal gradients in terms of magnitude, a normalized forced scaling strategy is adopted: the nonlinear direction vector is calculated as follows:

[0098]

[0099] in, This represents the Hadamard product, which is an element-wise multiplication between matrices or vectors. The mathematical basis of this formula is that by squaring the gradient elements and preserving their original signs, high-amplitude features can be non-linearly amplified, typically corresponding to the main discriminative path of the model, while suppressing low-amplitude noise. This simulates the oversaturation update behavior of attackers to quickly implant backdoors into key parameters. The anomaly factor vector is calculated using the following formula:

[0100]

[0101] in, Set as As a numerical stabilizing term, its physical function is to prevent the nonlinear direction vector from... When the modulus is close to zero, such as in the region of dead neurons, a division-by-zero error occurs, ensuring the numerical stability of the computation process;

[0102] The scalar scaling factor is set to the nearest. The global model updates the gradient L2 norm mean by 3 times, based on 3 Anomaly detection criteria; Here, the global model update gradient is explicitly defined as the change in global model parameters after each round of aggregation relative to the previous round, and the calculation formula is:

[0103]

[0104] Among them, superscript Indicates the current training aggregation round, superscript This indicates the previous training aggregation epoch; this definition ensures that the scale of the outlier factors matches the model evolution rate in the current training phase; the selected outlier factor vector is synthesized with the theoretically optimal gradient direction, and its calculation formula is as follows:

[0105]

[0106] This embodiment utilizes knowledge base for proactive generation. and A twin verification mechanism based on subspace projection and nonlinear synthesis was constructed.

[0107] Example 4:

[0108] The dual-track differential feature extraction module calculates the actual deviation features, specifically including: performing element-wise tensor differencing operations on the local model gradient update vector and the quantization feature matrix; using the difference operation result vector as the actual deviation feature reflecting the deviation of the local calculation process from the expected optimization path at the multi-source data end; the dual-track differential feature extraction module calculates the theoretical drift deviation and theoretical anomaly deviation, specifically including: calculating the tensor difference vector between the gradient drift template and the quantization feature matrix, defined as the theoretical drift deviation; calculating the tensor difference vector between the gradient anomaly template and the quantization feature matrix, defined as the theoretical anomaly deviation.

[0109] This embodiment details the core computational logic of the dual-track difference feature extraction module, aiming to extract pure residual features; the system performs difference in the real-dimensional dimension, updating the local model gradient vectors uploaded from multiple data sources. With quantized feature matrix The formula for element-by-element subtraction is:

[0110]

[0111] in, The actual deviation feature vector originates from the differential calculation and its physical meaning is the actual deviation of the local calculation process at the multi-source data end relative to the theoretical quantization benchmark. The system performs differential calculations on the theoretical dimension, calculating the tensor difference between the generated template and the quantization feature matrix, respectively. The calculation formula is as follows:

[0112]

[0113]

[0114] in, This is the theoretical drift bias vector, derived from differential calculations, and its physical meaning is the gradient component caused purely by differences in data distribution. This is the theoretical anomaly deviation vector, derived from differential calculations, and its physical meaning is the gradient component purely caused by the attack behavior;

[0115] This embodiment successfully extracted the common information of the global model itself through differential operations. Only the variable is retained; this processing method significantly reduces the background noise in the high-dimensional feature space, allowing subsequent analysis to focus on the direction and magnitude of the deviation, rather than the absolute value of the gradient; this common-mode interference removal design greatly improves the signal-to-noise ratio of feature extraction, ensuring the accuracy of subsequent similarity judgment.

[0116] Example 5:

[0117] The situational coupling decision module executes a response based on the similarity matching results, specifically including: if the first structural similarity is higher than a preset safety threshold and the second structural similarity is lower than the safety threshold, the model update source is determined to be a reliable data offset; the corresponding local model gradient update vector is included in the global model aggregation update process; if the second structural similarity is higher than the safety threshold, the model update source is determined to have parameter-level malicious perturbation; the corresponding local model gradient update vector is rejected for use in global model aggregation, and its source participant is marked as a low-reliability node; if both the first and second structural similarities are lower than the safety threshold, the model update source is determined to be an undefined abnormal pattern; the corresponding local model gradient update vector is transferred to an isolated analysis environment for further pattern recognition and attribution analysis.

[0118] This embodiment details the multi-level decision logic and isolation attribution mechanism based on structural similarity; the system introduces an improved hierarchical weighted topology mapping mechanism to calculate the first structural similarity. Second structural similarity To address the logical flaw in traditional gradient-based weighted methods that may miss malicious perturbations in small gradient layers, this embodiment introduces a sensitivity smoothing strategy. The system divides the high-dimensional gradient vector into segments based on the network hierarchy of the global model. Subvectors, where the superscript Indicates the first The gradient sub-vectors corresponding to each neural network layer are calculated; the correction sensitivity weights for each layer are calculated, where the denominator represents the weights for the entire network. The formula for summing the gradient norms of each level is:

[0119]

[0120] in, For network level indexing, This is a pre-defined numerical stability term used to prevent computational overflow caused by the summation of gradient norms approaching zero at the end of model convergence. This is a balancing factor; the value is an empirically optimal value derived from statistics of 100 attack-defense adversarial experiments conducted on a benchmark dataset.

[0121] Experimental results show that when At this point, the system achieves a maximum ratio of F1-Score > 0.92 for the detection rate of minor perturbation attacks to the false alarm rate for normal data shifts, effectively balancing the focus on the salient feature layer and the silent layer. It should be noted that this balancing factor... It is not fixed. In other embodiments of the present invention, it can be adaptively adjusted according to the depth of the neural network. For example, for shallow networks, this value can be appropriately increased. To enhance the dependence on the main feature layer, the formula introduces a uniform distribution term. A weight lower bound is set, forcing the system to pay attention to the directional consistency of all network layers, thereby preventing attackers from injecting malicious updates undetected by using silent layers with low gradient norms; the weighted synthesis of global similarity is calculated using the following formula:

[0122]

[0123]

[0124] Based on this, the system introduces a preset security threshold. This threshold is not statically set, but dynamically calculated based on a sliding time window: the system maintains a window of length... Historically reliable update of the first structural similarity queue Real-time calculation of queue average with standard deviation and set This threshold setting is based on a normal distribution. The principle is designed to enable security assessments to cover 99.7% of similarity changes caused by normal business fluctuations;

[0125] The following hierarchical response logic is executed: The system executes conditional branch decisions according to the risk priority principle, with the following logical order: If... If the model update source is determined to have a malicious perturbation at the parameter level, gradient removal or isolation analysis of the response will be performed directly, and the process will not proceed to the next decision-making stage; if the aforementioned conditions are not met, further judgment will be made if... If the model update source is determined to be a reliable data offset, model gradient acceptance is performed; if none of the above conditions are met, i.e. If the model update originates from an undefined abnormal mode, isolation analysis will be performed.

[0126] For gradient vectors transferred to the isolated analysis environment, the system performs deep attribution analysis; it uses the DBSCAN density clustering algorithm to identify the existence of cooperative attack clusters; and to address the curse of dimensionality caused by the inability of benign feature spaces to effectively represent unknown anomalies, the system introduces Gaussian random projection techniques to construct a projection matrix. Its elements are independently and identically distributed in ;

[0127] According to the Johnson-Lindenstrauss lemma, through The resulting low-dimensional vectors are preserved in the Euclidean distance structure of the original space. - Isometry ensures the accuracy of clustering unknown anomaly patterns; if clustering is effective, hierarchical energy distribution attribution analysis is further performed: Energy calculation: calculates actual deviation characteristics. At each neural network layer The squared L2 norm of the hierarchical deviation energy is defined as the level deviation energy, and its calculation formula is as follows:

[0128]

[0129] Distribution statistics: The relative energy percentage of the fully connected layer is calculated using the following formula:

[0130]

[0131] in, This represents the set of all fully connected hierarchical indices in the global model, and , To prevent the term from being divided by zero; its value is slightly less than This is to maintain stricter numerical boundaries in normalization calculations and ensure the purity of energy percentage calculations; Decision logic: If This indicates that the gradient bias energy is mainly concentrated in the classification head, which is determined to be label tampering; similarly, the relative energy proportion of the convolutional layer is calculated using the following formula:

[0132]

[0133] in, For the set of indices of all convolutional layers; if If the threshold is exceeded, it is determined to be a backdoor implantation; the above threshold This method involves conducting 1000 simulated attack tests on common convolutional neural networks, statistically analyzing the energy distribution histograms of successful attack samples at different levels, and selecting the quantiles with a 99% confidence level. Compared to more complex methods... Value analysis has clear physical interpretability and It has low computational complexity.

[0134] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention.

Claims

1. A multi-source gradient data processing system based on manifold reconstruction, characterized in that, include: The data and knowledge acquisition module is used to receive high-dimensional local model gradient update vectors from multiple sources, and to retrieve the global model parameters after the previous round of aggregation and the preset model anomaly pattern knowledge base, which contains typical pattern features of gradient deviation. The ideal manifold reconstruction and simulation module is used to perform forward inference and loss calculation on a preset trusted verification set based on the global model parameters, generate the floating-point theoretical optimal gradient direction vector for the current training stage, and perform adaptive data quantization compression processing on it through the processor to extract a low-bit-width quantization feature matrix. Based on the model anomaly pattern knowledge base, a gradient drift template representing the shift in the distribution of reliable data and a gradient anomaly template representing abnormal perturbations are generated. The dual-track differential feature extraction module is used to perform low-level tensor difference operations, calculate the actual deviation features between the local model gradient update vector and the quantized feature matrix, and calculate the theoretical drift deviation and theoretical anomaly deviation between the gradient drift template, the gradient anomaly template and the quantized feature matrix, respectively. The situational coupling decision module is used to calculate the first structural similarity between the actual deviation feature and the theoretical drift deviation, and the second structural similarity with the theoretical abnormal deviation in a high-dimensional vector space. Based on the similarity matching results, it executes instructions for reliable writing of multi-source gradient data or removal, release or isolation of abnormal data, thereby completing the filtering of underlying feature data and updating of system memory. The situational coupling decision module executes a response based on the similarity matching result, including: If the first structural similarity is higher than the preset safety threshold and the second structural similarity is lower than the safety threshold, then the model update source is determined to be a trusted data offset. The corresponding local model gradient update vector is incorporated into the global model aggregation update process; The situational coupling decision module executes a response based on the similarity matching result, and also includes: If the similarity of the second structure is higher than the security threshold, it is determined that there is a parameter-level malicious perturbation in the model update source; Refuse to use the corresponding local model gradient update vector for global model aggregation, and mark its source participant as a low-reliability node; The situational coupling decision module executes a response based on the similarity matching result, and also includes: If both the first structural similarity and the second structural similarity are lower than the safety threshold, then the model update source is determined to be an undefined abnormal mode; The corresponding local model gradient update vectors are then transferred to an isolated analysis environment for further pattern recognition and attribution analysis.

2. The multi-source gradient data processing system based on manifold reconstruction according to claim 1, characterized in that, The ideal manifold reconstruction and simulation module generates the theoretically optimal gradient direction vector, including: The trusted verification set is input into the neural network model constructed from the global model parameters; Calculate the loss function between the model output and the true label; The gradients of the loss function with respect to each parameter of the model are obtained through the backpropagation algorithm, forming the theoretically optimal gradient direction vector.

3. The multi-source gradient data processing system based on manifold reconstruction according to claim 1, characterized in that, The ideal manifold reconstruction and simulation module generates gradient drift templates and gradient anomaly templates, including: The non-independent and identically distributed offset features of the data in the model's abnormal pattern knowledge base are called to generate a drift factor vector that conforms to a normal distribution offset. The drift factor vector is vector-synthesized with the theoretical optimal gradient direction to generate the gradient drift template. The parameter backdoor perturbation feature or gradient magnitude anomaly feature in the model anomaly pattern knowledge base are called to generate an anomaly factor vector with direction reversal or nonlinear amplification characteristics. The anomaly factor vector is combined with the theoretically optimal gradient direction to generate the gradient anomaly template.

4. The multi-source gradient data processing system based on manifold reconstruction according to claim 1, characterized in that, The dual-track differential feature extraction module calculates the actual deviation features, including: Perform element-wise tensor difference operations on the local model gradient update vector and the quantized feature matrix; The result vector of the difference operation is used as a feature to reflect the actual deviation of the local calculation process at the multi-source data terminal from the expected optimization path.

5. A multi-source gradient data processing system based on manifold reconstruction according to claim 1, characterized in that, The dual-track differential feature extraction module calculates theoretical drift bias and theoretical anomaly bias, including: The tensor difference vector between the gradient drift template and the quantized feature matrix is ​​calculated and defined as the theoretical drift deviation. The tensor difference vector between the gradient anomaly template and the quantized feature matrix is ​​calculated and defined as the theoretical anomaly deviation.

Images