A self-supervised model training security defense method, system, device and medium

Abstract

This application relates to a method, system, device, and medium for secure defense during self-supervised model training. The method includes: performing joint frequency and spatial preprocessing on a local training dataset with original labels to generate a preprocessed labeled training set; removing labels to generate an unlabeled dataset, and obtaining purified feature extraction layer parameters through self-supervised learning; fixing these parameters and minimizing the symmetric cross-entropy loss of the classification layer based on the labels to obtain basic model parameters; partitioning the dataset according to the forward propagation loss and obtaining the final model parameters and update gradients by combining them with a semi-supervised joint loss; generating federated aggregation control instructions based on the update gradients and sending them to the server to control global weighted aggregation, generating the next round of globally secure model parameters. This method can improve the security and robustness of self-supervised model training and ensure the reliability of model aggregation in federated learning.

Description

Technical Field

[0001] This invention belongs to the field of neural network security technology, and in particular relates to a method, system, device and medium for security defense of self-supervised model training. Background Technology

[0002] With the development of deep learning, self-supervised learning reduces the reliance on manual annotation by mining semantic representations from unlabeled data; federated learning enables distributed collaborative training while protecting data privacy. The fusion of these two technologies, Federated Self-Supervised Learning (FSSL), has become a cutting-edge technology in distributed intelligent systems and is widely used in fields such as computer vision and natural language processing. However, FSSL's open distributed architecture and unsupervised training characteristics make it vulnerable to severe backdoor attacks, which are more covert and destructive.

[0003] Existing FSSL security defense technologies face a dilemma in balancing security, accuracy, and efficiency: traditional poisoning suppression defenses sacrifice the normal accuracy of the model, while high-order defense methods have high computational overhead and are difficult to adapt to high-frequency communication scenarios in federated learning; at the same time, existing solutions are insufficient in dealing with new backdoor attacks based on feature entanglement, distributed decoupling, and multi-party collusion, and cannot effectively block covert attack paths. Summary of the Invention

[0004] Therefore, it is necessary to provide a federated self-supervised learning security defense method, system, device, and medium that can balance defense security and computational efficiency, accurately resist new backdoor attacks, and ensure the reliable operation of distributed intelligent systems in response to the above-mentioned technical problems.

[0005] Firstly, this application provides a method for secure defense during self-supervised model training, including:

[0006] S1. Perform joint frequency and spatial preprocessing on the local training dataset with original labels. Filter out high-frequency malicious signals in the training dataset by frequency domain masking and apply random erasure to the candidate regions in the local training dataset to generate a preprocessed labeled training set.

[0007] S2. Remove the label information from the preprocessed labeled training set to generate an unlabeled dataset. Input the unlabeled dataset into the feature extraction layer of the neural network model to be trained. Calculate the self-supervised loss based on the self-supervised learning mechanism to bring positive sample pairs closer and push negative sample pairs further apart. Minimize the self-supervised loss through gradient backpropagation to obtain the purified feature extraction layer parameters of the neural network model.

[0008] S3. Fix the parameters of the purified feature extraction layer in the neural network model, and combine the label information in the preprocessed labeled training set to minimize the symmetric cross-entropy loss of the classification layer in the neural network model to obtain the basic neural network model parameters.

[0009] S4. Calculate the forward propagation loss value of the samples in the preprocessed labeled training set based on the parameters of the basic neural network model. Based on the forward propagation loss value, divide the preprocessed labeled training set into a high-confidence set and a low-confidence set. Remove the labels from the low-confidence set to generate an auxiliary unlabeled dataset. Combine the high-confidence set to calculate the semi-supervised joint loss. Minimize the semi-supervised joint loss through gradient backpropagation to obtain the final neural network model parameters and the neural network model update gradient.

[0010] S5. Generate federated aggregation control instructions based on the neural network model update gradient, and send the federated aggregation control instructions and the neural network model update gradient to the federated server. Control the federated server to perform global weighted aggregation of the neural network model update gradient to generate the next round of global safe model parameters. The next round of global safe model parameters includes the local training dataset with original labels for the next period.

[0011] Secondly, this application also provides a self-supervised model training security defense system for implementing the method described in the first aspect, comprising:

[0012] The preprocessing module is used to perform joint frequency and spatial preprocessing on the local training dataset with original labels. It filters out high-frequency malicious signals in the training dataset through frequency domain mask filtering and applies random erasure to the candidate regions in the local training dataset to generate a preprocessed labeled training set.

[0013] The self-supervised feature training module is used to remove the label information from the preprocessed labeled training set, generate an unlabeled dataset, and input the unlabeled dataset into the feature extraction layer of the neural network model to be trained. Based on the self-supervised learning mechanism, the self-supervised loss is calculated by bringing positive sample pairs closer and pushing negative sample pairs further apart. The self-supervised loss is minimized through gradient backpropagation to obtain the purified feature extraction layer parameters of the neural network model.

[0014] The self-supervised classification training module is used to fix the parameters of the purified feature extraction layer in the neural network model, and combine the label information in the preprocessed labeled training set to minimize the symmetric cross-entropy loss of the classification layer in the neural network model to obtain the basic neural network model parameters.

[0015] The semi-supervised joint loss module is used to calculate the forward propagation loss value of samples in the preprocessed labeled training set based on the parameters of the basic neural network model. Based on the forward propagation loss value, the preprocessed labeled training set is divided into a high-confidence set and a low-confidence set. The labels in the low-confidence set are removed to generate an auxiliary unlabeled dataset. The semi-supervised joint loss is calculated in combination with the high-confidence set, and the semi-supervised joint loss is minimized through gradient backpropagation to obtain the final neural network model parameters and the neural network model update gradient.

[0016] The communication and interaction module is used to generate federated aggregation control instructions based on the gradient update of the neural network model, send the federated aggregation control instructions to the federated server, control the federated server to perform global weighted aggregation of the gradient update of the neural network model, generate the next round of global safe model parameters, and receive the next round of global safe model parameters; wherein, the next round of global safe model parameters includes the local training dataset with original labels for the next period.

[0017] Thirdly, this application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to perform any of the above steps.

[0018] Fourthly, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, performs any of the above steps.

[0019] The aforementioned self-supervised model training security defense method, system, device, and medium, through joint frequency and spatial preprocessing of the local training dataset, can disrupt the spatial structural integrity and high-frequency feature signals of potential backdoor triggers. By stripping label information and training a purified feature extraction layer based on a self-supervised learning mechanism, coupled with a strategy of fixing the parameters of this extraction layer to minimize the symmetric cross-entropy loss of the classification layer, the feature representation learning and label mapping process can be effectively deconstructed, thereby severing the spurious association between potential backdoor triggers and the target classification label. Dividing the forward propagation loss value into high and low confidence sets and performing semi-supervised joint loss fine-tuning can correct the model using the distribution information of auxiliary unlabeled data, helping to improve the feature generalization ability and defense robustness of the neural network model in a distributed federated aggregation training environment. Attached Figure Description

[0020] To more clearly illustrate the technical solutions in the embodiments or related technologies of this application, the accompanying drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1 This is a schematic diagram of an implementation environment provided in an embodiment of the present invention;

[0022] Figure 2 This is a flowchart illustrating a self-supervised model training security defense method according to an embodiment of the present invention;

[0023] Figure 3This is a schematic diagram of a self-supervised model training security defense system according to an embodiment of the present invention. Detailed Implementation

[0024] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0025] This application provides a self-supervised model training security defense method, which can be applied to, for example... Figure 1 In the application environment shown, training device 01 communicates with federated server 02 via a network. A data storage system can store data for both training device 01 and federated server 02. The data storage system can be integrated into training device 01 or located on federated server 02. Training device 01 can be, but is not limited to, various personal computers, laptops, and cloud servers. Federated server 02 can be implemented using a standalone server or a server cluster consisting of multiple servers.

[0026] In one exemplary embodiment, such as Figure 2 As shown, a method for training a self-supervised model to ensure security is provided, and this method is applied to... Figure 1 Taking training device 01 as an example, the process may include the following steps:

[0027] S1. Perform joint frequency and spatial preprocessing on the local training dataset with original labels. Use frequency domain masking to filter out high-frequency malicious signals in the training dataset and apply random erasure to the candidate regions in the local training dataset to generate a preprocessed labeled training set.

[0028] Specifically, the original label can be the initial classification identifier attached to the dataset. The local training dataset can be the data set currently stored on the training device for computation. Joint frequency and spatial preprocessing can be a data transformation and modification operation performed simultaneously on the data in the frequency and spatial domains. Frequency domain masking filtering can be a filtering behavior that applies a matrix in the frequency domain to block specific frequency components from passing through. High-frequency malicious signals can be backdoor trigger signals embedded in the high-frequency bands of the training data. Candidate regions can be spatial locations in the training dataset that are determined to have highly significant features. Random erasure can be an operation that covers a specified data region with randomly generated pixel values. The preprocessed labeled training set can be a dataset that retains the original classification identifier after frequency and spatial domain processing.

[0029] Optionally, the training device can perform a frequency domain transformation on the local training dataset, applying a mask in the frequency domain to filter out high-frequency malicious signals in the local training dataset. The training device can filter out high-frequency malicious signals through low-pass filtering or frequency domain masking. The training device can perform an inverse transformation on the frequency-domain processed data to return it to the spatial domain. The training device can apply random erasure to candidate regions in the local training dataset in the spatial domain, covering rectangular regions in the image with random pixel values. The training device can combine the processing results from the frequency domain and the spatial domain to generate a preprocessed labeled training set.

[0030] S2. Remove the label information from the preprocessed labeled training set to generate an unlabeled dataset. Input the unlabeled dataset into the feature extraction layer of the neural network model to be trained. Calculate the self-supervised loss based on the self-supervised learning mechanism to bring positive sample pairs closer and push negative sample pairs further apart. Minimize the self-supervised loss through gradient backpropagation to obtain the purified feature extraction layer parameters of the neural network model.

[0031] Specifically, an unlabeled dataset can be a collection that includes only the data ontology without category labels. The feature extraction layer is a network layer in a neural network model responsible for extracting deep semantic representations from the input data. A self-supervised learning mechanism can be a method for training a neural network by constructing an agent task based on the inherent structure of the data. Positive sample pairs can be feature representation pairs derived from the same original data. Negative sample pairs can be feature representation pairs derived from different original data. Self-supervised loss can be a numerical value that quantifies the distance between sample pairs in the feature space and the optimization target. Gradient backpropagation can be an algorithm that calculates the gradient of the neural network parameters based on the loss value and propagates it layer by layer to update the parameters. Purified feature extraction layer parameters can be the weights of the feature extraction layer that have not been associated with malicious target labels after training on unlabeled data.

[0032] Optionally, the training device can remove all label information from the preprocessed labeled training set and construct an unlabeled dataset based on the label-removed data. The training device can input the unlabeled dataset into the feature extraction layer of the neural network model to be trained, and run the feature extraction layer under a self-supervised learning mechanism. The training device can calculate the feature distance between positive sample pairs derived from the same original sample and perform operations to shorten the distance between positive sample pairs; the training device can calculate the feature distance between negative sample pairs derived from different samples and perform operations to widen the distance between negative sample pairs. The training device can calculate the self-supervised loss based on the distance metric results and minimize the self-supervised loss using the backpropagation algorithm. After completing gradient updates, the training device can obtain the purified feature extraction layer parameters of the neural network model.

[0033] S3. Fix the parameters of the purified feature extraction layer in the neural network model, and combine them with the label information in the preprocessed labeled training set to minimize the symmetric cross-entropy loss of the classification layer in the neural network model to obtain the basic neural network model parameters.

[0034] Specifically, the classification layer is the network layer in the neural network model responsible for mapping the extracted features to the target category. Symmetric cross-entropy loss can be a numerical value that combines standard cross-entropy and inverse cross-entropy to measure the difference between the model's predicted probability distribution and the true label distribution. The basic neural network model parameters can be the overall weight combination of the neural network model after the joint training of the feature extraction layer and the classification layer.

[0035] Optionally, the training device can lock the parameters of the purified feature extraction layer in the neural network model, stop calculating the gradient of the purified feature extraction layer, and sever the spurious association between the trigger and the target label. The training device can extract features from the preprocessed labeled training set using the fixed purified feature extraction layer. The training device can input the extracted features into the classification layer of the neural network model, and calculate the symmetric cross-entropy loss of the classification layer output by combining it with the label information from the preprocessed labeled training set. The training device can introduce an inverse cross-entropy term using symmetric cross-entropy. The training device can amplify the difference in loss values ​​between clean and poisoned samples using symmetric cross-entropy. The training device can execute a gradient backpropagation algorithm to minimize the symmetric cross-entropy loss of the classification layer. The training device can update the parameters of the classification layer, and combine the updated classification layer parameters with the fixed purified feature extraction layer parameters to obtain the basic neural network model parameters.

[0036] S4. Calculate the forward propagation loss value of the samples in the preprocessed labeled training set based on the parameters of the basic neural network model. Based on the forward propagation loss value, divide the preprocessed labeled training set into a high-confidence set and a low-confidence set. Remove the labels from the low-confidence set to generate an auxiliary unlabeled dataset. Combine the high-confidence set to calculate the semi-supervised joint loss. Minimize the semi-supervised joint loss through gradient backpropagation to obtain the final neural network model parameters and the neural network model update gradient.

[0037] Specifically, the forward propagation loss can be the divergence or error measure between the predicted output calculated based on the parameters after the data is input into the model and the true label. The high-confidence set can be a subset of data with low forward propagation loss values ​​and a high probability of being considered clean. The low-confidence set can be a subset of data with high forward propagation loss values ​​and a high risk of including noise or poisoning. The auxiliary unlabeled dataset can be a set used to provide data distribution information, formed by removing labels from the low-confidence set. The semi-supervised joint loss can be the overall optimization objective that combines the supervised learning error of labeled data and the regularization consistency error of unlabeled data. The final neural network model parameters can be the deep learning network weights determined after semi-supervised joint optimization. The neural network model update gradient can be the difference between the model parameters before and after semi-supervised joint optimization or the parameter change vector calculated by backpropagation.

[0038] Optionally, the training device can construct an evaluation network based on the parameters of the basic neural network model, inputting samples from the preprocessed labeled training set into the evaluation network. The training device can calculate the forward propagation loss value for each sample, assigning the samples with the minimum loss to the high-confidence set. The training device can assign samples with larger remaining loss values ​​to the low-confidence set, removing the label information from the low-confidence set. The training device can use the low-confidence set with removed labels as an auxiliary unlabeled dataset. The training device can combine the high-confidence set and the auxiliary unlabeled dataset to jointly optimize the entire neural network model under a semi-supervised learning architecture. The training device can calculate the semi-supervised joint loss and minimize it through backpropagation. The training device can clean up potential poisoning data and improve the generalization ability of the neural network model through the distribution information of unlabeled data. After completing the semi-supervised joint optimization, the training device obtains the final neural network model parameters. The training device can calculate the difference between the final neural network model parameters and the basic neural network model parameters to obtain the neural network model update gradient.

[0039] S5. Generate federated aggregation control instructions based on the neural network model update gradient, and send the federated aggregation control instructions and the neural network model update gradient to the federated server. Control the federated server to perform global weighted aggregation of the neural network model update gradient to generate the next round of global safe model parameters. The next round of global safe model parameters includes the local training dataset with original labels for the next period.

[0040] Specifically, federated aggregation control instructions can be control signaling used to instruct distributed parameter aggregation rules and anomaly screening logic. The federated server can be the central computing node in a distributed architecture responsible for collecting node parameters and performing global model updates. Global weighted aggregation can be the act of merging gradients or parameters of neural network models uploaded from multiple nodes according to dynamically assigned weight coefficients. The next round of global safe model parameters can be the set of neural network weights generated after completing this round of global weighted aggregation for use in the next communication cycle. The local training dataset with original labels for the next cycle can be the data with classification labels included in the next round of global safe model parameters for continued use in subsequent training iterations.

[0041] Optionally, the training device can encapsulate instructions based on the statistical features or dimensional information within the neural network model's update gradient to generate federated aggregation control instructions. The training device can establish a communication connection with the federated server. The training device can send the federated aggregation control instructions, including configuration information, and the neural network model's update gradient to the federated server. The training device can convey processing logic to the federated server through the federated aggregation control instructions, controlling the federated server to execute anomaly screening logic and weight allocation algorithms. The training device can control the federated server to perform global weighted aggregation of the neural network model update gradients of multiple nodes, including the current node, ensuring the aggregation process is protected against malicious node contamination. The training device can control the federated server to generate the next round of global security model parameters after aggregation is completed. The training device can receive the next round of global security model parameters from the federated server in the next communication round. The training device can extract the local training dataset information with original labels carried in the next round of global security model parameters for use in the next training cycle.

[0042] This embodiment provides a self-supervised model training security defense method. By performing joint frequency and spatial preprocessing on the local training dataset, the spatial structural integrity and high-frequency feature signals of potential backdoor triggers can be destroyed. By stripping label information and training a purified feature extraction layer based on a self-supervised learning mechanism, coupled with a strategy of fixing the parameters of this extraction layer to minimize the symmetric cross-entropy loss of the classification layer, the feature representation learning and label mapping process can be effectively deconstructed, thereby severing the spurious association between potential backdoor triggers and the target classification label. By dividing the high and low confidence sets based on the forward propagation loss value and performing semi-supervised joint loss fine-tuning, the model can be corrected by using the distribution information of auxiliary unlabeled data, which helps to improve the feature generalization ability and defense robustness of the neural network model in a distributed federated aggregation training environment.

[0043] In one embodiment, S1 may include:

[0044] S11. Transform the images in the training dataset with original labels to the frequency domain to obtain frequency domain images. Use a smoothing mask matrix to filter out potential malicious triggering signals in the high-frequency bands of the frequency domain images to obtain frequency domain filtered images. Reconstruct the frequency domain filtered images through inverse Fourier transform to generate malicious-removed images.

[0045] Specifically, a smoothing mask matrix can be a mathematically calculated matrix used to smoothly filter out components of a specific frequency band in the frequency domain. A potential malicious trigger signal can be an abnormal frequency component hidden in a high-frequency band to activate a backdoor in the network model.

[0046] Optionally, the training device can map and transform the images in the training dataset to the frequency domain, generating a frequency domain image. The training device can then apply a smoothing mask matrix to the frequency domain image to filter out potential malicious triggering signals in the high-frequency bands. The training device can generate a frequency-domain filtered image with high-frequency features removed and can perform an inverse Fourier transform on the frequency-domain filtered image to generate a malicious-removed image.

[0047] S12. Perform feature evaluation processing on the malicious image removal, calculate the absolute value of the difference between each pixel and its neighboring pixels in terms of color and spatial distribution, and use the frequency of pixel occurrence in the malicious image removal as a weight to perform weighted summation, generating a saliency map representing the contribution of the image to decision-making.

[0048] For example, the formula for calculating the saliency plot can be:

[0049]

[0050] in, For the first The significance value of each pixel; For the first The intensity of each pixel; Indicates pixel intensity level Pixel intensity; Indicates pixel intensity level The frequency of the pixels appearing in the frequency-domain filtered image.

[0051] Specifically, neighboring pixels can be the set of pixels that are adjacent to the target pixel in spatial coordinates. The absolute value of color and spatial distribution difference can be a quantified value of the difference between the target pixel and its neighboring pixels in terms of color channel values ​​and physical coordinate distance. Pixel occurrence frequency can be the statistical proportion of a specific pixel intensity level appearing in the overall image data. The saliency map can be a two-dimensional matrix representation reflecting the contribution of each pixel to the overall image features or model decision.

[0052] Optionally, the training device can extract the color channel values ​​of each pixel in the malicious image and extract the spatial physical coordinates of each pixel. The training device can determine the neighboring pixels of each pixel and calculate the absolute value of the difference between each pixel and its neighboring pixels in color and spatial distribution. The training device can statistically analyze the pixel frequency of each pixel intensity level in the malicious image and configure the pixel frequency as a weight parameter. The training device can calculate the product of the absolute value of the difference between each pixel and the corresponding weight parameter. The training device can calculate the saliency value of each pixel and generate a saliency map representing the image's decision contribution based on the saliency values ​​of all pixels.

[0053] S13. Based on the saliency map, regions with saliency values ​​greater than a preset saliency threshold are selected as candidate regions, and random pixel values ​​are used to cover the candidate regions to generate a preprocessed labeled training set; wherein, the saliency value is used to characterize the contribution of the corresponding pixel to the output decision of the neural network model.

[0054] Specifically, the saliency value can be a calculated value that quantifies the contribution of each pixel in the saliency map to the output decision of the neural network model. The preset saliency threshold can be a preset numerical limit used to determine highly saliency regions. Random pixel value overlay can be an operation that generates an irregular sequence of pixel values ​​and replaces the original pixels in a specified region.

[0055] Optionally, the training device can extract the saliency value of each pixel in the saliency map. The training device can compare the saliency value of each pixel with a preset saliency threshold, filtering out pixel locations with saliency values ​​greater than the preset threshold. The training device can define the spatial regions corresponding to the filtered pixel locations as candidate regions. The training device can generate a set of random pixel values ​​and cover the candidate regions with these generated random pixel values. The training device can replace the original pixels within the candidate regions with random pixel values ​​to disrupt the spatial pattern structure within the candidate regions. The training device can retain the original labels attached to the image and can integrate the spatially covered image and its original labels to generate a preprocessed labeled training set.

[0056] This embodiment provides a self-supervised model training security defense method. By converting the image to the frequency domain and using a smoothing mask matrix for filtering, potential malicious trigger signals hidden in high-frequency bands can be cleaned, helping to reduce the concealment of frequency domain backdoor features. By calculating the absolute value of pixel color and spatial distribution differences through feature evaluation processing, and combining this with a weighted summation of pixel occurrence frequencies to generate a saliency map, numerical localization of high-contribution decision regions in the image can be achieved. Based on the saliency map, candidate regions are selected and random pixel values ​​are applied to disrupt the trigger space structure based on local pixel patches. This helps the model learn global semantic features, improving the model's defense capabilities against collusion attack features and its robustness in data processing.

[0057] In one embodiment, S2 may include:

[0058] S21. Remove the target category annotation information from all samples in the preprocessed labeled training set to generate an unlabeled dataset.

[0059] Specifically, target category annotation information can be label data appended to samples to indicate their classification affiliation. Unlabeled datasets can be collections that retain only the data ontology after removing classification labels.

[0060] Optionally, the training device can traverse each sample in the preprocessed labeled training set to locate the target category annotation information attached to each sample. The training device can separate the target category annotation information from the corresponding sample record, thus decoupling the sample's pixel data from the classification label. The training device can retain the main pixel matrix of samples without attached label information and integrate all the main pixel matrices of samples stripped of target category annotation information to generate an unlabeled dataset.

[0061] S22. Randomly crop and Gaussian blur perturb the unlabeled dataset to generate enhanced sample pairs. Input the enhanced sample pairs into the feature extraction network in the neural network model to extract the deep representation vectors of the enhanced sample pairs. Combine the deep representation vectors from the same original sample into positive sample pairs and cross-pair the deep representation vectors from different original samples into negative sample pairs. Integrate to generate contrastive feature representation pairs.

[0062] Specifically, random cropping can be an operation that selects sub-regions with varying sizes and positions within an image pixel matrix. Gaussian blur perturbation can be an operation that applies a Gaussian kernel function to smooth image pixels, incorporating numerical variations. Augmented sample pairs can be combinations of data copies obtained after perturbing a single image. Positive sample pairs can be combinations of deep representation vectors originating from the same original sample. Negative sample pairs can be combinations of deep representation vectors originating from different original samples.

[0063] Optionally, the training device can perform random cropping on samples in the unlabeled dataset and apply Gaussian blur perturbation to the samples in the unlabeled dataset. The training device can generate augmented data with viewpoint variations for each sample and combine the augmented data with viewpoint variations into augmented sample pairs. The training device can input the augmented sample pairs into the feature extraction network in the neural network model and run the multiply-accumulate operation logic of the feature extraction network. The training device can compute the deep representation vector of the augmented sample pair and identify the source sample identifier of the deep representation vector. The training device can pair and combine deep representation vectors originating from the same original sample to establish positive sample pairs. The training device can extract deep representation vectors originating from different original samples and cross-pair and combine deep representation vectors originating from different original samples to establish negative sample pairs. The training device can put positive and negative sample pairs into a data structure sequence to generate contrastive feature representation pairs.

[0064] S23. Perform spatial distance measurement on the contrastive feature representations, calculate the cosine similarity of positive sample pairs derived from the same original sample in the feature space, and extend the feature distance of negative sample pairs derived from different samples to generate self-supervised loss values.

[0065] Specifically, spatial distance measurement can be an operation that quantifies the positional differences between two vectors in the feature vector space. Cosine similarity can be a numerical measure of directional consistency by calculating the cosine of the angle between two vectors. Feature distance can be a numerical indicator that characterizes the degree of difference between samples in the feature space. Self-supervised loss can be an error value used to guide the model to optimize parameters based on the comparative relationships within the data.

[0066] Optionally, the training device can perform spatial distance measurement on the contrastive feature representation pairs, read the two deep representation vectors in the positive sample pairs derived from the same original sample, and calculate the dot product of the two deep representation vectors in the positive sample pairs. The training device can calculate the product of the magnitudes of the two deep representation vectors in the positive sample pairs, and divide the dot product by the magnitude product to obtain the cosine similarity. The training device can set an optimization term in the objective function to make the cosine similarity approach the upper bound. The training device can read the deep representation vectors in the negative sample pairs derived from different samples, and calculate the feature distance of the deep representation vectors in the negative sample pairs. The training device can set an optimization term in the objective function to increase the feature distance. The training device can perform logical calculations to push away the feature distance of negative sample pairs derived from different samples. The training device can combine the cosine similarity term of the positive sample pairs and the feature distance term of the negative sample pairs, perform numerical solutions to the loss function, and generate a self-supervised loss value.

[0067] S24. Perform self-supervised gradient optimization based on the self-supervised loss value, minimize the self-supervised loss value through gradient backpropagation, and obtain the purified feature extraction layer parameters of the neural network model.

[0068] For example, the formula for calculating the parameters of the purified feature extraction layer can be:

[0069]

[0070] in, To purify the parameters of the feature extraction layer; These are the parameters for the feature extraction layer to be trained. Indicated by To optimize variables, we need to find parameter values ​​that minimize the objective function; This refers to data from an unlabeled dataset. This is an unlabeled dataset; This indicates a loss due to self-monitoring.

[0071] Specifically, self-supervised gradient optimization can be a process of calculating the partial derivatives of parameters based on the error value and updating the weights of the neural network. Gradient backpropagation can be an algorithm that calculates the gradient vector layer by layer from the output to the input of the neural network.

[0072] Optionally, the training device can perform self-supervised gradient optimization based on the self-supervised loss value. The training device can calculate the partial derivatives of the parameters layer by layer from the top to the bottom of the feature extraction network to obtain the parameters of the feature extraction layer to be trained and their corresponding gradient information. The training device can apply the learning rate and gradient information to perform parameter numerical updates, and through multiple iterations, find the parameter direction that reduces the objective function value. The training device can combine the calculation formula of the purified feature extraction layer parameters to perform variable solving calculations to find the parameter values ​​that minimize the objective function. After completing the numerical update of the parameters of the feature extraction layer to be trained, the training device outputs the converged set of network weights, obtaining the purified feature extraction layer parameters of the neural network model.

[0073] This embodiment provides a self-supervised model training security defense method. By removing target category annotation information from the preprocessed labeled training set to generate an unlabeled dataset, and combining this with random cropping and Gaussian blur perturbation of the unlabeled dataset to generate enhanced sample pairs, the isolation between feature learning input data and target category labels can be achieved. A self-supervised loss value is generated by calculating the cosine similarity of positive sample pairs and the feature distance of negative sample pairs, and this loss value is minimized based on gradient backpropagation. This prompts the feature extraction network to extract deep representations only based on the inherent distribution patterns of the data. Applying a self-supervised purification formula to update weight parameters can block the mapping and binding of potential backdoor features to target labels, helping to improve the model's immunity to high-order feature entanglement attacks and the lower bound of basic defense in distributed systems.

[0074] In one embodiment, S3 may include:

[0075] S31. Input the purified feature extraction layer parameters into the feature extraction layer of the neural network model, lock the gradient update channel of the corresponding layer, and generate the feature solidification model.

[0076] Specifically, the neural network model can be a deep learning network that includes a multi-layered computational structure. The gradient update path can be the computational path in the backpropagation algorithm that propagates errors and modifies weights. The feature-fixed model can be a neural network model in which the parameters of the feature extraction layer are frozen and no longer participate in the update.

[0077] Optionally, the training device can input the purified feature extraction layer parameter values ​​into the feature extraction layer of the neural network model. The training device can override the original initial weights of the feature extraction layer and operate on the backpropagation computation graph of the feature extraction layer. The training device can lock the gradient update channels of the corresponding layer, forcing the gradient values ​​of all neurons in the feature extraction layer to zero. The training device can stop making any numerical modifications to the purified feature extraction layer parameters. The training device can solidify the feature mapping relationships of the network's front end by blocking gradient propagation, encapsulating the entire neural network model with locked parameters to generate a feature-fixed model.

[0078] S32. Extract deep visual features from the preprocessed labeled training set using the feature solidification model, input the deep visual features into the classification layer of the neural network model, calculate the prediction confidence of the samples in the preprocessed labeled training set for each target category, and generate the preliminary prediction probability matrix of the neural network model.

[0079] Specifically, deep visual features can be high-dimensional vector data calculated by a feature solidification model. Prediction confidence can be the probability value of a sample belonging to each target category, calculated by a neural network model. The preliminary prediction probability matrix can be a set of two-dimensional arrays containing the predicted probabilities of all samples for all target categories.

[0080] Optionally, the training device can load samples from the preprocessed labeled training set into the feature solidification model in batches and perform calculations using the forward propagation algorithm of the feature solidification model. The training device can input deep visual features into the classification layer of the neural network model to perform linear mapping operations. The training device can measure the prediction confidence of each sample in the preprocessed labeled training set for each target category. The training device can integrate the prediction confidence data of all samples, arrange the data dimensions according to the sample sequence and category sequence, and generate a preliminary prediction probability matrix for the neural network model.

[0081] S33. Perform symmetric cross-entropy calculation on the preliminary prediction probability matrix of the neural network model and the preprocessed labeled training set. By combining the standard cross-entropy with the inverse cross-entropy, suppress the gradient update bias caused by potential noise labels in the preprocessed labeled training set, and generate the symmetric cross-entropy loss value.

[0082] Specifically, symmetric cross-entropy calculation can be a computational process that fuses the cross-entropy divergence of two relative directions to evaluate the difference in probability distributions. Standard cross-entropy can be a calculation function that evaluates the error in predicting the probability distribution based on the true label distribution. Inverse cross-entropy can be a calculation function that evaluates the error in the true label distribution based on the predicted probability distribution. Noisy labels can be classification labels in the dataset that do not match the true content of the samples or have been tampered with. Gradient update bias can be a deviation parameter update vector generated by outlier labels.

[0083] Optionally, the training device can perform symmetric cross-entropy calculations on the preliminary prediction probability matrix of the neural network model and the preprocessed labeled training set, calculating both standard cross-entropy and inverse cross-entropy. The training device can combine the standard and inverse cross-entropy calculations and perform backpropagation based on the combined calculation result to output the error. The training device can amplify the difference in loss values ​​between clean and noisy labeled samples using the inverse cross-entropy term. The training device can use the error generated by the combined calculation to guide parameter update calculations. The training device can suppress the guiding effect of potential noisy labels in the preprocessed labeled training set on the gradient vector, reducing gradient update bias caused by noisy labels and outputting a symmetric cross-entropy loss value.

[0084] S34. Based on the symmetric cross-entropy loss value, while keeping the feature-fixed model parameters locked, only the weights of the classification layer are updated to obtain the converged classification layer parameters. The feature-fixed model and the converged classification layer parameters are then integrated to obtain the basic neural network model parameters.

[0085] For example, the formula for calculating the converged classification layer parameters can be:

[0086]

[0087] in, These are the parameters of the classification layer after convergence; These are the parameters for the classification layer to be trained; Indicated by To optimize variables, we need to find parameter values ​​that minimize the objective function; Preprocess the labeled training set; To preprocess samples in the labeled training set; For the sample Corresponding tags; This represents a feature solidification model composed of purified feature extraction layer parameters and classification layer parameters to be trained; This represents the symmetric cross-entropy loss value.

[0088] Specifically, the converged classification layer parameters can be a set of classification layer weights whose numerical changes are below a set threshold after multiple iterations.

[0089] Optionally, the training device can perform computations while keeping the feature-fixed model parameters locked, importing the symmetric cross-entropy loss value into the backpropagation algorithm computation graph, and calculating only the error gradient of the classification layer through the backpropagation algorithm. The training device can update only the weights of the classification layer and execute the calculation formula for the classification layer parameters. The training device can extract the feature-fixed model parameters in the locked state and the converged classification layer parameters. The training device can concatenate the feature-fixed model parameters and the classification layer parameters to obtain the basic neural network model parameters.

[0090] This embodiment provides a self-supervised model training security defense method. By inputting purified feature extraction layer parameters into the feature extraction layer and locking the gradient update channel to generate a feature-fixed model, the feature representation learning stage and the classification label mapping stage can be decoupled, which helps to cut off the erroneous mapping path between the backdoor trigger and the target category label. By calculating the prediction confidence and performing symmetric cross-entropy calculation through the feature-fixed model, the combination of standard cross-entropy and inverse cross-entropy can be used to suppress gradient update bias caused by potential noisy labels. By updating only the classification layer weights while keeping the feature-fixed model parameters locked, parameter convergence for the classification task can be completed while preserving the pure deep visual feature distribution. This can improve the robustness of the neural network model against poisoned data attacks and help improve the training security level of the basic neural network model parameters.

[0091] In one embodiment, S4 may include:

[0092] S41. Input the preprocessed labeled training set into the evaluation network constructed based on the parameters of the basic neural network model, calculate the cross-entropy divergence between the predicted output and the real label, quantify the convergence difficulty of each sample in the preprocessed labeled training set, and generate the forward propagation loss value.

[0093] Specifically, cross-entropy divergence can be a computational function that measures the numerical difference between the predicted probability distribution and the true label distribution. Convergence difficulty can be a measure of the computational error level required for a neural network model to fit the features of the sample data. Forward propagation loss can be a numerical variable representing the magnitude of the sample classification error, obtained through unidirectional network operations.

[0094] Optionally, the training device can construct an evaluation network based on the parameters of the basic neural network model and determine the node connection weights of the evaluation network. The training device can input the preprocessed labeled training set into the evaluation network, triggering the matrix multiplication and addition logic within the evaluation network. The training device can calculate the predicted output for samples in the preprocessed labeled training set. The training device can extract the true labels corresponding to each sample in the preprocessed labeled training set, import the predicted output and the true labels into a mathematical calculation unit, and calculate the cross-entropy divergence between the predicted output and the true labels. The training device can use the cross-entropy divergence calculation result to measure the error distribution of the neural network model fitting the data. The training device can quantify the convergence difficulty of each sample in the preprocessed labeled training set through numerical calculations and recording mechanisms, converting the convergence difficulty into an objective calculated value and binding it to the corresponding data sample record table. The training device can integrate the quantified output results of samples in the preprocessed labeled training set, extract the error metric data calculated for each sample, and generate the forward propagation loss value.

[0095] S42. Samples with forward propagation loss values ​​below a preset quantile are assigned to the high confidence set, and samples with values ​​above a preset quantile are assigned to the low confidence set. Labels are then removed from the low confidence set to generate an auxiliary unlabeled dataset.

[0096] Specifically, the preset quantile refers to the proportional boundary parameter that the system pre-sets to truncate the distribution of the data sequence.

[0097] Optionally, the training device can sort the samples in the preprocessed labeled training set in ascending order according to their forward propagation loss values. The training device can calculate the truncation node positions of the data sequence based on preset quantiles. The training device can locate samples within the preset quantiles and classify them into the high-confidence set. The training device can locate samples exceeding the preset quantiles and classify them into the low-confidence set. The training device can remove the labels from the low-confidence set, disconnecting the low-confidence sample content matrix from the target label encoding. The training device can integrate the processed low-confidence sample content matrix to generate an auxiliary unlabeled dataset.

[0098] S43. Perform semi-supervised distribution alignment calculation on the neural network model based on the high confidence set and the auxiliary unlabeled dataset, jointly calculate the supervised loss of labeled data and the regularization consistency loss of unlabeled data, and generate a semi-supervised joint loss value.

[0099] Specifically, semi-supervised distribution alignment computation can be a hybrid computation mechanism that simultaneously utilizes labeled data for error backpropagation and unlabeled data for feature space constraints within the same training architecture. Joint computation can be a computational process that fuses multiple error values ​​into a single overall metric through mathematical operations such as scaling and addition. Regularized consistency loss can be a constraining value that measures the degree of difference in the network model's output prediction distribution for the same unlabeled sample before and after introducing perturbation. The semi-supervised joint loss value can be the final error variable used to guide network parameter updates, formed by fusing the supervised loss and the regularized consistency loss.

[0100] Optionally, the training device can perform semi-supervised distribution alignment computation on the neural network model based on a high-confidence set and an auxiliary unlabeled dataset, and perform error measurement using the high-confidence set samples and the classification labels retained by the high-confidence set samples. The training device can calculate the supervised loss for labeled data. The training device can apply data augmentation or feature perturbation to the auxiliary unlabeled dataset samples. The training device can measure the divergence of the neural network model's output predictions for samples before and after perturbation. The training device can calculate the regularized consistency loss for unlabeled data. The training device can perform coefficient multiplication and addition operations on the supervised loss for labeled data and the regularized consistency loss for unlabeled data. The training device can perform joint computation, integrating multi-dimensional error constraint information to generate a semi-supervised joint loss value.

[0101] S44. Fine-tune the network weights of the neural network model by minimizing the semi-supervised joint loss value through gradient backpropagation, and calculate the difference between the network weights and the parameters of the basic neural network model. Extract the model weight change information of the neural network model in the semi-supervised joint loss stage to obtain the final neural network model parameters and the neural network model update gradient.

[0102] For example, the formula for calculating the final neural network model parameters can be:

[0103]

[0104] in, These are the parameters for the final neural network model; This represents the parameters of the neural network model to be fine-tuned; Indicated by To optimize the variables, we need to find parameter values ​​that minimize the objective function; For high confidence sets; To assist unlabeled datasets; This represents the semi-supervised joint loss value.

[0105] Specifically, network weights can be coefficient matrices upon which signal transmission between computational nodes of a neural network depends. Model weight change information can be vector data representing the numerical offset trajectory experienced by the neural network parameters during the current training phase. Optimization variables can be parameter objects that are in an iteratively changing state during the process of solving for the minimum value and whose optimal corresponding values ​​need to be obtained. The objective function can be a mathematical expression that drives parameter updates and system state changes.

[0106] Optionally, the training device can calculate the partial derivative vectors of network nodes based on the semi-supervised joint loss value. The training device can minimize the semi-supervised joint loss value through gradient backpropagation. The training device can fine-tune the network weights of the neural network model under the guidance of the loss function. The training device can read the base neural network model parameter matrix saved before fine-tuning and the currently updated network weight matrix, and calculate the difference between the network weights and the base neural network model parameters. The training device can extract the model weight change information of the neural network model during the semi-supervised joint loss stage, apply the calculation formula of the final neural network model parameters to perform mathematical derivation and assignment operations, and obtain the final neural network model parameters. The training device can configure the neural network model weight change information as the gradient for updating the neural network model.

[0107] This embodiment provides a self-supervised model training security defense method. By calculating the forward propagation loss value of samples and setting quantiles, the data is divided into high-confidence and low-confidence sets, enabling numerical screening and isolation of potentially poisoned samples and abnormally high-frequency triggering samples in the dataset. An auxiliary unlabeled dataset is generated by stripping labels from the low-confidence set, and a semi-supervised joint loss value is generated by combining it with the high-confidence set for semi-supervised distribution alignment calculation. This utilizes the spatial structural features of the unlabeled data to constrain and correct the model's classification decision boundary, helping to reduce the interference of abnormal labels on the direction of network weight updates. The network weights are fine-tuned by minimizing this joint loss value based on gradient backpropagation, and the difference is extracted as the update gradient. This improves the purity of security features within the update gradient uploaded from the local device to the server, contributing to enhanced distributed defense robustness of the global model during the federated aggregation stage.

[0108] In one embodiment, S5 may include:

[0109] S51. Calculate the mean scalar for each level dimension of the high-dimensional gradient tensor that updates the gradient along the neural network model. Extract the statistical features of parameter distribution in the low-dimensional space based on the mean scalar and generate a low-dimensional feature vector.

[0110] Specifically, a high-dimensional gradient tensor can be a network parameter array structure that includes data in multiple dimensions. Layer dimension can be the coordinate axes of the parameter matrix corresponding to different depths of computational layers in a neural network. The mean scalar can be a single numerical value obtained by summing the values ​​within a specific dimension and dividing by the number of elements. The low-dimensional space can be a data representation vector space with reduced dimensions compared to the original tensor. Parameter distribution statistical characteristics can be measures reflecting the central tendency and dispersion of values ​​within a dataset. Low-dimensional feature vectors can be numerical sequences representing the attributes of model parameters in a low-dimensional space.

[0111] Optionally, the training device can traverse each level dimension of the high-dimensional gradient tensor corresponding to the gradient update of the neural network model, summing the values ​​within each level dimension. The training device can then divide the sum by the number of elements corresponding to each level dimension. The training device can calculate the mean scalar along each level dimension and use the mean scalar as the basic data input to the feature analysis engine. The training device can extract the parameter distribution statistical features in the low-dimensional space based on the mean scalar. The training device can quantify the distribution pattern of the parameter updates at the current node and arrange the extracted parameter distribution statistical features according to rules. The training device can integrate the above statistical feature data to generate a low-dimensional feature vector.

[0112] S52. Configure the low-dimensional feature vector as the anomaly screening parameter and configure the neural network model update gradient as the aggregation update parameter, generate the federated aggregation control command and send it to the federated server.

[0113] Specifically, anomaly screening parameters can be the basis variables used to perform malicious node detection on the server side. Aggregation update parameters can be local gradient data used to merge global model parameters on the server side. Federated aggregation control instructions can be communication signaling carrying configuration information and controlling server behavior. The federated server can be the central computing node in a distributed network responsible for overall management and parameter aggregation.

[0114] Optionally, the training device can configure low-dimensional feature vectors as anomaly screening parameters. The training device can configure the neural network model update gradient as aggregation update parameters. The training device can integrate the anomaly screening parameters and aggregation update parameters, and can encapsulate and compute the integrated data according to the protocol format to generate federated aggregation control instructions. The training device can establish a network communication link with the federated server, send the federated aggregation control instructions to the federated server, and synchronously send the neural network model update gradient to the federated server.

[0115] Specifically, the federation aggregation control instructions are used to control the federation server to perform the following steps:

[0116] S521. Perform spectral clustering analysis based on anomaly screening parameters to obtain the anomaly probability of each client, and assign dynamic aggregation weights to each client according to the anomaly probability.

[0117] Specifically, spectral clustering analysis can be an unsupervised learning algorithm that uses graph theory principles to partition data based on a similarity matrix. Anomaly probability can be a numerical value that quantifies the likelihood that data uploaded by a client contains malicious features or belongs to an anomalous distribution. Dynamic aggregation weights can be multiplier factors temporarily calculated and assigned to each client when merging parameters.

[0118] Optionally, the federated server can extract low-dimensional statistical features from the anomaly screening parameters. The federated server can perform spectral clustering analysis based on the anomaly screening parameters to calculate the similarity matrix of the anomaly screening parameters for each client in the low-dimensional space. The federated server can construct a Laplacian matrix based on the similarity matrix and calculate eigenvectors. The federated server can divide the data of each client into different data clusters and measure the spatial geometric distance of each client from the normal majority data cluster. The federated server can obtain the anomaly probability of each client through distance conversion, input the anomaly probability into a weight decay function, and calculate the corresponding weight value for each client. The federated server can reduce the weight coefficient of clients with high probability values ​​and can assign dynamic aggregation weights to each client based on the anomaly probability.

[0119] S522. Based on the dynamic aggregation weight, the aggregation update parameters of each client are globally weighted and aggregated to obtain the global security model parameters for the next round.

[0120] For example, the formula for calculating the parameters of the next round of global security model can be:

[0121]

[0122] in, For the next round of global security model parameters, Indicates the first Communication rounds; The total number of clients participating in federated aggregation; Indicates the first The client in the first The neural network model updates the gradient in rounds.

[0123] Specifically, dynamic aggregation weights can be coefficients calculated based on client anomaly probabilities to control the parameter merging ratio. Aggregated update parameters can be the local model parameter change matrices submitted by each client after training. Global weighted aggregation can be a distributed computing mechanism that integrates data from multiple computing nodes by proportionally multiplying and adding them according to their corresponding weight coefficients.

[0124] Optionally, the federated server can parse the communication receive queue to obtain the aggregated update parameters uploaded by each client and verify the dimensionality integrity of the aggregated update parameters. The federated server can associate the aggregated update parameters with their corresponding dynamic aggregation weights in a data structure. The federated server can perform global weighted aggregation of the aggregated update parameters from each client based on the dynamic aggregation weights, performing scalar and tensor multiplication operations between the dynamic aggregation weights and the corresponding aggregated update parameters. The federated server can perform element-wise summation on the tensor obtained from multiplying all clients. The federated server can execute the network parameter update calculation formula to obtain the parameters for the next round of global security model.

[0125] This embodiment provides a self-supervised model training security defense method. By calculating the mean scalar along each level dimension of the high-dimensional gradient tensor and extracting low-dimensional feature vectors, the high-dimensional parameter space can be compressed into a low-dimensional representation space. This helps reduce network bandwidth overhead and anomaly detection computation costs on the server side during federated communication. Sending the low-dimensional feature vectors as anomaly screening parameters to the server and controlling its spectral clustering analysis allows for statistical comparison based on parameter distribution differences between different clients, thereby improving the system's ability to identify hidden poisoned nodes and potential malicious gradients. Assigning dynamic aggregation weights to each client based on the calculated anomaly probability and performing global weighted aggregation can suppress the influence of anomaly node update parameters on the global model, helping to improve the robustness of the federated learning global security model in adversarial attack environments.

[0126] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0127] Based on the same inventive concept, this application also provides a system for implementing the self-supervised model training security defense method described above. The solution provided by this device is similar to the implementation scheme described in the above method; therefore, the specific limitations of one or more self-supervised model training security defense system embodiments provided below can be found in the limitations of a self-supervised model training security defense method described above, and will not be repeated here.

[0128] In one exemplary embodiment, such as Figure 3 As shown, a self-supervised model training security defense system 600 is provided to implement the methods in the above-described method embodiments. This device may include:

[0129] The preprocessing module 601 is used to perform joint frequency and spatial preprocessing on the local training dataset with original labels. It filters out high-frequency malicious signals in the training dataset by frequency domain masking and applies random erasure to the candidate regions in the local training dataset to generate a preprocessed labeled training set.

[0130] The self-supervised feature training module 602 is used to strip the label information from the preprocessed labeled training set to generate an unlabeled dataset. The unlabeled dataset is then input into the feature extraction layer of the neural network model to be trained. The self-supervised loss is calculated by narrowing the distance between positive sample pairs and widening the distance between negative sample pairs based on the self-supervised learning mechanism. The self-supervised loss is minimized through gradient backpropagation to obtain the purified feature extraction layer parameters of the neural network model.

[0131] The self-supervised classification training module 603 is used to fix the parameters of the purified feature extraction layer in the neural network model, and combine the label information in the preprocessed labeled training set to minimize the symmetric cross-entropy loss of the classification layer in the neural network model to obtain the basic neural network model parameters.

[0132] The semi-supervised joint loss module 604 is used to calculate the forward propagation loss value of samples in the preprocessed labeled training set based on the parameters of the basic neural network model. Based on the forward propagation loss value, the preprocessed labeled training set is divided into a high-confidence set and a low-confidence set. The labels in the low-confidence set are removed to generate an auxiliary unlabeled dataset. The semi-supervised joint loss is calculated in combination with the high-confidence set. The semi-supervised joint loss is minimized through gradient backpropagation to obtain the final neural network model parameters and the neural network model update gradient.

[0133] The communication interaction module 605 is used to generate federated aggregation control instructions based on the gradient update of the neural network model, send the federated aggregation control instructions to the federated server, control the federated server to perform global weighted aggregation of the gradient update of the neural network model, generate the next round of global security model parameters, and receive the next round of global security model parameters.

[0134] In one embodiment, the preprocessing module 601 may include:

[0135] The image filtering unit is used to transform the images in the training dataset with original labels to the frequency domain to obtain a frequency domain image. It calls a smoothing mask matrix to filter out potential malicious triggering signals in the high-frequency bands of the frequency domain image to obtain a frequency domain filtered image. Finally, it reconstructs the frequency domain filtered image through inverse Fourier transform to generate a malicious-removed image.

[0136] The saliency map unit is used to perform feature evaluation processing on the malicious image removal. It calculates the absolute value of the difference between each pixel and its neighboring pixels in terms of color and spatial distribution, and uses the frequency of pixel occurrence in the malicious image removal as a weight to perform weighted summation, generating a saliency map that represents the image's contribution to decision-making.

[0137] The training set preprocessing unit is used to select regions with saliency values ​​greater than a preset saliency threshold as candidate regions based on the saliency map, and to cover the candidate regions with random pixel values ​​to generate a preprocessed labeled training set.

[0138] In one embodiment, the self-supervised feature training module 602 may include:

[0139] The label removal unit is used to remove the target category annotation information from all samples in the preprocessed labeled training set, generating an unlabeled dataset.

[0140] The contrastive feature representation pair generation unit is used to randomly crop and Gaussian blur perturb the unlabeled dataset to generate enhanced sample pairs. The enhanced sample pairs are then input into the feature extraction network in the neural network model to extract the deep representation vectors of the enhanced sample pairs. The deep representation vectors from the same original sample are paired and combined into positive sample pairs, and the deep representation vectors from different original samples are cross-paired and combined into negative sample pairs, thus integrating to generate contrastive feature representation pairs.

[0141] The self-supervised loss unit is used to perform spatial distance measurement on the contrastive feature representation pairs, calculate the cosine similarity of positive sample pairs derived from the same original sample in the feature space, and extend the feature distance of negative sample pairs derived from different samples to generate self-supervised loss values.

[0142] The purification parameter unit is used for self-supervised gradient optimization based on the self-supervised loss value. By minimizing the self-supervised loss value through gradient backpropagation, the purified feature extraction layer parameters of the neural network model are obtained.

[0143] In one embodiment, the self-supervised classification training module 603 may include:

[0144] The feature solidification model unit is used to input the purified feature extraction layer parameters into the feature extraction layer of the neural network model, lock the gradient update channel of the corresponding layer, and generate the feature solidification model.

[0145] The probability matrix generation unit is used to extract deep visual features from the preprocessed labeled training set through the feature solidification model, input the deep visual features into the classification layer of the neural network model, measure the prediction confidence of the samples in the preprocessed labeled training set on each target category, and generate the preliminary prediction probability matrix of the neural network model.

[0146] The symmetric cross-entropy unit is used to calculate the symmetric cross-entropy of the initial prediction probability matrix of the neural network model and the preprocessed labeled training set. By combining the standard cross-entropy with the inverse cross-entropy, it suppresses the gradient update bias caused by potential noise labels in the preprocessed labeled training set and generates the symmetric cross-entropy loss value.

[0147] The basic model parameter unit is used to update only the weights of the classification layer based on the symmetric cross-entropy loss value, while keeping the feature-fixed model parameters locked, to obtain the converged classification layer parameters. The feature-fixed model and the converged classification layer parameters are then integrated to obtain the basic neural network model parameters.

[0148] In one embodiment, the semi-supervised joint loss module 604 may include:

[0149] The forward propagation unit is used to input the preprocessed labeled training set into the evaluation network built based on the parameters of the basic neural network model, calculate the cross-entropy divergence between the predicted output and the real label, quantify the convergence difficulty of each sample in the preprocessed labeled training set, and generate the forward propagation loss value.

[0150] The unlabeled dataset unit is used to classify samples with forward propagation loss values ​​below a preset quantile into a high-confidence set, and samples with values ​​above a preset quantile into a low-confidence set, and remove the labels from the low-confidence set to generate an auxiliary unlabeled dataset.

[0151] The semi-supervised computation unit is used to perform semi-supervised distribution alignment computation on the neural network model based on the high confidence set and the auxiliary unlabeled dataset. It jointly calculates the supervised loss of labeled data and the regularization consistency loss of unlabeled data to generate a semi-supervised joint loss value.

[0152] The model parameters and update gradient unit are used to fine-tune the network weights of the neural network model by minimizing the semi-supervised joint loss value through gradient backpropagation, and to calculate the difference between the network weights and the basic neural network model parameters. This extracts the model weight change information of the neural network model during the semi-supervised fine-tuning stage, and obtains the final neural network model parameters and neural network model update gradient.

[0153] In one embodiment, the communication interaction module 605 may include:

[0154] The low-dimensional feature vector unit is used to calculate the mean scalar at each level of the high-dimensional gradient tensor that updates the gradient along the neural network model. Based on the mean scalar, the statistical features of the parameter distribution in the low-dimensional space are extracted to generate a low-dimensional feature vector.

[0155] The control instruction generation unit is used to configure low-dimensional feature vectors as anomaly screening parameters and configure the neural network model update gradient as aggregation update parameters, generate federated aggregation control instructions, and send them to the federated server.

[0156] In one embodiment, a computer device is provided, including a memory and a processor, the memory storing a computer program, the processor executing the computer program to implement the steps of a self-supervised model training security defense method as described above.

[0157] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps in the above method embodiments.

[0158] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to in the description of the method embodiments. The device embodiments described above are merely illustrative. The components described as separate parts may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this disclosure according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0159] The above-described embodiments are merely illustrative of several implementation methods of the embodiments of this application, and their descriptions are relatively specific and detailed. However, they should not be construed as limiting the scope of the patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the embodiments of this application, and these modifications and improvements all fall within the protection scope of the embodiments of this application.

Claims

1. A method for training a self-supervised model for secure defense, characterized in that, Applied to a client, the method includes: S1. Perform joint frequency and spatial preprocessing on the local training dataset with original labels. Filter out high-frequency malicious signals in the training dataset by frequency domain masking and apply random erasure to the candidate regions in the local training dataset to generate a preprocessed labeled training set. S2. Remove the label information from the preprocessed labeled training set to generate an unlabeled dataset. Input the unlabeled dataset into the feature extraction layer of the neural network model to be trained. Calculate the self-supervised loss based on the self-supervised learning mechanism to bring positive sample pairs closer and push negative sample pairs further apart. Minimize the self-supervised loss through gradient backpropagation to obtain the purified feature extraction layer parameters of the neural network model. S3. Fix the parameters of the purified feature extraction layer in the neural network model, and minimize the symmetric cross-entropy loss of the classification layer in the neural network model by combining the label information in the preprocessed labeled training set, to obtain the basic neural network model parameters; S4. Calculate the forward propagation loss value of the samples in the preprocessed labeled training set based on the parameters of the basic neural network model. Divide the preprocessed labeled training set into a high-confidence set and a low-confidence set based on the forward propagation loss value. Remove the labels from the low-confidence set to generate an auxiliary unlabeled dataset. Calculate the semi-supervised joint loss in combination with the high-confidence set. Minimize the semi-supervised joint loss through gradient backpropagation to obtain the final neural network model parameters and the neural network model update gradient. S5. Generate federated aggregation control instructions based on the neural network model update gradient, and send the federated aggregation control instructions and the neural network model update gradient to the federated server to control the federated server to perform global weighted aggregation of the neural network model update gradient to generate the next round of global safe model parameters; wherein, the next round of global safe model parameters includes the local training dataset with original labels for the next period.

2. The method according to claim 1, characterized in that, S1 includes: S11. Convert the images in the training dataset with original labels to the frequency domain to obtain frequency domain images. Use a smoothing mask matrix to filter out potential malicious triggering signals in the high-frequency bands of the frequency domain images to obtain frequency domain filtered images. Reconstruct the frequency domain filtered images through inverse Fourier transform to generate malicious-removed images. S12. Perform feature evaluation processing on the malicious-removed image, calculate the absolute value of the difference between each pixel and its neighboring pixels in terms of color and spatial distribution, and use the pixel occurrence frequency in the malicious-removed image as a weight to perform weighted summation, generating a saliency map characterizing the image's decision contribution; wherein, the calculation formula for the saliency map is: in, For the first The significance value of each pixel; For the first The intensity of each pixel; Indicates pixel intensity level Pixel intensity; Indicates pixel intensity level The frequency at which the pixels appear in the frequency-domain filtered image; S13. Based on the saliency map, regions with saliency values ​​greater than a preset saliency threshold are selected as candidate regions, and the candidate regions are covered with random pixel values ​​to generate the preprocessed labeled training set; wherein, the saliency value is used to characterize the contribution of the corresponding pixel to the output decision of the neural network model.

3. The method according to claim 1, characterized in that, S2 includes: S21. Remove the target category annotation information from all samples in the preprocessed labeled training set to generate an unlabeled dataset; S22. Randomly crop and Gaussian blur perturb the unlabeled dataset to generate enhanced sample pairs, and input the enhanced sample pairs into the feature extraction network in the neural network model to extract the deep representation vectors of the enhanced sample pairs. By pairing and combining deep representation vectors from the same original sample into positive sample pairs, and cross-pairing and combining deep representation vectors from different original samples into negative sample pairs, the comparison feature representation pairs are integrated to generate the comparison feature representation pairs. S23. Perform spatial distance measurement processing on the contrast feature representation pairs, calculate the cosine similarity of the positive sample pairs derived from the same original sample in the feature space, and extend the feature distance of the negative sample pairs derived from different samples to generate a self-supervised loss value. S24. Based on the self-supervised loss value, perform self-supervised gradient optimization processing, minimize the self-supervised loss value through gradient backpropagation, and obtain the purified feature extraction layer parameters of the neural network model; wherein, the calculation formula of the purified feature extraction layer parameters is: in, The parameters for the purified feature extraction layer; These are the parameters for the feature extraction layer to be trained. Indicated by To optimize variables, we need to find parameter values ​​that minimize the objective function; The data in the unlabeled dataset; This refers to the unlabeled dataset; This represents the self-monitored loss.

4. The method according to claim 1, characterized in that, The S3 includes: S31. Input the purified feature extraction layer parameters into the feature extraction layer of the neural network model, lock the gradient update channel of the corresponding layer, and generate a feature solidification model; S32. Extract deep visual features from the preprocessed labeled training set using the feature solidification model, input the deep visual features into the classification layer of the neural network model, calculate the prediction confidence of the samples in the preprocessed labeled training set for each target category, and generate a preliminary prediction probability matrix for the neural network model. S33. Perform symmetric cross-entropy calculation on the preliminary prediction probability matrix of the neural network model and the preprocessed labeled training set. By combining the standard cross-entropy with the inverse cross-entropy, suppress the gradient update deviation caused by potential noise labels in the preprocessed labeled training set, and generate a symmetric cross-entropy loss value. S34. Based on the symmetric cross-entropy loss value, while keeping the feature-fixed model parameters locked, only the weights of the classification layer are updated to obtain the converged classification layer parameters. The feature-fixed model and the converged classification layer parameters are then integrated to obtain the basic neural network model parameters. The calculation formula for the converged classification layer parameters is as follows: in, These are the parameters of the classification layer after convergence; The parameters of the classification layer to be trained; Indicated by To optimize variables, we need to find parameter values ​​that minimize the objective function; The preprocessed labeled training set; The samples in the preprocessed labeled training set; For the sample Corresponding tags; This refers to a feature solidification model composed of the purified feature extraction layer parameters and the classification layer parameters to be trained; is the symmetric cross-entropy loss value.

5. The method according to claim 1, characterized in that, The S4 includes: S41. Input the preprocessed labeled training set into the evaluation network constructed based on the parameters of the basic neural network model, calculate the cross-entropy divergence between the predicted output and the real label, quantify the convergence difficulty of each sample in the preprocessed labeled training set, and generate the forward propagation loss value. S42. Samples with forward propagation loss values ​​below a preset quantile are assigned to the high confidence set, and samples with values ​​exceeding the preset quantile are assigned to the low confidence set. Labels are then removed from the low confidence set to generate the auxiliary unlabeled dataset. S43. Based on the high confidence set and the auxiliary unlabeled dataset, perform semi-supervised distribution alignment calculation on the neural network model, jointly calculate the supervised loss of labeled data and the regularization consistency loss of unlabeled data, and generate a semi-supervised joint loss value. S44. Fine-tune the network weights of the neural network model by minimizing the semi-supervised joint loss value through gradient backpropagation, and calculate the difference between the network weights and the parameters of the basic neural network model. Extract the model weight change information of the neural network model in the semi-supervised joint loss stage to obtain the final neural network model parameters and the neural network model update gradient; wherein, the calculation formula of the final neural network model parameters is: in, These are the parameters of the final neural network model; This represents the parameters of the neural network model to be fine-tuned; Indicated by To optimize the variables, we need to find parameter values ​​that minimize the objective function; For the high confidence set; This refers to the auxiliary unlabeled dataset; The value is the semi-supervised joint loss.

6. The method according to claim 1, characterized in that, The S5 includes: S51. Calculate the mean scalar along each level dimension of the high-dimensional gradient tensor that updates the gradient of the neural network model, and extract the parameter distribution statistical features in the low-dimensional space based on the mean scalar to generate a low-dimensional feature vector. S52. Configure the low-dimensional feature vector as an anomaly screening parameter, and configure the neural network model update gradient as an aggregation update parameter, generate the federated aggregation control command and send it to the federated server; The federation aggregation control command is used to control the federation server to perform the following steps: S521. Perform spectral clustering analysis based on the anomaly screening parameters to obtain the anomaly probability of each client, and assign dynamic aggregation weights to each client according to the anomaly probability. S522. Based on the dynamic aggregation weight, the aggregation update parameters of each client are globally weighted and aggregated to obtain the next round of global security model parameters; wherein, the calculation formula of the next round of global security model parameters is: in, For the parameters of the next round of global security model, Indicates the first Communication rounds; The total number of clients participating in federated aggregation; Indicates the first The client in the first The neural network model updates the gradient in rounds.

7. A self-supervised model training security defense system, used to implement the method of any one of claims 1 to 7, characterized in that, The device includes: The preprocessing module is used to perform joint frequency and spatial preprocessing on the local training dataset with original labels. It filters out high-frequency malicious signals in the training dataset by frequency domain masking and applies random erasure to the candidate regions in the local training dataset to generate a preprocessed labeled training set. The self-supervised feature training module is used to strip the label information from the preprocessed labeled training set to generate an unlabeled dataset. The unlabeled dataset is then input into the feature extraction layer of the neural network model to be trained. The self-supervised loss is calculated based on the self-supervised learning mechanism to bring positive sample pairs closer and push negative sample pairs further apart. The self-supervised loss is minimized through gradient backpropagation to obtain the purified feature extraction layer parameters of the neural network model. The self-supervised classification training module is used to fix the parameters of the purified feature extraction layer in the neural network model, and minimize the symmetric cross-entropy loss of the classification layer in the neural network model by combining the label information in the preprocessed labeled training set, so as to obtain the basic neural network model parameters. The semi-supervised joint loss module is used to calculate the forward propagation loss value of the samples in the preprocessed labeled training set based on the parameters of the basic neural network model, divide the preprocessed labeled training set into a high-confidence set and a low-confidence set based on the forward propagation loss value, remove the labels from the low-confidence set to generate an auxiliary unlabeled dataset, calculate the semi-supervised joint loss in combination with the high-confidence set, and minimize the semi-supervised joint loss through gradient backpropagation to obtain the final neural network model parameters and neural network model update gradient. The communication interaction module is used to generate federated aggregation control instructions based on the gradient update of the neural network model, send the federated aggregation control instructions to the federated server, control the federated server to perform global weighted aggregation of the gradient update of the neural network model, generate the next round of global safe model parameters, and receive the next round of global safe model parameters; wherein, the next round of global safe model parameters includes the local training dataset with original labels for the next period.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

Images