142 results about "Inference attack" patented technology

Approximate closed-form bounds of Bayes security against record-level inference attacks, in particular membership and attribute inference attacks, are provided. In various embodiments, these approximate bounds of Bayes security are used in conjunction with training neural-network models by differential-privacy stochastic gradient descent to create trained models that achieve a desired level of privacy.

The invention discloses a DNS anomaly detection and domain name hijacking early warning method based on a multi-source NLP, and belongs to the technical field of network security. According to the method, firstly, multi-source information collection and NLP semantic analysis are carried out, malicious entities are identified from collected external information, and an attack mode is reasoned; performing association scoring on the identified malicious entity and an abnormal mode detected in a preset time window, if the score exceeds a threshold value, judging that a domain name hijacking behavior or a DNS poisoning behavior exists, executing a domain name hijacking early warning process, and storing an abnormal event log; and updating a multi-source information acquisition and NLP semantic analysis module and a DNS anomaly detection model according to feedback of a system alarm result, a log and sample data output by information association analysis and hijacking judgment. According to the invention, domain name hijacking and DNS abnormity can be quickly found, accurately positioned and timely warned, and false alarm and missing alarm are reduced.

The invention discloses a federated learning method and system based on differential privacy, belongs to the technical field of federated learning, and aims to solve the technical problem of how to realize federated learning under the condition of ensuring local data privacy and defending FL inference attack at the same time. Comprising the following steps that a private generation model is constructed based on an auxiliary classifier GAN, and the private generation model comprises a generator and two discriminators; the classification loss of the real sample and the classification sample is introduced to carry out model training on the private generation model, and a trained generator is obtained; each client carries out local training on the local model based on the synthetic sample to generate local model parameters of the local model, and uploads the local model parameters to a server, and the server carries out aggregation calculation based on the local model parameters and weights corresponding to the clients to obtain global model parameters, and sends the global model parameters to the server; and returning the global model parameters to each client for iterative model training.

The invention relates to the technical field of machine learning, in particular to a user local data distribution reasoning attack method based on federal learning. The method comprises the steps of generating an auxiliary data set, constructing a simulated attack user data set, training a shadow model, training a user model, calculating cosine similarity, adjusting the data proportion of various labels in the simulated attack user data set, forming an updated federal learning global model, judging a training termination condition, and obtaining a training result. Obtaining local data distribution of the target attack user; according to the method, the data privacy security problem caused by federal learning distributed training characteristics is solved; by deploying an auxiliary data set and a shadow model at a server side, analyzing and simulating an attack user data set by using a gradient direction, the distribution of local data of a target attack user on each category is dynamically deduced, the label distribution of the target attack user is revealed, and meanwhile, the method has relatively high concealment, adaptability and effectiveness.

The data privacy risk measurement method for machine learning comprises the following steps: simulating an attack, simulating potential privacy leakage through a member reasoning attack, and testing the privacy protection capability of a model in a real attack scene; data feature quantification: based on HDBSCAN clustering analysis and quasi-identifier vulnerability analysis, separately determining privacy disclosure risk quantification, identifying privacy risk factors in the data, and especially evaluating the influence of similarity between data points and feature attributes on privacy disclosure; and calculating an overall privacy risk score, integrating various evaluation results, and quantifying the privacy risk from a global perspective. The method has the beneficial effect that a comprehensive and quantitative data privacy risk index is provided. The method is suitable for data in various fields, and can help a data owner maintain efficient utilization of the data while protecting privacy.

The invention discloses a classification model privacy disclosure risk black box detection method and device, and relates to the technical field of machine learning security, and the method comprises the steps: constructing an auxiliary data set; generating an auxiliary model covering different privacy risk levels; training a shadow model and a black box member reasoning attack model corresponding to the shadow model; evaluating the privacy risk score of each auxiliary model by using the attack model, and labeling a high-risk / low-risk label; calculating a correction prediction entropy difference, and screening the first k samples with the maximum difference to form a query data set; extracting a prediction result of each auxiliary model on the query set, calculating and correcting a prediction entropy and a mean value, a variance, a kurtosis and a skewness thereof, and splicing into a k + 4-dimensional risk feature vector; training a risk detection classifier; similar features of the model to be detected are extracted and input into the classifier, and the classifier performs five-level privacy disclosure risk level mapping according to the final risk probability prediction value and outputs the five-level privacy disclosure risk level mapping. The method is mainly applied to risk assessment of classification models in high-privacy sensitive fields such as financial credit investigation and medical diagnosis.

A data privacy system automatically determines quasi-identifiers in a database containing individuals' records. The data privacy system applies a machine learning model to the database, the model configured to classify each record in the database and output a measure of its confidence in its classification. The data privacy system determines, based on the measure of confidence, how important each attribute is to the model's classification. The data privacy system iteratively applies a machine learning model on a modified database that includes the highest ranked attributes to identify the quasi-identifiers in the records in the database. The data privacy system can use identified quasi-identifiers to determine if the database is susceptible to a membership inference attack, and in response to such a determination, can perform one or more data privacy operations on the database to reduce this risk.

The invention belongs to the technical field of intelligent traffic system data privacy protection, and discloses a traffic flow federated security prediction method based on distributed homomorphic encryption, and the method comprises the steps: constructing a vehicle-road cloud integrated network architecture, constructing a local model and a global model based on LSTM, and forming a federated model; cooperatively generating a distributed key based on a road infrastructure node and a cloud platform node, and performing homomorphic encryption and training on the federated model; and traffic flow prediction is carried out based on the trained federal model. According to the method, the confidentiality of the model training process is higher, the confidentiality of shared model parameters can be ensured through the distributed key generation and homomorphic encryption computing technology, and inference attack threats based on model parameter analysis are avoided. According to the invention, the defect of data privacy in an existing traffic flow prediction system can be overcome, and an effective support technology is provided for intelligent traffic management.

The invention discloses a neural network model reasoning method for improving intermediate data security, and relates to the field of neural networks, in the method, in the neural network model reasoning process of an NPU, intermediate feature map data generated by each operator layer is encrypted in real time and stored in an end-side SoC memory in an encrypted form, and the security of the intermediate data is improved. Intermediate data leakage caused by analysis of an attacker on memory data can be effectively prevented, the intermediate data output by the neural network model is protected against cracking, threats in privacy leakage aspects such as member inference attacks can be effectively resisted, user privacy data is ensured not to be stolen, and user experience is improved. The data security of the neural network model in the reasoning process is greatly improved, and a powerful guarantee is provided for user information security.

The invention relates to a privacy enhancement federated learning method based on a conditional generative adversarial network, which is used for improving the performance of federated learning and enhancing the privacy protection capability. By designing an auxiliary data generation mechanism based on the CGAN, the training data set of the client is expanded, and the problem of data overfitting is solved. Meanwhile, a double-model coexistence concept of a private model and a public model of the client is provided, and an improved deep mutual learning model interaction training method is designed to ensure that sensitive information is only stored locally. In addition, through a high-performance model aggregation method based on prediction distribution, the performance of the global model is effectively improved. Experimental results show that the method can effectively resist various inference attacks while keeping the performance of the federal learning model, and the privacy protection capability of user data is remarkably enhanced.

The invention relates to the technical field of cloud task unloading and reinforcement learning, and particularly provides a task scheduling optimization method based on a double-agent strategy in a computing network environment, and the method comprises the steps: obtaining the energy consumption and delay of a task in an unloading process from a local processor MSDn to an edge server ECs based on a communication model; constructing an optimization target of task unloading based on the calculation model; and according to the optimization target, adopting a DAPO algorithm based on a double-agent strategy to obtain an optimal unloading strategy of the cloud edge end cooperation task. According to the method, the stability of the algorithm in a dynamic and complex environment is enhanced through double-agent configuration, the complex system state, space and decision-making process can be managed more effectively, privacy is quantified through the information entropy model, the uncertainty of an unloading mode is quantified, and therefore the attack inference difficulty is quantified.

The invention relates to the field of federated learning privacy security, in particular to a method for realizing member inference attack by controlling convolution kernel parameters. Firstly, an attacker designs convolution kernel parameters according to feature distribution of a part of a convolution region of a target sample, then records a non-zero gradient position of the target sample on a convolution kernel after back propagation, and sends a global model to a participant for local training, and finally, when the participant uploads a model update, the target sample is updated. And an attacker compares whether the non-zero gradient position of the manipulated convolution kernel is consistent with the recorded non-zero gradient position, so that member inference attack is implemented in a federated learning scene, and the technical problem that the traditional member inference attack cannot fully utilize a federated learning framework to improve the attack performance is solved.

The invention relates to the technical field of federated learning, in particular to a federated learning security aggregation method based on vector space secret sharing, which comprises the following steps that: a client divides a local model gradient into a plurality of secret shares and distributes the secret shares to a plurality of servers; the server carries out aggregation operation on the secret share through a security computing protocol, wherein the aggregation operation comprises addition and multiplication operation; calculating the similarity between the clients based on the secret state data, and screening out a credible client set; and executing security aggregation on the gradient share of the trusted client, reconstructing a global model gradient and issuing the global model gradient. According to the method, client poisoning attacks can be resisted, server reasoning attacks can also be resisted, a federal learning global model security aggregation method is designed based on vector space secret sharing, and threats caused by server offline are reduced.

A method including: receiving a training dataset and a testing dataset each comprising samples that were used to train and to test, respectively, a certain machine learning model; dividing the training dataset into non-overlapping training subsets, and the testing dataset into non-overlapping testing subsets; assigning the training subsets and the testing subsets into pairs, such that each of the pairs is a distinct combination of one of the training subsets with one of the testing subsets; training different membership inference attack (MIA) models on the pairs; and evaluating sensitive information leakage from the certain machine learning model based on aggregated inferences by the trained MIA models.

The invention discloses a privacy protection method for resisting label reasoning attacks in longitudinal federated learning, which comprises the following steps: S1, acquiring intermediate parameters of a local model uploaded by each client, and uploading the parameters to a server, including preprocessing target data; s2, the server side optimizes a loss function of the top model; s3, the server side trains the top model by receiving the intermediate parameters of the clients, and updates the gradient of the model through a gradient descent algorithm; in the process, the server side calculates the gradient of each client side, a proper amount of noise is introduced into the gradient, and the size of the local gradient is adjusted in a self-adaptive mode, so that iterative updating of model parameters is achieved. The process aims to realize collaborative optimization of the model while protecting data privacy. By means of the method, model optimization and updating can be effectively carried out, the model optimization efficiency is improved, the defense capacity on label reasoning attacks can be enhanced, and data privacy and safety are ensured.

The invention provides an unsupervised black box member reasoning attack method and system based on generative collaborative learning. A generator, a discriminator and a collaborative framework are introduced. Inputting the sample into a plurality of heuristic tasks, and calculating a task score of each heuristic task; aggregating all task scores by using a generator, generating a pseudo label of the sample, and filtering a noise sample based on a dynamic threshold value; calculating an offset vector between the sample input prompt and the output response of the large language model to represent the internal state of the large language model; and taking the offset vector as the input of a discriminator, and training the discriminator by means of the pseudo tag generated by the generator, so that the discriminator can predict the member probability of the sample according to the offset vector. And taking the member probability generated by the discriminator as a new pseudo tag of the sample to help the generator to perform training optimization. And iterating the cross supervision training until the discriminator converges. The discriminator obtained by training based on the method can realize high-performance member reasoning under complete black box and unsupervised conditions.

The invention discloses an anti-inference attack differential privacy protection method for a task unloading strategy in an edge computing environment, and belongs to the field of edge computing and privacy protection. The method comprises the following steps: constructing an unloading decision model framework based on a Markov decision process (MDP) in a mobile edge computing scene, a system model, a transmission model, a calculation model and a user task model; constructing a privacy model for the inference attack, speculating an unloading strategy of equipment through a system state, and evaluating privacy loss; on the basis, calculating stage smooth sensitivity, and disturbing an unloading strategy by adopting a differential privacy method of an exponential mechanism in combination with privacy loss; and dynamically adjusting the noise intensity according to the privacy loss and smooth sensitivity of the whole iteration period until the model converges. According to the method, the risk of leakage of the task unloading strategy is reduced, the method is suitable for the task unloading strategy of deep reinforcement learning, and the safety of the task unloading strategy in the edge computing environment can be effectively protected.

The application discloses a kind of inference system and method for sequence recommendation system member inference attack, including label data generation module, difference feature construction module and attack model training module;Step 1, label data generation is carried out;Step 2, the difference feature construction of member and non-member is carried out;Step 3, the training of attack model is carried out.Compared with prior art, the application can guarantee the data privacy of user in a wider range of scenarios;Fill in the blank of member inference attack in more stringent scenarios;Significantly improve the attack inference effect.

Disclosed concepts provide improved security of a deployed neural network (i.e. improved security during inferencing of a neural network), in particular against membership inference attacks. Specifically, a data scrambler and a descrambling layer are provided. The data scrambler receives an input from an entity (e.g., a user) and only scrambles the input if the entity is authenticated and authorized. The descrambling layer is attached to the deployed neural network, and descrambles the input so that the neural network may provide a correct output corresponding to the initial (unscrambled) input from the entity. Of course, if the entity is not authenticated and / or authorized, then the input will not be scrambled and so the descrambling layer will provide an incorrect input to the neural network. Accordingly, unauthenticated, or unauthorized entities will not receive the correct output corresponding to the input from the entity from the deployed neural network (as the input provided by such entities will not be scrambled by the data scrambler).

The present invention relates to a database (DB) inference attack control device and method and, more specifically, to a probabilistic logic-based database inference attack control device and method, in which: when receiving result data output from a database management system (DBMS) and inferring sensitive information from each result data record constituting the result data, data attributes are restricted on the basis of a sensitive inference attack element threshold, wherein the data attributes correspond to an inference attack element preconfigured to have a more critical impact compared to other inference attack elements; a risk probability of inferring the sensitive information is calculated on the basis of a preconfigured probabilistic inference attack logic; when the calculated risk probability exceeds a risk determination threshold, detection of an inference attack is determined, and one or more of data attributes corresponding to an inference attack element associated with the inference attack are restricted; and on the basis of an inference-possible logic, which is preconfigured as being usable for an inference attack, in addition to the probabilistic inference attack logic, data attributes corresponding to the inference attack element included in the inference-possible logic are restricted on the basis of an inference-possible logic threshold.

This invention provides a clustering-based method for attacking interaction-level membership in a recommender system. It involves constructing a shadow dataset with a distribution consistent with the training dataset of the target recommender system, and training a shadow recommender model based on it. For users in the shadow dataset, the method obtains the corresponding recommendation list provided by the shadow recommender model. The recommendation list is then clustered to generate user representation vectors representing users' multiple interests. Furthermore, based on the user representation vectors, the vector representations of candidate items, and the similarity between candidate items and each cluster, an attack feature vector is constructed. This attack feature vector is then used to train the attack model. The interaction to be audited in the target recommender system is input into the trained attack model to determine whether the interaction belongs to the training data of the target recommender system. Thus, this invention mitigates ranking bias by constructing multi-interest user representations through clustering, achieving accurate and reliable auditing of the use of single interaction data in the recommender system.

The invention discloses an illegal device detection method and system based on optimal dynamic differential privacy budget, and the method comprises the steps: collecting a device identifier of each device and network data, and carrying out the fusion of the device identifier and network data, thereby obtaining a fusion feature; and illegal equipment detection is carried out by using an illegal equipment detection model obtained by federal learning based on an optimal dynamic differential privacy budget determination algorithm, and a corresponding security policy is executed. Performance accuracy change caused by privacy budget is used as a data practicability index, inference accuracy change of a model under member inference attack is used as a data privacy protection index, and the influence of privacy budget value on data practicability and privacy protection can be accurately evaluated. According to the method, the optimal differential privacy budget can be selected for different federated participants in each round of federated local learning, the privacy security of the participants is protected to the greatest extent, meanwhile, the influence of differential privacy on the model performance is reduced to the greatest extent, and the optimal balance between privacy protection and model performance optimization can be realized.

The invention belongs to the technical field of data security management, and discloses a sensitive data security desensitization method and system.The association structure features of a numeric sensitive field set are analyzed, a joint random noise vector with a corresponding association structure is generated based on the features, and when associated numeric sensitive fields are desensitized, the associated numeric sensitive fields are desensitized, so that the sensitive data security desensitization efficiency is improved. Noise with a specific association mode is superposed on original data, and the effect of maintaining an association structure between original fields as much as possible while fuzzy individual values are achieved; according to the method, the association structure between sensitive fields can be effectively kept while individual privacy is protected, the availability of desensitized data is improved, and the inference attack risk is reduced.

The invention discloses a differentiated privacy protection method and system based on social propagation force perception, and the method comprises the following steps: S1, constructing a social graph, and carrying out the preprocessing and community division of the graph, and obtaining community tags; s2, calculating the propagation degree, the influence and the propagation weight of the node based on the community label; s3, grouping the nodes according to the comprehensive risk score, and presetting an initial differential privacy parameter for each group; s4, based on a grouping result, adaptively adjusting the noise intensity of each group through a water level type strategy, and carrying out differential training; and S5, after training is completed, constructing a strong black box node member to infer attacks, calculating node-level attack advantages and spreading weighted privacy risks, and evaluating and visualizing a privacy protection effect. Under the same global privacy budget, the effective noise variance needing to be superposed is lower than that of an independent Gaussian noise scheme, so that the model training stability can still be maintained in a strong privacy scene.

The present application provides a computer-implemented method for preventing a chatbot from leaking personal privacy under a black-box personal attribute inference attack. The chatbot is provided via a neural network executed by a processor of a server. The method includes training, by the processor, a language model (LM) of the chatbot according to utility objectives; applying, by the processor, one or more defense objectives to the chatbot's target LM by fine-tuning using a fake attacker model and pre-defined attributes with annotated datasets with a personal attribute predictor; and defending, by the processor, against inference attacks using the chatbot's target LM such that the personal privacy of content inputted and sent to the chatbot cannot be predicted by an external predictor and the security level of the chatbot is guaranteed.

The invention discloses an active authorization control scheme for splitting learning model copyright and user dual verification, and relates to the field of artificial intelligence. For a split learning model, dual verification of the model and a user identity is realized by constructing a fingerprint and embedding the user identity (ID). In the training stage, the fingerprints participate in a low proportion, so that the model can generate specific classification behaviors for the fingerprints while learning normal tasks. In the verification link, the model conducts reasoning on the fingerprint set, and the model copyright can be verified. And the client matches the extracted ID with the label output by the server to verify the identity of the user. The method supports active authorization control, and a normal model can be accessed when user identity verification is passed; and if not, the server is automatically switched to the shadow model added with the noise, so that high performance cannot be obtained. The method has high robustness and can resist pruning, fine tuning and label reasoning attacks. And a technical means is provided for copyright verification and user identity management of the split learning model.

The present invention relates to the field of machine learning technology, and discloses a distributed federated learning aggregation method, terminal and readable storage medium, wherein the method comprises: obtaining local model parameters of this round of iteration, dividing the local model parameters into multiple secret shares corresponding to the participants, sending each of the secret shares to the corresponding participants, and receiving the secret shares sent by the participants; performing aggregation locally according to the secret shares to obtain a local aggregation vector, and sharing the local aggregation vector with a manager server; receiving a global aggregation model updated by the manager server, and updating the local model according to the global aggregation model. The present invention divides the local model parameters by secret division, so that each participant can only obtain the divided local model parameters, and the manager server can only obtain the final aggregation calculation result, but cannot obtain the intermediate local model parameters, thereby effectively defending against reasoning attacks.

A data privacy system automatically determines quasi-identifiers in a database containing individuals' records. The data privacy system applies a machine learning model to the database, the model configured to classify each record in the database and output a measure of its confidence in its classification. The data privacy system determines, based on the measure of confidence, how important each attribute is to the model's classification. The data privacy system iteratively applies a machine learning model on a modified database that includes the highest ranked attributes to identify the quasi-identifiers in the records in the database. The data privacy system can use identified quasi-identifiers to determine if the database is susceptible to a membership inference attack, and in response to such a determination, can perform one or more data privacy operations on the database to reduce this risk.

The invention discloses a defense method and device for member reasoning attacks in large model fine tuning and a medium, when the accuracy difference of a target classifier on training data and verification data is larger than a threshold value, the defense method for the member reasoning attacks in large model fine tuning is executed, and the defense method comprises the steps that before gradient descent of the target classifier, the target classifier is subjected to gradient descent; according to the cardinal number change of the label set corresponding to each label between the kth batch and the (k-1) th batch, performing data fusion on samples in the label set corresponding to each label between the kth batch and the (k-1) th batch; and / or, in the gradient descending process of the target classifier, averaging gradient updating parameters of the pth layer in the target classifier, and updating each gradient element value in the gradient matrix corresponding to the pth layer by using the average value; the gradients of other hidden layers except the pth layer in the target classifier are updated based on gradient updating parameters obtained through calculation of a gradient descent method.

The invention discloses a data privacy protection method and device, equipment, a storage medium and a computer program product, and the method comprises the steps: determining a confidence coefficient upper bound and a confidence coefficient lower bound of a purchase intention set according to the number of real intentions included in the purchase intention set, based on the purchase intention set, the confidence coefficient upper bound, the confidence coefficient lower bound, the data element background knowledge and the purchase record of the data element purchaser, establishing a release intention-based attack model, an efficiency maximization attack model and a purchase record inference attack model for the data element purchaser; a uniform attack defense model, an efficiency maximization attack defense model and a purchase record inference attack defense model based on release intention are constructed by taking a confidence coefficient threshold as a constraint condition, and privacy protection is performed on a data element purchaser according to the attack defense models, so that the privacy security of the data element purchaser is guaranteed, and the security of the data element purchaser is improved. And thus, the privacy leakage risk of the data element purchaser is reduced.