79 results about "Inference attack" patented technology

The invention discloses a classification model privacy disclosure risk black box detection method and device, and relates to the technical field of machine learning security, and the method comprises the steps: constructing an auxiliary data set; generating an auxiliary model covering different privacy risk levels; training a shadow model and a black box member reasoning attack model corresponding to the shadow model; evaluating the privacy risk score of each auxiliary model by using the attack model, and labeling a high-risk / low-risk label; calculating a correction prediction entropy difference, and screening the first k samples with the maximum difference to form a query data set; extracting a prediction result of each auxiliary model on the query set, calculating and correcting a prediction entropy and a mean value, a variance, a kurtosis and a skewness thereof, and splicing into a k + 4-dimensional risk feature vector; training a risk detection classifier; similar features of the model to be detected are extracted and input into the classifier, and the classifier performs five-level privacy disclosure risk level mapping according to the final risk probability prediction value and outputs the five-level privacy disclosure risk level mapping. The method is mainly applied to risk assessment of classification models in high-privacy sensitive fields such as financial credit investigation and medical diagnosis.

The invention belongs to the technical field of intelligent traffic system data privacy protection, and discloses a traffic flow federated security prediction method based on distributed homomorphic encryption, and the method comprises the steps: constructing a vehicle-road cloud integrated network architecture, constructing a local model and a global model based on LSTM, and forming a federated model; cooperatively generating a distributed key based on a road infrastructure node and a cloud platform node, and performing homomorphic encryption and training on the federated model; and traffic flow prediction is carried out based on the trained federal model. According to the method, the confidentiality of the model training process is higher, the confidentiality of shared model parameters can be ensured through the distributed key generation and homomorphic encryption computing technology, and inference attack threats based on model parameter analysis are avoided. According to the invention, the defect of data privacy in an existing traffic flow prediction system can be overcome, and an effective support technology is provided for intelligent traffic management.

The invention discloses a neural network model reasoning method for improving intermediate data security, and relates to the field of neural networks, in the method, in the neural network model reasoning process of an NPU, intermediate feature map data generated by each operator layer is encrypted in real time and stored in an end-side SoC memory in an encrypted form, and the security of the intermediate data is improved. Intermediate data leakage caused by analysis of an attacker on memory data can be effectively prevented, the intermediate data output by the neural network model is protected against cracking, threats in privacy leakage aspects such as member inference attacks can be effectively resisted, user privacy data is ensured not to be stolen, and user experience is improved. The data security of the neural network model in the reasoning process is greatly improved, and a powerful guarantee is provided for user information security.

The invention relates to the technical field of cloud task unloading and reinforcement learning, and particularly provides a task scheduling optimization method based on a double-agent strategy in a computing network environment, and the method comprises the steps: obtaining the energy consumption and delay of a task in an unloading process from a local processor MSDn to an edge server ECs based on a communication model; constructing an optimization target of task unloading based on the calculation model; and according to the optimization target, adopting a DAPO algorithm based on a double-agent strategy to obtain an optimal unloading strategy of the cloud edge end cooperation task. According to the method, the stability of the algorithm in a dynamic and complex environment is enhanced through double-agent configuration, the complex system state, space and decision-making process can be managed more effectively, privacy is quantified through the information entropy model, the uncertainty of an unloading mode is quantified, and therefore the attack inference difficulty is quantified.

The invention relates to the field of federated learning privacy security, in particular to a method for realizing member inference attack by controlling convolution kernel parameters. Firstly, an attacker designs convolution kernel parameters according to feature distribution of a part of a convolution region of a target sample, then records a non-zero gradient position of the target sample on a convolution kernel after back propagation, and sends a global model to a participant for local training, and finally, when the participant uploads a model update, the target sample is updated. And an attacker compares whether the non-zero gradient position of the manipulated convolution kernel is consistent with the recorded non-zero gradient position, so that member inference attack is implemented in a federated learning scene, and the technical problem that the traditional member inference attack cannot fully utilize a federated learning framework to improve the attack performance is solved.

The invention relates to the technical field of federated learning, in particular to a federated learning security aggregation method based on vector space secret sharing, which comprises the following steps that: a client divides a local model gradient into a plurality of secret shares and distributes the secret shares to a plurality of servers; the server carries out aggregation operation on the secret share through a security computing protocol, wherein the aggregation operation comprises addition and multiplication operation; calculating the similarity between the clients based on the secret state data, and screening out a credible client set; and executing security aggregation on the gradient share of the trusted client, reconstructing a global model gradient and issuing the global model gradient. According to the method, client poisoning attacks can be resisted, server reasoning attacks can also be resisted, a federal learning global model security aggregation method is designed based on vector space secret sharing, and threats caused by server offline are reduced.

The application discloses a kind of inference system and method for sequence recommendation system member inference attack, including label data generation module, difference feature construction module and attack model training module;Step 1, label data generation is carried out;Step 2, the difference feature construction of member and non-member is carried out;Step 3, the training of attack model is carried out.Compared with prior art, the application can guarantee the data privacy of user in a wider range of scenarios;Fill in the blank of member inference attack in more stringent scenarios;Significantly improve the attack inference effect.

This invention provides a clustering-based method for attacking interaction-level membership in a recommender system. It involves constructing a shadow dataset with a distribution consistent with the training dataset of the target recommender system, and training a shadow recommender model based on it. For users in the shadow dataset, the method obtains the corresponding recommendation list provided by the shadow recommender model. The recommendation list is then clustered to generate user representation vectors representing users' multiple interests. Furthermore, based on the user representation vectors, the vector representations of candidate items, and the similarity between candidate items and each cluster, an attack feature vector is constructed. This attack feature vector is then used to train the attack model. The interaction to be audited in the target recommender system is input into the trained attack model to determine whether the interaction belongs to the training data of the target recommender system. Thus, this invention mitigates ranking bias by constructing multi-interest user representations through clustering, achieving accurate and reliable auditing of the use of single interaction data in the recommender system.

The invention belongs to the technical field of data security management, and discloses a sensitive data security desensitization method and system.The association structure features of a numeric sensitive field set are analyzed, a joint random noise vector with a corresponding association structure is generated based on the features, and when associated numeric sensitive fields are desensitized, the associated numeric sensitive fields are desensitized, so that the sensitive data security desensitization efficiency is improved. Noise with a specific association mode is superposed on original data, and the effect of maintaining an association structure between original fields as much as possible while fuzzy individual values are achieved; according to the method, the association structure between sensitive fields can be effectively kept while individual privacy is protected, the availability of desensitized data is improved, and the inference attack risk is reduced.

The invention discloses a differentiated privacy protection method and system based on social propagation force perception, and the method comprises the following steps: S1, constructing a social graph, and carrying out the preprocessing and community division of the graph, and obtaining community tags; s2, calculating the propagation degree, the influence and the propagation weight of the node based on the community label; s3, grouping the nodes according to the comprehensive risk score, and presetting an initial differential privacy parameter for each group; s4, based on a grouping result, adaptively adjusting the noise intensity of each group through a water level type strategy, and carrying out differential training; and S5, after training is completed, constructing a strong black box node member to infer attacks, calculating node-level attack advantages and spreading weighted privacy risks, and evaluating and visualizing a privacy protection effect. Under the same global privacy budget, the effective noise variance needing to be superposed is lower than that of an independent Gaussian noise scheme, so that the model training stability can still be maintained in a strong privacy scene.

The present application provides a computer-implemented method for preventing a chatbot from leaking personal privacy under a black-box personal attribute inference attack. The chatbot is provided via a neural network executed by a processor of a server. The method includes training, by the processor, a language model (LM) of the chatbot according to utility objectives; applying, by the processor, one or more defense objectives to the chatbot's target LM by fine-tuning using a fake attacker model and pre-defined attributes with annotated datasets with a personal attribute predictor; and defending, by the processor, against inference attacks using the chatbot's target LM such that the personal privacy of content inputted and sent to the chatbot cannot be predicted by an external predictor and the security level of the chatbot is guaranteed.

The invention discloses an active authorization control scheme for splitting learning model copyright and user dual verification, and relates to the field of artificial intelligence. For a split learning model, dual verification of the model and a user identity is realized by constructing a fingerprint and embedding the user identity (ID). In the training stage, the fingerprints participate in a low proportion, so that the model can generate specific classification behaviors for the fingerprints while learning normal tasks. In the verification link, the model conducts reasoning on the fingerprint set, and the model copyright can be verified. And the client matches the extracted ID with the label output by the server to verify the identity of the user. The method supports active authorization control, and a normal model can be accessed when user identity verification is passed; and if not, the server is automatically switched to the shadow model added with the noise, so that high performance cannot be obtained. The method has high robustness and can resist pruning, fine tuning and label reasoning attacks. And a technical means is provided for copyright verification and user identity management of the split learning model.

A data privacy system automatically determines quasi-identifiers in a database containing individuals' records. The data privacy system applies a machine learning model to the database, the model configured to classify each record in the database and output a measure of its confidence in its classification. The data privacy system determines, based on the measure of confidence, how important each attribute is to the model's classification. The data privacy system iteratively applies a machine learning model on a modified database that includes the highest ranked attributes to identify the quasi-identifiers in the records in the database. The data privacy system can use identified quasi-identifiers to determine if the database is susceptible to a membership inference attack, and in response to such a determination, can perform one or more data privacy operations on the database to reduce this risk.

The invention discloses a defense method and device for member reasoning attacks in large model fine tuning and a medium, when the accuracy difference of a target classifier on training data and verification data is larger than a threshold value, the defense method for the member reasoning attacks in large model fine tuning is executed, and the defense method comprises the steps that before gradient descent of the target classifier, the target classifier is subjected to gradient descent; according to the cardinal number change of the label set corresponding to each label between the kth batch and the (k-1) th batch, performing data fusion on samples in the label set corresponding to each label between the kth batch and the (k-1) th batch; and / or, in the gradient descending process of the target classifier, averaging gradient updating parameters of the pth layer in the target classifier, and updating each gradient element value in the gradient matrix corresponding to the pth layer by using the average value; the gradients of other hidden layers except the pth layer in the target classifier are updated based on gradient updating parameters obtained through calculation of a gradient descent method.

This application relates to the field of data anonymization technology, specifically to a method and system for anonymizing sensitive information in enterprise information databases. The method includes: extracting sensitive fields from the table structure of an enterprise's information database; calculating the first and second correlation degrees between any two sensitive fields to determine their correlation evaluation value; dividing all sensitive fields into multiple sets of related fields, obtaining the anonymization strength of each set, determining the anonymization level of each set, and selecting and executing the corresponding collaborative anonymization strategy. This application can completely block combined inference attack paths and ensure logical consistency of data during the anonymization process, achieving integrated collaboration between security protection and data utility.

The invention discloses a split reasoning member reasoning attack method and device based on knowledge migration, and the method comprises the steps: extracting part of knowledge of a target model through knowledge migration, and carrying out member reasoning attack on this basis. Specifically, the method comprises the following steps: firstly, constructing a shadow model, and carrying out knowledge migration on the shadow model to obtain a reconstructed shadow model; and then training an attack model by using the plurality of reconstructed shadow models, thereby realizing symmetry between an attack training process and an actual application scene. Then, knowledge migration is carried out on the target model, a plurality of reconstructed target models are generated, information of the reconstructed models is extracted through the attack model, and therefore member reasoning attack is completed. Compared with the prior art, the method has the advantages that the success rate of attacks is remarkably increased through integrated symmetric design of attack model training and actual attacks.

The application discloses a multi-party joint training method of a performance prediction model based on material data privacy protection. In the joint training process, a plurality of participants transmits the parameters of a local graph learning model to other participants after noise disturbance, so that the privacy and safety of the local material data are ensured, and the parameters of the local graph learning model are disturbed by noise before transmission, so that reasoning attacks by an enemy are avoided. In addition, the application obtains a global graph node relationship matrix through secure calculation, so that the local graph node relationship matrix of each participant is not leaked, and the performance of the local graph learning model can be improved. The application avoids leakage of material data and model parameters in the joint training process of the plurality of participants.

According to one embodiment, a method, computer system, and computer program product for assessing privacy risk is provided. The embodiment may include identifying a data set and a machine learning model. The embodiment may also include selecting a target feature set comprising one or more target features of the data set. The embodiment may further include conducting one or more differential inference attacks on the machine learning model based on the target feature set. The embodiment may also include determining a privacy risk score for the target feature set based on results of the one or more differential inference attacks.

The application discloses a kind of based on hybrid strategy's federal learning model training privacy protection method and system, combine to the local model training process of federal learning participant confounding auto-encoder network, data label is confused mapping, to this cut off the relationship between gradient information and data information, prevent attack party reconstructs out user original data using gradient information;Local differential privacy mechanism is combined to the parameter transmission process of federal learning, add Gaussian noise satisfying (epsilon, delta) local differential privacy to gradient parameter, to this gradient information is disturbed, while in the center server aggregation process is approximated average aggregation by randomization mechanism, hide single participant contribution, to this prevent inference attack carried out by attack party.The application can construct a privacy-safe federal learning system, resist various privacy risks caused by gradient leakage in the process of federal learning model training, while better balance between model performance and privacy security is achieved.

The invention relates to an attribute inference attack defense method and device, electronic equipment, a storage medium and a computer program product. The method comprises the steps of obtaining a target text; inputting the target text into a fine-grained anonymization module to eliminate privacy clues in the target text so as to obtain an anonymized text; and inputting the anonymized text into a privacy protection optimization module to add a target perturbation suffix for the anonymized text so as to obtain a perturbation text, the target perturbation suffix being used for preventing the attribute inference attack model from obtaining contents related to user privacy in the target text. In this way, by combining two mechanisms of eliminating privacy clues and preventing privacy inference, the advantage of each mechanism can be fully utilized to defend attribute inference attacks. Compared with a single defense means, the defense method provided by the invention can defend the attribute inference attack more comprehensively and more stably, so that the risk of user privacy disclosure can be effectively reduced, and safer and more thorough privacy protection can be provided for the user.

The invention relates to an inference attack-oriented track privacy protection method, which comprises the following steps of: identifying a vehicle parking area in a vehicle track according to a set distance threshold value and a time threshold value, combining all track points in the parking area into a parking point, and extracting all the parking points to construct a parking point set of the vehicle track; calculating the sensitivity of each parking point according to the parking time of the vehicle at the parking point; constructing a candidate confusion position set of each stop point according to the road network and the maximum reasonable speed of the vehicle; calculating an inference error threshold value and a differential privacy budget of each staying point according to the sensitivity of the staying points; dividing the candidate confusion position set by using a Hilbert space filling curve according to the inference error threshold value of the stay point and the differential privacy budget, and generating a protection position set meeting privacy constraints; and selecting a protection position from the protection position set to replace the stop point based on a trajectory disturbance strategy of an exponential mechanism to obtain a vehicle trajectory after privacy protection. According to the method, the data availability can be ensured, meanwhile, the protection capability on inference attacks is remarkably improved, and the safe release requirement of the trajectory data in practical application is met.

The application discloses a social perception recommendation method under multi-category sensitive link relationship protection and relates to the technical field of big data analysis. The application generates vector representations of users and items by using a heterogeneous graph neural network, introduces a mask protection mechanism of multi-type edges, solves the privacy leakage problem of multiple types of sensitive links, and thus prevents reasoning attacks of attackers based on background knowledge. A heterogeneous social perception model based on a session is constructed based on a session and multi-type nodes, recommendation performance is improved, the problem that multi-type node information cannot effectively act on a recommendation model is solved, and the problem of limited data use in a real scene is solved.

The application belongs to the technical field of intelligent transportation system data privacy protection, and discloses a traffic flow federal security prediction method based on distributed homomorphic encryption, which forms a federal model by constructing a vehicle-road cloud integrated network architecture, constructing a local model and a global model based on LSTM; the distributed key is generated based on the cooperation of the road infrastructure node and the cloud platform node, the federal model is homomorphically encrypted and trained; and the trained federal model is used for traffic flow prediction. The method has stronger confidentiality in the model training process, can guarantee the confidentiality of the shared model parameters through distributed key generation and homomorphic encryption calculation technology, and can avoid the threat of inference attack based on model parameter analysis. The application can solve the deficiency of data privacy in the existing traffic flow prediction system and provide effective support technology for intelligent transportation management.

The application discloses a kind of federal learning method and system, federal learning server sends gradient weighted aggregation strategy with byzantine robustness to all federal learning participants;Wherein, the gradient weighted aggregation strategy is generated by deep reinforcement learning;Federal learning server utilizes gradient weighted aggregation strategy and weights and aggregates perturbed model parameter update, utilizes the model parameter update after weighting and aggregation and updates global model;Federal learning server carries out quality assessment to the global model after updating according to own standard verification set, based on the evaluation result, utilizes deep reinforcement learning and outputs the gradient weighted aggregation strategy of next round.The purpose of the present application is to resist the various byzantine attacks including customized byzantine attack in federal learning, while realizing the defense of privacy inference attack to honest but curious server, so as to realize the federal learning of safe privacy.

The invention provides a member reasoning attack defense method and system based on multi-model collaborative regularization, and the method comprises the following steps: (1) dividing a data set into a plurality of non-overlapping subsets, and training an independent sub-model for each subset; and (2) in the training process of the sub-models, regularizing the currently trained model by using the output of other sub-models so as to reduce the output difference of the training sub-models on the training set and the test set. And (3) respectively inputting the original training data set into the plurality of trained sub-models to obtain a new label. And (4) carrying out distillation training by using the one-hot code of the original data and the new label to obtain a final model. The method provided by the invention not only retains the effectiveness of the model, but also enhances the defense capability of the model for member reasoning attacks.

The invention discloses a safe and efficient federated learning method and device based on error feedback and norm perception and a storage medium, and belongs to the field of artificial intelligence safety and distributed machine learning. According to the method, important gradient information is screened and accumulated through gradient compression and an error feedback mechanism, information integrity and estimation unbiasedness are ensured, a gradient norm-based adaptive differential privacy mechanism is fused, and noise intensity is dynamically allocated according to importance so as to optimize privacy budget; and the server aggregates the processed gradients and broadcasts and updates the gradients to complete iterative training. According to the method, (epsilon, delta)-differential privacy and convergence guarantee is met theoretically, the communication overhead can be remarkably reduced, member reasoning attacks can be effectively resisted, high model precision and convergence speed are kept in multiple learning tasks, and the method is suitable for large-scale popularization and application. And a reliable solution with triple balance of communication efficiency, model utility and privacy security is provided for federal learning application in a privacy sensitive scene.

A code privacy-preserving inference method and apparatus based on model segmentation and random activation, wherein the method includes the following steps: Step 1, segmenting the original code audit model into an original client and a server; Step 2, expanding the feedforward neural network of the Transformer block inside the original client to generate a pseudo client; Step 3, establishing a key authentication mechanism on the pseudo client; if key authentication is successful, the expanded neuron part automatically does not participate in the processing of input data, and the pseudo client automatically transforms into a client; if key authentication fails, the pseudo client randomly activates the expanded neuron part according to a random activation strategy; Step 4, establishing an encrypted channel connection mechanism between the pseudo client and the server. This invention can greatly defend against member inference attacks and inversion reconstruction attacks, while not affecting the model performance during normal user use.

A data privacy system automatically determines quasi-identifiers in a database containing individuals' records. The data privacy system applies a machine learning model to the database, the model configured to classify each record in the database and output a measure of its confidence in its classification. The data privacy system determines, based on the measure of confidence, how important each attribute is to the model's classification. The data privacy system iteratively applies a machine learning model on a modified database that includes the highest ranked attributes to identify the quasi-identifiers in the records in the database. The data privacy system can use identified quasi-identifiers to determine if the database is susceptible to a membership inference attack, and in response to such a determination, can perform one or more data privacy operations on the database to reduce this risk.

The application provides a kind of mixed blockchain under safe and reliable cryptocurrency transaction fraud detection model design method, including steps: building light mixed blockchain cryptocurrency transaction fraud detection model based on federated learning;According to the user credibility score, a consensus mechanism based on light mixed blockchain is proposed;Based on gradient similarity and attack heterogeneity, an adaptive privacy budget and sensitivity calculation differential privacy method is designed, and based on adaptive privacy budget allocation and sensitivity calculation method, a cryptocurrency transaction fraud detection model based on federated learning framework based on mixed blockchain is constructed.The federated learning framework based on mixed blockchain can improve the throughput and reduce the memory usage, and can also defend against information inference attacks through adaptive privacy allocation and sensitivity calculation, ensure that the privacy of transaction participants is not disclosed, while reducing the performance loss of fraud detection.

The invention provides a space trajectory differential privacy enhancement method based on a spatio-temporal context and a GAN. The method comprises the following steps: converting an original trajectory data set into a spatio-temporal diagram structure; aggregating space-time neighbor information of the nodes, and learning a low-dimensional embedded vector of each node; dynamically evaluating privacy sensitivities of different areas and track segments, and non-uniformly distributing differential privacy budget according to the privacy sensitivities; constructing a GAN generation model with gradient penalty; in a loss function, introducing a loss item based on a track overall semantic feature; the GAN generation model is trained, calibrated Gaussian noise is added to the gradient of a generator, and after training is completed, synthetic trajectory data sets are generated in batches; and carrying out post-processing on the generated track, and verifying the availability of the generated data. Through the adaptive privacy budget allocation and differential privacy training mechanism, sensitive position information can be protected in a targeted manner, and various privacy attacks including member inference attacks can be effectively resisted.