44 results about "Inference attack" patented technology

The application discloses a kind of inference system and method for sequence recommendation system member inference attack, including label data generation module, difference feature construction module and attack model training module;Step 1, label data generation is carried out;Step 2, the difference feature construction of member and non-member is carried out;Step 3, the training of attack model is carried out.Compared with prior art, the application can guarantee the data privacy of user in a wider range of scenarios;Fill in the blank of member inference attack in more stringent scenarios;Significantly improve the attack inference effect.

This invention provides a clustering-based method for attacking interaction-level membership in a recommender system. It involves constructing a shadow dataset with a distribution consistent with the training dataset of the target recommender system, and training a shadow recommender model based on it. For users in the shadow dataset, the method obtains the corresponding recommendation list provided by the shadow recommender model. The recommendation list is then clustered to generate user representation vectors representing users' multiple interests. Furthermore, based on the user representation vectors, the vector representations of candidate items, and the similarity between candidate items and each cluster, an attack feature vector is constructed. This attack feature vector is then used to train the attack model. The interaction to be audited in the target recommender system is input into the trained attack model to determine whether the interaction belongs to the training data of the target recommender system. Thus, this invention mitigates ranking bias by constructing multi-interest user representations through clustering, achieving accurate and reliable auditing of the use of single interaction data in the recommender system.

The invention discloses a differentiated privacy protection method and system based on social propagation force perception, and the method comprises the following steps: S1, constructing a social graph, and carrying out the preprocessing and community division of the graph, and obtaining community tags; s2, calculating the propagation degree, the influence and the propagation weight of the node based on the community label; s3, grouping the nodes according to the comprehensive risk score, and presetting an initial differential privacy parameter for each group; s4, based on a grouping result, adaptively adjusting the noise intensity of each group through a water level type strategy, and carrying out differential training; and S5, after training is completed, constructing a strong black box node member to infer attacks, calculating node-level attack advantages and spreading weighted privacy risks, and evaluating and visualizing a privacy protection effect. Under the same global privacy budget, the effective noise variance needing to be superposed is lower than that of an independent Gaussian noise scheme, so that the model training stability can still be maintained in a strong privacy scene.

The present application provides a computer-implemented method for preventing a chatbot from leaking personal privacy under a black-box personal attribute inference attack. The chatbot is provided via a neural network executed by a processor of a server. The method includes training, by the processor, a language model (LM) of the chatbot according to utility objectives; applying, by the processor, one or more defense objectives to the chatbot's target LM by fine-tuning using a fake attacker model and pre-defined attributes with annotated datasets with a personal attribute predictor; and defending, by the processor, against inference attacks using the chatbot's target LM such that the personal privacy of content inputted and sent to the chatbot cannot be predicted by an external predictor and the security level of the chatbot is guaranteed.

The invention discloses an active authorization control scheme for splitting learning model copyright and user dual verification, and relates to the field of artificial intelligence. For a split learning model, dual verification of the model and a user identity is realized by constructing a fingerprint and embedding the user identity (ID). In the training stage, the fingerprints participate in a low proportion, so that the model can generate specific classification behaviors for the fingerprints while learning normal tasks. In the verification link, the model conducts reasoning on the fingerprint set, and the model copyright can be verified. And the client matches the extracted ID with the label output by the server to verify the identity of the user. The method supports active authorization control, and a normal model can be accessed when user identity verification is passed; and if not, the server is automatically switched to the shadow model added with the noise, so that high performance cannot be obtained. The method has high robustness and can resist pruning, fine tuning and label reasoning attacks. And a technical means is provided for copyright verification and user identity management of the split learning model.

This application relates to the field of data anonymization technology, specifically to a method and system for anonymizing sensitive information in enterprise information databases. The method includes: extracting sensitive fields from the table structure of an enterprise's information database; calculating the first and second correlation degrees between any two sensitive fields to determine their correlation evaluation value; dividing all sensitive fields into multiple sets of related fields, obtaining the anonymization strength of each set, determining the anonymization level of each set, and selecting and executing the corresponding collaborative anonymization strategy. This application can completely block combined inference attack paths and ensure logical consistency of data during the anonymization process, achieving integrated collaboration between security protection and data utility.

According to one embodiment, a method, computer system, and computer program product for assessing privacy risk is provided. The embodiment may include identifying a data set and a machine learning model. The embodiment may also include selecting a target feature set comprising one or more target features of the data set. The embodiment may further include conducting one or more differential inference attacks on the machine learning model based on the target feature set. The embodiment may also include determining a privacy risk score for the target feature set based on results of the one or more differential inference attacks.

The invention relates to an attribute inference attack defense method and device, electronic equipment, a storage medium and a computer program product. The method comprises the steps of obtaining a target text; inputting the target text into a fine-grained anonymization module to eliminate privacy clues in the target text so as to obtain an anonymized text; and inputting the anonymized text into a privacy protection optimization module to add a target perturbation suffix for the anonymized text so as to obtain a perturbation text, the target perturbation suffix being used for preventing the attribute inference attack model from obtaining contents related to user privacy in the target text. In this way, by combining two mechanisms of eliminating privacy clues and preventing privacy inference, the advantage of each mechanism can be fully utilized to defend attribute inference attacks. Compared with a single defense means, the defense method provided by the invention can defend the attribute inference attack more comprehensively and more stably, so that the risk of user privacy disclosure can be effectively reduced, and safer and more thorough privacy protection can be provided for the user.

The application discloses a social perception recommendation method under multi-category sensitive link relationship protection and relates to the technical field of big data analysis. The application generates vector representations of users and items by using a heterogeneous graph neural network, introduces a mask protection mechanism of multi-type edges, solves the privacy leakage problem of multiple types of sensitive links, and thus prevents reasoning attacks of attackers based on background knowledge. A heterogeneous social perception model based on a session is constructed based on a session and multi-type nodes, recommendation performance is improved, the problem that multi-type node information cannot effectively act on a recommendation model is solved, and the problem of limited data use in a real scene is solved.

The application belongs to the technical field of intelligent transportation system data privacy protection, and discloses a traffic flow federal security prediction method based on distributed homomorphic encryption, which forms a federal model by constructing a vehicle-road cloud integrated network architecture, constructing a local model and a global model based on LSTM; the distributed key is generated based on the cooperation of the road infrastructure node and the cloud platform node, the federal model is homomorphically encrypted and trained; and the trained federal model is used for traffic flow prediction. The method has stronger confidentiality in the model training process, can guarantee the confidentiality of the shared model parameters through distributed key generation and homomorphic encryption calculation technology, and can avoid the threat of inference attack based on model parameter analysis. The application can solve the deficiency of data privacy in the existing traffic flow prediction system and provide effective support technology for intelligent transportation management.

The invention provides a member reasoning attack defense method and system based on multi-model collaborative regularization, and the method comprises the following steps: (1) dividing a data set into a plurality of non-overlapping subsets, and training an independent sub-model for each subset; and (2) in the training process of the sub-models, regularizing the currently trained model by using the output of other sub-models so as to reduce the output difference of the training sub-models on the training set and the test set. And (3) respectively inputting the original training data set into the plurality of trained sub-models to obtain a new label. And (4) carrying out distillation training by using the one-hot code of the original data and the new label to obtain a final model. The method provided by the invention not only retains the effectiveness of the model, but also enhances the defense capability of the model for member reasoning attacks.

The invention discloses a safe and efficient federated learning method and device based on error feedback and norm perception and a storage medium, and belongs to the field of artificial intelligence safety and distributed machine learning. According to the method, important gradient information is screened and accumulated through gradient compression and an error feedback mechanism, information integrity and estimation unbiasedness are ensured, a gradient norm-based adaptive differential privacy mechanism is fused, and noise intensity is dynamically allocated according to importance so as to optimize privacy budget; and the server aggregates the processed gradients and broadcasts and updates the gradients to complete iterative training. According to the method, (epsilon, delta)-differential privacy and convergence guarantee is met theoretically, the communication overhead can be remarkably reduced, member reasoning attacks can be effectively resisted, high model precision and convergence speed are kept in multiple learning tasks, and the method is suitable for large-scale popularization and application. And a reliable solution with triple balance of communication efficiency, model utility and privacy security is provided for federal learning application in a privacy sensitive scene.

A code privacy-preserving inference method and apparatus based on model segmentation and random activation, wherein the method includes the following steps: Step 1, segmenting the original code audit model into an original client and a server; Step 2, expanding the feedforward neural network of the Transformer block inside the original client to generate a pseudo client; Step 3, establishing a key authentication mechanism on the pseudo client; if key authentication is successful, the expanded neuron part automatically does not participate in the processing of input data, and the pseudo client automatically transforms into a client; if key authentication fails, the pseudo client randomly activates the expanded neuron part according to a random activation strategy; Step 4, establishing an encrypted channel connection mechanism between the pseudo client and the server. This invention can greatly defend against member inference attacks and inversion reconstruction attacks, while not affecting the model performance during normal user use.

The application provides a kind of mixed blockchain under safe and reliable cryptocurrency transaction fraud detection model design method, including steps: building light mixed blockchain cryptocurrency transaction fraud detection model based on federated learning;According to the user credibility score, a consensus mechanism based on light mixed blockchain is proposed;Based on gradient similarity and attack heterogeneity, an adaptive privacy budget and sensitivity calculation differential privacy method is designed, and based on adaptive privacy budget allocation and sensitivity calculation method, a cryptocurrency transaction fraud detection model based on federated learning framework based on mixed blockchain is constructed.The federated learning framework based on mixed blockchain can improve the throughput and reduce the memory usage, and can also defend against information inference attacks through adaptive privacy allocation and sensitivity calculation, ensure that the privacy of transaction participants is not disclosed, while reducing the performance loss of fraud detection.

The invention provides a space trajectory differential privacy enhancement method based on a spatio-temporal context and a GAN. The method comprises the following steps: converting an original trajectory data set into a spatio-temporal diagram structure; aggregating space-time neighbor information of the nodes, and learning a low-dimensional embedded vector of each node; dynamically evaluating privacy sensitivities of different areas and track segments, and non-uniformly distributing differential privacy budget according to the privacy sensitivities; constructing a GAN generation model with gradient penalty; in a loss function, introducing a loss item based on a track overall semantic feature; the GAN generation model is trained, calibrated Gaussian noise is added to the gradient of a generator, and after training is completed, synthetic trajectory data sets are generated in batches; and carrying out post-processing on the generated track, and verifying the availability of the generated data. Through the adaptive privacy budget allocation and differential privacy training mechanism, sensitive position information can be protected in a targeted manner, and various privacy attacks including member inference attacks can be effectively resisted.

The invention discloses a position privacy protection method and device based on semantic indistinguishability, equipment and a medium, and relates to the technical field of privacy protection. The method comprises the steps of firstly obtaining an interest point library, a semantic attribute library and historical movement data of a user; according to the interest point library and the semantic attribute library, obtaining a plurality of interest points similar to the semantic attributes of the actual positions as observation positions; forming an anonymous set by the actual position and the plurality of observation positions; according to historical mobile data, judging whether the anonymous set meets semantic indistinguishable constraints or not by adopting a forward algorithm of an HMM (Hidden Markov Model); a server is accessed using an anonymous set that meets semantic indistinguishable constraints. According to the method and the device, the inference attack based on the behavior pattern is simulated, so that an attacker cannot distinguish the impending (or current) behavior of the user by the anonymous position track with semantics indistinguishable, and the privacy risk caused by observing the release track is eliminated, so as to achieve the effect of resisting the attack.

A machine learning method including a first learning phase for training parameters θ of a learning model f by performing machine learning using a first dataset as a training data with a correct answer label; and a second learning phase for training parameters τ of a defender u and parameters ω of identifier h by performing machine learning using member data contained in the first dataset and non-member data contained in a second dataset. The second learning phase alternately performs, a first step for updating the parameters ω of the identifier h using the identification result when the first input result and the second input result are input to the identifier h; and a second step for updating the parameters τ of the defender u using the first output result, the second output result and the identification result.

The embodiment of the application relates to the technical field of vehicles, and discloses a privacy protection method, device and equipment for vehicle data, which comprises the following steps: performing multi-dimensional feature extraction on an original trajectory data set uploaded by a target vehicle, performing clustering processing on the extracted multi-dimensional features, determining a group feature label, performing aggregation processing on a road section in the original trajectory data set based on the group feature label, and obtaining road section aggregation data; calculating a target privacy budget corresponding to each group based on the group feature label and a preset total privacy budget, adding noise to the road section aggregation data based on the target privacy budget, and obtaining target privacy protection data, which can avoid the problem that current privacy protection technology cannot effectively defend against group feature inference attacks, thereby leading to the problem of insufficient protection and the risk of privacy leakage, realizes privacy protection in units of groups, and realizes optimal allocation of privacy protection resources through dynamic adjustment of the privacy budget.

This invention relates to a secure computation method and system for nonlinear functions based on unintentional shuffling and splitting outsourcing, belonging to the field of computer software technology. The method includes: generating dynamic permutation parameters for inference requests via a client; performing unintentional shuffling on the ciphertext through two mutually distrustful servers based on the dynamic permutation parameters; splitting the unintentionally shuffled data into a first part and a second part, with each server reconstructing the first and second parts respectively to obtain partial plaintext; calculating a nonlinear function using the partial plaintext and merging the results; and performing inverse unintentional shuffling on the merged results to obtain the final nonlinear layer output. This invention transforms expensive online secure matrix multiplication into low-cost vector addition and single-round communication operations, achieving "one-time pad" dynamic permutation at extremely low cost, fundamentally resisting statistical inference attacks.

The invention relates to the technical field of big data resource service and data security, and discloses a multi-source data security desensitization and efficient analysis integrated method, which comprises the following steps that: a security policy gateway is arranged, after a global analysis task is intercepted, the security policy gateway sends out a statistical response of a probe to query and obtain a data subset, and generates a disposable confusion token according to the response; the method comprises the following steps: generating a one-time token which is destroyed after use and is adaptive to real-time data statistical characteristics for each query, rewriting an original task by using the token, embedding a desensitization parameter into the token, and completing instant desensitization processing by a data source in an execution period, so that the data is in an unpredictable desensitization pattern in each response, and the desensitization accuracy is improved. Therefore, stable anchor points for information association in a cross-query mode are eliminated, and advanced reasoning attacks are avoided.

A method including: receiving a training dataset and a testing dataset each comprising samples that were used to train and to test, respectively, a certain machine learning model; dividing the training dataset into non-overlapping training subsets, and the testing dataset into non-overlapping testing subsets; assigning the training subsets and the testing subsets into pairs, such that each of the pairs is a distinct combination of one of the training subsets with one of the testing subsets; training different membership inference attack (MIA) models on the pairs; and evaluating sensitive information leakage from the certain machine learning model based on aggregated inferences by the trained MIA models.

The invention discloses a member reasoning attack method and device oriented to category fairness. According to the method, under the black box access condition, through joint modeling of a shadow model and a reference model, difficulty calibration and category fairness enhancement are realized. Firstly, a shadow model is used for fitting behaviors of a target model and generating data with member tags; secondly, training a plurality of reference models, and calculating difficulty calibration scores for input samples to correct member score deviations; then, target member scores, calibration scores and category labels are fused to construct attack features, a supervised comparative learning (MSCL) mechanism based on member identities is introduced, and sample pairs which are the same members or non-members are used as positive sample pairs for feature alignment; furthermore, adaptive weights are distributed for different categories according to the estimated value of the category memory degree, so that the low-memory degree category obtains higher attention in training, and the problem of vulnerability imbalance among the categories is relieved. And finally, sample member identity judgment is realized through the member probability output by the attack model. According to the method, the inter-category vulnerability difference can be remarkably reduced while the overall attack performance is maintained, and more fair privacy risk assessment is realized; and meanwhile, the number of required reference models is small, the calculation overhead is low, and good expandability and practical value are achieved.

A data privacy system automatically determines quasi-identifiers in a database containing individuals' records. The data privacy system applies a machine learning model to the database, the model configured to classify each record in the database and output a measure of its confidence in its classification. The data privacy system determines, based on the measure of confidence, how important each attribute is to the model's classification. The data privacy system iteratively applies a machine learning model on a modified database that includes the highest ranked attributes to identify the quasi-identifiers in the records in the database. The data privacy system can use identified quasi-identifiers to determine if the database is susceptible to a membership inference attack, and in response to such a determination, can perform one or more data privacy operations on the database to reduce this risk.

The application provides a power grid data security protection method and system, relates to the technical field of data security, and comprises the following steps: constructing a data inference relation graph, recording historical access information of an access subject, and when an access request is received, performing two-dimensional risk assessment based on a derived knowledge set inferred by combination of target data and historical data, a correlation measurement value of historical access and an operating state parameter, and generating an access control instruction. The application can effectively prevent inference attacks based on historical data correlation analysis and improve the power grid data security protection capability.

A data privacy system automatically determines quasi-identifiers in a database containing individuals' records. The data privacy system applies a machine learning model to the database, the model configured to classify each record in the database and output a measure of its confidence in its classification. The data privacy system determines, based on the measure of confidence, how important each attribute is to the model's classification. The data privacy system iteratively applies a machine learning model on a modified database that includes the highest ranked attributes to identify the quasi-identifiers in the records in the database. The data privacy system can use identified quasi-identifiers to determine if the database is susceptible to a membership inference attack, and in response to such a determination, can perform one or more data privacy operations on the database to reduce this risk.

The invention discloses a cross-domain member reasoning attack method based on causal reasoning. The method comprises the following steps of 1, inputting data; 2, data enhancement; 3, abandoning the influence of the environment on the graph data, and obtaining posterior distribution without environmental contingency; 4, introducing structure entropy regularization; 5, taking posterior output of the shadow model as a training set of an attack model, wherein nodes of different posterior types are provided with different labels; 6, distinguishing posterior distribution differences between members and non-members by the attack model, and popularizing the back-end attack model trained on the shadow data set to a target domain by adopting a risk extrapolation method; and step 7, cross-domain member reasoning attack prediction. According to the method, a causal inference technology is used, and environment mixed items among different domains are removed, so that the model can also achieve cross-domain attack in different domains. The invention further discloses a cross-domain member reasoning attack system based on causal reasoning.

The application discloses a data privacy protection method and device, equipment, a storage medium and a computer program product. The method comprises the following steps: determining an upper bound of confidence and a lower bound of confidence of a purchase intention set according to the number of real intentions included in the purchase intention set; establishing a publication intention-based attack model, an efficiency maximization attack model and a purchase record inference attack model for a data element purchaser based on the purchase intention set, the upper bound of confidence, the lower bound of confidence, data element background knowledge and the purchase record of the data element purchaser; constructing a uniform publication intention-based attack defense model, an efficiency maximization attack defense model and a purchase record inference attack defense model according to the attack models, and taking a confidence threshold as a constraint condition; and protecting the privacy of the data element purchaser according to the attack defense models, so that the privacy safety of the data element purchaser is ensured, and the risk of privacy leakage of the data element purchaser is reduced.

The invention discloses a large model member reasoning attack method based on embedded layer disturbance, which comprises the following steps of: inputting an original vector and a disturbance vector into a target model by adding disturbance to an embedded layer vector of a target sample by utilizing geometric structure characteristics of a semantic space of an embedded layer of a large model; and calculating the result difference of the model on the two vectors to judge whether a target sample belongs to a member data set, and performing member reasoning attack. The scheme is low in overhead, does not need to train an additional auxiliary model, does not need to obtain an additional shadow data set, can complete attack only by depending on an original model, has high accuracy and precision, can be suitable for a large-parameter large model, and can improve the attack efficiency under the condition of low calculation overhead and implementation cost. According to the method, the memory characteristics of the model to the training sample are fully described, so that the method still has relatively high discrimination accuracy and stability in a complex model structure and diversified task scenes, and the actual requirements of large model privacy disclosure risk assessment and data compliance detection are met.

A computer-implemented method for preventing leaking a personal privacy from a chatbot under black-box personal attribute inference attack is provided. The chatbot is provided via a neural network executed by a processor of a server. The method includes: training, by the processor, a Language Model (LM) of the chatbot according to utility objectives; applying, by the processor, one or more defense objectives with personal attribute predictor to fine-tune a target LM of the chatbot by using a fake attacker model and pre-define attributes with annotated datasets; and using, by the processor, the target LM on the chatbot to defend inference attack, such that the personal privacy of content inputted and sent to the chatbot cannot be predicted by external predictor and a security level of the chatbot is assured.

The application discloses a member inference attack prevention method and device, and a nonvolatile storage medium. The method comprises the following steps: obtaining multiple groups of local samples and multiple target test samples; determining a first label of each target test sample in the multiple target test samples, wherein the first label is used for indicating a classification result of the target test sample; obtaining a federated learning model stored locally by a member; generating shared parameters shared by other members and an attack model according to the multiple first labels, the multiple target test samples, and the federated learning model; and sending the shared parameters to the attack model. The application solves the technical problem that, in the related art, an attack model can identify local samples participating in distributed learning through a realistic sample generated directly by using the local samples, establish a corresponding relationship between the realistic sample and other members, and learn a training process of the local samples, thereby failing to prevent member inference attacks.