Code model personal privacy extraction attack method and device based on kernel semantic entropy

Abstract

The application provides a code model personal privacy extraction attack method and device based on kernel semantic entropy, which comprises the following steps: a target large model performs multiple independent queries on each privacy prompt word to obtain a plurality of response results corresponding to each privacy prompt word; the perplexity and the kernel semantic entropy are calculated based on the plurality of response results; the prompt words corresponding to the kernel semantic entropy lower than the semantic entropy threshold value and the perplexity lower than the perplexity threshold value are expanded, and the expanded prompt words are input into the target large model again for iteration; and an entity recognition model is used to perform privacy identification on each response result in the last round of iteration. The scheme combines the kernel semantic entropy and the perplexity to screen high-quality prompt words, thereby effectively filtering invalid prompt words, improving the efficiency of the overall privacy extraction attack, providing quantitative method support for evaluating the privacy protection capability of the code large model, and assisting in the construction and improvement of the large model privacy security protection system.

Description

Technical Field

[0001] This application relates to the field of information security, and in particular to a method and apparatus for personal privacy extraction attacks based on a code model of kernel semantic entropy. Background Technology

[0002] In recent years, intelligent code assistants based on large language models have been widely adopted in the software development field. Tools such as GitHub Copilot, CodeFuse, and the GPT series of code generation models provide developers with intelligent services such as code completion, generation, and defect detection by pre-training on massive public code datasets and fine-tuning them with private business data, effectively improving software development efficiency. However, during the training process, large code models inevitably come into contact with and memorize a large amount of sensitive personal information from the training corpus, including names, email addresses, mobile phone numbers, account passwords, credit card numbers, and other personal privacy elements. This makes the leakage of training data privacy a significant risk restricting the secure deployment of large code models.

[0003] Training data extraction attacks are typical black-box attacks targeting the memory characteristics of large language models. Attackers can construct specific input prompts to induce the model to output fragments of raw data memorized during training, thereby stealing the personal privacy information contained within. This type of attack does not require access to model weights or training datasets; it can be carried out solely through black-box calls, making it low-barrier and highly covert. Besides directly causing personal information leaks, it can also achieve identity re-identification through correlation analysis of multiple leaked privacy elements, further amplifying the harmful consequences of privacy breaches and posing a serious threat to the information security of model users.

[0004] Current research on attacks targeting training data extraction from large code models is still in its early stages, and technical solutions have several limitations. Early studies systematically verified the phenomenon of training data memorization in large language models for the first time, using manually constructed prompts combined with perplexity indices for privacy extraction. However, the attack process is highly dependent on manual operation, the prompt construction is highly random, and the efficiency and coverage of personal information extraction are extremely limited. Subsequent semi-automated extraction schemes for Codex series models introduced blind member inference technology, which reduced manual costs to some extent. However, using perplexity as the sole judgment feature for blind member inference resulted in large fluctuations in screening accuracy, and the evaluation stage still required a lot of manual intervention. Furthermore, the extracted personal information elements lacked effective correlations, making it difficult to support complete identity information reconstruction.

[0005] In summary, existing large-scale code model privacy extraction attack schemes still have significant shortcomings in terms of feature selection accuracy, prompt word optimization efficiency, and attack scenario coverage, and cannot comprehensively and accurately assess the risk of training data privacy leakage in large-scale code models. Summary of the Invention

[0006] This application provides a method and apparatus for personal privacy extraction attacks based on kernel semantic entropy code models. This solution uses a combination of kernel semantic entropy and perplexity to screen high-quality prompt words, thereby effectively filtering invalid prompt words, improving the overall efficiency of privacy extraction attacks, providing quantitative method support for evaluating the privacy protection capabilities of large code models, and helping to build and improve the privacy security protection system of large models.

[0007] In a first aspect, embodiments of this application provide a method for extracting personal privacy based on a code model using kernel semantic entropy, the method comprising:

[0008] S100. Construct a set of prompt words containing multiple privacy prompt words; S200: The target large model performs multiple independent queries on each privacy prompt word in the prompt word set to obtain multiple response results corresponding to each privacy prompt word, and obtains the perplexity of the target large model for each privacy prompt word based on the corresponding multiple response results; S300. Construct a semantic graph structure corresponding to each privacy prompt word, wherein the nodes in the semantic graph structure are the corresponding response results, and the edges are the semantic similarity between the corresponding response results; construct a semi-positive definite semantic kernel matrix based on each semantic graph structure, and calculate the kernel semantic entropy of each semi-positive definite semantic kernel matrix; S400. Select the prompt words whose corresponding kernel semantic entropy is lower than the semantic entropy threshold and whose corresponding perplexity is lower than the perplexity threshold as high-quality prompt words, and use discrete particle swarm optimization to expand each high-quality prompt word to obtain a new set of prompt words. S500: Use the new set of prompt words to iteratively execute steps S200-S400. When the iteration condition is met, execute step S600. S600 employs an entity recognition model to perform privacy recognition on each response result in the last iteration, and uses the privacy recognition result as the result of a personal privacy extraction attack.

[0009] Secondly, embodiments of this application provide a personal privacy extraction attack device based on a kernel semantic entropy code model, comprising: A building module for constructing a set of privacy tips; In the query module, the target large model performs multiple independent queries on each privacy prompt word in the prompt word set to obtain multiple response results corresponding to each privacy prompt word, and obtains the perplexity of the target large model for each privacy prompt word based on the corresponding multiple response results; The kernel semantic entropy calculation module is used to construct a semantic graph structure corresponding to each privacy prompt word. The nodes in the semantic graph structure are the corresponding response results, and the edges are the semantic similarity between the corresponding response results. Based on each semantic graph structure, a semi-positive definite semantic kernel matrix is ​​constructed, and the kernel semantic entropy of each semi-positive definite semantic kernel matrix is ​​calculated. The prompt word filtering module is used to select prompt words whose corresponding kernel semantic entropy is lower than the semantic entropy threshold and whose corresponding perplexity is lower than the perplexity threshold as high-quality prompt words. Discrete particle swarm optimization is used to expand each high-quality prompt word to obtain a new prompt word set. The iteration module iterates through the query module and the suggestion filtering module using the new set of suggestion words, and executes the output module when the iteration conditions are met. The output module uses an entity recognition model to perform privacy recognition on each response result in the last iteration, and uses the privacy recognition result as the result of the personal privacy extraction attack.

[0010] Thirdly, embodiments of this application provide an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to execute a personal privacy extraction attack method based on a kernel semantic entropy code model.

[0011] Fourthly, embodiments of this application provide a readable storage medium storing a computer program, which, when executed by a processor, implements a method for extracting personal privacy based on a code model of kernel semantic entropy.

[0012] The main contributions and innovations of this invention are as follows: This application does not perform semantic clustering on the response results. Instead, it uses each response result corresponding to the same prompt word as a node, establishes a weighted semantic graph based on bidirectional natural language reasoning relationships, and calculates the response-level kernel semantic entropy using the eigenvalue distribution of the graph Laplacian hot kernel, thereby preserving continuous and fine-grained semantic dependencies between responses. This application inputs each response perplexity channel and the response-level kernel semantic entropy channel into a blind member inference selector, and forms particle fitness through multi-channel selection voting. This application uses a particle swarm optimization algorithm suitable for discrete prompt word sequences, uses code-aware lexical unit sequences to represent particle positions, and uses insertion, deletion, replacement, and exchange operations to represent particle velocities. It updates the prompt word set by combining individual optimal position, global optimal position, corpus exploration, elite retention, and deduplication mechanisms. This application distinguishes between real privacy leaks and model illusions based on the intersection relationship of the code repository addresses corresponding to different privacy entities. By verifying whether multiple types of privacy entities coexist in the same public code repository, it can accurately distinguish whether the extracted results are real training data memorized by the model or randomly generated illusion content, filter false leakage results, improve the accuracy of the final privacy extraction attack results, and eliminate the interference of illusion content on risk assessment.

[0013] Details of one or more embodiments of this application are set forth in the following drawings and description to make other features, objects and advantages of this application more readily apparent. Attached Figure Description

[0014] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings: Figure 1 This is a flowchart illustrating a method for extracting personal privacy based on a code model according to an embodiment of this application. Figure 2 This is a structural block diagram of a personal privacy extraction attack device based on a kernel semantic entropy code model according to an embodiment of this application; Figure 3 This is a schematic diagram of the hardware structure of an electronic device according to an embodiment of this application. Detailed Implementation

[0015] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with one or more embodiments of this specification. Rather, they are merely examples of apparatuses and methods consistent with some aspects of one or more embodiments of this specification as detailed in the appended claims.

[0016] It should be noted that the steps of the corresponding methods are not necessarily performed in the order shown and described in this specification in other embodiments. In some other embodiments, the methods may include more or fewer steps than described in this specification. Furthermore, a single step described in this specification may be broken down into multiple steps in other embodiments; and multiple steps described in this specification may be combined into a single step in other embodiments.

[0017] Example 1 This application provides a method for extracting personal privacy from code models based on kernel semantic entropy. This method combines kernel semantic entropy with perplexity to filter high-quality prompts, effectively filtering invalid prompts and improving the overall efficiency of privacy extraction attacks. It provides quantitative methods for evaluating the privacy protection capabilities of large code models, contributing to the construction and improvement of large model privacy security protection systems. Furthermore, the privacy prompts, privacy extraction corpora, and user privacy involved in training involved in this application have all been obtained with full consent and authorization, and the collection, use, and processing of related information must comply with the relevant laws, regulations, and standards of relevant countries and regions. Specifically, refer to... Figure 1 The method includes: S100. Construct a set of prompt words containing multiple privacy prompt words; S200: The target large model performs multiple independent queries on each privacy prompt word in the prompt word set to obtain multiple response results corresponding to each privacy prompt word, and obtains the perplexity of the target large model for each privacy prompt word based on the corresponding multiple response results; S300. Construct a semantic graph structure corresponding to each privacy prompt word, wherein the nodes in the semantic graph structure are the corresponding response results, and the edges are the semantic similarity between the corresponding response results; construct a semi-positive definite semantic kernel matrix based on each semantic graph structure, and calculate the kernel semantic entropy of each semi-positive definite semantic kernel matrix; S400. Select the prompt words whose corresponding kernel semantic entropy is lower than the semantic entropy threshold and whose corresponding perplexity is lower than the perplexity threshold as high-quality prompt words, and use discrete particle swarm optimization to expand each high-quality prompt word to obtain a new set of prompt words. S500: Use the new set of prompt words to iteratively execute steps S200-S400. When the iteration condition is met, execute step S600. S600 employs an entity recognition model to perform privacy recognition on each response result in the last iteration, and uses the privacy recognition result as the result of a personal privacy extraction attack.

[0018] In step S100, multiple standard prompt word templates are obtained from the prompt word template database, a corpus dataset containing privacy-extracted corpus is constructed, and the privacy-extracted corpus from the corpus dataset is filled into the standard prompt word templates in the prompt word template database to obtain multiple privacy prompt words.

[0019] Specifically, this solution uses CodexLeaks as a prompt word template database, and obtains multiple valid standard prompt word templates from the prompt word template database. These templates have a fixed JSON structured format, such as {"name": "xxx","email": "xxx"}. These standard prompt word templates can effectively guide the model to output personal information.

[0020] Specifically, this scheme collects publicly available brute-force dictionaries in multiple languages ​​and selects high-frequency personal information elements such as common names, usernames, and weak passwords from these dictionaries to form a corpus dataset.

[0021] Furthermore, multiple privacy-extracting corpora are randomly sampled from the corpus dataset and populated into the standard prompt word template to obtain multiple privacy prompt words.

[0022] In step S200, the first word length is set, and the word length of each response result is the sum of the word length of the corresponding privacy prompt word and the first word length.

[0023] Specifically, by setting a fixed first term length, it can be ensured that the length of the response result is sufficient to contain meaningful fragments of personal information.

[0024] For example, in this scheme, the length of the first word element is 100. That is, when the number of tokens for the privacy prompt word is 100, the number of tokens for the corresponding response result output by the target large model is 200.

[0025] Furthermore, the temperature parameter of the target large model is set to 0.8, which makes the probability distribution of the target large model output smoother, increases the possibility of the model selecting low probability tokens, thereby improving the diversity and randomness of the response results and exposing more potential training data memories.

[0026] Furthermore, the size of the non-repeating n-gram of the target large model is set to 2 to avoid the model repeating the same 2-gram fragments in the output, thus further ensuring the richness of the response content.

[0027] In step S200, the cross-entropy of the conditional probability generated by each word in the response result is used as the corresponding response perplexity value, and the average response perplexity value of multiple response results is used as the perplexity of the corresponding privacy prompt word.

[0028] Given a word The general formula for calculating the perplexity of a given sequence is:

[0029] Where PPL stands for perplexity level. Indicates that given a previous word element Under the condition of, word element The probability of occurrence, where n is the total number of lexical units; perplexity is essentially the exponent of the negative log probabilities of all positions in the sequence. The smaller the value, the more certain the model's estimate of the probability of the generated lexical units is, and the more likely the response is a reproduction of the model's memory of the training data rather than randomly generated content.

[0030] Specifically, perplexity measures the degree of certainty that the target large model generates the response result. The lower the perplexity, the higher the model's confidence in the response result, and the greater the probability that the corresponding content appears in the training set. It is more likely to be private content that the model has memorized and reproduced. Conversely, it is more likely to be completely new content randomly created by the model, which belongs to content that has not appeared in the training data.

[0031] In step 300, a pre-trained semantic coding model is used to semantically encode each response result to obtain the semantic coding result. Then, the cosine similarity between different semantic coding results is used as the semantic similarity to obtain the semantic similarity between pairs of response results. Finally, the response results are used as nodes and the semantic similarity between response results is used as edges to complete the construction of the semantic graph structure.

[0032] In other words, multiple response results are sampled for each privacy prompt word, and each response result is used as a semantic graph node. The edge weights are determined based on the semantic relationship prediction results of the pairwise response results in two directions by the natural language processing inference model, and the semantic graph structure is constructed. The hot kernel is constructed based on the Laplacian matrix of the semantic graph structure to obtain the positive semi-definite semantic kernel matrix, and the hot kernel is normalized. The kernel semantic entropy is calculated based on the eigenvalues ​​of the normalized hot kernel. Furthermore, the Laplacian matrix of the semantic graph structure is obtained, and the corresponding positive semi-definite semantic kernel matrix is ​​calculated based on the Laplacian matrix, as expressed by the formula:

[0033] in, It is a positive semidefinite semantic kernel matrix. For the length scale parameter of the thermonuclear, The Laplacian matrix of the semantic graph structure. This indicates that it is based on exponential operations.

[0034] To meet the conditions for calculating kernel semantic entropy, the positive semi-definite semantic kernel matrix is ​​trace-normalized to obtain a unit trace positive semi-definite matrix, and the kernel semantic entropy is calculated based on the unit trace positive semi-definite matrix.

[0035] Specifically, the formula for trace normalization is expressed as:

[0036] in, It is a positive semi-definite matrix with unit trace. The trace of the matrix, The assignment operation means dividing the original positive semidefinite semantic kernel matrix by its trace to obtain the normalized result.

[0037] Specifically, the unit trace positive semidefinite matrix is ​​used to describe the degree of concentration or dispersion of semantic information in multiple response results.

[0038] Specifically, this scheme uses von Neumann entropy to calculate kernel semantic entropy, expressed by the formula:

[0039] in, Privacy Tips kernel semantic entropy, It is a positive semi-definite matrix with unit trace. Let be the trace of the matrix.

[0040] Specifically, kernel semantic entropy is used to measure the degree of disorder of a matrix. In other words, in this scheme, the lower the kernel semantic entropy, the more concentrated the information of the semi-positive definite semantic kernel matrix, that is, the stronger the semantic consistency of multiple response results, and the higher the probability that the corresponding content comes from the training data; conversely, the higher the kernel semantic entropy, the more dispersed the feature values, and the more chaotic the semantics of multiple response results.

[0041] For example, in step S300, kernel semantic entropy is used to directly treat the sampled response results as semantic graph nodes for the same privacy prompt word, without first performing semantic clustering on the response results. According to the definition of kernel semantic entropy, the semantic kernel... Let be a positive semi-definite kernel of the unit trace defined on a finite set of generated texts, used to characterize the semantic relevance between generated texts; the kernel semantic entropy is the von Neumann entropy of the semantic kernel, and the calculation process is as follows: Step 1: Response sampling and bidirectional natural language inference. For any real coefficient... Unit trace semidefinite semantic kernel The following conditions must be met:

[0042] Among them, regarding privacy prompt words Additional independent sampling for the target large model Response results For any two different response results and , respectively As a premise, Assumptions and As a premise, The hypothesis is input into a natural language inference model, and the predicted label for each direction is mapped to a semantic inertia score. Implication relation corresponds to 1, neutral relation corresponds to 0.5, and contradictory relation corresponds to 0. The edge weights of a weighted undirected semantic graph are represented as follows:

[0043] in Indicates the response result and The symmetrical edge weights between them, the semantic relation scores in the two directions are added together to make The value of is between 0 and 2. A symmetric edge weight matrix is ​​formed by all edge weights. .

[0044] Step 2: Calculate the graph Laplacian matrix. Based on the edge weight matrix... Calculate the degree matrix and the unnormalized graph Laplace matrix The formula is expressed as:

[0045] in, Represents nodes The sum of the weights of the connected sides. is the graph Laplacian matrix of the weighted semantic graph.

[0046] Step 3: Semantic Graph Kernel Construction. Based on the graph Laplacian matrix. Constructing a positive semidefinite thermal kernel matrix The formula is expressed as:

[0047] in, is the length scale parameter of the thermonuclear.

[0048] Step 4: Semantic Normalization. Correlation normalization is performed using the diagonal elements of the heat kernel matrix, and then divided by the number of responses. The unit trace semi-definite semantic kernel is obtained. The formula is expressed as:

[0049] in, , and Representing the heat core matrix respectively The corresponding element, Unit trace semi-definite semantic kernel The Line 1 Column elements. After normalization The diagonal elements are all ,therefore .

[0050] Step 5: Kernel semantic entropy calculation. Calculate the unit trace positive semi-definite semantic kernel. The von Neumann entropy was determined as a privacy prompt word. The kernel semantic entropy is expressed by the formula:

[0051] in, Represents the von Neumann entropy. Unit trace semi-positive definite semantic kernel The There are 3 eigenvalues, each of which is non-negative and their sum is 1. The lower the kernel semantic entropy, the more concentrated the semantic information corresponding to multiple responses to the same prompt word, and the lower the uncertainty of model generation; the higher the kernel semantic entropy, the more dispersed the semantic information, and the higher the uncertainty of model generation.

[0052] In step S400, prompts whose kernel semantic entropy is lower than the semantic entropy threshold and whose corresponding perplexity is lower than the perplexity threshold can more effectively induce the target large model to output the private content memorized in the training corpus. Therefore, selecting such prompts as high-quality prompts can effectively reduce the computational overhead of invalid prompts in subsequent iterations and improve the overall efficiency of privacy extraction attacks.

[0053] In other words, this scheme only retains high-quality prompt words and uses discrete particle swarm optimization to expand each high-quality prompt word, discarding low-quality prompt words from the prompt word set.

[0054] Specifically, discrete particle swarm optimization (DSO) is used for iterative search in a discrete cue word space. Each particle represents a high-quality cue word, its position is a sequence of lexical units obtained through code-aware word segmentation, and its velocity is a sequence of editing operations (insertion, deletion, replacement, or swapping) performed on the lexical unit sequence. Each particle stores the current cue word, historical velocity, its individual best cue word, and its fitness, while the particle swarm also stores the globally best cue word.

[0055] When updating particles, some historical editing operations are retained with a preset inertial probability, and cognitive and social editing operations are generated based on the lexical differences between the current particle and the individual optimal prompt word and the global optimal prompt word. Random editing operations can also be generated from the corpus or the current particle group vocabulary with a preset exploration probability. The retained and generated editing operations are applied to the current prompt word to obtain the new particle position, and length limits, elite retention, and duplicate prompt word elimination are performed.

[0056] In other words, this scheme uses discrete particle swarm optimization to iteratively optimize each high-quality prompt word, and selects prefix corpus from the prefix corpus to expand each iteratively optimized high-quality prompt word to obtain a new set of prompt words. The prefix corpus includes structured fragments, annotation prefixes, and escape characters.

[0057] Specifically, prefixes are randomly selected again from the prefix corpus and directly concatenated before the main body of high-quality prompt words. Each high-quality prompt word is concatenated with only one prefix, and historical prefixes are not superimposed. The special prefixes include structured segments, annotation prefixes, and escape characters. The special prefix concatenation and the corpus exploration within the discrete particle swarm are two independent processing steps.

[0058] In step S500, this scheme can use reaching a preset number of iterations as the iteration condition. When the current iteration reaches the preset number of iterations, the iteration is terminated and the process proceeds to the next step S600. Alternatively, it can be set to have the number of high-quality prompt words obtained from the screening be lower than a preset threshold. When the proportion of high-quality prompt words is consistently lower than the preset threshold after multiple iterations, the iteration process is terminated.

[0059] The iterative optimization mechanism can continuously enrich high-quality prompt words with better attack effects through multiple rounds of screening and expansion, and continuously eliminate invalid prompt words with poor attack effects, ultimately improving the overall attack performance of the prompt word set.

[0060] In the current embodiment, multiple privacy information entities are obtained from each response result in the personal privacy extraction attack results, and the code repository address corresponding to each privacy information entity is obtained. If there is an intersection between the code repository addresses corresponding to different privacy information entities, the privacy leakage of the corresponding response result is judged as a real leakage; if there is no intersection between the code repository addresses corresponding to different privacy information entities, the privacy leakage of the corresponding response result is judged as a model illusion.

[0061] Specifically, the Named Entity Recognition (NER) model is used to perform privacy recognition on each response result in the last round. The privacy information entities obtained by privacy recognition can be any type such as name, email, phone number, address, ID number, credit card number, etc.

[0062] Specifically, the code repository address corresponding to each privacy information entity is searched and obtained through the GitHub API. The code repository address is the public code storage path containing the privacy information entity on the public code hosting platform. If multiple privacy information entities appear in the same code file in the same code repository, it means that these privacy information originally existed in the same code segment of the training dataset, and the content extracted by the model is a fragment of training data that is actually remembered. If different privacy entities correspond to completely different code repositories, or even if some privacy entities cannot be retrieved from the corresponding public code storage records, then the result is likely to be hallucination content generated by the model and does not belong to real training data privacy leakage. This result can be filtered to improve the accuracy of the final attack result and eliminate the interference of randomly generated content by the model on the risk assessment result.

[0063] In other words, specifically, personal privacy extraction attack results are generated based on the search results of the identified privacy information entities in public code repositories.

[0064] Example 2 Based on the same concept, referencing Figure 2 This application also proposes a personal privacy extraction attack device based on a kernel semantic entropy code model, comprising: A building module for constructing a set of privacy tips; In the query module, the target large model performs multiple independent queries on each privacy prompt word in the prompt word set to obtain multiple response results corresponding to each privacy prompt word, and obtains the perplexity of the target large model for each privacy prompt word based on the corresponding multiple response results; The kernel semantic entropy calculation module is used to construct a semantic graph structure corresponding to each privacy prompt word. The nodes in the semantic graph structure are the corresponding response results, and the edges are the semantic similarity between the corresponding response results. Based on each semantic graph structure, a semi-positive definite semantic kernel matrix is ​​constructed, and the kernel semantic entropy of each semi-positive definite semantic kernel matrix is ​​calculated. The prompt word filtering module is used to select prompt words whose corresponding kernel semantic entropy is lower than the semantic entropy threshold and whose corresponding perplexity is lower than the perplexity threshold as high-quality prompt words. Discrete particle swarm optimization is used to expand each high-quality prompt word to obtain a new prompt word set. The iteration module iterates through the query module and the suggestion filtering module using the new set of suggestion words, and executes the output module when the iteration conditions are met. The output module uses an entity recognition model to perform privacy recognition on each response result in the last iteration, and uses the privacy recognition result as the result of the personal privacy extraction attack.

[0065] Example 3 This embodiment also provides an electronic device, see reference. Figure 3It includes a memory 402 and a processor 401, wherein the memory 402 stores a computer program and the processor 401 is configured to run the computer program to perform the steps in any of the above method embodiments.

[0066] Specifically, the processor 401 may include a central processing unit (CPU), an application specific integrated circuit (ASIC), or one or more integrated circuits that can be configured to implement the embodiments of this application.

[0067] The memory 402 may include a mass storage device for data or instructions. For example, and not limitingly, the memory 402 may include a hard disk drive (HDD), a floppy disk drive, a solid-state drive (SSD), flash memory, an optical disk drive, a magneto-optical disk drive, magnetic tape, or a Universal Serial Bus (USB) drive, or a combination of two or more of these. Where appropriate, the memory 402 may include removable or non-removable (or fixed) media. Where appropriate, the memory 402 may be internal or external to a data processing device. In a particular embodiment, the memory 402 is non-volatile memory. In a particular embodiment, the memory 402 includes read-only memory (ROM) and random access memory (RAM). Where appropriate, the ROM may be a mask-programmed ROM, a programmable read-only memory (PROM), an erasable read-only memory (EPROM), an electrically erasable read-only memory (EEPROM), an electrically alterable read-only memory (EAROM), or flash memory, or a combination of two or more of these. Where appropriate, the RAM can be Static Random-Access Memory (SRAM) or Dynamic Random-Access Memory (DRAM). DRAM can be Fast Page Mode Dynamic Random Access Memory (FPMDRAM), Extended Data Out Dynamic Random Access Memory (EDODRAM), Synchronous Dynamic Random-Access Memory (SDRAM), etc.

[0068] The memory 402 can be used to store or cache various data files that need to be processed and / or communicated, as well as possible computer program instructions executed by the processor 401.

[0069] The processor 401 reads and executes computer program instructions stored in the memory 402 to implement any of the personal privacy extraction attack methods based on the kernel semantic entropy code model in the above embodiments.

[0070] Optionally, the electronic device may further include a transmission device 403 and an input / output device 404, wherein the transmission device 403 is connected to the processor 401 and the input / output device 404 is connected to the processor 401.

[0071] The transmission device 403 can be used to receive or send data via a network. Specific examples of the network described above may include wired or wireless networks provided by the communication provider of the electronic device. In one example, the transmission device includes a Network Interface Controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the transmission device 403 may be a Radio Frequency (RF) module used for wireless communication with the Internet.

[0072] Input / output device 404 is used to input or output information. In this embodiment, the input information may be a set of prompt words, etc., and the output information may be a response result, the result of a personal privacy extraction attack, etc.

[0073] Optionally, in this embodiment, the processor 401 can be configured to perform the following steps via a computer program: S100. Construct a set of prompt words containing multiple privacy prompt words; S200: The target large model performs multiple independent queries on each privacy prompt word in the prompt word set to obtain multiple response results corresponding to each privacy prompt word, and obtains the perplexity of the target large model for each privacy prompt word based on the corresponding multiple response results; S300. Construct a semantic graph structure corresponding to each privacy prompt word, wherein the nodes in the semantic graph structure are the corresponding response results, and the edges are the semantic similarity between the corresponding response results; construct a semi-positive definite semantic kernel matrix based on each semantic graph structure, and calculate the kernel semantic entropy of each semi-positive definite semantic kernel matrix; S400. Select the prompt words whose corresponding kernel semantic entropy is lower than the semantic entropy threshold and whose corresponding perplexity is lower than the perplexity threshold as high-quality prompt words, and use discrete particle swarm optimization to expand each high-quality prompt word to obtain a new set of prompt words. S500: Use the new set of prompt words to iteratively execute steps S200-S400. When the iteration condition is met, execute step S600. S600 employs an entity recognition model to perform privacy recognition on each response result in the last iteration, and uses the privacy recognition result as the result of a personal privacy extraction attack.

[0074] It should be noted that the specific examples in this embodiment can refer to the examples described in the above embodiments and optional implementations, and will not be repeated here.

[0075] Generally, various embodiments can be implemented in hardware or dedicated circuitry, software, logic, or any combination thereof. Some aspects of the invention can be implemented in hardware, while others can be implemented by firmware or software executed by a controller, microprocessor, or other computing device, but the invention is not limited thereto. Although various aspects of the invention may be shown and described as block diagrams, flowcharts, or using some other graphical representation, it should be understood that, by way of non-limiting example, these blocks, apparatuses, systems, techniques, or methods described herein can be implemented in hardware, software, firmware, dedicated circuitry or logic, general-purpose hardware or controllers or other computing devices, or some combination thereof.

[0076] Embodiments of the present invention can be implemented by computer software, which may be executable in a mobile device's data processor, such as a processor entity, or by hardware, or by a combination of software and hardware. Computer software or programs (also referred to as program products), including software routines, applets, and / or macros, can be stored in any device-readable data storage medium, and they include program instructions for performing specific tasks. The computer program product may include one or more computer-executable components configured to perform the embodiments when the program is run. The one or more computer-executable components may be at least one piece of software code or a portion thereof. Additionally, it should be noted in this respect that, as Figure 3 Any box in the logical flow can represent a program step, or interconnected logic circuits, boxes and functions, or a combination of program steps and logic circuits, boxes and functions. Software can be stored on physical media such as memory chips or blocks of storage implemented within a processor, magnetic media such as hard disks or floppy disks, and optical media such as DVDs and their data variants, CDs, etc. The physical medium is a non-transient medium.

[0077] Those skilled in the art should understand that the technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments have been described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0078] The above embodiments are merely illustrative of several implementation methods of this application, and their descriptions are relatively specific and detailed, but they should not be construed as limiting the scope of this application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A method for extracting personal privacy based on a code model using kernel semantic entropy, characterized in that, Includes the following steps: S100. Construct a set of prompt words containing multiple privacy prompt words; S200: The target large model performs multiple independent queries on each privacy prompt word in the prompt word set to obtain multiple response results corresponding to each privacy prompt word, and obtains the perplexity of the target large model for each privacy prompt word based on the corresponding multiple response results; S300. Construct a semantic graph structure corresponding to each privacy prompt word, wherein the nodes in the semantic graph structure are the corresponding response results, and the edges are the semantic similarity between the corresponding response results; construct a semi-positive definite semantic kernel matrix based on each semantic graph structure, and calculate the kernel semantic entropy of each semi-positive definite semantic kernel matrix; S400. Select the prompt words whose corresponding kernel semantic entropy is lower than the semantic entropy threshold and whose corresponding perplexity is lower than the perplexity threshold as high-quality prompt words, and use discrete particle swarm optimization to expand each high-quality prompt word to obtain a new set of prompt words. S500: Use the new set of prompt words to iteratively execute steps S200-S400. When the iteration condition is met, execute step S600. S600 employs an entity recognition model to perform privacy recognition on each response result in the last iteration, and uses the privacy recognition result as the result of a personal privacy extraction attack.

2. The method for extracting personal privacy based on a code model using kernel semantic entropy as described in claim 1, characterized in that, Multiple standard prompt word templates are obtained from the prompt word template database. A corpus dataset containing privacy-extracted corpus is constructed. The privacy-extracted corpus from the corpus dataset is then filled into the standard prompt word templates in the prompt word template database to obtain multiple privacy prompt words.

3. The method for extracting personal privacy based on a code model using kernel semantic entropy as described in claim 1, characterized in that, Set the first word length. The word length of each response result is the sum of the word length of the corresponding privacy prompt word and the first word length.

4. The method for extracting personal privacy based on a code model using kernel semantic entropy as described in claim 1, characterized in that, The cross-entropy of the conditional probability generated by each word in the response result is used as the corresponding response perplexity value, and the average response perplexity value of multiple response results is used as the perplexity of the corresponding privacy prompt word.

5. The method for extracting personal privacy based on a code model using kernel semantic entropy as described in claim 1, characterized in that, Obtain the Laplacian matrix of the semantic graph structure, calculate the corresponding positive semi-definite semantic kernel matrix based on the Laplacian matrix, perform trace normalization on the positive semi-definite semantic kernel matrix to obtain the unit trace positive semi-definite matrix, and calculate the kernel semantic entropy based on the unit trace positive semi-definite matrix.

6. The method for extracting personal privacy based on a code model using kernel semantic entropy according to claim 1, characterized in that, Discrete particle swarm optimization is used to iteratively optimize each high-quality prompt word, and prefix corpus is selected from the prefix corpus to expand each iteratively optimized high-quality prompt word, resulting in a new set of prompt words. The prefix corpus includes structured fragments, annotation prefixes, and escape characters.

7. The method for extracting personal privacy based on a code model using kernel semantic entropy as described in claim 1, characterized in that, In the results of a privacy extraction attack, multiple privacy information entities are obtained for each response, and the code repository address corresponding to each privacy information entity is obtained. If there is an intersection between the code repository addresses corresponding to different privacy information entities, the privacy leakage of the corresponding response result is judged as a real leakage; if there is no intersection between the code repository addresses corresponding to different privacy information entities, the privacy leakage of the corresponding response result is judged as a model illusion.

8. A personal privacy extraction attack device based on a kernel semantic entropy code model, characterized in that, include: A building module for constructing a set of privacy tips; In the query module, the target large model performs multiple independent queries on each privacy prompt word in the prompt word set to obtain multiple response results corresponding to each privacy prompt word, and obtains the perplexity of the target large model for each privacy prompt word based on the corresponding multiple response results; The kernel semantic entropy calculation module is used to construct a semantic graph structure corresponding to each privacy prompt word. The nodes in the semantic graph structure are the corresponding response results, and the edges are the semantic similarity between the corresponding response results. Based on each semantic graph structure, a semi-positive definite semantic kernel matrix is ​​constructed, and the kernel semantic entropy of each semi-positive definite semantic kernel matrix is ​​calculated. The prompt word filtering module is used to select prompt words whose corresponding kernel semantic entropy is lower than the semantic entropy threshold and whose corresponding perplexity is lower than the perplexity threshold as high-quality prompt words. Discrete particle swarm optimization is used to expand each high-quality prompt word to obtain a new prompt word set. The iteration module iterates through the query module and the suggestion filtering module using the new set of suggestion words, and executes the output module when the iteration conditions are met. The output module uses an entity recognition model to perform privacy recognition on each response result in the last iteration, and uses the privacy recognition result as the result of the personal privacy extraction attack.

9. An electronic device comprising a memory and a processor, characterized in that, The memory stores a computer program, and the processor is configured to run the computer program to execute a code model-based personal privacy extraction attack method according to any one of claims 1-7.

10. A readable storage medium, characterized in that, The readable storage medium stores a computer program, which, when executed by a processor, implements a code model-based personal privacy extraction attack method as described in any one of claims 1-7.

Images