Large model code member inference method based on code grammar constraint lexical element filtering
By constructing a set of grammatical conventions and removing grammatically necessary tokens at the lexical level while retaining author intent tokens, the problem of member inference affected by grammatical noise in existing technologies is solved, achieving more efficient source code member detection.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-12-15
- Publication Date
- 2026-03-27
AI Technical Summary
Existing methods ignore the syntactic structure of the code in source code member inference tasks, which inevitably leads to tokens being mixed into member scores, reducing discriminative power. Furthermore, the lack of real member/non-member benchmark datasets affects detection performance.
Construct a set of grammatical conventions based on the target programming language, identify and remove lexical units determined solely by grammatical rules through grammatical structure analysis, retain lexical units that carry the author's intent, and calculate member scores using the model's output probability.
It significantly improves the separability between members and non-members, enhances the accuracy of member inference and detection performance, reduces syntactic noise interference, and is applicable to various programming languages and large language models.
Smart Images
Figure CN121745294A_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application relates to the technical field of open source code detection, and in particular to a large model code member inference method based on code syntax constraint token filtering. BACKGROUND
[0002] In recent years, large language models have shown strong generation capabilities in natural language processing, automatic programming and other tasks. The wide application of models in code generation, automatic completion and intelligent development assistants and other scenarios benefits from their pre-training on large-scale open source code corpus. However, open source code usually carries open source licenses such as GPL, which clearly requires the use range and distribution method, and randomly using code with strong copyright restrictions for commercial model pre-training may cause legal and compliance risks.
[0003] To evaluate whether the training data of a large language model contains a specific sample, the academic community has proposed a membership inference attack (MIA), which is to determine whether a certain data is in the model training set in a black box or gray box environment. Existing membership inference methods for language models mostly treat input as natural language text, and distinguish members from non-members by analyzing token probability, compression length, minimum K% probability token and other indicators output by the model. These methods have achieved certain results in the natural language scene, but there are the following problems in the membership inference task for source code: Ignoring the syntactic structure characteristics of code: Source code is different from natural language text, and its writing must follow formal syntax. A large number of tokens are only used to meet the syntax constraints, such as parentheses, commas, colons, and indentation, etc. The appearance of these tokens is determined by syntax rules, rather than the personal style or semantic choices of the developer.
[0004] Grammar certain tokens are mixed into member scores, reducing discriminability: Existing methods usually treat all tokens equally when calculating member scores, and include grammar certain tokens and tokens carrying author's intent in the calculation, which dilutes the difference between members and non-members in the score, reducing detection performance.
[0005] Lack of real member / non-member benchmark dataset for code scenarios: Some works use synthetic or postulated member / non-member division, which is difficult to truly reflect the actual problem of "whether it appears in the pre-training data" in the industry.
[0006] Therefore, there is an urgent need for a source code member inference method that can fully utilize the syntactic features of programming languages, distinguish between syntactically deterministic tokens and tokens that reflect the author's creative intent, thereby improving the detection performance of member relationships in pre-trained code of large language models and providing technical support for copyright compliance and transparency auditing of model training data. Summary of the Invention
[0007] In view of this, the present invention provides a method for inferring code members of a large model based on code syntax constraint lexical filtering to solve the above problems.
[0008] This invention provides a method for inferring code members in a large language model based on code syntax constraint lexical filtering, comprising: constructing a set of syntax conventions based on the syntax specifications of the target programming language, wherein each syntax convention includes antecedent conditions and corresponding successor lexical units; inputting the source code to be detected into a large language model to obtain the lexical sequence output by the model and the predicted probability of each lexical unit; performing syntactic structure analysis on the lexical sequence, identifying and removing lexical units that are determined solely by syntax rules and are unrelated to the author's personality based on the set of syntax conventions, thereby obtaining a set of retained lexical units; calculating a member score based on the predicted probability of each lexical unit in the set of retained lexical units; and determining whether the source code to be detected belongs to the pre-training data members of the large language model based on the comparison result of the member score and a preset threshold.
[0009] In another implementation of the present invention, the set of syntax conventions includes at least one of the following syntax categories: data model syntax conventions, expression syntax conventions, simple statement syntax conventions, and compound statement syntax conventions.
[0010] In another implementation of the present invention, the method further includes: decomposing the original lexical units in the lexical sequence, segmenting the original lexical units containing multiple grammatical symbols or keywords according to character or keyword boundaries to obtain sub-lexical units; generating an abstract syntax tree of the source code to be detected using a code parser, aligning the sub-lexical units with AST nodes, and verifying whether their context satisfies the antecedent conditions of the grammatical convention.
[0011] In another implementation of the present invention, the step of performing grammatical structure analysis on the lexical sequence, and identifying and removing grammatical convention lexical units that are determined solely by grammatical rules and are unrelated to the author's personality based on the grammatical convention set, to obtain a retained lexical unit set, includes: traversing the lexical sequence; if all sub-lexical units of a certain original lexical unit satisfy the subsequent lexical unit features in the grammatical convention set, and its context environment meets the corresponding antecedent conditions, then it is marked as a grammatical convention lexical unit and removed to obtain a retained lexical unit set.
[0012] In another implementation of the present invention, the set of reserved lexical units is represented as:
[0013] in, 0, 1} are token retention tags, where 1 means to retain and 0 means to remove.
[0014] In another implementation of the invention, the member score is the average of the negative log probabilities of the retained lexical units:
[0015] in, To preserve the index set of tokens, Let be the predicted probability of the large language model under test for the i-th retained word.
[0016] In another implementation of the present invention, determining whether the source code to be detected belongs to the pre-training data member of the large language model based on the comparison result of the member score and the preset threshold includes: when SPP(x) > ε, determining that the source code sample to be detected belongs to the pre-training data member of the large language model; when SPP(x) ≤ ε, determining that the source code sample to be detected does not belong to the pre-training data member of the large language model.
[0017] In another aspect, this invention provides a large-scale model code member inference system based on code syntax constraint lexical filtering, comprising: a syntax convention construction module: constructing a set of syntax conventions based on the syntax specifications of the target programming language, each syntax convention including antecedent conditions and corresponding successor lexical units; a lexical unit probability acquisition module: inputting the source code to be detected into the large language model, obtaining the lexical unit sequence output by the model and the predicted probability of each lexical unit; a syntax filtering module: performing syntactic structure analysis on the lexical unit sequence, identifying and removing lexical unit terms determined solely by syntax rules and unrelated to the author's personality based on the set of syntax conventions, obtaining a set of retained lexical units; a scoring calculation module: calculating a member score based on the predicted probability of each lexical unit in the set of retained lexical units; and a judgment output module: determining whether the source code to be detected belongs to the pre-training data member of the large language model based on the comparison result of the member score and a preset threshold.
[0018] In another aspect, the present invention provides an electronic device comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the computer program, implements the steps of a large model code member inference method based on code syntax constraint lexical filtering as described in any of the preceding claims. In another aspect, the present invention provides a computer storage medium storing a computer program that, when executed by a processor, implements the steps of a large model code member inference method based on code syntax constraint lexical filtering as described in any of the preceding claims.
[0019] The present invention provides a large-scale code member inference method based on code grammar constraint lexical filtering. By explicitly constructing a set of grammatical conventions and performing grammatical pruning at the lexical level, a large number of lexical terms determined by grammatical rules are removed from member scores, making member scores more focused on lexical terms that carry the author's style, thereby improving the separability between members and non-members. Post-processing is based on the output probabilities of the trained model, eliminating the need to retrain shadow models or build complex auxiliary models, resulting in lower additional computational costs. It more accurately preserves lexical terms that reflect the code author's intent and style, reduces the interference of grammatical noise on member inference, and significantly improves member inference performance under various large language models and various member / non-member ratio settings. Attached Figure Description
[0020] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. By reading the detailed description of the embodiments below, the advantages and benefits of the solutions will become clear to those skilled in the art. The accompanying drawings are only for illustrating preferred embodiments and are not intended to limit the present invention. In the accompanying drawings: Figure 1 This is a schematic diagram of a large model code member inference method based on code syntax constraint lexical filtering, according to an embodiment of the present invention.
[0021] Figure 2 This is a flowchart illustrating a large model code member inference method based on code syntax constraint lexical filtering, according to an embodiment of the present invention. Detailed Implementation
[0022] To enable those skilled in the art to better understand the technical solutions in the embodiments of the present invention, the technical solutions in the embodiments of the present invention will be clearly and thoroughly described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art should fall within the protection scope of the present invention.
[0023] Figure 1 A schematic diagram of a large model code member inference method based on code syntax constraint lexical filtering provided in an embodiment of the present invention is shown below. Figure 1 As shown, this embodiment mainly includes: S101. Based on the syntax specification of the target programming language, construct a set of syntax conventions, each of which includes antecedent conditions and corresponding successor terms.
[0024] For example, based on the official syntax specification of the target programming language (such as Python), the syntactic elements such as data models, expressions, simple statements, and compound statements are organized to obtain several syntactic conventions, each of which is defined by... Precondition, successor token The formal representation forms a set of grammatical conventions. Each grammatical convention describes the successor token that must or is highly certain to appear under a specific grammatical structure, such as parentheses, colons, indentation blocks, fixed keyword sequences, etc.
[0025] S102. Input the source code to be detected into the large language model and obtain the word sequence output by the model and the predicted probability of each word.
[0026] For example, the source code sample to be detected is input into the large language model under test, and the token sequence and its corresponding output probability are obtained through the built-in token segmenter of the model, where each token corresponds to its conditional probability in a given context.
[0027] S103. Perform grammatical structure analysis on the lexical sequence, identify and remove grammatical convention lexical units that are determined solely by grammatical rules and are unrelated to the author's personality based on the set of grammatical conventions, and obtain the set of retained lexical units.
[0028] S104. Calculate the member score based on the predicted probability of each word in the reserved word set.
[0029] S105. Based on the comparison result between the member score and the preset threshold, determine whether the source code to be detected belongs to the pre-training data member of the large language model.
[0030] The present invention provides a large-scale code member inference method based on code grammar constraint lexical filtering. By explicitly constructing a set of grammatical conventions and performing grammatical pruning at the lexical level, a large number of lexical terms determined by grammatical rules are removed from member scores, making member scores more focused on lexical terms that carry the author's style, thereby improving the separability between members and non-members. Post-processing is based on the output probabilities of the trained model, eliminating the need to retrain shadow models or build complex auxiliary models, resulting in lower additional computational costs. It more accurately preserves lexical terms that reflect the code author's intent and style, reduces the interference of grammatical noise on member inference, and significantly improves member inference performance under various large language models and various member / non-member ratio settings.
[0031] In another implementation of the present invention, the set of syntax conventions includes at least one of the following syntax categories: data model syntax conventions, expression syntax conventions, simple statement syntax conventions, and compound statement syntax conventions.
[0032] For example, the syntax conventions of data model classes include: syntactic constructs such as lists, slices, dictionaries, sets, strings, byte sequences, objects, and tuples, and corresponding opening and closing delimiters and access symbols, such as "[" and "]", "{" and "}", and "(" and ")".
[0033] Expression syntax conventions include syntax nodes such as function calls, lambda expressions, conditional expressions, comprehensions, and chained comparisons, with subsequent tokens including right parentheses, commas, and comparison operators.
[0034] Simple statement syntax conventions: including import, from-import, assert, global, etc., the corresponding keyword sequences, and the occurrence patterns of the optional alias as.
[0035] Compound statement syntax conventions include statements such as if, for, while, try-except, with, class, def, and match. Their characteristics are that a colon and an indented block must follow the keyword at the beginning of the statement, and they may be accompanied by branching structures such as elif, else, except, and finally.
[0036] In another implementation of the present invention, the method further includes: decomposing the original lexical units in the lexical sequence, segmenting the original lexical units containing multiple grammatical symbols or keywords according to character or keyword boundaries to obtain sub-lexical units; generating an abstract syntax tree of the source code to be detected using a code parser, aligning the sub-lexical units with AST nodes, and verifying whether their context satisfies the antecedent conditions of the grammatical convention.
[0037] For example, such as Figure 2 As shown, the obtained token sequence is decomposed into sub-tokens. If a single token contains a substring corresponding to a predecessor or successor token in the syntax convention, then the token is split into multiple ordered sub-tokens. Based on the decomposed sub-token sequence and the abstract syntax tree (AST) of the source code, each sub-token is traversed to determine whether it appears as a successor token in the syntax convention, and the context of its AST node position is verified to ensure that the antecedent condition of the syntax convention is met. For example, when the target programming language is Python, the syntax convention set is built based on the official Python language reference documentation, and the abstract syntax tree (AST) is generated by the language's built-in parser.
[0038] In another implementation of the present invention, the step of performing grammatical structure analysis on the lexical sequence, and identifying and removing grammatical convention lexical units that are determined solely by grammatical rules and are unrelated to the author's personality based on the grammatical convention set, to obtain a retained lexical unit set, includes: traversing the lexical sequence; if all sub-lexical units of a certain original lexical unit satisfy the subsequent lexical unit features in the grammatical convention set, and its context environment meets the corresponding antecedent conditions, then it is marked as a grammatical convention lexical unit and removed to obtain a retained lexical unit set.
[0039] For example, if all sub-tokens of an original token are marked as satisfying the syntax convention, then the original token is marked as a "syntax convention token" and is not included in the member scoring calculation; the remaining tokens that are not marked as syntax convention tokens are formed into a reserved token set.
[0040] In another implementation of the present invention, the set of reserved lexical units is represented as:
[0041] in, 0, 1} are token retention tags, where 1 means to retain and 0 means to remove.
[0042] For example, the source code x to be detected is input into the target large language model to obtain the word sequence. and their corresponding prediction probabilities The code parser generates an Abstract Syntax Tree (AST), and the word sequence is aligned with the AST nodes. The word sequence is traversed; if a certain word... Set of syntactic conventions If a word has the characteristic of being a "successor word" and its context meets the corresponding "precedence condition" (e.g., the current word is self and it is in the parameter list of the function defined by def), then it is marked as a "syntactic convention word" and removed (Pruned).
[0043] In another implementation of the invention, the member score is the average of the negative log probabilities of the retained lexical units:
[0044] in, To preserve the index set of tokens, Let be the predicted probability of the large language model under test for the i-th retained word.
[0045] For example, a membership score for grammatical trimming is calculated based on the token probabilities in the retained token set, where the membership score is the average of the negative log probabilities of the retained tokens. Membership scores are calculated only on the "non-grammatically necessary lexical units" retained after grammatical trimming to exclude interference from grammatical noise.
[0046] In another implementation of the present invention, determining whether the source code to be detected belongs to the pre-training data member of the large language model based on the comparison result of the member score and the preset threshold includes: when SPP(x) > ε, determining that the source code sample to be detected belongs to the pre-training data member of the large language model; when SPP(x) ≤ ε, determining that the source code sample to be detected does not belong to the pre-training data member of the large language model.
[0047] For example, the direction here depends on the threshold definition. Generally, the lower the negative log probability, the more likely it is to be a member, or the higher the negative value, the more likely it is to be a member. Here, it is sufficient to keep the logic consistent with the specification. Generally, determining it as a member means that the perplexity is low enough.
[0048] Starting with a large language model pre-trained on The Pile dataset (which is publicly declared to be available), this invention constructs a realistic membership inference benchmark by using real Python functions from The Pile as member samples and functions released later than the model release, whose originality has been verified through GitHub API retrieval and manual review, as non-member samples. Based on this, the method of this invention achieves an average AUROC improvement of over 10% compared to existing methods such as LOSS, ZLIB, MIN-K%, and DC-PDD under multiple large language models and various member / non-member ratio settings, validating the effectiveness and universality of the syntax pruning approach in code membership inference scenarios. This invention's method is used for batch membership detection of function-level source code in open-source code repositories to evaluate whether the pre-training data of large language models contains code samples with specific license constraints.
[0049] Example 1 Based on the official syntax reference documentation of the target programming language (taking Python as an example), a set of syntax conventions was manually compiled and constructed. Each convention is described in the form of {Precondition, Consequent}. A total of 47 grammatical conventions are summarized, covering four main categories: data models, expressions, simple statements, and compound statements. Examples of specific grammatical conventions are shown in the table below:
[0050] Example 2 To illustrate the working process of this invention more intuitively, the open-source large language model Pythia-2.8B is used as the model under test to perform member inference detection on a piece of Python unit test code.
[0051] 1. Sample code to be detected The following Python function-level source code is selected as the input sample x: def testbanOK(self): ticket_str = 'FailTicket: ip=193.168.0.128 time=1167605999.0 bantime=None' ticket.setTime(1000002000.0) self.assertEqual(ticket.getTime(), 1000002000.0) 2. Lexicalization and Probability Acquisition Input the above code into the Pythia-2.8B model. The model first uses its built-in token segmenter to divide the code into token sequences and outputs the predicted probability of each token given the context. For example, for the word sequence of the function definition line ['def', 'test', 'ban', 'OK', '(', 'self', ')', ':'], the model, based on the memory of training with massive amounts of code, will usually give extremely high prediction probabilities for parts with fixed grammatical structures (e.g., p(self)≈0.99, p(:)≈0.999).
[0052] 3. Syntax Convention Matching and Pruning The method of this invention combines an Abstract Syntax Tree (AST) to traverse lexical units and match them with grammatical rules: 1) The term "self": The system identifies it as currently within a FunctionDef node and at the first position in the parameter list. According to the syntax convention of "compound statement classes": the first parameter of a Python class method definition is necessarily or with a very high probability being self. This term satisfies the syntax convention and is judged as "syntactic noise," so it is removed.
[0053] 2) Lexicons and colons: After the parameter list in a function definition, the syntax mandates closing parentheses and colons. These two lexicons are entirely determined by the syntax rules and are therefore excluded.
[0054] 3) The lexical in or = (if present): For example, in the code ticket_str = ..., the assignment symbol = is part of the syntactic structure and is removed.
[0055] 4) Lexicon testbanOK: This is a developer-defined function name that is not subject to any mandatory syntax conventions and is therefore reserved.
[0056] 5) The string content: 'FailTicket: ip=193.168.0.128 ...' contains a specific IP address and timestamp, which is high-entropy semantic information. It depends entirely on the author's intention and is therefore retained.
[0057] 4. Member rating calculation After the above trimming steps, the system obtains a set of retained lexical units. (Mainly includes semantically meaningful terms such as testbanOK, ticket_str, 193.168.0.128, ticket, setTime, 1000002000.0). Using formulas Calculate the score.
[0058] Comparative explanation: If the traditional perplexity (PPL) method is used, the high prediction probability (extremely low loss) of a large number of grammatical terms (such as self, :, ()) will lower the overall average loss, making the model's "memory" of this code seem stronger than it actually is (or making it difficult to distinguish between members and non-members).
[0059] The benefits of this invention: By eliminating these syntactic terms, the SPP score focuses only on specific variable names, IP address values, and function call logic. If the model has "seen" this specific code (member) during the pre-training phase, its prediction probability for a specific number sequence like 193.168.0.128 will be significantly higher than for unseen cases.
[0060] 5. Judgment Result The calculated SPP score is 0.008. This is compared to the preset threshold ε = 0.015. Since 0.008 < 0.015 (i.e., the mean negative log probability is extremely low, indicating that the model predicts the preserved semantic parts very accurately), the system determines that the function `testbanOK` belongs to the pre-training data members of Pythia-2.8B. This conclusion is consistent with the actual situation (this code snippet does indeed come from an open-source project in The Pile dataset).
[0061] Compared with the prior art, the present invention has the following beneficial effects: 1. Fully leverage code syntax structure to improve the signal-to-noise ratio of detection. By explicitly constructing a set of syntax conventions and performing syntax pruning at the token level, a large number of tokens determined by syntax rules are removed from member scoring, making member scoring more focused on tokens that embody the author's style, thereby improving the separability between members and non-members.
[0062] 2. The method is universal and easily scalable to various programming languages and models. Based on a universal framework of "syntax conventions + AST matching + token probabilistic pruning," this invention can be extended to multiple programming languages such as Java, C / C++, and JavaScript, provided the syntax reference document and parser are replaced. Furthermore, it is independent of the model's internal structure and can be used with large language models of various architectures and scales.
[0063] 3. Seamless integration with existing compliance audit processes. This invention uses function-level source code as the unit of detection, allowing for easy embedding into enterprise open-source compliance audits and model training data governance processes. It is used to determine whether a target large language model uses code samples with specific license constraints, providing quantitative evidence for legal and compliance teams.
[0064] This invention significantly improves MIA performance without increasing model training costs. It performs post-processing only based on the output probabilities of the already trained model, eliminating the need to retrain shadow models or construct complex auxiliary models. Therefore, the additional computational cost is low, yet it achieves significantly better detection results than existing methods across multiple model and data ratio settings.
[0065] Another aspect of the present invention provides a large model code member inference system based on code syntax constraint lexical filtering, comprising: Syntax convention building module: Based on the syntax specification of the target programming language, a set of syntax conventions is built. Each syntax convention includes antecedent conditions and corresponding successor terms.
[0066] Lexical probability acquisition module: Input the source code to be detected into the large language model, and obtain the lexical sequence and the predicted probability of each lexical from the model output.
[0067] Syntax filtering module: Performs syntactic structure analysis on the lexical sequence, identifies and removes lexical terms that are determined solely by grammatical rules and are unrelated to the author's personality based on the set of grammatical conventions, and obtains a set of retained lexical terms.
[0068] Scoring Calculation Module: Calculates member scores based on the predicted probabilities of each word in the reserved word set.
[0069] Determination output module: Based on the comparison result between the member score and the preset threshold, determine whether the source code to be detected belongs to the pre-training data member of the large language model.
[0070] The present invention provides a large-scale code member inference system based on code grammar constraint lexical filtering. By explicitly constructing a set of grammatical conventions and performing grammatical pruning at the lexical level, a large number of lexical terms determined by grammatical rules are removed from member scoring, making member scoring more focused on lexical terms that carry the author's style, thereby improving the separability between members and non-members. Post-processing is based on the output probability of the trained model, eliminating the need to retrain shadow models or build complex auxiliary models, resulting in lower additional computational costs. It more accurately preserves lexical terms that reflect the code author's intent and style, reduces the interference of grammatical noise on member inference, and significantly improves member inference performance under various large language models and various member / non-member ratio settings.
[0071] In another aspect of the present invention, the electronic device includes: a processor, a memory, and a communication bus and a communication interface.
[0072] in: The processor, memory, and communication interface communicate with each other via a communication bus.
[0073] A communication interface is used to communicate with other electronic devices or servers.
[0074] The processor is used to execute programs, specifically, to perform any of the steps of the large model code member inference method based on code syntax constraint lexical filtering in the above embodiments.
[0075] Specifically, the program may include program code, which includes computer operation instructions.
[0076] The processor may be a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of this application. The one or more processors included in the smart device may be processors of the same type, such as one or more CPUs; or they may be processors of different types, such as one or more CPUs and one or more ASICs.
[0077] Memory is used to store programs. Memory may include high-speed RAM, and may also include non-volatile memory, such as at least one disk drive.
[0078] Specifically, the program can be used to cause the processor to execute steps to implement any of the code syntax constraint lexical filtering methods for large model code membership inference described in the embodiments. The specific implementation of each step in the program can be found in the corresponding descriptions of the steps and units executed by any of the code syntax constraint lexical filtering methods for large model code membership inference described above, and will not be repeated here. Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the devices and modules described above can be referred to the corresponding process descriptions in the foregoing method embodiments.
[0079] An exemplary embodiment of this application also provides a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to cause a computer to perform the methods of various embodiments of this application.
[0080] The methods described above according to embodiments of the present invention can be implemented in hardware, firmware, or as software or computer code that can be stored in a recording medium (such as a CD-ROM, RAM, floppy disk, hard disk, or magneto-optical disk), or as computer code originally stored on a remote recording medium or a non-transitory machine-readable medium and subsequently stored on a local recording medium, downloaded via a network. Thus, the methods described herein can be processed by software stored on a recording medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware (such as an ASIC or FPGA). It is understood that the computer, processor, microprocessor controller, or programmable hardware includes storage components (e.g., RAM, ROM, flash memory, etc.) capable of storing or receiving software or computer code, which, when accessed and executed by the computer, processor, or hardware, implements the methods described herein. Furthermore, when a general-purpose computer accesses code used to implement the methods shown herein, the execution of the code transforms the general-purpose computer into a dedicated computer for executing the methods shown herein.
[0081] Specific embodiments of the present invention have now been described. Other embodiments are within the scope of the appended claims. In some cases, the actions described in the claims can be performed in a different order and still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require a specific or sequential order to achieve the desired result.
[0082] It should be noted that all directional indications (such as up, down, left, right, back, etc.) in the embodiments of the present invention are only used to explain the relative positional relationship between the components in a certain order (as shown in the figure). If the specific order changes, the directional indication will also change accordingly.
[0083] In the description of this invention, the terms "first" and "second" are used only for convenience in describing different components or names, and should not be construed as indicating or implying a sequential relationship, relative importance, or implicitly specifying the number of technical features indicated. Thus, a feature defined with "first" and "second" may explicitly or implicitly include at least one of that feature.
[0084] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used herein in the description of the invention is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention.
[0085] It should be noted that although specific embodiments of the present invention have been described in detail with reference to the accompanying drawings, this should not be construed as limiting the scope of protection of the present invention. Various modifications and variations that can be made by those skilled in the art without inventive effort within the scope described in the claims still fall within the scope of protection of the present invention.
[0086] The examples of the embodiments of the present invention are intended to concisely illustrate the technical features of the embodiments of the present invention, so that those skilled in the art can intuitively understand the technical features of the embodiments of the present invention, and are not intended to be an improper limitation of the embodiments of the present invention.
[0087] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims
1. A method for inferring code membership in a large model based on code syntax constraint lexical filtering, characterized in that, include: Based on the syntax specifications of the target programming language, a set of syntax conventions is constructed, and each syntax convention includes an antecedent condition and its corresponding successor lexical. Input the source code to be detected into the large language model and obtain the word sequence output by the model and the predicted probability of each word; A grammatical structure analysis is performed on the lexical sequence. Based on the set of grammatical conventions, lexical terms that are determined solely by grammatical rules and are unrelated to the author's personality are identified and eliminated, resulting in a set of retained lexical terms. Based on the predicted probability of each word in the reserved word set, the member score is calculated; Based on the comparison between the member score and the preset threshold, it is determined whether the source code to be detected belongs to the pre-trained data member of the large language model.
2. The method according to claim 1, characterized in that, The set of grammatical conventions includes at least one of the following grammatical categories: Syntax conventions for data models, expressions, simple statements, and compound statements.
3. The method according to claim 1, characterized in that, Also includes: The original lexical units in the lexical unit sequence are decomposed, and the original lexical units containing multiple grammatical symbols or keywords are segmented according to character or keyword boundaries to obtain sub-lexical units; The code parser generates an abstract syntax tree for the source code to be tested, and aligns the sub-words with the AST nodes to verify whether the context satisfies the antecedent conditions of the syntax convention.
4. The method according to claim 3, characterized in that, The grammatical structure analysis of the lexical sequence involves identifying and removing grammatical convention lexical units that are solely determined by grammatical rules and unrelated to the author's personality, based on the set of grammatical conventions, to obtain a retained lexical unit set, including: Traverse the sequence of lexical units. If all sub-lexical units in a certain original lexical unit satisfy the features of the successor lexical units in the grammatical convention set and their context environment meets the corresponding antecedent conditions, then mark them as grammatical convention lexical units and remove them to obtain the set of retained lexical units.
5. The method according to claim 4, characterized in that, The set of reserved lexical units is represented as follows: in, 0, 1} are token retention tags, where 1 means to retain and 0 means to remove.
6. The method according to claim 1, characterized in that, The member score is the average of the negative log probabilities of the retained lexical units: in, To preserve the index set of tokens, Let be the predicted probability of the large language model under test for the i-th retained word.
7. The method according to claim 6, characterized in that, The step of determining whether the source code to be detected belongs to the pre-training data member of the large language model based on the comparison result of the member score and the preset threshold includes: When SPP(x) > ε, it is determined that the source code sample to be detected belongs to the pre-training data member of the large language model; When SPP(x)≤ε, it is determined that the source code sample to be detected does not belong to the pre-training data members of the large language model.
8. A large-model code membership inference system based on code syntax constraint lexical filtering, characterized in that, include: Syntax convention building module: Based on the syntax specification of the target programming language, a set of syntax conventions is built. Each syntax convention includes an antecedent condition and its corresponding successor lexical. Lexical probability acquisition module: Input the source code to be detected into the large language model, and obtain the lexical sequence output by the model and the predicted probability of each lexical; Syntax filtering module: performs syntactic structure analysis on the lexical sequence, identifies and removes lexical terms that are determined solely by grammatical rules and are unrelated to the author's personality based on the set of grammatical conventions, and obtains a set of retained lexical terms; The scoring calculation module calculates the member score based on the predicted probability of each word in the reserved word set. Determination output module: Based on the comparison result between the member score and the preset threshold, determine whether the source code to be detected belongs to the pre-training data member of the large language model.
9. An electronic device, characterized in that, include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the computer program, implements the steps of the large model code member inference method based on code syntax constraint lexical filtering as described in any one of claims 1 to 7.
10. A computer storage medium, characterized in that, The computer storage medium stores a computer program, which, when executed by a processor, implements the steps in the large model code member inference method based on code syntax constraint lexical filtering as described in any one of claims 1 to 7.