Code processing method and code processing system

Abstract

The embodiment of the present specification provides a code processing method and a code processing system, wherein the code processing method comprises: in response to a code generation request of a front end for original code, parsing the original code to obtain an original abstract syntax tree and a symbol table; determining a target symbol based on preset sensitive symbol rules traversing each symbol in the symbol table; generating a marked symbol corresponding to the target symbol; replacing a node corresponding to the target symbol in the original abstract syntax tree with the marked symbol to obtain a desensitization abstract syntax tree; generating desensitization code based on the desensitization abstract syntax tree; and uploading the desensitization code to a server, so that a code generation model deployed by the server generates code based on the desensitization code. The code generation result with higher applicability can be obtained under the premise of ensuring the safety of sensitive information, and the problem of poor applicability of the completed code based on the desensitization code generation is solved.

Description

Technical Field

[0001] The embodiments in this specification relate to the field of data processing technology, and in particular to a code processing method and a code processing system. Background Technology

[0002] With the development of artificial intelligence (AI), AI is increasingly being applied to assisted code generation, greatly improving code writing efficiency. In practical applications of AI-assisted code generation, it is often necessary to input a portion of the source code into the generative model as a reference context for the model's output code. However, source code frequently contains core corporate secrets or sensitive information, and directly uploading the source code to the code generation model carries the risk of leaking corporate secrets. Therefore, de-identifying the source code before uploading it becomes a necessary step.

[0003] Existing source code anonymization solutions suffer from poor applicability of the generated code based on the anonymized source code. Summary of the Invention

[0004] In view of this, embodiments of this specification provide a code processing method. One or more embodiments of this specification also relate to a code processing system, a computing device, and a computer-readable storage medium, to address the technical deficiencies existing in the prior art.

[0005] According to a first aspect of the embodiments of this specification, a code processing method is provided, including: In response to the front-end's code generation request for the original code, the original code is parsed to obtain the original abstract syntax tree and symbol table; The target symbol is determined by traversing the symbol table based on preset sensitive symbol rules. Generate the tag symbol corresponding to the target symbol; Replace the nodes corresponding to the target symbols in the original abstract syntax tree with the tag symbols to obtain the de-identified abstract syntax tree; Generate de-identified code based on the de-identification abstract syntax tree; Upload the de-identified code to the server so that the code generation model deployed on the server can generate code based on the de-identified code.

[0006] According to a second aspect of the embodiments of this specification, a code processing system is provided, including a front-end, a client, and a server; The front end is used to send code generation requests to the client based on the original code; The client is used to execute the above code processing methods; The server is used to call the code generation model and generate code snippets based on the de-identified code.

[0007] According to a third aspect of the embodiments of this specification, a computing device is provided, comprising: Memory and processor; The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions. When the computer-executable instructions are executed by the processor, they implement the steps of the above code processing method.

[0008] According to a fourth aspect of the embodiments of this specification, a computer-readable storage medium is provided that stores computer-executable instructions that, when executed by a processor, implement the steps of the above-described code processing method.

[0009] In one or more embodiments of this specification, in response to a front-end request for code generation of the original code, the original code is parsed to obtain an original abstract syntax tree and symbol table; each symbol in the symbol table is traversed based on preset sensitive symbol rules to determine the target symbol; a marker symbol corresponding to the target symbol is generated; the node corresponding to the target symbol in the original abstract syntax tree is replaced with the marker symbol to obtain a desensitized abstract syntax tree; desensitized code is generated based on the desensitized abstract syntax tree; and the desensitized code is uploaded to the server so that the code generation model deployed on the server can generate code based on the desensitized code. This method can accurately identify sensitive symbols through the symbol table, avoiding cross-scope conflicts caused by the inability to distinguish scopes in regular expression replacement. By replacing nodes at the abstract syntax tree level rather than text replacement, the consistency of the syntax structure of the desensitized code and the original code is ensured, thus avoiding misalignment during restoration due to position offsets or format changes. Because the desensitized code retains a complete syntax structure and the marker symbols have clear mapping relationships, the code generation model can correctly understand the context and output complete code with excellent readability, thereby achieving more applicable code generation results while ensuring the security of sensitive information, solving the problem of poor applicability of complete code generated based on desensitized code. Attached Figure Description

[0010] Figure 1 This is a flowchart illustrating a code processing method provided in one embodiment of this specification; Figure 2 This is a sequence diagram illustrating the flow architecture of a code processing method provided in one embodiment of this specification. Figure 3 This is a sequence diagram of the flow architecture of another code processing method provided in one embodiment of this specification; Figure 4 This is a schematic diagram of the structure of a code processing system provided in one embodiment of this specification; Figure 5 This is a schematic diagram of the structure of another code processing system provided in one embodiment of this specification; Figure 6This is a structural block diagram of a computing device provided in one embodiment of this specification. Detailed Implementation

[0011] Many specific details are set forth in the following description to provide a full understanding of this specification. However, this specification can be implemented in many other ways than those described herein, and those skilled in the art can make similar extensions without departing from the spirit of this specification. Therefore, this specification is not limited to the specific implementations disclosed below.

[0012] The terminology used in one or more embodiments of this specification is for the purpose of describing particular embodiments only and is not intended to be limiting of the one or more embodiments of this specification. The singular forms “a,” “described,” and “the” as used in one or more embodiments of this specification and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used in one or more embodiments of this specification refers to and includes any or all possible combinations of one or more associated listed items.

[0013] It should be understood that although the terms first, second, etc., may be used to describe various information in one or more embodiments of this specification, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, first may also be referred to as second without departing from the scope of one or more embodiments of this specification, and similarly, second may also be referred to as first. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to a determination."

[0014] Furthermore, it should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in one or more embodiments of this specification are all information and data authorized by the user or fully authorized by all parties. Moreover, the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation entry points are provided for users to choose to authorize or refuse.

[0015] First, the terms and concepts used in one or more embodiments of this specification will be explained.

[0016] AI code generation tools: AI code generation tools are code completion / generation / refactoring services based on large language models, including GitHub Copilot, Amazon CodeWhisperer, Cursor, Claude Code, StarCoder, etc.

[0017] Reversible Code Desensitization (RCD): This method replaces sensitive identifiers with temporary tags locally and generates a mapping table that can be restored locally without changing the compilation result.

[0018] S2 digest: S2 digest refers to the SHA-256 hash value calculated from the topology and data flow dependencies of the Abstract Syntax Tree (AST) of the anonymized code. It is used for fast matching of the mapping table during the restoration stage.

[0019] Incremental Fusion Engine (IFE): IFE is used to restore the desensitized fragments returned by the model, and then perform a three-way merging with the local working copy and common ancestor. It also automatically repairs syntax-level conflicts and prompts semantic-level conflicts to make the merged result compileable and runnable as much as possible.

[0020] An Integrated Development Environment (IDE) is an application that provides a program development environment, which may include tools such as a code editor, compiler, debugger, and graphical user interface. It integrates code writing, analysis, compilation, and debugging functions to assist developers in writing, testing, debugging, and deploying code.

[0021] Front-end: The front-end refers to the code layer that runs on the user's device (such as a browser or application) and is responsible for building and rendering the user interface (UI) that the user directly sees and interacts with.

[0022] Server-side: The server-side refers to the backend system that runs on a remote server, is responsible for handling core project logic, data storage, permission verification, and providing data interfaces to clients.

[0023] Client: A client is a program installed on a user's device that is responsible for initiating network requests, receiving server responses, and managing the local user state and interaction logic.

[0024] Enterprise source code often contains sensitive information such as keys and project rules. If it is submitted directly to the cloud AI code generation service, there is a risk of leakage: (1) Source code leakage is irreversible: Once the AI ​​model obtains the real identifier, it can remember it permanently and send it back with prompts. The traditional "post-deletion" cannot prove that it has been forgotten. (2) Single point of failure → global failure: After the key and algorithm constants are restored, the attacker can directly bypass all runtime protections. The existing "simple regular expression desensitization" solution cannot guarantee: (1) The generated code can be compiled directly. (2) The variable names after restoration are consistent with the original code and there is no conflict when merging with the local undesensitized code.

[0025] To address the aforementioned problems, this specification provides a code processing method, a code processing system, a computing device, and a computer-readable storage medium, which will be described in detail in the following embodiments.

[0026] See Figure 1 , Figure 1 A flowchart of a code processing method according to an embodiment of this specification is shown, including the following specific steps: Step 102: In response to the front end's code generation request for the original code, parse the original code to obtain the original abstract syntax tree and symbol table.

[0027] The embodiments in this specification are applied to scenarios that require code processing, including but not limited to: code desensitization, symbol replacement, abstract syntax tree generation, and symbol table construction.

[0028] The original code is either a fragment of source code or complete code to be processed. For example, the original code could be a source code file containing variable definitions, functions, and sensitive information such as keys and internal interface addresses.

[0029] A code generation request is an instruction that performs code completion or code generation based on the original code. For example, a code generation request can be a network request triggered by a user pressing a shortcut key in a code editor.

[0030] A primitive abstract syntax tree (API) is a tree-like data structure obtained after lexical and syntactic analysis of the original code. It reflects the syntactic structure contained in the original code. For example, an API can include variable declaration nodes, function call nodes, and literal nodes.

[0031] A symbol table is a data structure used to record the semantic information of each identifier (variable name, function name, class name, etc.) in the original code, including its scope, type, and reference location. It is generated after semantic analysis of the original code. For example, a symbol table can contain multiple entries, each corresponding to an identifier and recording the scope level to which that identifier belongs.

[0032] Indicatively, Figure 2 A timing diagram illustrating the flow architecture of a code processing method according to an embodiment of this specification is shown.

[0033] See Figure 2 Developers / users send code generation requests for the original code to the client through the front end. The client responds to the request, parses the original code, and obtains the original abstract syntax tree and symbol table.

[0034] In response to a code generation request from the frontend, the original code is parsed to obtain the original abstract syntax tree and symbol table. This can be achieved in several ways: First, by intercepting the code generation request through an integrated development environment (IDE) plugin and calling a local parser to parse the original code. Second, by receiving the request on the server side and using a pre-loaded compiler frontend component to parse the original code. Third, by directly generating the abstract syntax tree and symbol table on the client side using a lightweight parser library (such as a tree parser). This approach is not limited to this method.

[0035] For example, when a user is writing code in an integrated development environment, they select a function body and click the "AI Completion" button, and the front end generates a code generation request. The client plugin receives this request, calls the locally deployed syntax parsing tool to parse the original code, obtains the original abstract syntax tree (containing function declaration nodes, variable nodes, return statement nodes, etc.), and at the same time builds a symbol table (recording function names, parameter names, local variable names and their scope information).

[0036] In response to the front-end's code generation request for the original code, the original code is parsed to obtain the original abstract syntax tree and symbol table, providing a structured and semantic foundation for subsequent accurate identification of sensitive symbols based on the symbol table and node replacement based on the abstract syntax tree.

[0037] Step 104: Traverse each symbol in the symbol table based on the preset sensitive symbol rules to determine the target symbol.

[0038] The preset sensitive symbol rules are matching conditions used to determine whether a symbol contains sensitive information. For example, the preset sensitive symbol rules can be a list of regular expressions (such as matching keywords such as "password", "secret", "token", "apikey", etc.), or they can be judgment rules based on symbol type (such as string constants, database column names).

[0039] The target symbol is an identifier in the symbol table that meets the preset sensitive symbol rules and needs to be de-identified. For example, the target symbol can be the variable name "adminPassword" or the function name "getSecretKey".

[0040] See Figure 2After obtaining the original abstract syntax tree and symbol table, the client traverses each symbol in the symbol table based on the preset sensitive symbol rules to determine the target symbol.

[0041] The specific methods for determining the target symbol by traversing the symbol table based on preset sensitive symbol rules can be as follows: Each symbol in the symbol table is read sequentially, and its name is matched against a preset sensitive word dictionary. Alternatively, the symbol type information and annotation tags can be combined to determine whether it is a sensitive symbol using weighted rules. Another method is to use a local small language model to semantically classify the symbol names and filter out symbols belonging to the sensitive category; this is not limited here.

[0042] For example, the default sensitive symbol rule is a list containing keywords such as "token", "secret", "pwd", and "key". Traversing the symbol list, if a variable symbol named "dbPassword" is found that contains "pwd", then that symbol is identified as a target symbol; another variable symbol named "tempIndex" does not contain any sensitive words, so it is not identified as a target symbol.

[0043] Based on the preset sensitive symbol rules, the system traverses each symbol in the symbol table to determine the target symbol. It can use the semantic information provided by the symbol table (such as symbol name, type, and scope) to achieve more accurate sensitive identification than plain text matching, avoiding mismatches of ordinary words in comments or strings.

[0044] Step 106: Generate the marker symbol corresponding to the target symbol.

[0045] The marker symbol is a placeholder used to temporarily replace the original target symbol during the desensitization stage. It is unique and corresponds one-to-one with the original target symbol. For example, the marker symbol can be "_T{0}_", "_T{1}_", "_T{2}_", etc.

[0046] See Figure 2 After determining the target symbol, the client generates the corresponding marker symbol.

[0047] The specific methods for generating the marker symbols corresponding to the target symbols can be as follows: First, generate them sequentially according to a preset naming template (e.g., "_T{serial number}_"). Second, use a hash function to calculate the original symbol name and scope information to obtain a fixed-length marker. Third, use an incrementing counter combined with a random suffix to generate a globally unique marker string; this is not limited here.

[0048] For example, for the target symbol "dbPassword", a tag symbol "_T{0}_" is generated; for another target symbol "secretKey", a tag symbol "_T{1}_" is generated.

[0049] Generating the corresponding marker symbol for the target symbol can provide replaceable placeholder nodes for the subsequent construction of the desensitized abstract syntax tree, while retaining the ability to restore the original symbol through the mapping table.

[0050] Step 108: Replace the nodes corresponding to the target symbols in the original abstract syntax tree with the tag symbols to obtain the de-identified abstract syntax tree.

[0051] The node corresponding to the target symbol is the identifier node representing that symbol in the original abstract syntax tree. For example, for a variable name node, this node stores the variable name string; for a function name node, this node stores the function name string.

[0052] The desensitized abstract syntax tree is a new abstract syntax tree obtained by replacing all target symbol nodes in the original abstract syntax tree with corresponding tag symbol nodes. Its syntax structure is completely consistent with the original abstract syntax tree, except that the identifier names are changed.

[0053] See Figure 2 After generating the tag symbol corresponding to the target symbol, the client replaces the node corresponding to the target symbol in the original abstract syntax tree with the tag symbol to obtain the de-identified abstract syntax tree.

[0054] The specific methods for obtaining a desensitized abstract syntax tree (API) by replacing the nodes corresponding to the target symbols in the original API with marker symbols can be as follows: First, perform a depth-first traversal of the original API; whenever an identifier node is encountered whose name matches any target symbol, create a new marker symbol node to replace the original node. Second, use an API transformation library (such as a rewriter) to batch replace all matching nodes. Third, generate sensitive symbol nodes directly using marker symbols during the construction of the API; this is not limited here.

[0055] For example, in the original abstract syntax tree, there is an identifier node storing "dbPassword". This node is replaced with a new node storing "_T{0}_"; another identifier node storing "secretKey" is replaced with a new node storing "_T{1}_". After the replacement, all sensitive symbols in the de-identified abstract syntax tree become meaningless marker symbols, but the function call relationships and variable reference structures remain unchanged.

[0056] By replacing the nodes corresponding to the target symbols in the original abstract syntax tree with the marker symbols, a desensitized abstract syntax tree is obtained. This can hide sensitive information without destroying the code's syntactic structure, so that the subsequently generated desensitized code can still be correctly parsed and understood by the code generation model.

[0057] Step 110: Generate de-identified code based on the de-identification abstract syntax tree.

[0058] The de-identified code is the source code text obtained by deserializing (or parsing) the de-identified abstract syntax tree according to the syntax rules. Sensitive symbols have been replaced with marked symbols, while other code (including comments, formatting, and non-sensitive symbols) remains consistent with or syntactically equivalent to the original code. For example, in the de-identified code, the original variable name "dbPassword" becomes "_T{0}_", while the rest of the code remains unchanged.

[0059] See Figure 2 After the client obtains the de-identification abstract syntax tree, it generates de-identified code based on the de-identification abstract syntax tree.

[0060] Based on the desensitized abstract syntax tree, the specific methods for generating desensitized code can be: using a code generator provided by the compiler backend to traverse each node of the desensitized abstract syntax tree and output the corresponding text fragment; or using a formatted printing library (such as a code beautifier) ​​to output the code according to the indentation style of the original code; or recording node position information and only replacing the text range of sensitive symbols in the original code, thereby preserving the original format, which is not limited here.

[0061] For example, the original code is "let dbPassword='123456';", in which the variable name node in the de-identified abstract syntax tree has been replaced with "_T{0}_". Calling the code generator outputs "let _T{0}_='123456';", which is the de-identified code. If the original code contains comments and blank lines, the code generator will try to maintain the layout.

[0062] Based on the desensitized abstract syntax tree, desensitized code is generated, which can convert the desensitized syntax structure into transmittable and executable code text for subsequent use by the code generation model uploaded to the server.

[0063] Step 112: Upload the de-identified code to the server so that the code generation model deployed on the server can generate code based on the de-identified code.

[0064] The server-side can be a cloud server, a local inference server, or a third-party AI service platform that provides code generation models. For example, the server-side can be a remote server cluster that has deployed large code models (such as code completion models).

[0065] Code generation models are large language models based on deep learning. Their input is code text, and their output is completed or generated code snippets. For example, a code generation model can be a generative pre-trained transform model specifically designed for code completion and code generation tasks.

[0066] See Figure 2After generating the de-identified code, the client uploads it to the server. Upon receiving the de-identified code, the server uses the code generation model to generate its own code based on it.

[0067] The specific methods for uploading anonymized code to the server can be as follows: 1) Encapsulate the anonymized code in the request body and send it to the server interface via a Hypertext Transfer Protocol (HTTP) request. 2) Establish a persistent connection using WebSockets and upload the code in chunks. 3) Upload the code by calling the server's Application Programming Interface (API) via a command-line tool; this is not a specific method.

[0068] The specific methods by which the server-deployed code generation model generates code based on anonymized code can be as follows: The model uses the anonymized code as a contextual hint and predicts subsequent code word by word using an autoregressive approach. Alternatively, the model can generate complete fragments that are syntactically compatible with the anonymized code based on placeholders and contextual structure within the anonymized code. Another method is for the model to reconstruct or interpret the anonymized code in conjunction with user-specified natural language instructions; this is not limited here.

[0069] For example, the client uploads the anonymized code "let_T{0}_='123456'; let_T{1}_=getSecret();" to the server via an HTTP POST request. The server-side code generation model receives this anonymized code, uses it as context, and completes the subsequent code: "console.log(_T{0}_); return _T{1}_;". The server then returns the completed code to the client.

[0070] Upload the de-identified code to the server so that the code generation model deployed on the server can generate code based on the de-identified code. This can leverage the powerful generation capabilities of the large cloud model. At the same time, since the uploaded content is de-identified code that does not contain the original sensitive symbols, it effectively prevents the leakage of corporate secrets or sensitive information during transmission and processing.

[0071] In this embodiment, in response to a front-end request for code generation of the original code, the original code is parsed to obtain an original abstract syntax tree and symbol table; based on preset sensitive symbol rules, each symbol in the symbol table is traversed to determine the target symbol; a marker symbol corresponding to the target symbol is generated; the node corresponding to the target symbol in the original abstract syntax tree is replaced with the marker symbol to obtain a desensitized abstract syntax tree; desensitized code is generated based on the desensitized abstract syntax tree; and the desensitized code is uploaded to the server so that the code generation model deployed on the server can generate code based on the desensitized code. This method can accurately identify sensitive symbols through the symbol table, avoiding cross-scope conflicts caused by the inability to distinguish scopes in regular expression replacement. Furthermore, by replacing nodes in the abstract syntax tree rather than text replacement, the consistency of the syntax structure of the desensitized code and the original code is ensured, thereby avoiding misalignment during restoration due to positional offsets or format changes. Because the desensitized code retains the complete syntactic structure and the marker symbols have clear mapping relationships, the code generation model can correctly understand the context and output complete code with excellent readability. This achieves more applicable code generation results while ensuring the security of sensitive information, and solves the problem of poor applicability of complete code generated based on desensitized code.

[0072] In one optional embodiment of this specification, step 102 includes the following specific steps: The symbol table is traversed and matched based on a preset sensitive symbol dictionary. If a match is successful, it is identified as the target symbol. And / or, The symbol table is traversed and matched using a pre-defined regular expression. If a match is found, the symbol is identified as the target symbol.

[0073] A sensitive symbol dictionary is a pre-built collection of sensitive words used for precise matching of symbol names. For example, a sensitive symbol dictionary may contain words such as "password", "token", "secret", and "apikey".

[0074] Regular expressions are expressions used to describe string matching patterns, which can match symbol names that conform to a specific pattern. For example, the regular expression ".[Pp]wd." can be used to match symbol names that contain "pwd" or "Pwd".

[0075] The system iterates through the symbol table based on a pre-defined sensitive symbol dictionary, matching each symbol. If a match is found, the symbol is identified as the target symbol. Specifically, this can be done by performing a complete string match between the name of each symbol in the symbol table and the entry in the sensitive symbol dictionary. For example, if the symbol name is "dbPassword" and the dictionary contains "password", then the match is successful. Alternatively, the symbol name can be segmented and partially matched against dictionary entries. For example, splitting "user_secret_key" into "user", "secret", and "key" will match "secret" and "key". Another approach is to ignore case sensitivity and specific prefixes and suffixes during matching, such as matching "ApiToken" in "getApiToken" against "token" in the dictionary; this is not a limitation here.

[0076] The system iterates through the symbol table using predefined regular expressions, matching each symbol. If a match is found, the symbol is identified as the target. Specifically, this can be done by sequentially inputting each symbol name into the regular expression engine for matching; if a match is found, it's identified as the target symbol. For example, using the regular expression ".[Kk]ey.", the symbol name "privateKey" is matched successfully. Alternatively, the symbol name can be preprocessed (e.g., removing numeric suffixes) before regular expression matching. For example, the symbol "var_key_123" matches ".*_key" after removing numeric suffixes. Another approach is to combine multiple regular expressions and match them sequentially; any successful match identifies the target symbol. For example, first matching the pattern containing "pwd," then matching the pattern containing "secret," without further limitation.

[0077] For example, the preset sensitive symbol dictionary contains "pwd" and "token". The symbol "userPwd" exists in the symbol table and is successfully matched, so it is determined as a target symbol. And / or the symbol "publicKey" is successfully matched using the regular expression ".[Kk]ey." and is also determined as a target symbol. For the symbol "counter", neither the sensitive symbol dictionary nor the regular expression matches, so it is not considered a target symbol.

[0078] In the embodiments of this specification, each symbol in the symbol table is traversed and matched based on a preset sensitive symbol dictionary; if a match is successful, it is determined to be the target symbol. And / or, each symbol in the symbol table is traversed and matched based on a preset regular expression; if a match is successful, it is determined to be the target symbol. This allows for accurate identification of various sensitive symbols through multiple flexible matching methods, adapting to the naming conventions of different code libraries, and improving the recall and accuracy of desensitization.

[0079] In one optional embodiment of this specification, step 106 includes the following specific steps: Generate sequentially increasing integers to serve as the marker symbols corresponding to the target symbols.

[0080] Sequentially increasing integers are sequences of integers that start from a primary value (such as 0 or 1) and increment by 1 after each target symbol is processed. For example, sequentially increasing integers could be 0, 1, 2, 3, and so on.

[0081] Generating sequentially increasing integers as marker symbols for the target symbols can be achieved in several ways. One approach is to maintain a global counter, reading the current counter value and incrementing it each time a marker symbol needs to be generated. For example, the counter could start at 0, the first target symbol would be marked with the symbol "_T{0}_", and then the counter would become 1. Another approach is to assign incrementing numbers based on the traversal order of the target symbols in the symbol table. For example, the first matched target symbol could be assigned a smaller number. A third approach is to use thread-safe atomic integers to generate unique incrementing numbers in a multi-threaded environment; this approach is not limited here.

[0082] For example, when traversing the symbol table, three target symbols "password", "apiKey", and "secret" are matched in sequence. First, "password" is assigned the index 0, generating the symbol "_T{0}_"; then "apiKey" is assigned the index 1, generating "_T{1}_"; and finally "secret" is assigned the index 2, generating "_T{2}_".

[0083] In the embodiments of this specification, sequentially increasing integers are generated as marker symbols corresponding to the target symbols. This can obtain globally unique placeholders with extremely low computational cost, while maintaining the compactness and readability of the marker symbols, facilitating fast lookup during the storage and restoration of the mapping table.

[0084] In one optional embodiment of this specification, the following specific steps are included before step 110: Identify nodes of string literals in the de-identified abstract syntax tree; Encrypt string literals to generate encrypted ciphertext; Replace the nodes of the string literals with encrypted ciphertext to obtain the updated de-identified abstract syntax tree.

[0085] String literals are syntactic units in source code used to represent fixed text content, such as UI prompts, configuration parameters, data values, or paths. String literals are typically sequences of characters enclosed in quotation marks (used to represent text data). For example, string literals can be "https: / / internal.api.com" or "sk-123456".

[0086] A string literal node is a node in the de-identified abstract syntax tree that represents a string literal. This node can contain the value and position information of the original string. For example, in the abstract syntax tree, a string literal node typically has a type identifier "StringLiteral" and a field that stores the actual string.

[0087] Indicatively, Figure 3 A flow architecture timing diagram of another code processing method provided according to an embodiment of this specification is shown.

[0088] See Figure 3 After obtaining the de-identified abstract syntax tree, the client identifies the nodes of string literals in the de-identified abstract syntax tree; encrypts the string literals to generate encrypted ciphertext; and replaces the nodes of the string literals with the encrypted ciphertext to obtain the updated de-identified abstract syntax tree.

[0089] Identifying string literal nodes in the de-identified abstract syntax tree can be done in several ways. One approach is to perform a depth-first traversal of the de-identified abstract syntax tree, checking if each node is a string literal. For example, in a Python abstract syntax tree, this could be done by checking if a node is of type "ast.Str". Another approach is to use the visitor pattern to register handlers for string literal nodes and automatically collect all such nodes. A third approach is to mark all string literal nodes from the original code while generating the de-identified abstract syntax tree; this is not a specific limitation.

[0090] Encrypting a string literal to generate encrypted ciphertext can be done in several ways. Specifically, one method is to use a symmetric encryption algorithm (such as the Advanced Encryption Standard AES) to encrypt the original value of the string literal, obtaining binary ciphertext, and then converting the binary ciphertext into a printable encoded string (such as hexadecimal or Base64). For example, encrypting the string "secret" yields the hexadecimal string "a1b2c3d4". Another method is to use a hash function (such as SHA-256) to perform an irreversible hash on the string, but this method is irreversible and suitable for scenarios where recovery is not required. A third method is to use a format-preserving encryption algorithm, ensuring that the ciphertext and plaintext have the same length and remain in string form; this method is not limited here.

[0091] The updated de-identified abstract syntax tree is the abstract syntax tree obtained by replacing the original string literal nodes with encrypted ciphertext nodes, where the encrypted ciphertext serves as the new string literal value.

[0092] The updated de-identified abstract syntax tree (API) is obtained by replacing the nodes of string literals with encrypted ciphertext. This can be achieved by: traversing all string literal nodes, creating new string literal nodes with encrypted ciphertext values, and then replacing the original nodes with the new ones. For example, an API rewriting tool can be used to directly modify node values. Alternatively, during the API generation process, when a string literal node is encountered, the ciphertext is dynamically calculated and a new node is generated to avoid subsequent replacements. Another approach is to record the original node positions and directly output the ciphertext during code generation without actually modifying the API structure; this is not limited to this method.

[0093] For example, the de-identified abstract syntax tree contains a string literal node with the value "https: / / internal.api.com". This string is encrypted using the AES-256-GCM algorithm to obtain binary ciphertext, which is then converted into the corresponding hexadecimal string.

[0094] In the embodiments of this specification, the nodes of string literals in the de-identified abstract syntax tree are identified; the string literals are encrypted to generate encrypted ciphertext; and the nodes of the string literals are replaced with the encrypted ciphertext to obtain an updated de-identified abstract syntax tree. By reversibly encrypting and embedding string literals, sensitive strings in the code (such as keys, Uniform Resource Locators (URLs), and Structured Query Language (SQL) statements) can be prevented from being directly obtained by the generation model, further improving the security of de-identification. It also allows the model to perform string concatenation, formatting, and other operations normally, avoiding damage to code semantics due to incompatible ciphertext formats.

[0095] In one optional embodiment of this specification, encrypting a string literal to generate encrypted ciphertext includes: A symmetric encryption algorithm is used to encrypt the string literal to generate encrypted ciphertext.

[0096] Symmetric encryption algorithms are encryption algorithms that use the same key for both encryption and decryption. For example, symmetric encryption algorithms can be Advanced Encryption Standard (AES), Data Encryption Standard (DES), etc., and are not limited here.

[0097] For example, the same symmetric key is pre-configured on the client and server sides, and the string literal "db_connection_string" is encrypted using the AES-256 algorithm to obtain a 64-bit hexadecimal encrypted ciphertext string corresponding to the above string literal.

[0098] In the embodiments of this specification, a symmetric encryption algorithm is used to encrypt string literals to generate encrypted ciphertext, which can achieve efficient encryption and decryption, ensure the reversibility of ciphertext, and thus accurately restore the original string after the model returns the result, while avoiding the transmission or storage of sensitive strings in plaintext.

[0099] In one optional embodiment of this specification, the following specific steps are included before step 112: A mapping table is generated based on the target symbol, the marker symbol, and the scope chain of the target symbol. Store the mapping table.

[0100] The scope chain of a target symbol is a scope identifier from the innermost scope to the global scope, which can be used to uniquely determine the semantic context of the symbol.

[0101] A mapping table is a data structure that records the correspondence between target symbols and marker symbols, as well as the scope chain of each target symbol. For example, a mapping table can be a hash table where the key is the marker symbol and the value is a structure containing the original symbol name and scope chain.

[0102] See Figure 3 After generating the de-identified code, the client generates a mapping table based on the target symbol, the tag symbol, and the scope chain of the target symbol, and stores the mapping table in the local database.

[0103] Based on the target symbol, the marker symbol, and the target symbol's scope chain, a mapping table is generated. Specifically, this can be achieved by: simultaneously determining the target symbol and generating the marker symbol, recording the target symbol's original name, marker symbol, and current scope chain (obtainable from the symbol table), and storing this information as a mapping record. For example, a mapping table entry could be {"_T{0}_": {"original": "password", "scope": "global"}}. Alternatively, the mapping table can be persistently stored using a database or file system for later restoration. Another approach is to serialize the mapping table into JSON format and store it in memory or on disk; this is not limited to this method.

[0104] For example, the scope chain of the target symbol "dbPassword" is "function connectDb → global", a marker symbol "_T{0}_" is generated for it, and a record is added to the mapping table: the marker symbol "_T{0}_" corresponds to the original symbol "dbPassword" and the scope chain "connectDb: global". The mapping table is stored in the local file "mapping.json".

[0105] In the embodiments of this specification, before uploading the de-identified code to the server, a mapping table is generated based on the target symbol, the marker symbol, and the scope chain of the target symbol, and then the mapping table is stored. This provides an accurate semantic basis for the subsequent restoration of the marker symbol. By distinguishing the same-named symbols under different scopes through the scope chain, restoration errors are effectively avoided.

[0106] In one optional embodiment of this specification, after step 112, the following specific steps are further included: Receive code snippets generated by the code generation model and returned by the server; Based on the mapping table, replace the marker symbols in the code snippet with the target symbols to obtain the restored code snippet; The target code is determined based on the restored code snippet, and the target code is fed back to the front end.

[0107] The code snippet is the code output by the code generation model based on the de-identified code. It may contain marker symbols and encrypted ciphertext, which are used to replace or supplement the target symbols and string literals in the original code.

[0108] See Figure 3 After the client uploads the de-identified code to the server, it receives the code snippet generated by the code generation model from the server. According to the mapping table, the client replaces the marker symbols in the code snippet with the target symbols to obtain the restored code snippet. Based on the restored code snippet, the client determines the target code and feeds the target code back to the front end.

[0109] The client receives code snippets generated by the code generation model from the server. This can be achieved in several ways: First, by receiving text data returned by the server via the Hypertext Transfer Protocol (HTTP), which serves as the code snippet. For example, after sending a request, the client waits for a JSON response from the server, where the "completion" field stores the code snippet. Alternatively, it can be achieved by continuously receiving streamed code snippets via WebSocket and piecing them together step by step. No specific limitation is specified here.

[0110] The restored code snippet is the code text obtained by replacing all the marker symbols in the code snippet back to the original target symbols according to the mapping table. For example, "_T{0}_" in the original code snippet is replaced with "dbPassword", and "_T{1}_" is replaced with "secretKey".

[0111] Based on the mapping table, the marker symbols in the code snippet are replaced with the target symbols to obtain the restored code snippet. Specifically, this can be done by: parsing the code snippet, identifying the marker symbols (e.g., matching the pattern "_T{\d+}_" using regular expressions), then looking up the corresponding original symbol and scope chain in the mapping table, and performing the replacement. For example, a global string replacement function can be used to replace each marker symbol with its corresponding original symbol. Alternatively, a temporary abstract syntax tree can be constructed on the code snippet, traversing the identifier nodes and replacing them to avoid replacing coincidental matches in comments or strings. Another approach is to combine scope chain information and only perform replacements when the scope of the marker symbol matches the scope recorded in the mapping table; this is not limited here.

[0112] The target code is determined based on the restored code snippet and then fed back to the front end. This can be done in several ways: First, the restored code snippet can be directly returned to the front end as the target code to replace the code region selected in the user's editor. For example, the restored code snippet can be inserted into the corresponding position in the original code. Second, the restored code snippet can be merged with the original code (e.g., a three-way merge) before being returned to resolve conflicts. Third, the restored code snippet can be formatted (e.g., with automatic indentation) before being fed back; this is not a specific limitation.

[0113] For example, the server returns the code snippet "let result=_T{0}_+_T{1}_". The mapping table records "_T{0}_" as "dbPassword" and "_T{1}_" as "secretKey". After replacement, the result is "let result=dbPassword+secretKey". The client sends this code snippet to the front-end integrated development environment plugin, which then replaces the user-selected code block with this code.

[0114] In this embodiment, the code snippet generated by the code generation model and returned by the server is received. Based on a mapping table, the marker symbols in the code snippet are replaced with target symbols to obtain the restored code snippet. The target code is determined based on the restored code snippet and fed back to the front end. By utilizing the correspondence between marker symbols and target symbols recorded in the mapping table, placeholders in the model's de-identification result are accurately restored to the original sensitive symbols, avoiding scope misalignment or incorrect replacement of symbols with the same name caused by simple text replacement. Since the symbol names in the restored code snippet are completely consistent with those in the original code and the complete syntactic structure is preserved, the final target code can be directly compiled and run. Furthermore, the existence of the mapping table makes the restoration process deterministic and reversible, thus achieving lossless recovery from de-identified code to original code while ensuring restoration accuracy, providing users with high-quality code that is ready to use immediately.

[0115] In one optional embodiment of this specification, the original code is a code snippet from the local workspace code; The target code was determined based on the restored code snippet, including: The restored code snippet, the original code, and the code in the local workspace are merged to obtain the target code.

[0116] The local workspace code is the content of a complete source code file stored on the user's local disk. It can include the original code as well as other code that the user has not uploaded. For example, the local workspace code could be a Python file containing multiple function definitions, while the original code is only the body of one of those functions.

[0117] The restored code snippet, the original code, and the local workspace code are merged to obtain the target code. This can be achieved using a three-way merge algorithm, with the original code as the common ancestor. The restored code snippet is treated as a modification of one branch, and the local workspace code as a modification of another branch. Conflict-free parts are automatically merged, and conflicts are marked for user resolution. For example, a variant of the Global Information Tracker (Git) merge strategy can be used for text merging. Alternatively, semantic merging based on an abstract syntax tree can be used, replacing the corresponding parts of the local workspace code with the restored code snippet. Another approach is to directly replace the local workspace code only if there are no other modifications relative to the original code; otherwise, the user is prompted to manually merge. The common ancestor serves as the baseline for the original code before the two modified versions fork during the version merging process, used to compare and calculate the differences between them. In code completion scenarios, the common ancestor is the original code snippet submitted by the user to the code generation model before completion; it is the common starting point for de-identification processing, model generation, and subsequent merging operations.

[0118] For example, the local workspace code is a file containing functions "foo" and "bar". The original code is the incomplete code for function "foo", and the restored code snippet is the completed code for the "foo" function generated by the model. During merging, the completed code replaces the original "foo" function body in the local workspace code, while the "bar" function remains unchanged, resulting in the target code.

[0119] In this embodiment, the restored code snippet, the original code, and the local workspace code are merged to obtain the target code. By using the original code as the merging benchmark and treating the restored code snippet as one party's modification and the local workspace code as another party's modification in a three-way merging process, it can automatically identify and retain other modifications in the local workspace that have not been uploaded. This avoids directly overwriting code content that the user did not participate in de-identification, ensuring the integrity of the merged target code and maintaining the continuity of the user's original work progress, without requiring the user to manually restore the overwritten code. In addition, the comparison mechanism of the three-way merging can automatically detect and repair syntax gaps that may be introduced by the de-identification-restoration process. By completing missing declarations or adjusting code positions, it ensures that the merged target code can be compiled without manual repair, achieving the effect of "compile immediately upon generation".

[0120] In one optional embodiment of this specification, the restored code snippet, the original code, and the local workspace code are merged to obtain the target code, including: The restored code snippet, the original code, and the code in the local workspace are merged, and a check for merge conflicts is triggered. If not, obtain the target code.

[0121] A merge conflict occurs when, during the merge process, the restored code snippet and the local working directory code make different modifications to the same location in the original code, preventing automatic merging from completing. For example, the local working directory code modifies a variable name in the original code, while the restored code snippet uses the original variable name, resulting in a conflict.

[0122] The restored code snippet, the original code, and the local workspace code are merged, triggering a check for merge conflicts. This can be achieved using a three-way merge algorithm, comparing the original code (base version), the local workspace code (current version), and the restored code snippet (another version) to detect differences. For example, a line-by-line comparison can be performed; if a line is modified in both the local workspace code and the restored code snippet, and the modifications differ, it is marked as a conflict. Alternatively, conflict detection can be based on the node positions in the abstract syntax tree; if two modifications involve the same syntax node (like a single statement), they are considered a conflict. Fine-grained conflict detection can also be performed using a difference analysis library, which is not limited here.

[0123] For example, the original code contains the line "int x=0", which is modified to "int x=1" in the local workspace code, and then modified to "int x=2" in the restored code snippet. During the merge, a conflict is detected, but instead of generating target code, a conflict is reported to the user. Conversely, if the local workspace code did not modify the line, there is no conflict, and the target code is obtained directly.

[0124] In the embodiments described in this specification, the restored code snippet, the original code, and the local working directory code are merged, and a check for merge conflicts is triggered; if no conflict occurs, the target code is obtained. This method proactively detects merge conflicts during the merging process, generating target code only when there are no conflicts, thereby avoiding logical errors or inconsistent code that may be introduced by automatic merging and ensuring the correctness of the final code logic. By pre-judging conflicts, problematic code is avoided from being directly merged into the working directory, reducing the risk of compilation failures or runtime exceptions due to improper merging, and improving the security and reliability of code merging.

[0125] In one optional embodiment of this specification, the merging conflict includes at least one of a syntax-level conflict and a semantic-level conflict; After triggering the determination of whether a merge conflict has occurred, at least one of the following is also included: If so, and the merge conflict is a syntax-level conflict, the missing declaration is inserted in the local workspace copy at the position corresponding to the original code using the incremental merging engine.

[0126] If so, and the merge conflict is a semantic-level conflict, highlight the issue in the integrated development environment.

[0127] Syntax-level conflicts are those where the merged code fails compilation or syntax checks due to missing necessary declarations (such as variable declarations, function declarations, and import statements). For example, the restored code snippet uses the variable "responseData," but this variable is not declared in the local workspace code and does not exist in the original code.

[0128] Semantic conflicts occur when, despite the code being syntactically correct, logical contradictions (such as type mismatches, scope overriding, redefinition, or inconsistent assignment and usage) lead to unexpected runtime behavior. For example, the restored code snippet assigns the boolean value "true" to the variable "flag", while the "flag" in the same scope in the local workspace code has been defined as an integer and assigned the value "1".

[0129] The incremental merging engine is a component used to automatically fix syntax-level conflicts during the merging process. It can infer the type and scope of missing declarations based on the context and insert the corresponding declaration statements.

[0130] The content corresponding to semantic-level conflicts is the specific code element that caused the semantic conflict and its context information, including but not limited to: the name of the symbol involved in the conflict, the line of code where the conflict occurs, the definition location of the conflicting symbol in the local workspace code, the way the conflicting symbol is used in the code completion, and the description information of the conflict type, etc., which are not limited here.

[0131] In an integrated development environment (IDE), highlighting semantic-level conflicts can be achieved in several ways. First, through an IDE plugin, the background color of the conflicting line of code can be highlighted (e.g., light red), and a warning icon can be displayed next to the line number. Second, a red wavy line can be displayed below the conflict symbol, and a tooltip showing the conflict description (e.g., "Semantic-level conflict: variable 'count' type mismatch, local string, complete to integer") can be displayed. Third, all semantic conflicts can be listed in the IDE's "Issues" panel, each including the file path, line number, and conflict description, with double-click navigation supported. This method is not limited to specific methods.

[0132] If so, and the merge conflict is a syntax-level conflict, the missing declaration is inserted into the scope of the local working directory code using the incremental merging engine. Specifically, this can be done by resolving the scope of the original code in the local working directory (such as function scope, block scope, or global scope) to determine the appropriate insertion location for the missing declaration. For example, if a variable declaration is missing, it is inserted on the first line of the function body containing the original code; if an import statement is missing, it is inserted in the import section at the top of the file. Alternatively, when the type cannot be automatically inferred, a declaration with a type placeholder is inserted, marked as requiring user confirmation. Another approach is to insert the function prototype at the end of the current scope if the missing declaration is a function declaration; this is not limited to this method.

[0133] If so, and the merge conflict is a semantic-level conflict, the corresponding content of the semantic-level conflict should be highlighted in the IDE. Specifically, this can be done by using an IDE plugin to locate the line and column numbers of the conflicting code in the editor, setting the background color of that line to light red, and displaying a red wavy line below the conflict symbol. When the mouse hovers over it, a tooltip should appear: "Semantic-level conflict: Symbol 'flag' has an inconsistent type; it is an integer type in the local workspace code, but the completed code attempts to use it as a boolean type." Simultaneously, a record should be added to the "Issues" view in the IDE, containing the file path, line number, and conflict description. Double-clicking the record will automatically jump to the conflict location. Alternatively, quick fix suggestions (such as "change the local variable to a boolean type" or "change the usage in the completed code") can be provided in the highlighted area; this is not limited to this approach.

[0134] For example, during the merging process, it was discovered that the restored code snippet contained the statement "console.log(responseData)", while "responseData" was not declared in the local workspace code, and the variable did not exist in the original code. After detecting this syntax-level conflict, the incremental merging engine inserted "let responseData=null" as the first line of the function body containing the original code in the local workspace code, thereby automatically resolving the conflict.

[0135] In another example, the restored code snippet assigns the variable "isValid" the value "true", while in the local workspace code, "isValid" is defined as the integer "0". This conflict is determined to be a semantic conflict. In this case, the integrated development environment (IDE) highlights the line containing "isValid=true" in red and displays a red wavy line below "isValid". A tooltip appears when the mouse hovers over the line: "Semantic conflict: isValid type mismatch (integer in local workspace, boolean in completed code)". The user then manually modifies the code based on the tooltip.

[0136] In the embodiments of this specification, by distinguishing between syntax-level conflicts and semantic-level conflicts, and handling them respectively by automatically inserting missing declarations and highlighting hints, the system can automatically complete missing declarations for syntax-level conflicts, eliminating the tedious manual addition by users. For semantic-level conflicts, the system uses highlighting hints to intuitively locate problematic code, enabling users to quickly understand the cause of the conflict and make the correct decision, thereby significantly reducing the burden on users to manually resolve conflicts. Automatic insertion of missing declarations ensures the syntactic correctness of the merged code, while highlighting semantic conflicts guides users to correct logical inconsistencies. Thus, while ensuring the semantic consistency of the merged code, the system improves the intelligence level of the code processing method and the user experience.

[0137] In one optional embodiment of this specification, the code snippet is modified by replacing the marker symbols in the code snippet with the target symbols according to the mapping table to obtain the restored code snippet, including: Based on the mapping table, replace the marker symbols in the code snippet with the target symbols to obtain the intermediate code snippet; The encrypted ciphertext in the intermediate code snippet is traversed, and each encrypted ciphertext is decrypted into a string literal to obtain the restored code snippet.

[0138] An intermediate code snippet is the code text obtained by replacing the marker symbols in the code snippet with the target symbols. It may still contain string literals in encrypted ciphertext form. For example, an intermediate code snippet could be "let password=_T{0}_".

[0139] Traversing the encrypted ciphertext in an intermediate code snippet involves scanning the snippet, identifying all encrypted ciphertext placeholders (such as strings with a specific format), and processing them one by one. For example, traversal could be achieved by matching all strings that conform to the ciphertext format using regular expressions.

[0140] Based on the mapping table, the marker symbols in the code snippet are replaced with target symbols to obtain an intermediate code snippet. Specifically, this can be done by: parsing the code snippet, identifying the marker symbols (e.g., using the regular expression "_T{\d+}_"), looking up the corresponding target symbols in the mapping table, and then performing string replacement. For example, replacing "_T{0}_" with "password". Alternatively, a temporary abstract syntax tree can be constructed on the code snippet, traversing the identifier nodes and looking up the mapping table based on the node name to avoid replacing comments or coincidental matches in the string. No specific limitation is imposed here.

[0141] The encrypted ciphertext in the intermediate code snippet is traversed, and each ciphertext is decrypted into a string literal to obtain the restored code snippet. Specifically, this can be done by: using regular expressions to match the encrypted ciphertext format (such as hexadecimal or Base64 strings), calling a decryption algorithm (such as AES decryption) on each matched ciphertext to obtain the original string, and then replacing the ciphertext with the original string. For example, "7b8d9f0e......" is decrypted to "https: / / internal.api.com". Alternatively, the string literal nodes can be decrypted and replaced during the traversal of the abstract syntax tree. Another approach is to pre-build a mapping table from ciphertext to the original string and directly look up and replace the ciphertext; this is not limited here.

[0142] For example, the code snippet is "let result=_T{0}_+'a1b2c3d4'". Replacing "_T{0}_" with "apiKey" according to the mapping table yields the intermediate code snippet "let result=apiKey+'a1b2c3d4'". Then, the intermediate code snippet is traversed, the ciphertext "a1b2c3d4" is identified, decrypted to obtain "secret", and replaced to obtain the restored code snippet "let result=apiKey+'secret'".

[0143] In the embodiments of this specification, according to the mapping table, the marker symbols in the code segment are replaced with target symbols to obtain intermediate code segments; the encrypted ciphertext in the intermediate code segments is traversed, and each encrypted ciphertext is decrypted into a string literal to obtain the restored code segment. By first restoring the symbol names based on the mapping table and then decrypting the encrypted ciphertext in the intermediate code segments, the symbol restoration and string decryption are isolated from each other, avoiding the problem of mistakenly treating marker symbols as ciphertext or mistakenly replacing ciphertext as marker symbols that may occur when directly mixing and replacing them. This ensures that the restored code can completely restore the original information, guaranteeing the accuracy and reliability of the restoration process.

[0144] In one optional embodiment of this specification, after determining the target code based on the restored code snippet, the method further includes: Invoke the compilation hook to compile the target code; If compilation succeeds, the unit test hooks are invoked to test the target code; If the test passes, a runnable instruction for the target code is returned to the front end.

[0145] Compiler hooks are pre-defined extension points in the code compilation process. They can be callback functions or event interfaces provided by the compiler at specific points in time during compilation, allowing developers to insert custom logic at various stages of compilation (such as preprocessing, compilation, assembly, and linking) to intervene in or modify the compilation process. For example, a compiler hook could be used in an integrated development environment to call the underlying compiler interface to perform custom compilation checks or code injection.

[0146] Unit test hooks are pre-defined extension points in the unit test execution process. They can be callback interfaces provided by the test framework at specific stages of the unit test lifecycle, allowing developers to insert custom logic at various stages of the test (such as test suite initialization, before test case execution, after execution, overall cleanup, etc.) to control the preparation, execution, and closing processes of the test.

[0147] The runnable indicator for target code is a message used to indicate that the code currently generated by the front end has passed compilation and testing and is safe to use. For example, the runnable indicator can be a green checkmark icon or a "code is ready" message.

[0148] See Figure 3 After the client determines the target code based on the restored code snippet, it calls the compilation hook to compile the target code; if the compilation is successful, it calls the unit test hook to test the target code; if the test is successful, it returns a runnable indication of the target code to the front end.

[0149] The compilation hook is invoked to compile the target code. This can be done in several ways: First, by using an integrated development environment (IDE) plugin to call a local compiler command, writing the target code to a temporary file and executing the compilation. For example, for Java code, this would be done by calling `javac TargetCode.java`. Second, by using an in-memory compilation tool (such as the Java Compiler API) to directly compile the code string. Third, by sending the target code to a remote compilation service and receiving the compilation results; this method is not limited to these methods.

[0150] If compilation succeeds, unit test hooks are invoked to test the target code. This can be achieved by: automatically triggering the unit test framework after successful compilation to run test cases related to the target code; for example, using JUnit's test runner to execute test classes. Alternatively, unit test hooks can be used to collect metrics such as test coverage and execution time. This step can also be skipped if the project does not have unit tests configured; this is not a specific limitation.

[0151] If the test passes, a runnable indication of the target code is returned to the front end. This can be achieved by displaying a "Compilation and testing passed" message in the integrated development environment (IDE) interface, such as a green marker in the status bar. Alternatively, a notification box can pop up informing the user that the code is safe to run. Another approach is to automatically apply the target code to the user's project and mark it as verified; this is not limited to this method.

[0152] For example, after obtaining the target code, the compilation hook is invoked to call the local Python compiler to perform syntax checks on the target code. After successful compilation, the unit test hook is invoked to run the associated test cases in the project (such as test_function_name.py). After all tests pass, the integrated development environment plugin pops up a notification in the lower right corner of the editor: "The target code has passed compilation and unit tests and can be run safely," and displays a green checkmark next to the line number.

[0153] In the embodiments described in this specification, after determining the target code based on the restored code snippet, a compilation hook is invoked to compile the target code; if compilation succeeds, a unit test hook is invoked to test the target code; if the test succeeds, a runnable indication of the target code is returned to the front end. This can automatically verify the correctness and stability of the de-identified generated code, reduce the cost of manual debugging for users, and improve the credibility and usability of code completion. Furthermore, by performing the "restoration-compilation-unit testing" process locally, it can be ensured that any injected code must be built locally, thus reducing the explosion radius from the entire network to a single project.

[0154] In one optional embodiment of this specification, before storing the mapping table, the following is further included: Based on the structural topology of the original abstract syntax tree, the summary value is calculated as the index identifier of the mapping table; Storage mapping table, including: Based on the index identifier, the mapping table is stored in the local database.

[0155] The structural topology of the primitive abstract syntax tree (API) is a data structure that represents the connections, hierarchical structure, and sequence information between nodes in the API, excluding the specific content of the nodes (such as variable names or literals). For example, the structural topology can be represented as a sequence of node types or a normalized string of a tree structure.

[0156] The digest value is a string obtained by compressing the structural topology of the original abstract syntax tree (API), and it can be used to uniquely identify that structural topology. For example, a hash algorithm can be used to compress the structural topology of the original API to obtain the digest value.

[0157] The index of the mapping table is a key-value pair used to uniquely identify and quickly retrieve the corresponding mapping table in the local database. This key-value pair allows direct access to the de-identified mapping relationship associated with a specific code structure. It is typically a fixed-length string or numeric value, such as a hash digest (e.g., SHA-256), UUID, or an auto-incrementing integer, and has global uniqueness or high-probability uniqueness.

[0158] Based on the structural topology of the original abstract syntax tree, a digest value is calculated as the index identifier of the mapping table. Specifically, this can be done by: traversing the original abstract syntax tree, ignoring specific identifier names and literal values, and only retaining node types and parent-child relationships to generate a normalized topology string, which is then hashed. For example, for a function definition node, "FunctionDef" and its child node type sequence can be recorded. Alternatively, a tree hash algorithm can be used to directly calculate the structural hash. Another approach is to serialize the abstract syntax tree and then calculate the hash; this is not limited here.

[0159] Based on the index identifier, the mapping table is stored in a local database. Specifically, this can be done in several ways: First, the digest value is used as the key, and the contents of the mapping table (such as the mapping from tokenized symbols to primitive symbols, scope chain information, etc.) are used as the values, stored in a local key-value database. Second, the mapping table file can be named after its digest value and stored in the file system. Third, an index can be created in a memory cache and simultaneously persisted to disk; this is not a specific limitation.

[0160] For example, the original abstract syntax tree's structural topology is "Program→FunctionDef→Assign→Name→Load", ignoring the specific variable name "x" and the value "1". Calculating the SHA-256 hash of this topology string yields the digest value "9f86d081884c7d659a2feaa0c55ad015a3bf4f1b2b0b822cd15d6c15b0f00a08". Using this digest value as the key, the mapping table (containing records such as "_T{0}_" corresponding to "password") is stored in the "mapping" table of the local database.

[0161] In the embodiments of this specification, before storing the mapping table, a digest value is calculated as the index identifier of the mapping table based on the structural topology of the original abstract syntax tree; based on the index identifier, the mapping table is stored in a local database. By associating the mapping table with the syntactic structure of the code through the digest value, when processing code with the same structural topology subsequently, the existing mapping table can be quickly reused directly from the local database based on the digest value, without having to re-perform sensitive symbol identification and marker symbol generation operations, thus avoiding repeated desensitization processing and effectively improving desensitization efficiency. Furthermore, since the digest value depends only on the syntactic structure and not the specific code content, it enables code fragments with the same structure to be quickly matched to the same mapping table in multi-file, multi-version concurrent scenarios, without having to generate it repeatedly for each file or version, improving matching speed and resource utilization during concurrent processing. In addition, by storing the mapping table and the original code on the local device at all times, the cloud service only accesses the desensitized code fragments, ensuring that the original sensitive information does not leave the user's controllable environment, thereby meeting mainstream compliance requirements such as general data protection regulations and network security level protection, providing protection for enterprises in terms of data security and privacy protection.

[0162] In one optional embodiment of this specification, calculating the digest value as the index identifier of the mapping table based on the structural topology of the original abstract syntax tree includes: The hash algorithm is used to calculate the string sequence corresponding to the structural topology of the original abstract syntax tree, and the string sequence is used as the index identifier of the mapping table.

[0163] The string sequence corresponding to the structure topology of the abstract syntax tree is the conversion of the structural information of the abstract syntax tree (such as node type, parent-child relationship, sibling order, etc.) into a hashable string representation.

[0164] The hash algorithm is used to calculate the string sequence corresponding to the structure and topology of the original abstract syntax tree (Abstract Syntax Tree). This string sequence is then used as the index identifier of the mapping table. Specifically, this can be done by performing a depth-first traversal of the Abstract Syntax Tree, appending the type name of each node to the string, recursively processing child nodes, and finally applying the hash algorithm to the entire string. For example, for the expression "a+b", the string "BinOp(Load, Add, Load)" is generated, and then the hash is calculated. Alternatively, the string sequence can be normalized (e.g., removing spaces and sorting child nodes) before calculating the hash, ensuring that Abstract Syntax Trees with the same structure produce the same index; this is not a limitation here.

[0165] For example, the original code is "let x=y+z", and its abstract syntax tree structure topology corresponds to the string sequence "Program(VariableDeclaration(Identifier, BinaryExpression(Identifier, Identifier)))". The SHA-256 algorithm is used to calculate the hash of this string, resulting in "e5a3f2b1......", which serves as the index identifier for the mapping table.

[0166] In the embodiments of this specification, a hash algorithm is used to calculate the string sequence corresponding to the structural topology of the original abstract syntax tree, and the string sequence is used as the index identifier of the mapping table. By compressing the complex tree structure into a fixed-length hash value, it is easier to store and compare quickly. At the same time, code snippets with the same structure will generate the same hash value, so that the corresponding mapping table can be reused, avoiding the overhead of repeatedly generating mapping tables.

[0167] The above code processing method will be further explained below through complete embodiments: Step 1: Local Desensitization 1. Parse the selected code to obtain the AST and symbol table (which can be implemented based on tree-sitter or the official compiler frontend).

[0168] 2. For each symbol, if it matches the local sensitive dictionary (which can be customized) or conforms to a regular expression (such as .*key.*, .*secret.*), then: a) Generate a temporary marker __T{n}__ (n increments); b) Record <original identifier, tag, scope chain> to the mapping table; c) Replace all occurrences of the symbol in the AST.

[0169] 3. Encrypt the string literal using AES-256-GCM (key stored locally only), generate __S{base64_ciphertext}__ and replace the original node.

[0170] 4. Calculate S2 digest = SHA-256 (AST topological serialization string).

[0171] 5. Using the S2 digest as the key, store the mapping table in a local encrypted database (SQLite+SQLCipher).

[0172] Step 2: Cloud generation: 1. Send the anonymized code to the AI ​​code generation service via HTTPS (fully compatible with the official API, no modification required).

[0173] 2. The completed fragments returned by the model still use the __T{n}__ style variable name.

[0174] Step 3, Local Restore and Merge: 1. After receiving the completed fragment, use S2 digest to retrieve the mapping table from the local database.

[0175] 2. Scan the segment, and replace the original identifier with __T{n}__ when it is encountered; decrypt the original string with __S{ciphertext}__ when it is encountered.

[0176] 3. Merge the "Restored Fragment" with the "Local Workspace Copy" and the "Common Ancestor" in a three-way manner: a) Syntax-level conflicts (such as undeclared variables) are automatically corrected by IFE by inserting the missing declaration; b) Semantic conflicts (such as type mismatches) are highlighted in the IDE and confirmed by the developer.

[0177] 4. The merged results are written back to disk, and the compilation / unit test hooks are triggered to ensure that the program can run.

[0178] For example, the application of each step in steps one through three above to the code completion of the cheat detection plugin will be used as an example for illustration: Suppose a developer is writing an anti-cheat plugin for a game. The original selected code contains a function for verifying player data. This function contains an internal API address and a key string, both of which are sensitive information.

[0179] Step 1: Local Desensitization: The client parses the function's code, generating an abstract syntax tree (AST) and a symbol table. Based on a pre-defined sensitive dictionary (containing keywords such as "url," "key," and "secret") and regular expressions, the API address variable name and key variable name are identified as target symbols. For each target symbol, sequentially incrementing temporary tags are generated, and the original identifier, tag symbol, and their scope chain are recorded in the mapping table. Simultaneously, all occurrences of target symbol nodes in the AST are replaced with their corresponding tag symbols. Next, string literal nodes (i.e., the actual string values ​​of the API address and key) in the function are identified, and AES-256-GCM encryption is performed using the local key to generate ciphertext placeholders and replace the original string nodes. Subsequently, the structure and topology of the desensitized AST are serialized, and a SHA-256 digest is calculated. Using this digest as the key, the mapping table is stored in a local encrypted database.

[0180] Step 2, Cloud Generation: The client sends the anonymized code (where sensitive variable names are replaced with markers like "T{0}" and sensitive strings are replaced with placeholders like "S{base64 ciphertext}") to the AI ​​code generation service via HTTPS. The server-side large model returns a completed code snippet based on the context of the anonymized code, which still uses the same markers.

[0181] Step 3: Local Restoration and Merging: After receiving the completed code snippet, the client retrieves the mapping table from the local database using the previously calculated digest value. It scans the completed snippet, replacing any marked symbols with the original identifiers and decrypting the ciphertext placeholders to restore the original string. Then, it performs a three-way merge with the local workspace copy (i.e., the complete plugin source code file) and the common ancestor (i.e., the original incomplete function code). During the merge process, if a syntax conflict is detected (e.g., the completed code uses undeclared variables or unimported modules), the incremental merging engine automatically inserts the missing declaration or import statement at the appropriate location in the local workspace copy; if a semantic conflict is detected (e.g., type mismatch or scope overriding), the conflict location is highlighted in the integrated development environment, prompting the developer for confirmation. The merged code is written back to disk, triggering compilation hooks and unit test hooks for compilation and unit testing, ultimately ensuring the code is compileable and runnable.

[0182] The above scheme accurately identifies sensitive symbols through a symbol table, avoiding scope conflicts in regular expression replacement. Through node-level replacement in an abstract syntax tree and a topologically based digest index, it ensures that the de-identified code maintains structural consistency with the original code and that there are no misalignments during restoration. It also supports rapid reuse of the mapping table in multi-file, multi-version scenarios. String encryption and symbol mapping ensure that the cloud only accesses the de-identified fragments while the original sensitive information remains local, meeting compliance requirements such as GDPR. Through three-way merging and hierarchical handling of syntax / semantic conflicts, it automatically repairs any syntax gaps that may be introduced during de-identification and restoration, achieving "generate and compile immediately." Ultimately, while ensuring the security of sensitive information, it obtains highly applicable and readable code generation results, improving the security and development efficiency of AI-assisted coding.

[0183] Corresponding to the above method embodiments, this specification also provides code processing system embodiments. Figure 4 A schematic diagram of the structure of a code processing system provided in one embodiment of this specification is shown. Figure 4 As shown, the system includes: Front-end 402, client-side 404, server-side 406; The front-end 402 is used to send a code generation request for the original code to the client 404; Client 404 is used to execute the code processing method described above; The server-side 406 error is used to invoke the code generation model, which generates code snippets based on the de-identified code.

[0184] Optionally, Figure 5 A schematic diagram of the structure of another code processing system provided in one embodiment of this specification is shown. For example... Figure 5 As shown, the aforementioned client 404 has a local database 408, which is used to store mapping tables based on index identifiers.

[0185] In this embodiment, in response to a front-end request for code generation of the original code, the original code is parsed to obtain an original abstract syntax tree and symbol table; based on preset sensitive symbol rules, each symbol in the symbol table is traversed to determine the target symbol; a marker symbol corresponding to the target symbol is generated; the node corresponding to the target symbol in the original abstract syntax tree is replaced with the marker symbol to obtain a desensitized abstract syntax tree; desensitized code is generated based on the desensitized abstract syntax tree; and the desensitized code is uploaded to the server so that the code generation model deployed on the server can generate code based on the desensitized code. This method can accurately identify sensitive symbols through the symbol table, avoiding cross-scope conflicts caused by the inability to distinguish scopes in regular expression replacement. Furthermore, by replacing nodes in the abstract syntax tree rather than text replacement, the consistency of the syntax structure of the desensitized code and the original code is ensured, thereby avoiding misalignment during restoration due to positional offsets or format changes. Because the desensitized code retains the complete syntactic structure and the marker symbols have clear mapping relationships, the code generation model can correctly understand the context and output complete code with excellent readability. This achieves more applicable code generation results while ensuring the security of sensitive information, and solves the problem of poor applicability of complete code generated based on desensitized code.

[0186] The above is an illustrative scheme of a code processing system according to this embodiment. It should be noted that the technical solution of this code processing system and the technical solution of the code processing method described above belong to the same concept. For details not described in detail in the technical solution of the code processing system, please refer to the description of the technical solution of the code processing method described above.

[0187] Figure 6 A structural block diagram of a computing device according to one embodiment of this specification is shown. The components of the computing device 600 include, but are not limited to, a memory 610 and a processor 620. The processor 620 is connected to the memory 610 via a bus 630, and a database 650 is used to store data.

[0188] The computing device 600 also includes an access device 640, which enables the computing device 600 to communicate via one or more networks 660. Examples of these networks include Public Switched Telephone Network (PSTN), Local Area Network (LAN), Wide Area Network (WAN), Personal Area Network (PAN), or combinations of communication networks such as the Internet. The access device 640 may include one or more of any type of wired or wireless network interface (e.g., a network interface card (NIC)), such as an IEEE 802.11 Wireless Local Area Network (WLAN) wireless interface, a Wi-MAX (Worldwide Interoperability for Microwave Access) interface, an Ethernet interface, a Universal Serial Bus (USB) interface, a cellular network interface, a Bluetooth interface, or a Near Field Communication (NFC) interface.

[0189] In one embodiment of this specification, the above-described components of the computing device 600 and Figure 6 Other components, not shown, can also be connected to each other, for example, via a bus. It should be understood that... Figure 6 The block diagram of the computing device shown is for illustrative purposes only and is not intended to limit the scope of this specification. Those skilled in the art can add or replace other components as needed.

[0190] The computing device 600 can be any type of stationary or mobile computing device, including mobile computers or mobile computing devices (e.g., tablet computers, personal digital assistants, laptop computers, notebook computers, netbooks, etc.), mobile phones (e.g., smartphones), wearable computing devices (e.g., smartwatches, smart glasses, etc.) or other types of mobile devices, or stationary computing devices such as desktop computers or personal computers (PCs). The computing device 600 can also be a mobile or stationary server.

[0191] The processor 620 is used to execute the following computer-executable instructions, which, when executed by the processor, implement the steps of the above-described code processing method.

[0192] The above is an illustrative scheme of a computing device according to this embodiment. It should be noted that the technical solution of this computing device and the technical solution of the code processing method described above belong to the same concept. For details not described in detail in the technical solution of the computing device, please refer to the description of the technical solution of the code processing method described above.

[0193] An embodiment of this specification also provides a computer-readable storage medium storing computer-executable instructions that, when executed by a processor, implement the steps of the above-described code processing method.

[0194] The above is an illustrative scheme of a computer-readable storage medium according to this embodiment. It should be noted that the technical solution of this storage medium and the technical solution of the code processing method described above belong to the same concept. For details not described in detail in the technical solution of the storage medium, please refer to the description of the technical solution of the code processing method described above.

[0195] The foregoing has described specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired result. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

[0196] The computer instructions include computer program code, which may be in the form of source code, object code, executable file, or certain intermediate forms. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording media, USB flash drive, portable hard drive, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content included in the computer-readable medium may be appropriately added or removed according to the requirements of patent practice. For example, in some regions, according to patent practice, computer-readable media may not include electrical carrier signals and telecommunication signals.

[0197] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that the embodiments in this specification are not limited to the described order of actions, because according to the embodiments in this specification, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in this specification are all preferred embodiments, and the actions and modules involved are not necessarily essential to the embodiments in this specification.

[0198] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0199] The preferred embodiments disclosed above are merely illustrative of this specification. The optional embodiments do not exhaustively describe all details, nor do they limit the invention to the specific implementations described. Clearly, many modifications and variations can be made based on the embodiments described herein. These embodiments are selected and specifically described in this specification to better explain the principles and practical applications of the embodiments, thereby enabling those skilled in the art to better understand and utilize this specification. This specification is limited only by the claims and their full scope and equivalents.

Claims

1. A code processing method, applied to a client, comprising: In response to the front-end's code generation request for the original code, the original code is parsed to obtain the original abstract syntax tree and symbol table; The target symbol is determined by traversing each symbol in the symbol table based on preset sensitive symbol rules. Generate the tag symbol corresponding to the target symbol; Replace the node corresponding to the target symbol in the original abstract syntax tree with the tag symbol to obtain a de-identified abstract syntax tree; Based on the aforementioned de-identification abstract syntax tree, generate de-identification code; Upload the de-identified code to the server so that the code generation model deployed on the server can generate code based on the de-identified code.

2. The method according to claim 1, wherein the step of traversing each symbol in the symbol table based on preset sensitive symbol rules to determine the target symbol includes: Based on a preset sensitive symbol dictionary, each symbol in the symbol table is traversed and matched. If a match is successful, it is determined to be the target symbol. And / or, The symbol table is traversed and matched based on a preset regular expression. If a match is successful, it is determined to be the target symbol.

3. The method according to claim 1, wherein generating the marker symbol corresponding to the target symbol comprises: Generate sequentially increasing integers as the tag symbols corresponding to the target symbol.

4. The method according to claim 1, further comprising, before generating de-identified code based on the de-identification abstract syntax tree: Identify the nodes of string literals in the desensitized abstract syntax tree; The string literal is encrypted to generate encrypted ciphertext; Replace the nodes of the string literal with encrypted ciphertext to obtain the updated de-identified abstract syntax tree.

5. The method according to claim 4, wherein encrypting the string literal to generate encrypted ciphertext includes: The string literal is encrypted using a symmetric encryption algorithm to generate encrypted ciphertext.

6. The method according to any one of claims 1-5, further comprising, before uploading the de-identified code to the server: A mapping table is generated based on the target symbol, the marker symbol, and the scope chain of the target symbol; Store the mapping table.

7. The method according to claim 6, further comprising, after uploading the de-identified code to the server: Receive the code snippet generated by the code generation model and returned by the server; According to the mapping table, the marker symbols in the code snippet are replaced with the target symbols to obtain the restored code snippet; The target code is determined based on the restored code fragment, and the target code is fed back to the front end.

8. The method according to claim 7, wherein the original code is a code snippet in the local workspace code; The step of determining the target code based on the restored code fragment includes: The restored code snippet, the original code, and the local workspace code are merged to obtain the target code.

9. The method according to claim 8, wherein merging the restored code fragment, the original code, and the local workspace code to obtain the target code comprises: The restored code snippet, the original code, and the local workspace code are merged, and a check is triggered to determine whether a merge conflict has occurred. If not, obtain the target code.

10. The method of claim 9, wherein the merging conflict includes at least one of a syntax-level conflict and a semantic-level conflict; After triggering the determination of whether a merge conflict has occurred, at least one of the following is also included: If so, and the merge conflict is a syntax-level conflict, the missing declaration is inserted into the scope of the local workspace code using the incremental fusion engine; If so, and the merge conflict is a semantic-level conflict, the content corresponding to the semantic-level conflict will be highlighted in the integrated development environment.

11. The method according to claim 7, wherein replacing the marker symbols in the code segment with the target symbols according to the mapping table to obtain the restored code segment comprises: According to the mapping table, the marker symbols in the code snippet are replaced with the target symbols to obtain the intermediate code snippet; The encrypted ciphertext in the intermediate code fragment is traversed, and each encrypted ciphertext is decrypted into the string literal to obtain the restored code fragment.

12. The method of claim 7, further comprising, after determining the target code based on the restored code fragment: The compilation hook is invoked to compile the target code; If compilation succeeds, the unit test hook is invoked to test the target code; If the test passes, a runnable instruction for the target code is returned to the front end.

13. The method of claim 6, further comprising, before storing the mapping table: Based on the structural topology of the original abstract syntax tree, the summary value is calculated as the index identifier of the mapping table; The storage of the mapping table includes: Based on the index identifier, the mapping table is stored in the local database.

14. The method according to claim 12, wherein calculating the digest value as the index identifier of the mapping table based on the structural topology of the original abstract syntax tree includes: A hash algorithm is used to calculate the string sequence corresponding to the structural topology of the original abstract syntax tree, and the string sequence is used as the index identifier of the mapping table.

15. A code processing system, comprising a front-end, a client, and a server; The front-end is used to send a code generation request for the original code to the client; The client is used to perform the method according to any one of claims 1 to 13; The server is used to call the code generation model and generate code snippets based on the de-identified code.

16. The system according to claim 15, wherein the client has a local database for storing mapping tables based on index identifiers.

17. A computing device, comprising: Memory and processor; The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions, which, when executed by the processor, implement the steps of the method according to any one of claims 1 to 13.

18. A computer-readable storage medium storing computer-executable instructions that, when executed by a processor, implement the steps of the method according to any one of claims 1 to 13.

Images