Code generation method based on graph alignment coding large model and multi-agent collaboration

Abstract

The application discloses a ST code generation method based on a graph alignment coding large model and multi-agent cooperation, a demand analysis module receives an ST code programming demand input by a user, refines and analyzes the demand based on multi-round interactive dialogues of the user and insight agents, and generates a standardized ST code programming demand; a retrieval module receives the standardized ST code programming demand, retrieves and acquires relevant knowledge by means of a retrieval agent and in combination with an ST code knowledge base; a double-agent cooperative self-correcting code generation module receives the standardized ST code programming demand and the retrieved relevant knowledge, a graph alignment coding large model constructed based on a graph neural network and a cross-modal alignment technology, and the module is used as a coding agent and an inspection agent to work cooperatively, injects ST code structure information into the large model, and generates final ST code. The application can realize high-accuracy and high-reliability ST code automatic generation and improve the development efficiency of PLC programs in the industrial control field.

Description

Technical Field

[0001] The present invention belongs to the field of computer technology, and specifically relates to a code generation method based on a graph alignment coding large model and multi-agent collaboration. Background Art

[0002] Programmable logic controllers (PLCs) are core components of automated factories. Structured Text (ST), a high-level language, is crucial for PLCs due to its ability to concisely express logic and seamlessly integrate with other languages ​​within the same standard. However, traditional ST programming often involves complex logic design and extensive manual configuration. The programming process is tedious and time-consuming, and prone to logical and repetitive errors. When faced with complex process flows or large-scale systems, developers need to spend a considerable amount of time and effort writing and debugging code.

[0003] With the rapid development of artificial intelligence technology, deep learning and artificial intelligence technology have improved the performance of automatic program generation, but they still face the challenges of relying on large amounts of training data and low program generation quality. Recently, large language models have achieved breakthrough results in the fields of natural language processing and artificial intelligence. Large language models have strong generalization performance and provide new ideas for solving these challenges. However, in complex industrial scenarios, they are still faced with multiple challenges due to the diversity of processes and the complexity of system requirements: (1) User requirement descriptions are often ambiguous and lack dynamic interaction mechanisms to accurately analyze requirements; (2) The code generation process does not fully integrate knowledge bases such as equipment parameters, industry specifications, and safety rules, resulting in mismatches between generated code and hardware configurations or safety standards; (3) Existing large models lack a deep semantic understanding of code structures (such as abstract syntax trees and data flow graphs), making it difficult to ensure the logical completeness of generated code. (4) Code verification often relies on independent compilation links, lacks iterative self-correction capabilities, and has a long error correction cycle. Summary of the Invention

[0004] To address the deficiencies of existing technologies, improve the consistency between user needs and code semantics, and enhance the quality, standardization, adaptability, grammatical compliance, logical completeness, and reliability of generated code, the present invention adopts the following technical solutions:

[0005] The code generation method based on graph alignment coding large model and multi-agent collaboration includes the following steps:

[0006] Step 1: Analyze the requirements for code programming;

[0007] Step 2: Search and obtain relevant knowledge based on demand analysis;

[0008] Step 3: The parsed code programming requirements and related knowledge are combined through the collaboration of the coding agent and the review agent to perform dual-agent collaborative code self-correction to generate the final code. The coding agent uses a graph alignment coding model, and the implementation process includes the following steps:

[0009] Step 3.1.1 Dataset construction;

[0010] Step 3.1.2 Model Architecture Design: The graph alignment encoding large model includes a graph neural network encoder, an adapter, and a large language model decoder. The graph neural network encoder processes the abstract syntax tree (AST) and the data flow graph (DFG) to extract node features. A single-layer cross-attention layer is used as the adapter to project the graph features output by the graph neural network encoder into the embedding space of the large language model decoder. The large language model decoder receives the concatenated output of the graph tags and text tags from the adapter to generate the final output. The large language model decoder is trained to generate source code based on the graph information and answer questions about the graph structure, thereby aligning the code representation in the large language model with the graph structure and giving it a deeper understanding of the source code.

[0011] Step 3.1.3 model training includes the graph neural network encoder pre-training stage and the graph and large language model alignment stage; in the graph neural network encoder pre-training stage, the large language model decoder weights are frozen, and the graph neural network encoder and adapter are trained based on the graph alignment dataset; in the graph and large language model alignment training stage, the large language model decoder weights are unfrozen, and the graph neural network encoder, adapter, and large language model decoder are trained based on the constructed datasets (graph alignment dataset, graph question and answer dataset, and downstream code dataset).

[0012] Furthermore, the graph neural network encoder in step 3.1.2 is executed as follows:

[0013] Based on the code embedding model, each node corresponding to the code snippet is converted into a feature vector, and a size of The node feature matrix ,in represents the number of nodes in the graph, Represents the feature dimension of each node;

[0014] Construct an edge index matrix that can represent the connection relationship between nodes in the graph , the size is ,in represents the number of edges in the graph, represents a set of integers;

[0015] The node feature matrix The edge index matrix is ​​input into the graph neural network encoder. After multiple layers of graph neural network iteration, the node representation is updated. The formula is as follows:

[0016]

[0017]

[0018] The final output is the node representation matrix of the last layer:

[0019]

[0020] in, Representing the hidden layer of a graph neural network and output layer Dimensions, Indicates the number of hidden layers;

[0021] Use the cross attention layer to represent the node matrix output by the graph neural network Projection as graph tokens , the formula is as follows:

[0022]

[0023] in, represents a randomly initialized learnable query vector, represents a cross attention layer.

[0024] Furthermore, the graph neural network encoder pre-training phase in step 3.1.3 includes the following steps:

[0025] First, the abstract syntax tree AST and data flow graph DFG are extracted from the source code, and the text sequence of the corresponding source code is obtained; the context node representation is output through the graph neural network encoder , using a cross-attention layer to project the output of the graph neural network encoder into graph tokens , the graph token with text tokens Splicing into graphic tokens , text token The embedding after code segmentation;

[0026] Afterwards, the graphic token Input the frozen large language model decoder and generate the target code through autoregressive training, that is, given a graph token and generated text tokens Predict the next text token under the condition , and optimize it in a way that maximizes the conditional probability. The formula is as follows:

[0027]

[0028] in Indicates the text tokens, represents the frozen large language model parameters, Indicates the length of the text token sequence;

[0029] When calculating the loss during training, the loss of the image token is masked and only the cross entropy loss of the text part is calculated. The formula is as follows:

[0030]

[0031] in represents the actual target text token sequence, Represents the text token sequence predicted by the model. This phase forces the model to reconstruct the code from the graph structure and implicitly learn semantic associations such as grammar and data flow.

[0032] Furthermore, in step 3.1.1, the graph alignment coding large model training dataset includes a graph alignment dataset, and the graph alignment dataset is constructed by obtaining the code program and extracting the abstract syntax tree AST and the data flow graph DFG to obtain the code graph pair;

[0033] In step 3.1.3, during the graph and large language model alignment training phase, for the graph alignment task, the abstract syntax tree AST and data flow graph DFG are processed by the graph neural network encoder to generate a node representation matrix , the node representation matrix is ​​transformed through the cross attention layer adapter Projection as graph tokens , the graph token with text tokens Concatenate, input large language model decoder, generate code through autoregressive training, text token Embeddings for code tokens and question answering questions.

[0034] Furthermore, in step 3.1.1, the graph alignment coding large model training dataset includes a graph question answering dataset, and the designed prediction types include edge prediction, parent node prediction, and child node prediction;

[0035] The edge prediction, given the graph label and the source code of two nodes, the task of the model is to predict whether there is an edge between them. This task is only built based on the data flow graph DFG data;

[0036] The parent node prediction, given the mark in the graph and the source code of a node, the task of the model is to predict the parent node of the node in the graph. This task is built based on the abstract syntax tree AST and data flow graph DFG data;

[0037] The child node prediction, given the tags in the graph and the source code of a node, the model's task is to predict the child nodes of the node in the graph. This task is built based on the abstract syntax tree AST and data flow graph DFG data;

[0038] In step 3.1.3, during the graph alignment and large language model training phase, for graph alignment and graph question answering tasks, the abstract syntax tree AST and data flow graph DFG are processed by the graph neural network encoder to generate a node representation matrix. , the node representation matrix is ​​transformed through the cross attention layer adapter Projection as graph tokens , the graph token with text tokens Splicing, input large language model decoder, generate graph question answering tasks through autoregressive training, including answers to three types of tasks: edge prediction, parent node prediction, and child node prediction, and text tokens Embeddings for code tokens and question-answering questions;

[0039] In the graph question answering task, the training goal of the graph alignment encoding model is to maximize the , question text token and generated answer text token Generate answer text tokens under the conditions The probability of , its training objective can be expressed as:

[0040]

[0041] in represents the loss function of the graph question answering task training objective, represents the length of the answer text token sequence, Represents the frozen large language model parameters.

[0042] Furthermore, in step 3.1.1, the graph alignment coding large model training dataset includes a downstream task dataset, which comprehensively considers two types of discriminant and generative code tasks, including code translation, code clone detection, defect detection, code summarization, and code repair;

[0043] In step 3.1.3, for downstream code tasks, the downstream task data is directly input into the large language model decoder (without passing through the graph neural network encoder and adapter) and processed according to the standard text generation process. At this stage, the graph question-answering task is used to explicitly strengthen the large language model decoder's understanding of the graph structure. By jointly training the downstream task and the graph alignment task, the LLM is prompted to integrate structural information with text semantics. By unfreezing the LLM weights, it adapts to the representation space of code structure perception, thereby improving the robustness of code generation and understanding.

[0044] Furthermore, the self-correction of the code by the dual-agent collaboration in step 3 includes the following steps:

[0045] Step 3.2.1: Input the coding requirement text and related knowledge into the coding agent to generate the initial code, marked as code 0;

[0046] Step 3.2.2: Enter the code 0 into the review agent and perform the following verification process:

[0047] Call the compiler tool for compilation verification;

[0048] Analyze the compilation results to determine the correctness of the code; if the review agent determines that the code is correct, output code 0 as the final result; if the review agent determines that the code has an error, record the compilation result 0 and error diagnosis information;

[0049] Step 3.2.3: When code 0 fails the review, the original requirements, the relevant knowledge, the initial code, and the compilation result 0 are fed back to the coding agent to generate a revised code, which is marked as code 1;

[0050] Step 3.2.4: Perform recursive verification on the corrected code:

[0051] Input code n, n ≥ 1 into the review agent and repeat the verification process in step 3.2.2;

[0052] If the verification passes, the current code is output;

[0053] If errors still exist, an enhanced feedback message containing the newly added compilation result n is generated;

[0054] Step 3.2.5: Repeat steps 3.2.3 to 3.2.4 until any of the following conditions is met:

[0055] The code has passed compilation and logic verification;

[0056] The preset maximum number of iterations has been reached.

[0057] Furthermore, in step 3.2.5, the code generation and correction process satisfies the Markov decision process, and the formula is as follows:

[0058]

[0059]

[0060]

[0061] in, represents the code generated by the nth iteration, Indicates that the review agent calls the compiler to compile the code, Indicates the error message returned by the compiler. Indicates the initial code, Indicates the error message returned by the initial compiler. Represents a collection of historical codes and errors, Represents the original requirement text, Represents relevant knowledge text, represents the coding agent performing code generation, Indicates that the code has not passed verification. Indicates the preset maximum number of iterations.

[0062] Furthermore, in step 1, the coding requirements are refined and analyzed through multiple rounds of interactive dialogues between the user and the insight agent to obtain standardized coding requirements, including the following steps:

[0063] Step 1.1: The user inputs the code requirement description into the Insight Agent, which first performs a preliminary analysis of the user requirement to determine whether further clarification is needed.

[0064] The insight agent is implemented based on a large language model and preset system prompts, and identifies ambiguities and missing information in requirements based on the following criteria:

[0065] Keyword detection: Checks whether user input contains ambiguous words or phrases;

[0066] Entity recognition: Identify whether the key entities in the requirements (such as variable names, device models, and operating instructions) are complete;

[0067] Logical structure analysis: Determine whether the control logic in the description is complete, such as whether there are missing conditions or actions;

[0068] Industry specification compliance: Check whether it complies with industry standards or specifications for PLC programming;

[0069] Step 1.2: If the insight agent determines in step 1.1 that the user's needs are clear and do not require clarification, it directly outputs the standardized code programming requirements. If the insight agent determines in step 1.1 that the user's needs are vague or ambiguous, it uses proactive guided questioning based on preset intelligent questioning templates and ambiguity resolution templates to guide the user to clarify the vague needs. Among them, intelligent questioning templates include parameter questioning, entity questioning, and logical closed-loop questioning; ambiguity resolution templates include term clarification, unit system confusion, and other ambiguity types.

[0070] Step 1.3: The user supplements the details of the requirement description based on the questions asked by the insight agent in step 1.2 and feeds back to the insight agent. The insight agent again determines whether the user's requirements need to be clarified, and then repeats steps 1.2 and 1.3 until the standardized code programming requirements are output.

[0071] Furthermore, in step 2, by combining the search agent and the code knowledge base, the relevant knowledge is retrieved according to the parsed code programming requirements, and the relevant knowledge includes code cases, industrial control equipment information and relevant industry specifications. The retrieval process includes the following steps:

[0072] Step 2.1: Collect code documents, industrial control equipment information documents, and relevant industry specification documents, and segment them into fixed-length code or text snippets according to the rules. Then, input each code or text snippet into the semantic vector model to obtain the code and text snippet vectors. Finally, construct a vector database based on the code and text snippet vectors.

[0073] Step 2.2: Segment the industrial control equipment information documents and related industry specification documents collected in Step 2.1 into fixed-length text segments according to the rules. Use Elasticsearch to build an inverted index on the segmented text segments to construct a text database.

[0074] Step 2.3: Input the standardized coding requirements into the retrieval agent, which constructs a query phrase based on the coding requirements. ,in They represent device-level query phrases, logic-level query phrases, and security-level query phrases respectively. Then, a hybrid search method is used to retrieve relevant knowledge text fragments from the vector database and text database based on the query phrases.

[0075] The retrieval of relevant knowledge text fragments adopts a hierarchical retrieval strategy including the following steps:

[0076] The first round of search (device level) is used to obtain basic device parameters and specifications. It searches for device models (such as S7-1200), I / O module configurations, and communication protocol support, and returns results such as device specification documents, I / O address allocation tables, and communication parameter configurations.

[0077] The second round of search (logical level) is used to obtain control logic templates and algorithm implementations, and return results such as control logic code templates, algorithm parameter configurations, and related application cases;

[0078] The final search (safety level) is used to obtain safety verification rules and compliance requirements. It searches for safety integrity level (SIL) requirements, safety function implementation, and fault diagnosis mechanisms. The final result is safety verification rules, compliance check instructions, and safety function code snippets.

[0079] The hybrid search method is used to retrieve relevant knowledge text fragments from the vector database and the text database. Specifically, the method is implemented as follows: based on the query phrase, the BM25 text search method is used to retrieve relevant text knowledge from the text database, and based on the query phrase vector, the vector search method is used to retrieve relevant text knowledge from the vector database. The knowledge texts retrieved by the two methods are merged to obtain the relevant knowledge texts retrieved in the first round. The formula is as follows:

[0080]

[0081]

[0082]

[0083] in, Represents a collection of text snippets, represents a text database, Indicates a query phrase (which can refer to a device-level query phrase, a logical-level query phrase, or a security-level query phrase). Indicates that relevant text fragments are returned based on BM25 score retrieval , represents the text segment vector, represents a vector database, represents a query phrase vector (which can refer to a device-level query phrase vector, a logic-level query phrase vector, or a security-level query phrase vector). Indicates that relevant text fragments are returned based on vector similarity retrieval , Indicates that the relevant text fragments obtained by text retrieval and vector retrieval are merged to obtain the relevant knowledge text of hybrid retrieval .

[0084] The advantages and beneficial effects of the present invention are:

[0085] (1) Through a multi-round interactive requirements parsing mechanism, combined with a large language model and intelligent questioning templates, it effectively eliminates requirements ambiguity and improves the semantic consistency between requirements and code;

[0086] (2) A hybrid retrieval strategy combining vector database and text database is used to accurately match device parameters, logic templates, and safety rules from the knowledge base, providing contextual knowledge enhancement for the graph alignment coding model and improving the quality, standardization, and adaptability of the generated ST code;

[0087] (3) A large graph alignment encoding model is proposed. The structured features of the abstract syntax tree (AST) and data flow graph (DFG) of the ST code are encoded through a graph neural network and aligned with the embedding space of the large language model. The model is trained jointly with graph question answering tasks (edge ​​prediction, parent / child node prediction) and code generation to achieve a deep integration of grammatical rules, data dependencies, and ST code logic, thereby improving the grammatical compliance and logical completeness of the generated ST code.

[0088] (4) The dual-agent collaborative mechanism significantly reduces manual debugging costs and improves code reliability through a closed loop of "generate-verify-iterate" and combines compilation verification with knowledge enhancement feedback. BRIEF DESCRIPTION OF THE DRAWINGS

[0089] Figure 1 is a flow chart of a method in an embodiment of the present invention.

[0090] Figure 2 It is a schematic diagram of the structure of the demand analysis module in an embodiment of the present invention.

[0091] Figure 3 This is an example diagram of a specific implementation of the demand analysis process in an embodiment of the present invention.

[0092] Figure 4 It is a schematic diagram of the structure of the retrieval module in an embodiment of the present invention.

[0093] Figure 5 This is an implementation example of the retrieval module and code generation process in an embodiment of the present invention.

[0094] Figure 6 It is a schematic diagram of the structure of the large model of alignment coding in an embodiment of the present invention.

[0095] Figure 7 It is a schematic diagram of the structure of the self-correcting code generation module for dual-agent collaboration in an embodiment of the present invention. DETAILED DESCRIPTION

[0096] The following describes the specific embodiments of the present invention in detail with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are only used to illustrate and explain the present invention and are not intended to limit the present invention.

[0097] The code generation method based on the graph alignment coding big model and multi-agent collaboration solves the problems of requirement ambiguity, knowledge gaps and logical deviations in industrial control ST code generation through the three-layer architecture of requirement analysis-knowledge retrieval-self-correction generation. Specifically, first, through multiple rounds of interactive requirement analysis modules (combined with big models and intelligent questioning templates), user requirements are dynamically clarified to generate standardized coding requirements; secondly, a hybrid retrieval mechanism is used to accurately match device parameters, logic templates and safety rule knowledge from the knowledge base; finally, through dual-agent collaboration (graph alignment coding big model generates code + review agent iterative verification) and combined with the retrieved relevant knowledge, code generation and self-correction are achieved. The graph alignment coding big model integrates graph neural networks and big language models to enhance the alignment capabilities of code semantics and logical structures. Figure 1 As shown, the method of the present invention comprises the following steps:

[0098] Step 1: The user inputs the ST code programming requirements into the requirements analysis module, which refines and analyzes the requirements through multiple rounds of interactive dialogues between the user and the insight agent to obtain standardized ST code programming requirements.

[0099] like Figure 2 As shown, the implementation of the demand parsing module in step 1 includes the following steps:

[0100] Step 1.1: The user inputs the ST code requirement description into the Insight Agent. The Insight Agent first performs a preliminary analysis of the user's requirements to determine whether further clarification is needed.

[0101] The Insight Agent is implemented based on the Tongyi Qianwen language model and preset system prompts. It identifies ambiguities and missing information in requirements based on the following criteria:

[0102] (1) Keyword detection: Check whether the user input contains ambiguous words or phrases;

[0103] (2) Entity identification: Identify whether the key entities in the requirements (such as variable names, equipment models, and operating instructions) are complete;

[0104] (3) Logical structure analysis: Determine whether the control logic in the description is complete, such as whether there are missing conditions or actions;

[0105] (4) Industry standard compliance: Check whether it complies with the industry standards or specifications for PLC programming.

[0106] Step 1.2: If the insight agent in step 1.1 determines that the user's needs are clear and do not need to be clarified, it directly outputs the standardized ST code programming requirements; if the insight agent in step 1.1 determines that the user's needs are vague or ambiguous, it uses the preset intelligent questioning template and ambiguity resolution template to actively guide the user to clarify the vague needs. An example of this process is as follows: Figure 3 shown.

[0107] Among them, the intelligent question template is:

[0108] (1) Parameter-asking question: “Please clarify the trigger threshold of {signal name}”;

[0109] (2) Entity-based questioning: “Please clarify the model of {device name}”;

[0110] (3) Logical closed loop type: "What action should be performed when {condition} is not met?"

[0111] The ambiguity resolution template is:

[0112] (1) Terminology clarification: “The {fuzzy term} you mentioned refers to: 1) {standard term 1}, 2) {standard term 2}, 3) {other definitions}”;

[0113] (2) Unit system confusion: "The input {pressure value 50} unit is: kPa, MPa, psi";

[0114] (3) Other types of ambiguity, such as control type ambiguity, logical relationship ambiguity, etc.

[0115] Step 1.3: The user supplements the details of the requirement description based on the questions asked by the insight agent in step 1.2 and feeds back to the insight agent. The insight agent again determines whether the user's requirements need to be clarified, and then repeats steps 1.2 and 1.3 until the standardized ST code programming requirements are output.

[0116] Step 2: Input the standardized ST code programming requirements output from Step 1 into the retrieval module. During this process, the retrieval agent combines the ST code knowledge base to retrieve and obtain relevant knowledge, including ST code cases, industrial control equipment information, and relevant industry specifications.

[0117] like Figure 4 As shown, the implementation of the retrieval module includes the following steps:

[0118] Step 2.1: Collect ST code documents, industrial control equipment information documents, and industry specification documents, and segment them into fixed-length code or text fragments according to the rules. Then, input each code or text fragment into the semantic vector model to obtain the code and text fragment vectors. Finally, construct a vector database based on the code and text fragment vectors.

[0119] Step 2.2: Segment the industrial control equipment information documents and industry specification documents collected in Step 2.1 into fixed-length text segments according to the rules. Use Elasticsearch to build an inverted index on the segmented text segments to construct a text database.

[0120] Step 2.3: Input the standardized ST code programming requirements into the search agent, and the search agent constructs a query phrase based on the ST code programming requirements ,in They represent device-level query phrases, logic-level query phrases, and security-level query phrases respectively. Then, a hybrid search method is used to retrieve relevant knowledge text fragments from the vector database and text database based on the query phrases. The process is shown in the following example. Figure 5 As shown;

[0121] The retrieval of relevant knowledge text fragments adopts a hierarchical retrieval strategy including the following steps:

[0122] (1) The first round of search (device level) is used to obtain basic device parameters and specifications. The device model (such as S7-1200), IO module configuration, and communication protocol support are searched, and the device specification document, IO address allocation table, and communication parameter configuration are returned.

[0123] (2) Second round of retrieval (logical level) is used to obtain control logic templates and algorithm implementations, and return results such as control logic code templates, algorithm parameter configurations, and related application cases;

[0124] (3) The final round of retrieval (safety level) is used to obtain safety verification rules and compliance requirements, search for safety integrity level (SIL) requirements, safety function implementation and fault diagnosis mechanism, and finally obtain the return results such as safety verification rules, compliance inspection instructions and safety function code snippets. The hybrid retrieval method is used to retrieve relevant knowledge text snippets from the vector database and the text database as follows: based on the query phrase, the BM25 text retrieval method is used to retrieve relevant text knowledge from the text database, and based on the query phrase vector, the vector retrieval method is used to retrieve relevant text knowledge from the vector database. The knowledge texts retrieved by the two are merged to obtain the relevant knowledge text of the first round of retrieval. The formula is as follows:

[0125]

[0126]

[0127]

[0128] in, Represents a collection of text snippets, represents a text database, Indicates a query phrase (which can refer to a device-level query phrase, a logical-level query phrase, or a security-level query phrase). Indicates that relevant text fragments are returned based on BM25 score retrieval , represents the text segment vector, represents a vector database, represents a query phrase vector (which can refer to a device-level query phrase vector, a logic-level query phrase vector, or a security-level query phrase vector). Indicates that relevant text fragments are returned based on vector similarity retrieval , Indicates that the relevant text fragments obtained by text retrieval and vector retrieval are merged to obtain the relevant knowledge text of hybrid retrieval .

[0129] Step 3: The standardized ST code programming requirements output from step 1 and the relevant knowledge retrieved from step 2 are input into the dual-agent collaborative self-correcting code generation module. This module works collaboratively with the coding agent (graph alignment coding model) and the review agent to generate the final ST code.

[0130] The graph alignment encoding model uses a graph neural network to extract the structural information of the ST code. Using an adapter, the output features of the graph neural network are projected into the embedding space of a large language model (LLM). The large language model is trained to generate source code based on the graph information and answer questions about the graph structure. These goals align the code representation in the language model with the graph structure and provide a deeper understanding of the source code. The specific implementation of the graph alignment encoding model is as follows:

[0131] Step 3.1.1 Dataset Construction: The training dataset for the graph alignment encoding model includes the graph alignment dataset, the graph question answering dataset, and the downstream task dataset. The construction methods of various datasets are as follows:

[0132] Graph alignment dataset construction: obtain the ST code program and extract the abstract syntax tree (AST) and data flow graph (DFG) to obtain code graph pairs;

[0133] Graph Question Answering Dataset: Three types of questions are designed:

[0134] (1) Edge prediction: Given a graph label and the source code of two nodes, the model’s task is to predict whether there is an edge between them. This task is built only on data flow graph (DFG) data;

[0135] (2) Parent Node Prediction: Given the tokens in the graph and the source code of a node, the model’s task is to predict the parent node of the node in the graph. This task is built on the abstract syntax tree (AST) and data flow graph (DFG) data.

[0136] (3) Child node prediction: Given the tokens in the graph and the source code of a node, the model’s task is to predict the child nodes of the node in the graph. This task is built based on the abstract syntax tree (AST) and data flow graph (DFG) data.

[0137] Downstream Code Task Dataset: This dataset comprehensively considers both discriminative and generative code tasks, including code translation, code clone detection, defect detection, code summarization, and code repair.

[0138] Step 3.1.2 Model architecture design: Figure 7 As shown in the figure, the model architecture mainly consists of three modules: Graph Neural Network (GNN) Encoder: Use the graph neural network encoder to process the abstract syntax tree and data flow graph to extract node features; Adapter: Use a single-layer cross-attention layer as an adapter to project the graph features output by the graph neural network into the embedding space of the large language model; Large Model (LLM) Decoder: Receive the concatenated input of graph tags and text tags to generate the final output.

[0139] The graph neural network encoder processes the abstract syntax tree and data flow graph to extract node features. The specific implementation is as follows:

[0140] (1) Based on the code embedding model, each node corresponding to the ST code fragment is converted into a feature vector, and finally a feature vector of size The node feature matrix ,in is the number of nodes in the graph, is the characteristic dimension of each node;

[0141] (2) Construct an edge index matrix that can represent the connection relationship between nodes in the graph , the size is ,in is the number of edges in the graph, represents a set of integers;

[0142] (3) Input the node feature matrix and edge index matrix into the GNN encoder. After multiple layers of GNN iteration, update the node representation. The formula is as follows:

[0143]

[0144]

[0145] The final output is the node representation matrix of the last layer:

[0146]

[0147] in, are the dimensions of the GNN hidden and output layers.

[0148] Among them, in step 3.1.2, a single-layer cross attention layer is used as an adapter to project the graph features output by the graph neural network into the embedding space of the large language model. The specific implementation is: use the cross attention layer to project the node representation matrix output by the GNN Projection as graph tokens , the formula is as follows:

[0149]

[0150] in, is a randomly initialized learnable query vector, represents a cross attention layer.

[0151] Step 3.1.3 Model training: Model training is divided into two stages: graph encoder pre-training stage and graph-LLM alignment stage. The specific implementation is:

[0152] Graph Neural Network Encoder Pre-training: In this stage, the LLM weights are frozen and the GNN and adapter are trained based on the graph dataset. First, AST and DFG are extracted from the source code, and the text sequence of the corresponding source code is obtained. The context node representation is output by GNN The output of the GNN is projected into graph tokens using a cross-attention layer. , the graph token with text tokens (Embedding after code segmentation) spliced ​​into image and text tokens , then the graphic token Input the frozen LLM decoder and generate the target code through autoregressive training, that is, given a graph token and generated text tokens Predict the next text token under the condition , and optimize it in a way that maximizes the conditional probability. The formula is as follows:

[0153]

[0154] in Indicates the text tokens, represents the frozen large language model parameters, Indicates the length of the text token sequence;

[0155] When calculating the loss during training, the loss of the image token is masked and only the cross entropy loss of the text part is calculated. The formula is as follows:

[0156]

[0157] in represents the actual target text token sequence, Represents the text token sequence predicted by the model. This phase forces the model to reconstruct the code from the graph structure and implicitly learn semantic associations such as grammar and data flow.

[0158] Graph and LLM alignment training: In this stage, the LLM weights are unfrozen and GNN, adapter and LLM are trained based on the graph alignment dataset, graph question answering dataset and downstream code dataset. For graph alignment and graph question answering tasks, AST and DFG are processed by GNN to generate node representations. , through the cross-attention layer adapter Projection as graph tokens , compare graph tokens with text tokens The code word embedding and the question-answering embedding are spliced ​​together and the input LLM is trained through autoregression to generate code or graph question-answering tasks, including edge prediction, parent node prediction, and child node prediction. In the graph question-answering task, the training goal of the graph alignment encoding model is to maximize the accuracy of the given graph token. , question text token and generated answer text token Generate answer text tokens under the conditions The probability of , its training objective can be expressed as:

[0159]

[0160] in represents the loss function of the graph question answering task training objective, represents the length of the answer text token sequence, Represents frozen large language model parameters. For downstream code tasks, downstream task data is directly fed into the LLM (without passing through the GNN and adapter) and processed according to the standard text generation pipeline. This phase explicitly strengthens the LLM's understanding of graph structure through graph question answering tasks. Joint training of downstream tasks with the graph alignment task enables the LLM to integrate structural information with textual semantics. Unfreezing the LLM weights allows it to adapt to a code structure-aware representation space, improving the robustness of code generation and comprehension. For the aforementioned large graph encoding model, the graph neural network model is based on DUPLEX, and the large language model is based on LLaMA3 8B.

[0161] In step 3, the dual-agent collaborative self-correcting code generation module is as follows: Figure 6As shown, the following steps are included:

[0162] Step 3.2.1: Input the coding requirement text and related knowledge into the coding agent to generate the initial code (marked as code 0);

[0163] Step 3.2.2: Enter code 0 into the review agent and perform the following verification process:

[0164] (1) Call the compiler tool for compilation verification;

[0165] (2) Analyze the compilation results to determine the correctness of the code: If the review agent determines that the code is correct, it outputs code 0 as the final result; if the review agent determines that the code has an error, it records the compilation result 0 and error diagnosis information;

[0166] Specifically, the review agent is based on the Tongyi Qianwen model, and the specific functions of the agent are defined through system prompt words.

[0167] Step 3.2.3: If code 0 fails the review, the four-tuple information (original requirement, relevant knowledge, code 0, compilation result 0) is fed back to the coding agent to generate a revised version of the code (code 1).

[0168] Step 3.2.4: Perform recursive verification on the corrected code:

[0169] (1) Input code n (n ≥ 1) into the review agent and repeat the verification process of step 3.2.2;

[0170] (2) If the verification passes, the current code is output;

[0171] (3) If errors still exist, an enhanced feedback message containing the newly added compilation result n is generated.

[0172] Step 3.2.5: Repeat steps 3.2.3 to 3.2.4 until any of the following conditions is met:

[0173] (1) The code passes compilation and logic verification.

[0174] (2) Reach the preset maximum number of iterations. The code generation and correction process satisfies the Markov decision process, and the formula is as follows:

[0175]

[0176]

[0177]

[0178] in, Indicates the error message returned by the compiler. Indicates that the review agent calls the compiler tool to compile the code, represents the code generated by the nth iteration, Coding agents perform code generation, Represents the original requirement text, Represents relevant knowledge text, Represents a collection of historical codes and errors, Indicates that the code has not passed verification. Indicates the preset maximum number of iterations.

[0179] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some or all of the technical features therein. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A code generation method based on graph alignment coding large model and multi-agent collaboration, characterized by The steps include: Step 1: Analyze the requirements for code programming; Step 2: Retrieve and acquire relevant knowledge based on the requirements analysis. By combining the retrieval agent and the code knowledge base, relevant knowledge is retrieved based on the parsed code programming requirements. The relevant knowledge includes code cases, industrial control equipment information, and relevant industry specifications. The retrieval process includes the following steps: Step 2.1: Collect code documents, industrial control equipment information documents, and relevant industry specification documents, and segment them into fixed-length code or text snippets according to the rules. Then, input each code or text snippet into the semantic vector model to obtain the code and text snippet vectors. Finally, construct a vector database based on the code and text snippet vectors. Step 2.2: Segment the industrial control equipment information documents and related industry specification documents collected in step 2.1 into text segments of fixed length according to the rules, and construct a text database for the segmented text segments; Step 2.3: Input the standardized coding requirements into the retrieval agent. The retrieval agent constructs a query phrase based on the coding requirements and then uses a hybrid search method to retrieve relevant knowledge text fragments from the vector database and text database based on the query phrase. Step 3: The parsed code programming requirements and related knowledge are combined through the collaboration of the coding agent and the review agent to perform dual-agent collaborative code self-correction to generate the final code. The coding agent uses a graph alignment coding model, and the implementation process includes the following steps: Step 3.1.1 Dataset construction; Step 3.1.2 Model architecture design: The graph alignment encoding model includes a graph neural network encoder, an adapter, and a large language model decoder; the graph neural network encoder extracts node features; The adapter projects the graph features output by the graph neural network encoder into the embedding space of the large language model decoder; The large language model decoder receives the concatenated output of the graph tags and text tags from the adapter and generates the final output. The large language model decoder is trained to generate source code based on the graph information and answer questions about the graph structure. Step 3.1.3: Model training, including the graph neural network encoder pre-training phase and the graph and large language model alignment phase. In the graph neural network encoder pre-training phase, the large language model decoder weights are frozen, and the graph neural network encoder and adapter are trained based on the graph alignment dataset. During the graph and large language model alignment training phase, the large language model decoder weights are unfrozen, and the graph neural network encoder, adapter, and large language model decoder are trained based on the constructed dataset.

2. The code generation method based on graph alignment coding large model and multi-agent collaboration according to claim 1 is characterized by: The graph neural network encoder in step 3.1.2 is executed as follows: Convert each node corresponding to the code snippet into a feature vector and get a size of The node feature matrix ,in represents the number of nodes in the graph, Represents the feature dimension of each node; Construct an edge index matrix that can represent the connection relationship between nodes in the graph , the size is ,in represents the number of edges in the graph, represents a set of integers; The node feature matrix and edge index matrix Input to the graph neural network encoder, after multiple layers of graph neural network iteration, update the node representation, the formula is as follows: The final output is the node representation matrix of the last layer: in, Representing the hidden layer of a graph neural network and output layer Dimensions, Indicates the number of hidden layers; Use the cross attention layer to represent the node matrix output by the graph neural network Projection as graph tokens , the formula is as follows: in, represents a randomly initialized learnable query vector, represents a cross attention layer.

3. The code generation method based on graph alignment coding large model and multi-agent collaboration according to claim 2 is characterized by: The graph neural network encoder pre-training phase in step 3.1.3 includes the following steps: First, an abstract syntax tree and data flow graph are extracted from the source code, and the text sequence of the corresponding source code is obtained. The context node representation is output through a graph neural network encoder. The output of the graph neural network encoder is projected into graph tokens using a cross-attention layer. The graph tokens and text tokens are concatenated into graph-text tokens, where the text tokens are the embeddings after code segmentation. After that, the graph-text token is input into the frozen large language model decoder, and the target code is generated through autoregressive training, that is, given the graph token and generated text tokens Predict the next text token under the condition , and optimize in a way that maximizes the conditional probability; When calculating the loss during training, the loss of the graph token is masked and only the cross entropy loss of the text part is calculated.

4. The code generation method based on graph alignment coding large model and multi-agent collaboration according to claim 2 is characterized by: In step 3.1.1, the graph alignment coding large model training dataset includes a graph alignment dataset. The graph alignment dataset is constructed by obtaining a code program and extracting an abstract syntax tree and a data flow graph to obtain a code graph pair; In the step 3.1.3, in the graph and large language model alignment training phase, for the graph alignment task, the abstract syntax tree and data flow graph are processed by the graph neural network encoder to generate a node representation matrix, which is projected into graph tokens through the cross-attention layer adapter. The graph tokens are concatenated with the text tokens and input into the large language model decoder. Code is generated through autoregressive training. The text tokens are the embeddings of the code after word segmentation and the embeddings of the question and answer questions.

5. The code generation method based on graph alignment coding large model and multi-agent collaboration according to claim 1 is characterized by: In step 3.1.1, the graph alignment coding large model training dataset includes a graph question answering dataset; In step 3.1.3, during the graph alignment and large language model training phase, for graph alignment and graph question answering tasks, the abstract syntax tree and data flow graph are processed by the graph neural network encoder to generate a node representation matrix, and the node representation matrix is ​​converted to a node representation matrix by the cross attention layer adapter. Project the graph tokens, concatenate the graph tokens with the text tokens, input them into the large language model decoder, and generate the answers to the code or graph question-answering tasks through autoregressive training. The text tokens are the embeddings of the code after word segmentation and the embeddings of the question-answering questions. In the graph question answering task, the training goal of the graph alignment encoding model is to maximize the , question text token and generated answer text token Generate answer text tokens under the conditions probability.

6. The code generation method based on graph alignment coding large model and multi-agent collaboration according to claim 2 is characterized by: In step 3.1.1, the training dataset of the graph alignment coding large model includes a downstream task dataset, which comprehensively considers two types of discriminant and generative code tasks, including code translation, code clone detection, defect detection, code summarization, and code repair; In step 3.1.3, for downstream code tasks, downstream task data is directly input into the large language model decoder.

7. The code generation method based on graph alignment coding large model and multi-agent collaboration according to claim 1 is characterized by: The dual-agent collaborative code self-correction in step 3 includes the following steps: Step 3.2.1: Input the coding requirement text and related knowledge into the coding agent to generate the initial code, marked as code 0; Step 3.2.2: Enter the code 0 into the review agent and perform the following verification process: Call the compiler tool for compilation verification; Analyze compilation results to determine code correctness; If the review agent determines that the code is correct, it outputs code 0 as the final result; if the review agent determines that the code has an error, it records the compilation result 0 and error diagnosis information; Step 3.2.3: When code 0 fails the review, the original requirements, the relevant knowledge, the initial code, and the compilation result 0 are fed back to the coding agent to generate a revised code, which is marked as code 1; Step 3.2.4: Perform recursive verification on the corrected code: Input code n, n ≥ 1 into the review agent and repeat the verification process in step 3.2.2; If the verification passes, the current code is output; If errors still exist, an enhanced feedback message containing the newly added compilation result n is generated; Step 3.2.5: Repeat steps 3.2.3 to 3.2.4 until any of the following conditions is met: The code has passed compilation and logic verification; The preset maximum number of iterations has been reached.

8. The code generation method based on graph alignment coding large model and multi-agent collaboration according to claim 7 is characterized by: In step 3.2.5, the code generation and correction process satisfies the Markov decision process, and the formula is as follows: in, represents the code generated by the nth iteration, Indicates that the review agent calls the compiler to compile the code, Indicates the error message returned by the compiler. Indicates the initial code, Indicates the error message returned by the initial compiler. Represents a collection of historical codes and errors, Represents the original requirement text, Represents relevant knowledge text, represents the coding agent performing code generation, Indicates that the code has not passed verification. Indicates the preset maximum number of iterations.

9. The code generation method based on graph alignment coding large model and multi-agent collaboration according to claim 1 is characterized by: In step 1, the coding requirements are refined and analyzed through multiple rounds of interactive dialogues between the user and the insight agent to obtain standardized coding requirements, including the following steps: Step 1.1: The user inputs the code requirement description into the Insight Agent, which first performs a preliminary analysis of the user requirement to determine whether further clarification is needed. Step 1.2: If the insight agent determines in step 1.1 that the user's needs are clear and do not require clarification, it directly outputs the standardized coding requirements. If the insight agent determines in step 1.1 that the user's needs are vague or ambiguous, it uses proactive guided questioning based on the preset intelligent questioning template and ambiguity resolution template to guide the user to clarify the vague needs. Step 1.3: The user supplements the details of the requirement description based on the questions asked by the insight agent in step 1.2 and feeds back to the insight agent. The insight agent again determines whether the user's requirements need to be clarified, and then repeats steps 1.2 and 1.3 until the standardized code programming requirements are output.

10. The code generation method based on graph alignment coding large model and multi-agent collaboration according to claim 1 is characterized by: In step 2, the hybrid retrieval method is used to retrieve relevant knowledge text fragments from the vector database and the text database, which is specifically implemented as follows: based on the query phrase, the text retrieval method is used to retrieve relevant text knowledge from the text database, and based on the query phrase vector, the vector retrieval method is used to retrieve relevant text knowledge from the vector database, and the knowledge texts retrieved by the two are merged to obtain the relevant knowledge text of the first round of retrieval.

Images