Code generation method and apparatus based on code graph

Abstract

Provided in the embodiments of the present description is a code generation method based on a code graph. The method comprises: receiving first question text for codes in a code repository; on the basis of the first question text, performing relevance-based retrieval in a pre-constructed code graph, and determining a first context sub-graph, wherein the code graph comprises nodes, which represent code elements in the code repository, and directed edges, which represent code relationships between the nodes; performing graph representation processing on the first context sub-graph, so as to obtain a mapping representation of each node being mapped into an embedding space of a large language model; performing encoding on the first question text in an embedding layer of the large language model, so as to obtain a text representation; and on the basis of the mapping representation and the text representation, performing target reasoning in a reasoning layer of the large language model, so as to obtain target code text with respect to the first question text.

Description

A code generation method and apparatus based on code graphs

[0001] This application claims priority to Chinese Patent Application No. 202411612892.0, filed on November 12, 2024, entitled "A Code Generation Method and Apparatus Based on Code Graphs", the entire contents of which are incorporated herein by reference. Technical Field

[0002] This specification relates to one or more embodiments in the field of large language modeling and code generation technology, and in particular to a code generation method and apparatus based on code graphs. Background Technology

[0003] With the rapid development of information technology, the software industry is facing unprecedented challenges, including the continuous increase in the complexity of software projects and the rapid iteration of market demands. Against this backdrop, the industry has developed a variety of tools to assist developers in writing code. For example, Large Language Models (LLMs), represented by ChatGPT, have been widely used in code generation scenarios in recent years.

[0004] Code generation technology refers to the process by which a model automatically generates program code based on the input problem and requirements, thereby improving the coding efficiency of developers. Early code generation technologies were mainly aimed at simple business scenarios, generating small-scale code that implements a single function in a single code file, such as generating sorting algorithms or generating SQL query fragments, etc.

[0005] However, as the complexity of software projects continues to rise, the references and calls between functional modules contained in the source code files become intricate. Traditional code generation techniques struggle to understand the code structure of modern software projects, resulting in errors in identifying and locating code files that need modification, thus affecting the accuracy of code generation.

[0006] Therefore, we hope to find a solution that can improve the model's ability to understand code structure through technical means, thereby accurately locating the code files that need to be modified and improving the accuracy of code generation. Summary of the Invention

[0007] This specification describes one or more embodiments of a code generation method and apparatus based on code graphs, which solves the problem that existing large language models have difficulty understanding the code structure in code repositories, improves the ability of large language models to locate target code files, and thus better completes repository-level code generation tasks.

[0008] According to the first aspect, a code generation method based on code graphs is provided, including:

[0009] Receive the first question text for the code in the code repository.

[0010] Based on the first question text, a relevance search is performed in a pre-constructed code graph to determine a first context subgraph. The code graph contains nodes representing code elements in the code repository and directed edges representing code relationships between nodes.

[0011] The first context subgraph is processed by graph representation to obtain the mapping representation of each node to the large language model embedding space.

[0012] In the embedding layer of the large language model, the first question text is encoded to obtain a text representation.

[0013] In the inference layer of the large language model, target inference is performed based on the mapping representation and the text representation to obtain the target code text for the first question text.

[0014] According to one implementation, the code elements include at least one of the following: code module, code file, class definition, method declaration, variable declaration; the code relationships include at least one of the following: hierarchical relationship, import relationship, reference relationship, inheritance relationship, and call relationship.

[0015] According to one implementation, the first issue text includes one of the following: code location, bug fix, new feature addition, or dependency update.

[0016] According to one implementation, determining the first context subgraph includes:

[0017] Based on the related nodes obtained by the relevance retrieval, a target search is performed. The target search includes, for any related node, searching along the directed edges in the code graph within a preset number of hops to obtain several extended nodes.

[0018] The first context subgraph is constructed based on the aforementioned associated nodes and their respective extended nodes.

[0019] According to one implementation, the relevance retrieval in the pre-built code graph includes:

[0020] From the first question text, several first identifiers are extracted. The first identifiers include code element names and / or functional description words. In the code graph, several associated nodes are retrieved based on text fuzzy matching. The text similarity between the code element names of the associated nodes and the first identifiers is higher than a preset threshold.

[0021] According to one implementation, the relevance retrieval in the pre-built code graph includes:

[0022] The first question text is input into the first model for inference to obtain the first extended text, which includes the functional description text of the code elements related to the first question text; the first extended text and the code graph are input into the second model for encoding to obtain the first code corresponding to the first extended text and the semantic code corresponding to the source code of each node in the code graph; in the code graph, several related nodes are retrieved, and the representation similarity between the semantic code corresponding to the related nodes and the first code is higher than a preset threshold.

[0023] In one scenario of the above implementation, the second model is the CGE-Large model.

[0024] According to one implementation, the method further includes:

[0025] For each file node in the first context subgraph, several first code files are determined based on their relevance to the first question text. The target inference includes: performing target inference based on the first code files, the mapping representation, and the text representation in the inference layer of the large language model.

[0026] In one scenario of the above implementation, determining several first code files includes:

[0027] The file nodes and thought chain hint text in the first context subgraph are input into the third model for reasoning to select the several first code files; the thought chain hint text is used to instruct the third model to perform reasoning in multiple steps to gradually determine the code files corresponding to the K file nodes with the highest relevance to the first question text.

[0028] According to one implementation, the graph representation processing of the first context subgraph includes:

[0029] The code text content contained in each node in the first context subgraph is represented to obtain the encoding representation corresponding to each node; each encoding representation is input into a mapping network for mapping processing to obtain the mapping representation.

[0030] In one scenario of the above implementation, the code graph includes a first node, and the target code text content contained in the first node is long text; the process of representing the code text content contained in each node of the first context subgraph includes:

[0031] The target code text content is segmented to obtain multiple text blocks.

[0032] Each text block is processed to obtain the text block encoding representation corresponding to each text block.

[0033] Based on the encoding representation of each text block, the encoding representation of the first node is determined.

[0034] According to one implementation, the target reasoning includes:

[0035] Using the large language model, the adjacency matrix of the code graph is used as an attention mask to perform attention-based encoding processing.

[0036] According to one implementation, the method further includes:

[0037] If the target code text contains target modifications to historical code, then the target modifications are packaged and submitted.

[0038] According to one implementation, the code relationships include structural relationships and semantic relationships, and the code graph is constructed through the following steps:

[0039] Construct a first abstract syntax tree based on the first source code contained in the code repository.

[0040] Traverse the first abstract syntax tree and add each node representing a code element and the directed edges representing the structural relationships between nodes to the code graph.

[0041] Semantic analysis is performed on the code elements contained in the first source code to obtain several first semantic relationships.

[0042] In the code graph, the source node and the target node are connected by directed edges according to the first semantic relationship.

[0043] According to a second aspect, a code generation apparatus based on a code graph is provided, the apparatus comprising:

[0044] The receiving module is configured to receive the first question text for the code in the code repository.

[0045] The retrieval module is configured to perform a relevance search in a pre-constructed code graph based on the first question text to determine a first context subgraph. The code graph includes nodes representing code elements in the code repository and directed edges representing code relationships between nodes.

[0046] The first encoding module is configured to perform graph representation processing on the first context subgraph to obtain the mapping representation of each node to the large language model embedding space.

[0047] The second encoding module is configured to encode the first question text in the embedding layer of the large language model to obtain a text representation.

[0048] The reasoning module is configured to perform target reasoning based on the mapping representation and the text representation in the reasoning layer of the large language model to obtain target code text for the first question text.

[0049] According to a third aspect, a computer program product is provided, including a computer program / instructions that, when executed by a processor, implement the steps of the method described in the first aspect.

[0050] According to a fourth aspect, a computing device is provided, including a memory and a processor, characterized in that the memory stores executable code, and when the processor executes the executable code, it implements the method described in the first aspect.

[0051] In summary, the methods and apparatus disclosed in the embodiments of this specification can perform graph analysis on the code graph representing the code repository based on the problem text, extract the context subgraph related to the problem text, and input it into a large language model. This improves the accuracy of the large language model in understanding the code structure and correctly generates the target code text for the problem text. The embodiments of this specification can solve the problem that existing large language models have difficulty understanding the code structure due to excessively long code repository context, helping the model to better complete repository-level code tasks. Attached Figure Description

[0052] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are merely some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without any creative effort.

[0053] Figure 1 is a schematic diagram of a code generation method framework based on code graphs disclosed in this specification;

[0054] Figure 2 is a flowchart of a code generation method based on a code graph according to an embodiment of this specification;

[0055] Figure 3 is an exemplary first problem text provided in the embodiments of this specification;

[0056] Figure 4 is an exemplary code diagram provided according to an embodiment of this specification;

[0057] Figure 5A is an exemplary prompt template for extracting keywords according to an embodiment of this specification;

[0058] Figure 5B shows an exemplary result output for keyword extraction provided according to an embodiment of this specification;

[0059] Figure 6A is an exemplary prompt template for an extended question text provided according to an embodiment of this specification;

[0060] Figure 6B shows an exemplary output of an extended problem text provided according to an embodiment of this specification;

[0061] Figure 7A is an exemplary prompt template containing mind chain hint text provided according to an embodiment of this specification;

[0062] Figure 7B shows an exemplary output of a mind chain text prompt template provided according to an embodiment of this specification;

[0063] Figure 8 is a schematic diagram of a code generation device based on a code graph according to an embodiment of this specification. Detailed Implementation

[0064] The solutions provided in the embodiments of this specification will now be described with reference to the accompanying drawings.

[0065] In one or more embodiments of this specification, the Python programming language will be used as an example to illustrate the process by which a large language model generates corresponding code text for a problem text. Combined with the Python programming framework, a technical solution that can solve this problem will be explained. It should be noted that although Python is used as an example in some embodiments of this specification, this does not imply a limitation on the application scenarios or technical tools of the embodiments of this invention. The technical concepts embodied in the various embodiments of this specification can be applied to code projects built using various large language models that support code generation and their supported programming languages.

[0066] As mentioned earlier, to improve the efficiency of developers during the R&D process, Large Language Models (LLMs) have been applied to the automatic generation of program code. Thanks to training on massive amounts of code text, LLMs have shown considerable potential in understanding and generating code. Numerous related experimental results also demonstrate that LLMs can locate code and automatically generate target code text to complete the code task given in the question text input into the LLM.

[0067] In code generation applications, LLMs typically focus on function-level tasks. This means receiving a specified code snippet and a code task, and the LLM generates the corresponding target code around that snippet. However, in today's increasingly complex modern software engineering, there is a need for LLMs to comprehensively understand the structural and functional relationships between different source code files and function calls within a code repository, and to handle code generation tasks holistically based on the code context. This scenario is also known as "repository-level code generation." In this scenario, LLMs still have certain limitations in handling comprehensive code generation across files and function calls.

[0068] The reason for this is that repository-level code generation differs from small-scale function-level code generation. Function-level code generation often requires the LLM to first locate the source code context relevant to the target task from a large code repository. Then, based on this source code context, it understands the structural dependencies between codes, locates the target files that need modification, and finally generates target code text that satisfies the target task. However, due to the input length limitations of LLM, inputting the entire program's code repository into LLM is impractical. Even with some related technologies, the source code context initially located in the code repository may still fall into the category of long texts that LLM struggles to understand accurately. Furthermore, due to the high complexity of source code in large software engineering projects and the high uncertainty of target code generation tasks, the source code context selected from the code repository based on human experience, as disclosed in some related technologies, is highly likely to lose key information relevant to the code generation task and may also introduce a large amount of noise information unrelated to the code generation task. All of the factors listed above can affect the accuracy of target code generation by LLM in repository-level code generation scenarios.

[0069] To address the aforementioned technical problems, the inventors propose a code generation method based on a code graph. This method utilizes graph search technology to locate code within a code repository, assisting LLM in code generation and improving accuracy. Specifically, Figure 1 illustrates a framework of the code generation method based on a code graph in one embodiment. First, based on the user-provided question text regarding code in the code repository, the code graph of the repository is searched to obtain several nodes related to the question text and the directed edges between these nodes, forming a context subgraph. Since this subgraph is extracted from the code graph, it also carries various source code information from the code repository. Next, the context subgraph undergoes graph representation processing to obtain the representation vectors of each node. Finally, the question text is input into the embedding layer of a large language model for encoding. The resulting text representation, along with the representation vectors of each node in the context subgraph, is input into the inference layer of the large language model for inference, yielding the target code text for the question text. Since the representation vectors of each node in the input context subgraph can help LLM understand the semantic and structural information of the source code related to the problem text in the code repository during the inference process, implementing this technique can improve LLM's understanding of the code repository and enhance the accuracy of generating target code.

[0070] Following the above technical concept, Figure 2 shows a flowchart of a code generation method based on a code graph according to an embodiment of this specification. It is understood that this method can be executed by any device, equipment, platform, or cluster of devices with computing and processing capabilities. Referring to Figure 2, in one embodiment, the method includes at least the following steps: S201: Receive a first question text for code in a code repository. S203: Based on the first question text, perform a relevance search in a pre-constructed code graph to determine a first context subgraph, the code graph containing nodes representing code elements in the code repository and directed edges representing code relationships between nodes. S205: Perform graph representation processing on the first context subgraph to obtain a mapping representation of each node to the embedding space of a large language model. S207: Encode the first question text in the embedding layer of the large language model to obtain a text representation. S209: In the inference layer of the large language model, perform target inference based on the mapping representation and the text representation to obtain target code text for the first question text.

[0071] The specific execution method of each of the above steps will be described in detail below with reference to the accompanying drawings.

[0072] In step S201, a first issue text for the code in the code repository is received.

[0073] A code repository can contain program code written in any programming language, such as C++, Java, Python, etc. In some software engineering projects that employ hybrid programming methods, the code repository may also contain program code written in multiple programming languages.

[0074] In the embodiments described in this specification, Python code will be used as an example. However, it should be understood that one or more embodiments in this specification are intended to provide a code generation method based on code graphs, and are not limited to a specific programming language. The method provided by the embodiments of this invention can be applied to any scenario involving the technical concept provided in the embodiments of this invention.

[0075] The first issue text can be a natural language description of various issues related to the code in the code repository, such as: code location issues, bug fixing, adding new features, updating dependencies, etc. Depending on the issue, the first issue text can contain different contextual information corresponding to the issue. For example, when LLM is needed for code location, the first issue text can contain a functional description related to the code to be located; when LLM is needed for bug fixing, the first issue text can provide error messages and error outputs related to program runtime errors; when LLM is needed to generate code to add new features, the first issue text can include a functional description of the new features.

[0076] For example, taking fixing a code error as an example, the first problem text can be as shown in Figure 3. Referring to the figure, in this example, lines 1-3 of the first problem text provide a background overview of the problem. In a specific example, the source code that can reproduce the code problem can be given as context, for example:

[0077] from astropy.modeling import models as m

[0078] from astropy.modeling.separable import separability_matrix

[0079] cm=m.Linear1D(10)&m.Linear1D(5)

[0080] The source code shown above references two modules (astropy.modeling and astropy.modeling.separable) from the code repository and declares a model variable cm.

[0081] The expected output is given in rows 5-6 of the attached diagram. In a specific example, it could be:

[0082] The function `separability_matrix` is called to calculate the separability matrix based on the given variable parameters. Lines 8-9 of the attached diagram show the actual output of the program. In a specific example, it could be:

[0083] In this case, by equating the given variable parameters with the parameters of the model variable cm and replacing them with cm, the same function call outputs a matrix that is different from the expected result and is not separable.

[0084] Finally, in line 11 of the attached diagram, a summary description of the problem can be provided to instruct the LLM to analyze the error results generated by the code based on the first problem text and generate fix code text. In a specific example, the problem description could be:

[0085] "The above is a code defect in the program. The task is to analyze the code defect and provide the code text that can fix it."

[0086] This example demonstrates that the first question text can include various forms of text content. For instance, the description of the question may be given in natural language, while examples of program code (the [code snippet] in the attached figure) may be given in the target programming language. Leveraging the semantic understanding capabilities of a large language model, the first question text containing various formats can be processed. This specification does not specifically limit the text format of the input first question text in the embodiments.

[0087] In step S203, based on the first question text, a relevance search is performed in a pre-constructed code graph to determine a first context subgraph. The code graph contains nodes representing code elements in the code repository and directed edges representing code relationships between nodes.

[0088] As can be seen from the example in the first question above, real-world software engineering is usually not limited to a single function body or a single source code file. Application source code is typically built into a code repository containing multiple interdependent files, modules, and libraries. Therefore, when LLM performs code generation tasks on such projects, it needs a comprehensive understanding of the relationships between various program elements in the code repository, locating the target code file within a code repository containing massive amounts of source code files, and ensuring that the generated code can be seamlessly integrated into the code repository, completing the code generation task without introducing new problems. This highlights the difficulty of extending code generation tasks from a function-level understanding to a code repository-level understanding.

[0089] To help LLM quickly and accurately locate target code files, this step can abstract the complex code repository into a code graph structure. Then, based on similarity comparison, a contextual subgraph related to the first question text is extracted from the code graph. This subgraph can assist LLM in understanding the structural and semantic relationships between code elements in subsequent steps.

[0090] Next, we will briefly introduce the code diagram. A code repository mainly contains the structural and semantic relationships of the code. Structural relationships refer to the organization of the code, such as the hierarchical structure (directory hierarchy) between source code files, and the program architecture described by the various code elements written in the programming language (such as modules, classes, functions, etc.). Semantic relationships consist of the various code elements in the code repository and the code relationships between them, such as inheritance relationships between classes, calling relationships between functions, import relationships between modules, and reference relationships between variables, etc.

[0091] Figure 4 illustrates an exemplary code graph. As can be seen, during the construction of the code graph, code elements can be abstracted as nodes in the graph structure, and the code relationships between code elements can be abstracted as directed edges in the graph structure (for ease of understanding, structural relationships are identified with solid directed edges, and semantic relationships with dashed directed edges). Accordingly, a code repository can be abstracted as a code graph represented by a graph structure. In a specific implementation, the code elements include at least one of the following: code modules, code files, class definitions, method declarations, and variable declarations; the code relationships include at least one of the following: hierarchical relationships, import relationships, reference relationships, inheritance relationships, and call relationships.

[0092] The above is a brief explanation of the code graph. The method for constructing the code graph will be described in detail below, and will not be elaborated on here. Based on the code graph corresponding to the code repository, in this step, according to the first question text, several nodes and directed edges related to the first question text are selected from the code graph containing a large number of code elements and code relationships to form a context subgraph.

[0093] In graph search, there are various methods for retrieving related nodes and directed edges based on similarity.

[0094] According to one implementation, the relevance retrieval can be performed by comparing the cosine similarity or Euclidean distance between the node representation vector and the first question text representation vector to assess their relevance.

[0095] In specific application scenarios, as mentioned earlier, the initial question text may involve a variety of issues, such as code location, bug fixing, adding new features, and updating dependencies. However, the code information relevant to solving this problem is usually not explicitly provided in the initial question text. Furthermore, since the initial question text is primarily expressed in natural language, it may contain redundant information unrelated to the problem. These factors lead to a semantic gap between the initial question text and the actual source code. Therefore, relying solely on the initial question text for relevance retrieval in the code graph may not only miss some key information but also introduce noise, ultimately affecting the comprehensiveness and accuracy of the search results.

[0096] Therefore, during the relevance retrieval process, the semantics of the first question text can be enhanced to broaden its information coverage, aiming to construct a richer and more accurate question representation vector. Furthermore, the keywords in the first question text can be analyzed to extract key terms highly relevant to the problem-solving process, allowing for a greater focus on these key clues and the filtering out of noise during relevance retrieval.

[0097] According to one implementation, several first identifiers can be extracted from the first question text. These first identifiers include: code element names and / or functional descriptions. Specifically, a keyword extraction prompt template is designed to guide the LLM to extract key terms mentioned in the first question text, such as code element keywords like source code file names and function names, and descriptive keywords related to code functionality like `test_loop` and `train_loop`. Figure 5A provides an exemplary keyword extraction prompt template, in which the question description (in words such as source code file names, function names, etc.) is given. <issue>The output includes the first question text (labeled with tags), the code repository name (shown as [astropy / astropy] in the attached figure), the task instructions (the "Task Instructions" paragraph marked in gray in the attached figure), the output format requirements (the "Output in the following format" paragraph marked in gray in the attached figure), and other additional requirements, such as notes, examples of expected / unexpected output, etc. Corresponding to Figure 5A, Figure 5B provides an exemplary output result for keyword extraction. As can be seen, LLM outputs the extracted key terms (first identifier) ​​related to the first question text according to the output format given in the prompt template. These include: specific analysis information (labeled with [Analysis_Start], [Analysis_End] tags), extracted related code elements (labeled with [Related Code Element_Start], [Related Code Element_End] tags), and extracted related keywords (labeled with [Keyword_Start], [Keyword_End] tags).

[0098] After guiding the LLM to extract several first identifiers from the first question text using a prompt template, several related nodes are retrieved in the code graph based on fuzzy text matching. The text similarity between the code element names of these related nodes and the first identifiers is higher than a preset threshold. Specifically, a code graph is a graph data structure where each node represents a code element in a code repository, such as a file, function, class, or variable. By matching the first identifier with the code element names corresponding to the nodes and calculating their text similarity, it is possible to determine which nodes have a high degree of matching with the first identifiers. In this process, a similarity threshold can be set; only when the text similarity between the code element names of a node and the first identifiers exceeds the preset similarity threshold can the node be considered as a related node. By performing such a relevance retrieval, it is ensured that the retrieved related nodes are similar to the first question text in terms of keywords, and the code elements represented by these related nodes may be the key to solving the code generation task.

[0099] The above describes a relevance retrieval in the code graph using keyword extraction technology in one implementation. According to another implementation, the first question text can also be input into the first model for inference to obtain the first extended text. The first extended text includes functional descriptions of code elements related to the first question text. Specifically, a prompt template for the extended text content is designed to guide the LLM to infer the first question text, outputting several more specific functional descriptions of related code, refining the question description, and further narrowing the semantic distance between the first question text and the specific source code. The inference result serves as the first extended text for searching related nodes in the code graph. Figure 6A provides an exemplary prompt template for the extended question text, in which the question description (in words such as...) is given. <issue>The output includes the first problem text (labeled), the code repository name (shown as [astropy / astropy] in the figure), the task instructions (the "Task" paragraph marked in gray in the figure), the output format requirements (the "Output in the following format" paragraph marked in gray in the figure), and other additional requirements, such as notes, examples of expected / unexpected output, etc. Corresponding to Figure 6A, Figure 6B provides an exemplary output of an expanded problem text. It can be seen that LLM outputs the inferences derived from the first problem text according to the output format given in the prompt template. This includes: specific analysis information (labeled with [Analysis_Start], [Analysis_End]) and several expanded inferences (labeled with [Related Inference_Start], [Related Inference_End]). It is easy to see that LLM further expands the problem description based on the first problem text, decomposing it into multiple actionable inferences, such as finding code files that implement a specific function, identifying references to a specific function, etc.

[0100] After guiding the LLM to infer the first extended text based on the first question text using a prompt template, the first extended text and the code graph are input into the second model for encoding, resulting in a first code corresponding to the first extended text and semantic codes corresponding to the source code of each node in the code graph. In the code graph, several associated nodes are retrieved, and the representational similarity between the semantic codes corresponding to these associated nodes and the first codes is higher than a preset threshold. Specifically, the second model can encode the first extended text into a high-dimensional semantic vector (the first code), which captures the core semantic information of the text; simultaneously, the second model also encodes each node in the code graph into a high-dimensional semantic vector (the semantic code), which reflects the semantic information of the source code text represented by the node. By matching the first code with the semantic codes corresponding to the nodes and calculating their representational similarity, it is possible to determine which nodes have a high degree of matching with the first extended text. In this process, a similarity threshold can be set; only when the representational similarity between a node and the first extended text exceeds the preset similarity threshold can the node be considered an associated node. By performing such a relevance retrieval, it is ensured that the retrieved associated nodes and the first question text are similar in semantics of the explicit text or the inferred text. In one application scenario, an encoding model suitable for code can be used to encode the first extended text and the code graph. Specifically, the second model is the CGE-Large model.

[0101] In some specific practices, the relevance retrieval methods exemplified in the two implementation methods above can be combined. That is, during the relevance retrieval of the code graph, not only is the search based on keywords extracted from the first question text, but also the inference results from the first question text, i.e., the expanded text, are used for deeper retrieval. This composite relevance retrieval allows for a more comprehensive capture of the semantic information of the first question text, improving the depth and breadth of the relevance retrieval.

[0102] The current execution steps also involve the construction of a context subgraph. Specifically, by performing the relevance retrieval described above, several related nodes relevant to the first question text can be extracted from the code graph. Based on these related nodes, the context subgraph can be extracted from the code graph.

[0103] In one implementation, the associated nodes and the directed edges between them can be directly extracted to form a subgraph, namely the context subgraph.

[0104] In another implementation, considering that the call and called relationships between the code elements represented by nodes play an important guiding role in code generation, a target search can be performed on each associated node in the code graph by expanding along directed edges based on the associated nodes obtained from the relevance retrieval. The resulting nodes and their corresponding directed edges can form a subgraph, namely the context subgraph. This implementation can be summarized as follows: based on the associated nodes obtained from the relevance retrieval, a target search is performed, which includes, for any associated node, searching along directed edges in the code graph within a preset number of hops to obtain several extended nodes; and the first context subgraph is constructed based on the associated nodes and their corresponding extended nodes.

[0105] In one application scenario of this implementation, the target search is a breadth-first N-hop BFS search, where N is a preset range of hop counts. Typically, N is 1, meaning that each associated node and its one-hop neighbor are included in the nodes of the context subgraph.

[0106] After the above steps, the code graph, which originally contained a massive number of code elements and complex code relationships, has been effectively simplified into a context subgraph closely related to the first question text. Next, in step S205, the first context subgraph is subjected to graph representation processing to obtain the mapping representation of each node to the large language model embedding space.

[0107] In this step, in order to input the information contained in the context subgraph into the LLM code generation task, each node in the context subgraph can be encoded, and the encoded representations obtained from each node can be aligned semantically with the input dimension of the LLM. Therefore, in one implementation, the code text content contained in each node of the first context subgraph is first represented to obtain the encoded representation corresponding to each node; then, each encoded representation is input into a mapping network for mapping processing to obtain the mapped representation.

[0108] Since the text content contained in the nodes of the context subgraph is mainly source code, and the encoder often faces input length limitations when representing nodes, nodes containing long source code cannot have their entire source code input to the encoder during representation processing. This results in the encoded representation not fully reflecting the node's characteristics. Therefore, in one implementation, when any node is detected to contain long target code text, the target code text content can be segmented into multiple text blocks. Then, each text block is represented separately to obtain its corresponding text block encoded representation. Finally, based on the text block encoded representations, the encoded representation of the node is determined, thus completing the representation processing of the node. The text segmentation can be performed in various ways. For example, the target code text content can be segmented into multiple text blocks using a preset text length as the unit segmentation length, or it can be segmented into multiple text blocks using preset text delimiters (such as semicolons, newlines, curly braces, etc.) as segmentation points.

[0109] The mapping representations of each node in the first context subgraph obtained in this step will be input into the LLM in subsequent steps to perform code generation tasks for the first problem text.

[0110] Next, in steps S207 and S209, the first question text is encoded in the embedding layer of the large language model to obtain a text representation. In the inference layer of the large language model, target inference is performed based on the mapping representation and the text representation to obtain the target code text for the first question text.

[0111] Similar to the representation processing of the nodes of the first context subgraph in the previous step, the first question text also needs to be encoded into a semantic space that the LLM can understand. Therefore, the first question text is input into the embedding layer of the LLM for encoding to obtain a text representation aligned with the LLM input dimension.

[0112] Next, the text representation corresponding to the first question text can be concatenated with the mapping representations of each node in the first context subgraph, and the concatenated representation vector is input into the inference layer of the LLM. The LLM integrates the source code structure and semantic information provided by the first context subgraph, as well as the semantic content in the first question text, to perform comprehensive inference, identify code fragments related to the code task described by the first question text, and finally generate target code text that can achieve the code task.

[0113] In some implementations, the attention mechanism of the LLM can be enhanced to improve its ability to understand the structure of the source code. That is, the large language model can be used, with the adjacency matrix of the code graph as an attention mask, to perform attention-based encoding processing.

[0114] Specifically, in the attention mechanism of autoregressive large language models, a causal attention mask is typically used to ensure that the prediction at each position depends only on the preceding token. However, when using large language models to process code graphs, the structural information carried by the graph structure can also be incorporated into the attention mechanism to enhance the large language model's ability to understand code structure. The adjacency matrix describes the connections between nodes in the code graph and can be used as an attention mask input to the large language model. Using the adjacency matrix for attention-based reasoning allows the large language model to better capture structural information in the code graph, focusing on neighboring nodes directly connected to the current node. This leads to a deeper understanding of the call relationships, inheritance relationships, and so on between various code elements, further improving the performance of the large language model in code understanding and code generation tasks.

[0115] Furthermore, to further improve the accuracy of LLM in locating critical code files, the LLM can be guided to pay more attention to code files most likely to require modification during the input phase. Therefore, in addition to enhancing the LLM attention mechanism described above, the accuracy of LLM in locating code files can be improved by prioritizing the code files input by the LLM, which are selected through a specific ranking strategy. In other words, although the context subgraph can limit the code repository information that the LLM needs to refer to to a certain range, it still involves dozens or even hundreds of nodes representing different code files.

[0116] In order to enable the LLM to focus on the code files most likely to be modified during code generation, in one implementation, the file nodes in the first context subgraph can be sorted based on their relevance to the first question text to determine a number of first code files; then, in the inference layer of the large language model, target inference is performed based on the first code files, the mapping representation, and the text representation.

[0117] In this implementation, the relevance ranking can be evaluated and ranked by calculating the representational similarity between each associated node and the first question text. The code files represented by the top K associated nodes in the ranking result are classified as the first code files. The K first code files selected through ranking serve as one of the inputs for the LLM in subsequent steps when performing the target inference. They can help the LLM more accurately locate those code files that are crucial to solving the code task, thereby improving the efficiency and accuracy of the LLM in understanding the code repository and generating target code text that better meets expectations.

[0118] In a specific application scenario, the relevance ranking can also be achieved by designing a prompt template containing thought chain hint text, instructing the large model to perform step-by-step reasoning from the associated nodes to select the first code file. Figure 7A provides an exemplary prompt template containing thought chain hint text, in which the problem description (in words) is given. <issue>The first question text is labeled with tags, the code repository name (shown as [astropy / astropy] in the figure), the file list corresponding to the associated nodes contained in the first context subgraph (the "The following are the files retrieved from this repository" paragraph in the figure), the task instructions (the "Task" paragraph marked in gray in the figure), the output format requirements (the "Output in the following format" paragraph marked in gray in the figure), and other additional requirements, such as notes, expected / unexpected output examples, etc. It can be seen that the task instructions paragraph applies the concept of a thought chain, describing the sorting task step by step to guide the LLM in progressive reasoning. Corresponding to Figure 7A, Figure 7B provides an exemplary result output for a thought chain text prompt template. It can be seen that the LLM, following the output format given in the prompt template and the task instructions, progressively analyzes from the given file list, selects the first code file, and outputs the specific reasoning process and the first code file, including: specific analysis information (packaged with [Analysis_Start], [Analysis_End] tags), and the selected first code file (packaged with [Related Files_Start], [Related Files_End] tags). These initial code files are input into the LLM during subsequent target inference to help the LLM pay more attention to them.

[0119] Furthermore, when outputting target code text, LLM outputs different types of target code text depending on the purpose of the code task indicated by the first problem text. In one implementation, if the target code text contains target modifications to historical code, these modifications are packaged and committed. In some code modification-related tasks (e.g., fixing code defects, adding new features), the target code text generated by LLM needs to be merged into the source code of the code repository to be effective. In this scenario, the generated target code text can be output in Git Patch format (usually generated by the Git format-patch command), and the patch file can be applied to the current code repository using Git Apply, thus completing the code generation task for the first problem text. In downstream work, the correctness of the patch file can be verified by executing test cases.

[0120] The above describes the main flow of a code generation method based on a code graph provided in the embodiments of this specification. In some embodiments of this specification, a method for constructing a code graph is also provided. This construction method includes at least the following steps: constructing a first abstract syntax tree based on the first source code contained in the code repository; traversing the first abstract syntax tree and adding each node representing a code element and the directed edges representing the structural relationships between nodes to the code graph; performing semantic analysis on the code elements contained in the first source code to obtain several first semantic relationships; and connecting source nodes and target nodes through directed edges in the code graph according to the first semantic relationships.

[0121] Specifically, an Abstract Syntax Tree (AST) is a data structure that displays the various code elements and their relationships in a source code file in a tree structure. Each node represents a code element in the source code, such as a module, class declaration, function definition, etc. The connections between nodes reflect the syntactic hierarchy and code relationships between code elements, such as function calls, variable references, etc.

[0122] An AST analysis of the code files in the code repository yields the first abstract syntax tree (AST) corresponding to each module and package. Traversing the AST, each node representing a code element is added to the code graph, and the connections representing the structural relationships between these nodes are added as directed edges. The code graph is denoted as: G = (V, E) (1)

[0123] Wherein, V is the set of nodes representing code elements in the code graph, and E is the set of directed edges representing the code relationships between nodes in the code graph.

[0124] According to a specific implementation, after performing the above AST traversal operation, the set of nodes and the set of directed edges in the code graph can be represented as: V = {repo}∪V mod ∪V file ∪V class ∪V func ∪V field (2) E = E contains ∪E imports ∪E extends (3)

[0125] Where {repo} represents the current code repository, V mod V represents a module / library. file V represents a code file. class V represents a class. func V represents the function definition. field Indicates a field / variable. E contains Represents the directory hierarchy between source code files in a code repository, E imports This represents the import relationships between modules, classes, and methods in the source code file. extends It represents the inheritance relationship between classes.

[0126] For example, suppose the current code repository `repo` contains a code module `mod`, which contains a source code file `file` that defines class `c`, which inherits from class `d`. Correspondingly, in a code diagram, this can be represented as:

[0127] Based on the construction of the code graph through AST analysis, the semantic relationships between the nodes in the code graph can be analyzed based on the source code (the semantic analysis), the code relationships between code elements (the first semantic relationship) can be parsed, and the code graph can be added as directed edges.

[0128] According to one specific implementation, after performing the above semantic analysis, the set of directed edges in the code graph can be updated to: E = E contains ∪E imports ∪E extends ∪E calls ∪E uses (5)

[0129] Among them, E calls E represents the calling relationship between code elements. uses This represents the reference / usage relationship of code elements.

[0130] For example, suppose function m1 calls function m2, function m2 uses the variable field f, and this variable field f is also used by function m3. Correspondingly, in a code diagram, this can be represented as:

[0131] By performing the above steps, AST analysis and semantic analysis are conducted on each source code file in the code repository, and a code graph corresponding to the code repository can be gradually constructed. Each node represents a code element, and the directed edges represent the code relationships between code elements.

[0132] The foregoing description, based on one or more embodiments, details a code generation method based on a code graph. Using the method provided in the embodiments of this specification, during the LLM's code generation task, graph analysis is performed on the code graph representing the code repository based on the problem text. Contextual subgraphs related to the problem text are extracted and input into the LLM, improving the accuracy of the LLM's understanding of the code structure and correctly generating target code text for the problem text.

[0133] In this specification, the terms "first" in the first problem text, the first model, etc., as well as the corresponding "second" and "third" (if they exist) in the text, are merely for the convenience of distinction and description, and do not have any limiting meaning.

[0134] The foregoing description describes specific embodiments of this specification; other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than those shown in the embodiments, and the desired result may still be achieved. Furthermore, the processes depicted in the drawings do not necessarily need to follow the specific or sequential order shown to achieve the desired result. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

[0135] Figure 8 is a schematic diagram of a code generation device based on a code graph according to an embodiment of this specification. The device 800 is deployed in a computing device, which can be implemented using any device, equipment, platform, device cluster, etc., with computing and processing capabilities. This device embodiment corresponds to the method embodiment shown in Figure 2. The device 800 includes:

[0136] The receiving module 801 is configured to receive the first question text for the code in the code repository.

[0137] The retrieval module 802 is configured to perform a relevance search in a pre-constructed code graph based on the first question text to determine a first context subgraph, wherein the code graph contains nodes representing code elements in the code repository and directed edges representing code relationships between nodes.

[0138] The first encoding module 803 is configured to perform graph representation processing on the first context subgraph to obtain a mapping representation of each node to the large language model embedding space.

[0139] The second encoding module 804 is configured to encode the first question text in the embedding layer of the large language model to obtain a text representation.

[0140] The reasoning module 805 is configured to perform target reasoning based on the mapping representation and the text representation in the reasoning layer of the large language model to obtain target code text for the first question text.

[0141] According to another embodiment, this specification also provides a computer program product including a computer program / instructions that, when executed by a processor, implement the steps of the method described above in conjunction with FIG2.

[0142] According to yet another embodiment, this specification also provides a computing device including a memory and a processor, characterized in that the memory stores executable code, and when the processor executes the executable code, it implements the steps of the method described above in conjunction with FIG2.

[0143] Those skilled in the art will recognize that the functions described in the embodiments of the present invention in one or more of the above examples can be implemented using hardware, software, firmware, or any combination thereof. When implemented in software, these functions can be stored in a computer-readable medium or transmitted as one or more instructions or code on a computer-readable medium.

[0144] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above descriptions are merely specific embodiments of the present invention and are not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, or improvements made based on the technical solutions of the present invention should be included within the scope of protection of the present invention.< / issue> < / issue> < / issue>

Claims

1. A code generation method based on code graphs, comprising: Receive the first question text for the code in the code repository; Based on the first question text, a relevance search is performed in the pre-constructed code graph to determine the first context subgraph. The code graph contains nodes representing code elements in the code repository and directed edges representing code relationships between nodes. The first context subgraph is processed by graph representation to obtain the mapping representation of each node to the embedding space of the large language model. In the embedding layer of the large language model, the first question text is encoded to obtain a text representation; In the inference layer of the large language model, target inference is performed based on the mapping representation and the text representation to obtain the target code text for the first question text.

2. The method of claim 1, wherein, The code elements include at least one of the following: code module, code file, class definition, method declaration, variable declaration; the code relationships include at least one of the following: hierarchical relationship, import relationship, reference relationship, inheritance relationship, and call relationship.

3. The method of claim 1, wherein, The first issue text includes one of the following: code location, bug fix, new feature addition, or dependency update.

4. The method of claim 1, wherein, Determining the first context subgraph includes: Based on the related nodes obtained by the relevance retrieval, a target search is performed. The target search includes, for any related node, searching along the directed edges in the code graph within a preset number of hops to obtain several extended nodes. The first context subgraph is constructed based on the aforementioned associated nodes and their respective extended nodes.

5. The method of claim 1, wherein, The relevance retrieval in the pre-built code graph includes: Extract several first identifiers from the first question text. The first identifiers include code element names and / or function description words. In the code graph, based on text fuzzy matching, several associated nodes are retrieved, and the text similarity between the code element name of the associated node and the first identifier is higher than a preset threshold.

6. The method of claim 1, wherein, The relevance retrieval in the pre-built code graph includes: The first question text is input into the first model for inference to obtain the first extended text, which includes the functional description text of the code elements related to the first question text; The first extended text and the code graph are input into the second model for encoding to obtain the first encoding corresponding to the first extended text and the semantic encoding corresponding to the source code of each node in the code graph. In the code graph, several associated nodes are retrieved, and the semantic code corresponding to the associated node has a representation similarity to the first code that is higher than a preset threshold.

7. The method of claim 6, wherein, The second model is the CGE-Large model.

8. The method according to claim 1, further comprising: For the file nodes in the first context subgraph, several first code files are determined based on their relevance to the first question text; The target reasoning includes: performing target reasoning based on the first code file, the mapping representation, and the text representation in the reasoning layer of the large language model.

9. The method of claim 8, wherein, The determination of several first code files includes: The file nodes and thought chain hint text in the first context subgraph are input into the third model for reasoning to select the several first code files; the thought chain hint text is used to instruct the third model to perform reasoning in multiple steps to gradually determine the code files corresponding to the K file nodes with the highest relevance to the first question text.

10. The method of claim 1, wherein, The graph representation processing of the first context subgraph includes: The code text content contained in each node of the first context subgraph is represented to obtain the encoding representation of each node. Each encoded representation is input into a mapping network for mapping processing to obtain the mapped representation.

11. The method of claim 10, wherein, The code graph includes a first node, and the target code text content contained in the first node is long text; the process of representing the code text content contained in each node of the first context subgraph includes: The target code text content is segmented to obtain multiple text blocks; Each text block is processed to obtain the text block encoding representation corresponding to each text block; Based on the encoding representation of each text block, the encoding representation of the first node is determined.

12. The method of claim 1, wherein, The target reasoning includes: Using the large language model, the adjacency matrix of the code graph is used as an attention mask to perform attention-based encoding processing.

13. The method of claim 1, wherein, The method further includes: If the target code text contains target modifications to historical code, then the target modifications are packaged and submitted.

14. The method of claim 1, wherein, The code relationships include structural relationships and semantic relationships. The code graph is constructed through the following steps: Construct a first abstract syntax tree based on the first source code contained in the code repository; Traverse the first abstract syntax tree and add each node representing a code element and the directed edges representing the structural relationships between nodes to the code graph. Semantic analysis is performed on the code elements contained in the first source code to obtain several first semantic relationships; In the code graph, the source node and the target node are connected by directed edges according to the first semantic relationship.

15. A code generation apparatus based on a code graph, the apparatus comprising: The receiving module is configured to receive the first question text for the code in the code repository. The retrieval module is configured to perform a relevance search in a pre-constructed code graph based on the first question text to determine a first context subgraph, wherein the code graph contains nodes representing code elements in the code repository and directed edges representing code relationships between nodes; The first encoding module is configured to perform graph representation processing on the first context subgraph to obtain a mapping representation of each node to the large language model embedding space. The second encoding module is configured to encode the first question text in the embedding layer of the large language model to obtain a text representation; The reasoning module is configured to perform target reasoning based on the mapping representation and the text representation in the reasoning layer of the large language model to obtain target code text for the first question text.

16. A computer program product comprising a computer program / instructions that, when executed by a processor, implement the steps of the method according to any one of claims 1-14.

17. A computing device comprising a memory and a processor, wherein: The memory stores executable code, and the processor executes the executable code to implement the method in any one of claims 1-14.

Images