Code generation method and device based on code diagram

Abstract

The embodiment of the invention provides a code generation method based on a code graph. The code generation method comprises the steps that a first problem text for codes in a code warehouse is received; according to the first problem text, correlation retrieval is carried out in a pre-constructed code graph, a first context sub-graph is determined, and the code graph comprises nodes representing code elements in the code warehouse and directed edges representing code relations between the nodes. And performing graph representation processing on the first context sub-graph to obtain mapping representation of each node mapped to a large language model embedding space. And in an embedded layer of the large language model, encoding the first problem text to obtain text representation. And in a reasoning layer of the large language model, performing target reasoning based on the mapping representation and the text representation to obtain a target code text for the first problem text.

Description

Technical Field

[0001] One or more embodiments of the present specification relate to the technical field of large language models and code generation, and in particular, to a code generation method and device based on a code graph. Background Art

[0002] With the rapid development of information technology, the software industry is facing unprecedented challenges, including the continuous increase in the complexity of software projects and the rapid iteration of market demand. In this context, the industry has developed a variety of tools that can assist developers in code writing. For example, the Large Language Model (LLM) represented by ChatGPT has been widely used in code generation scenarios in recent years.

[0003] Code generation technology refers to the automatic generation of program code by the model based on the input problems and requirements, thereby improving the coding efficiency of developers. Early code generation technology was mainly aimed at simple business scenarios, generating small-scale code to implement a single function in a single code file, such as generating a sorting algorithm, generating SQL query fragments, and so on.

[0004] However, as the complexity of software projects continues to increase, the reference and calling relationships between the functional modules contained in the source code files become intricate. Traditional code generation technology finds it difficult to understand the code structure in modern software projects, resulting in errors in identifying and locating code files that need to be modified, which in turn affects the accuracy of code generation.

[0005] Therefore, we hope to have a solution that can improve the model's ability to understand the code structure through technical means, so as to accurately locate the code files that need to be modified and improve the accuracy of code generation. Summary of the invention

[0006] One or more embodiments of this specification describe a code graph-based code generation method and device to solve the problem that the existing large language model has difficulty understanding the code structure in the code repository, improve the large language model's ability to locate the target code file, and thus better complete the repository-level code generation task.

[0007] According to a first aspect, a code generation method based on a code graph is provided, comprising:

[0008] A first question text is received for a code in a code repository.

[0009] According to the first question text, a relevance search is performed in a pre-constructed code graph to determine a first context subgraph, wherein the code graph includes nodes representing code elements in the code repository and directed edges representing code relationships between nodes.

[0010] The first context subgraph is subjected to graph representation processing to obtain a mapping representation of each node mapped to an embedding space of a large language model.

[0011] In the embedding layer of the large language model, the first question text is encoded to obtain a text representation.

[0012] In the reasoning layer of the large language model, target reasoning is performed based on the mapping representation and the text representation to obtain a target code text for the first question text.

[0013] According to one implementation, the code elements include at least one of the following: code modules, code files, class definitions, method declarations, and variable declarations; the code relationships include at least one of the following: hierarchical relationships, import relationships, reference relationships, inheritance relationships, and call relationships.

[0014] According to one implementation, the first problem text includes one of the following: code location, error repair, new function addition, and dependency update.

[0015] According to an implementation manner, determining the first context subgraph includes:

[0016] Based on the several associated nodes obtained by the correlation retrieval, a target search is performed, wherein the target search includes, for any associated node, searching along the directed edge in the code graph within a preset hop count range to obtain several extended nodes;

[0017] The first context subgraph is constructed according to the plurality of associated nodes and their corresponding extended nodes.

[0018] According to one implementation, performing relevance retrieval in a pre-built code graph includes:

[0019] Extract a plurality of first identifiers from the first question text, wherein the first identifiers include code element names and / or function description words; in the code graph, retrieve a plurality of associated nodes based on text fuzzy matching, wherein the text similarity between the code element names of the associated nodes and the first identifiers is higher than a preset threshold.

[0020] According to one implementation, performing relevance retrieval in a pre-built code graph includes:

[0021] The first question text is input into a first model for inference to obtain a first extended text, wherein the first extended text includes a functional description text of code elements related to the first question text; the first extended text and the code graph are input into a second model for encoding to obtain a first code corresponding to the first extended text and a semantic code corresponding to the source code of each node in the code graph; in the code graph, a number of associated nodes are retrieved, and the representation similarity between the semantic code corresponding to the associated nodes and the first code is higher than a preset threshold.

[0022] In one scenario of the above implementation, the second model is a CGE-Large model.

[0023] According to one implementation, the method further includes:

[0024] For the file nodes in the first context subgraph, a plurality of first code files are determined based on their relevance ranking with the first question text. The target reasoning includes: in the reasoning layer of the large language model, target reasoning is performed based on the first code file, the mapping representation and the text representation.

[0025] In one scenario of the above implementation, the determining of the plurality of first code files includes:

[0026] The file nodes and thought chain prompt text in the first context subgraph are input into the third model for reasoning to select the first code files; the thought chain prompt text is used to instruct the third model to perform reasoning in multiple steps to gradually determine the code files corresponding to the K file nodes with the highest relevance to the first question text.

[0027] According to an implementation manner, performing graph representation processing on the first context subgraph includes:

[0028] The code text content contained in each node in the first context subgraph is represented and processed to obtain the encoding representation corresponding to each node; each encoding representation is input into the mapping network for mapping processing to obtain the mapping representation.

[0029] In one scenario of the above implementation, the code graph includes a first node, and the target code text content included in the first node is a long text; the characterization processing of the code text content included in each node in the first context subgraph includes:

[0030] The target code text content is segmented to obtain multiple text blocks.

[0031] Each text block is represented and processed to obtain a text block encoding representation corresponding to each text block.

[0032] Based on the encoding representations of each text block, the encoding representation of the first node is determined.

[0033] According to one implementation, the target reasoning includes:

[0034] Using the large language model, the adjacency matrix of the code graph is used as an attention mask to perform attention-based encoding processing.

[0035] According to one implementation, the method further includes:

[0036] If the target code text contains target modifications to the historical code, the target modifications are packaged and submitted.

[0037] According to one implementation, the code relationship includes a structural relationship and a semantic relationship, and the code graph is constructed by the following steps:

[0038] A first abstract syntax tree is constructed according to the first source code included in the code repository.

[0039] The first abstract syntax tree is traversed, and each node representing a code element and a directed edge representing a structural relationship between nodes are added to the code graph.

[0040] Semantic analysis is performed on code elements included in the first source code to obtain a plurality of first semantic relationships.

[0041] In the code graph, according to the first semantic relationship, the source node and the target node are connected through a directed edge.

[0042] According to a second aspect, a code generation device based on a code graph is provided, the device comprising:

[0043] The receiving module is configured to receive a first question text for the code in the code repository.

[0044] The retrieval module is configured to perform relevance retrieval in a pre-constructed code graph according to the first question text to determine a first context subgraph, wherein the code graph includes nodes representing code elements in the code repository and directed edges representing code relationships between nodes.

[0045] The first encoding module is configured to perform graph representation processing on the first context subgraph to obtain a mapping representation of each node mapped to the embedding space of the large language model.

[0046] The second encoding module is configured to encode the first question text in the embedding layer of the large language model to obtain a text representation.

[0047] The reasoning module is configured to perform target reasoning based on the mapping representation and the text representation in the reasoning layer of the large language model to obtain a target code text for the first question text.

[0048] According to a third aspect, a computer program product is provided, comprising a computer program / instruction, which, when executed by a processor, implements the steps of the method described in the first aspect.

[0049] According to a fourth aspect, a computing device is provided, comprising a memory and a processor, wherein an executable code is stored in the memory, and when the processor executes the executable code, the method described in the first aspect is implemented.

[0050] In summary, the above method and device disclosed in the embodiments of this specification can be used to perform graph analysis on the code graph representing the code repository based on the question text, extract the context subgraph related to the question text, and input it into the large language model to improve the accuracy of the large language model in understanding the code structure and correctly generate the target code text for the question text. The embodiments of this specification can solve the problem that the existing large language model is difficult to understand the code structure due to the long code repository context, and help the model better complete the repository-level code task. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention, and for ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0052] Figure 1 A schematic diagram of a code generation method framework based on a code graph disclosed in this specification;

[0053] Figure 2 A flow chart of a code generation method based on a code graph provided according to an embodiment of this specification;

[0054] Figure 3 An exemplary first question text provided for the embodiments of this specification;

[0055] Figure 4 An exemplary code diagram provided according to an embodiment of this specification;

[0056] Figure 5A An exemplary prompt template for extracting keywords provided according to an embodiment of this specification;

[0057] Figure 5BAn exemplary result output of extracting keywords according to an embodiment of this specification is provided;

[0058] Fig. 6A An exemplary prompt template for an extended question text provided according to an embodiment of the present specification;

[0059] Figure 6B An exemplary result output of an extended question text provided according to an embodiment of this specification;

[0060] Fig. 7A An exemplary prompt template including a thought chain prompt text provided according to an embodiment of this specification;

[0061] Figure 7B An exemplary result output for a thought chain text prompt template provided according to an embodiment of this specification;

[0062] Figure 8 The present invention is a schematic diagram of a code generation device based on a code graph according to an embodiment of the present specification. DETAILED DESCRIPTION

[0063] The solution provided by the embodiments of this specification is described below in conjunction with the accompanying drawings.

[0064] In one or more embodiments of this specification, the Python programming language will be used as an example to explain the process of a large language model generating corresponding code text for a problem text, and in combination with the Python programming framework, a technical solution that can solve the problem will be explained. It should be noted that although Python is used as an example in some embodiments of this specification, it does not represent a limitation on the application scenarios and technical tools of the embodiments of the present invention. The technical concepts embodied in the various embodiments of this specification can be applied to code projects constructed by various large language models that support code generation and the programming languages ​​they support.

[0065] As mentioned above, in order to improve the efficiency of developers in the R&D process, large language models (LLMs) have been applied to the automatic generation of program code. Thanks to the massive code text training, LLMs have shown certain potential in understanding and generating code. A large number of relevant experimental results also show that LLMs can complete code location and automatically generate target code texts to complete the code tasks given in the problem text input to LLMs.

[0066] In code generation applications, LLM usually focuses on processing function-level tasks, that is, receiving a specified code snippet and a code task, and LLM generates the corresponding target code around the code snippet. However, in today's increasingly complex modern software engineering, there is a need for LLM to be able to comprehensively understand the structural and functional relationships between different source code files and function calls in the code repository, and to fully process code generation tasks according to the code context. This scenario is also called "repository-level code generation". In this scenario, LLM still has certain limitations in processing comprehensive code generation across files and function calls.

[0067] The reason is that repository-level code generation is different from small-scale function-level code generation. It often requires LLM to first preliminarily locate the source code context related to the target task from a large code repository, and then based on the source code context, understand the structural dependencies between the codes, locate the target files that need to be modified, and then generate the target code text that can meet the target task. However, due to the input length limit of LLM, it is unrealistic to input the entire code repository of the program into LLM. Even if the source code context preliminarily located in the code repository through some related technologies, it may still belong to the category of long text that is difficult for LLM to accurately understand. In addition, due to the high complexity of the source code of large-scale software engineering and the high uncertainty of the target code generation task, the source code context that is disclosed in some related technologies and screened in the code repository based on manual experience is very likely to lose key information related to the code generation task, and may also introduce a large amount of noise information that is irrelevant to the code generation task. Many of the factors listed above may affect the accuracy of LLM in generating target code in the repository-level code generation task scenario.

[0068] In order to solve the above technical problems, the inventors proposed a code generation method based on code graph, which can use graph search technology to locate code in the code warehouse, assist LLM in code generation, and improve accuracy. Specifically, Figure 1The framework of a code generation method based on a code graph in an embodiment is disclosed. First, based on the question text given by the user for the code in the code warehouse, the code graph of the code warehouse is searched to obtain several nodes related to the question text and directed edges between the nodes to form a context subgraph. Since the subgraph is extracted from the code graph, it also carries various source code information of the code warehouse. After that, the context subgraph is processed by graph representation to obtain the representation vector of each node. Finally, the question text will be input into the embedding layer of the large language model for encoding, and the obtained text representation and the representation vector of each node in the context subgraph will be input into the reasoning layer of the large language model for reasoning to obtain the target code text for the question text. Since the representation vectors of each node of the input context subgraph can assist LLM in understanding the semantics and structural information of the source code related to the question text in the code warehouse during the reasoning process of LLM, the implementation of this technical method can improve LLM's understanding of the code warehouse and improve the accuracy of generating target code.

[0069] Following the above technical ideas, Figure 2 In FIG. 1 , a flow chart of a code generation method based on a code graph according to an embodiment of the present specification is shown. It can be understood that the method can be executed by any device, equipment, platform, or device cluster with computing and processing capabilities. Figure 2 In one embodiment, the method includes at least the following steps. S201: Receive a first question text for a code in a code repository. S203: Based on the first question text, perform a relevance search in a pre-constructed code graph to determine a first context subgraph, wherein the code graph includes nodes representing code elements in the code repository and directed edges representing code relationships between nodes. S205: Perform graph representation processing on the first context subgraph to obtain a mapping representation of each node mapped to the embedding space of a large language model. S207: In the embedding layer of the large language model, encode the first question text to obtain a text representation. S209: In the reasoning layer of the large language model, perform target reasoning based on the mapping representation and the text representation to obtain a target code text for the first question text.

[0070] The specific implementation methods of the above steps will be described in detail below with reference to the accompanying drawings.

[0071] In step S201, a first question text for a code in a code repository is received.

[0072] The code contained in the code repository can be program code written in any programming language, for example, C++, Java, Python, etc. In some software projects that apply hybrid programming methods, the code repository can also contain program code written in multiple programming languages.

[0073] In the embodiments of this specification, Python code will be used as an example for explanation. However, it should be understood that one or more embodiments in this specification are intended to provide a code generation method based on a code graph, and are not limited to a specific programming language. The method provided by the embodiments of the present invention can be applied to any scenario related to the technical concept provided by the embodiments of the present invention.

[0074] The first problem text may be a natural language description of various problems related to the code repository code, such as: code location problems, error repair, adding new functions, updating dependencies, etc. Depending on the problem, the first problem text may contain different context information corresponding to the problem. For example, when LLM is needed to locate the code, the first problem text may include a functional description related to the code to be located; when LLM is needed to fix errors, the first problem text may provide error information and error output related to program running errors; when LLM is needed to generate code to add new functions, the first problem text may include a functional description of the new function.

[0075] For example, taking fixing a code error as an example, the first question text may be Figure 3 Referring to the attached figure, in this example, lines 1 to 3 of the first question text provide a background overview of the problem. In a specific example, source code that can reproduce the code problem can be provided as context, for example:

[0076] from astropy.modeling import models as m

[0077] from astropy.modeling.separable import separability_matrix

[0078] cm=m.Linear1D(10)&m.Linear1D(5)

[0079] In the source code shown above, two modules in the code repository (astropy.modeling, astropy.modeling.separable) are referenced, and a model variable cm is declared.

[0080] In lines 5-6 of the accompanying figure, the expected output result is given. In a specific example, it can be:

[0081] >>>separability_matrix(m.Pix2Sky_TAN()&m.Linear1D(10)&m.Linear1D(5))

[0082] array([[True,True,False,False],

[0083] [True,True,False,False],

[0084] [False,False,True,False],

[0085] [False,False,False,True]])

[0086] The function separability_matrix is ​​called to calculate the given variable parameters and output the separable matrix. The actual output of the program is given in lines 8-9 of the attached figure. In a specific example, it can be:

[0087] >>>separability_matrix(m.Pix2Sky_TAN()&cm)

[0088] array([[True,True,False,False],

[0089] [True,True,False,False],

[0090] [False,False,True,True],

[0091] [False,False,True,True]])

[0092] Among them, the given variable parameters are equivalent to the parameters of the model variable cm. After replacing them with cm, the same function call outputs a matrix that is different from the expected result and has no separability.

[0093] Finally, in line 11 of the accompanying figure, a summary description of the problem can be given to instruct LLM to analyze the error results generated by the code based on the first problem text and generate a repair code text. In a specific example, the problem description can be:

[0094] "The above is a code defect in the program. The task is to analyze the code defect and give the code text that can fix the defect."

[0095] It can be seen from this example that the first question text may include text content in various forms, for example, the description of the question is given in natural language, and the example of the program code (the [code segment] in the accompanying figure) is given in the target programming language. With the semantic understanding ability of the large language model, the first question text containing text content in various formats can be processed. The embodiments of this specification do not specifically limit the text format of the input first question text.

[0096] In step S203, a relevance search is performed in a pre-constructed code graph according to the first question text to determine a first context subgraph, wherein the code graph includes nodes representing code elements in the code repository and directed edges representing code relationships between nodes.

[0097] It can also be seen from the example in the first question text above that real-world software engineering is usually not limited to a single function body or a single source code file. The source code of an application is usually built into a code repository containing multiple interdependent files, modules, and libraries. Therefore, when LLM performs code generation tasks for such projects, it needs to fully understand the relationship between each program element in the code repository, locate the target code file in the code repository containing a large number of source code files, and ensure that the generated code can be seamlessly integrated into the code repository, completing the code task without introducing new problems. This also highlights the difficulty of extending the code generation task from function-level understanding to code repository-level understanding.

[0098] In order to help LLM locate the target code file quickly and accurately, in this step, the complex code repository can be abstracted into a code graph expressed by a graph structure, and then based on the similarity comparison, the context subgraph related to the first question text is extracted from the code graph. This subgraph can assist LLM in understanding the structural and semantic relationships between code elements in subsequent steps.

[0099] Next, we will briefly introduce the code diagram. The code repository mainly contains the structural relationship and semantic relationship of the code, where the structural relationship represents the organization of the code, such as the hierarchical structure (directory level) between source code files, and the program architecture described by each code element (such as module, class, function, etc.) written in a programming language, and the semantic relationship is composed of the various code elements in the code repository and the code relationship between them, such as the inheritance relationship between classes, the calling relationship between functions, the import relationship between modules, the reference relationship to variables, etc.

[0100] Figure 4An exemplary code graph is shown. It can be seen that in the process of constructing the code graph, code elements can be abstracted as nodes in the graph structure, and the code relationships between code elements can be abstracted as directed edges in the graph structure (for ease of understanding, structural relationships are marked with solid directed edges, and semantic relationships are marked with dotted directed edges). In this way, the code repository can be abstracted into a code graph represented by a graph structure. In a specific implementation, the code elements include at least one of the following: code modules, code files, class definitions, method declarations, and variable declarations; the code relationships include at least one of the following: hierarchical relationships, import relationships, reference relationships, inheritance relationships, and call relationships.

[0101] The above is a brief description of the code graph. The construction method of the code graph will be described in detail below, so I will not elaborate on it here. Based on the code graph corresponding to the code repository, in this step, according to the first question text, in the code graph containing a large number of code elements and code relationships, several nodes and directed edges related to the first question text are selected to form a context subgraph.

[0102] In graph search, there are various methods to retrieve related nodes and directed edges based on similarity.

[0103] According to one implementation, the relevance retrieval may be performed by comparing the cosine similarity or Euclidean distance between the node representation vector and the first question text representation vector, thereby evaluating the relevance therebetween.

[0104] In some specific application scenarios, as mentioned above, the input first question text may involve a variety of issues, such as code location, error fixing, adding new features, updating dependencies, etc. Usually, the code information related to solving the problem is not explicitly given in the first question text. In addition, since the first question text is mainly expressed in natural language, it may be mixed with some redundant information that is not related to the problem. These factors will cause a certain semantic gap between the first question text and the actual source code. Therefore, if only the first question text is used as the basis for relevance retrieval in the code graph, not only may some key information be missed, but also noise information may be introduced, which will ultimately affect the comprehensiveness and accuracy of the retrieval results.

[0105] To this end, during the relevance search process, the first question text can be semantically enhanced to broaden its information coverage, in order to build a richer and more accurate question representation vector. Keywords in the first question text can also be analyzed to extract key terms that are highly relevant to solving the problem, so that during the relevance search process, these key clues can be more focused and some noise information can be shielded.

[0106] According to one implementation, several first identifiers may be extracted from the first question text, the first identifiers including: code element names and / or function descriptors. Specifically, a prompt template for extracting keywords is designed to guide the LLM to extract key terms mentioned in the first question text, such as code element keywords such as source code file names and function names, and descriptive keywords related to code functions such as test_loop and train_loop. Figure 5A An exemplary prompt template for extracting keywords is provided, in which the problem description (in <issue>The first question text wrapped in a tag), the code repository name (shown as [astropy / astropy] in the figure), task instructions (the "task instructions" paragraph marked in gray in the figure), output format requirements (the "output in the following format" paragraph marked in gray in the figure), other additional requirements, such as precautions, expected / unexpected output examples, etc. Figure 5A Correspondingly, Figure 5B An exemplary result output of keyword extraction is provided. It can be seen that LLM outputs the extracted key terms related to the first question text (the first identifier) ​​according to the output format given in the prompt template, including: specific analysis information (wrapped with [analysis_start], [analysis_end] tags), extracted related code elements (wrapped with [related code element_start], [related code element_end] tags), and extracted related keywords (wrapped with [keyword_start], [keyword_end] tags).

[0107] After using the prompt template to guide LLM to extract several first identifiers from the first question text, based on text fuzzy matching, in the code graph, several associated nodes are retrieved, and the text similarity between the code element name of the associated node and the first identifier is higher than the preset threshold. Specifically, a code graph is a graph data structure, in which each node represents a code element in a code warehouse, such as a file, function, class or variable. The first identifier is matched with the code element name corresponding to the node, and by calculating the text similarity between the two, it can be determined which nodes have a higher match with the first identifier. In this process, a similarity threshold can be set, and only when the text similarity between the code element name of the node and the first identifier exceeds the preset similarity threshold, the node can be regarded as the associated node. By performing such a correlation search, it can be ensured that the retrieved associated nodes have similarity in keywords with the first question text, and the code elements represented by these associated nodes may be the key to solving the code generation task.

[0108] The above describes, in one implementation, the relevance search performed in the code graph using keyword extraction technology. According to another implementation, the first question text may also be input into the first model for inference to obtain a first extended text, wherein the first extended text includes a functional description text of the code elements related to the first question text. Specifically, a prompt template for the extended text content is designed to guide the LLM to infer the first question text, output a number of more specific functional descriptions of related codes, refine the problem description, and further shorten the semantic distance between the first question text and the specific source code. The result obtained by inference is used as the first extended text for searching related nodes in the code graph. Fig. 6A An example prompt template for expanding the question text is provided, in which the question description (in <issue>The first question text wrapped in a tag), the code repository name (shown as [astropy / astropy] in the figure), the task instructions (the "task" paragraph marked in gray in the figure), the output format requirements (the "output in the following format" paragraph marked in gray in the figure), and other additional requirements, such as precautions, expected / unexpected output examples, etc. Fig. 6A Correspondingly, Figure 6B An example output of an expanded question text is provided. It can be seen that LLM outputs the inferences obtained based on the first question text in accordance with the output format given in the prompt template, including: specific analysis information (wrapped with [analysis_start], [analysis_end] tags), and several expanded inferences (wrapped with [related inference_start], [related inference_end] tags). It is not difficult to see that LLM further expands the problem description based on the first question text and decomposes it into multiple operational inferences, such as finding the code file that implements a certain function, finding the reference location of a certain function, and so on.

[0109] After using the prompt template to guide LLM to infer the first extended text based on the first question text, the first extended text and the code graph are input into the second model for encoding to obtain the first code corresponding to the first extended text and the semantic code corresponding to the source code of each node in the code graph; in the code graph, several associated nodes are retrieved, and the representation similarity between the semantic code corresponding to the associated node and the first code is higher than the preset threshold. Specifically, the second model can encode the first extended text into a high-dimensional semantic vector (the first code), which captures the core semantic information of the text; at the same time, the second model also encodes each node in the code graph into a high-dimensional semantic vector (the semantic code), which reflects the semantic information of the source code text represented by the node. The first code is matched with the semantic code corresponding to the node, and by calculating the representation similarity between the two, it can be determined which nodes have a higher matching degree with the first extended text. In this process, a similarity threshold can be set, and only when the representation similarity between the node and the first extended text exceeds the preset similarity threshold, the node can be regarded as the associated node. By performing such a correlation search, it can be ensured that the retrieved associated node has similarity with the first question text in the semantics of the explicit text or the semantics of the inferred text. In an application scenario, the first extended text and the code graph may be encoded using a coding model suitable for code. Specifically, the second model is a CGE-Large model.

[0110] In some specific practices, the relevance retrieval methods exemplified in the above two implementations can also be used in combination. That is to say, in the process of relevance retrieval of the code graph, not only is the retrieval based on the keywords extracted from the first question text, but also the inference result of the first question text, that is, the extended text, is used for deep retrieval. Through this composite relevance retrieval, the semantic information of the first question text can be captured more comprehensively, improving the depth and breadth of the relevance retrieval.

[0111] The current execution step also involves the construction of a context subgraph. Specifically, by executing the relevance search described above, several associated nodes related to the first question text can be extracted from the code graph, and based on these associated nodes, the context subgraph can be extracted from the code graph.

[0112] In one implementation, the associated nodes and the directed edges between the associated nodes may be directly extracted to form a subgraph, namely, the context subgraph.

[0113] In another implementation, considering the code elements represented by the nodes, the calling and called relationships between them will also play an important guiding role in code generation. Therefore, based on the associated nodes obtained by the correlation retrieval, each associated node can be expanded along the directed edges in the code graph to perform a target search. The nodes obtained after the expansion and the directed edges between them can constitute a subgraph, namely the context subgraph. This implementation can be summarized as follows: based on the several associated nodes obtained by the correlation retrieval, a target search is performed, and the target search includes, for any associated node, searching along the directed edges in the code graph within a preset hop range to obtain several extended nodes; and constructing the first context subgraph based on the several associated nodes and their corresponding extended nodes.

[0114] In an application scenario of this implementation, the target search is a breadth-first N-hop BFS search, where N is a preset hop range. Typically, N is 1, that is, each associated node and its one-hop neighbor node are included in the nodes of the context subgraph.

[0115] After the above steps, the code graph originally containing a large number of code elements and complex code relationships has been effectively simplified into a context subgraph closely related to the first question text. Next, in step S205, the first context subgraph is subjected to graph representation processing to obtain a mapping representation of each node mapped to the large language model embedding space.

[0116] In this step, in order to input the information contained in the context subgraph into the LLM to perform the code generation task, each node in the context subgraph can be encoded, and the encoding representation obtained by encoding each node is aligned with the input dimension of the LLM in the semantic space. Therefore, in one implementation, the code text content contained in each node in the first context subgraph is firstly represented to obtain the encoding representation corresponding to each node; then each encoding representation is input into the mapping network for mapping processing to obtain the mapping representation.

[0117] In view of the fact that the text content contained in the nodes in the context subgraph is mainly source code, when the nodes are represented, the encoder usually faces the limitation of input length, which makes it impossible to input the source code completely into the encoder when representing the nodes containing long text source code, so that the obtained encoding representation cannot fully reflect the characteristics of the node. Therefore, in one implementation, when it is detected that the target code text content contained in any node is a long text, the target code text content can be segmented to obtain multiple text blocks; then each text block is represented separately to obtain the text block encoding representation corresponding to each text block; finally, based on the encoding representation of each text block, the encoding representation of the node is determined to complete the representation of the node. The text segmentation can be performed in a variety of ways. For example, the target code text content can be segmented into multiple text blocks with a preset text length as the unit segmentation length, or the target code text content can be segmented into multiple text blocks with a preset text delimiter (such as a semicolon, a line break, a curly brace, etc.) as a segmentation point.

[0118] The mapping representation of each node of the first context subgraph obtained in this step will be input into the LLM in the subsequent step to perform the code generation task for the first question text.

[0119] Next, in step S207 and step S209, the first question text is encoded in the embedding layer of the large language model to obtain a text representation. In the reasoning layer of the large language model, target reasoning is performed based on the mapping representation and the text representation to obtain a target code text for the first question text.

[0120] Similar to the representation processing performed on the nodes of the first context subgraph in the previous step, the first question text also needs to be encoded into a semantic space that can be understood by LLM. Therefore, the first question text is input into the embedding layer of LLM for encoding to obtain a text representation aligned with the LLM input dimension.

[0121] Next, the text representation corresponding to the first question text and the mapping representation of each node of the first context subgraph can be vectorized and the spliced ​​representation vector can be input into the reasoning layer of LLM. LLM integrates the source code structure information and semantic information provided by the first context subgraph and the semantic content in the first question text, performs comprehensive reasoning, identifies the code fragments related to the code task described by the first question text, and finally generates the target code text that can realize the code task.

[0122] In some implementations, the attention mechanism of the LLM can be enhanced to improve the structural recognition ability of the LLM for the source code. That is, the large language model can be used to perform attention-based encoding processing using the adjacency matrix of the code graph as an attention mask.

[0123] Specifically, in the attention mechanism of the autoregressive large language model, a causal attention mask is usually used to ensure that the prediction of each position depends only on the previous token. However, when using a large language model to process code graphs, the structural information that the graph structure can carry can also be incorporated into the attention mechanism to improve the large language model's ability to recognize the code structure. The adjacency matrix describes the connection relationship between nodes in the code graph and can be input into the large language model as an attention mask. Using the adjacency matrix for reasoning based on the attention mechanism can enable the large language model to better capture the structural information in the code graph, focus on the neighboring nodes directly connected to the current node, and then have a deeper understanding of the calling relationship, inheritance relationship, etc. between the various code elements in the code, further improving the performance of the large language model in code understanding and code generation tasks.

[0124] In addition, in order to further improve the accuracy of LLM in locating key code files, LLM can be guided to pay more attention to some code files that are most likely to need to be modified during the input phase of LLM. Then, in addition to using the enhancements to the LLM attention mechanism described above, you can also improve the accuracy of LLM in locating code files by inputting code files that need to be focused on to LLM. These code files are selected through a certain sorting strategy. In other words, although the context subgraph can limit the code repository information that LLM needs to refer to to a certain range, it still involves dozens or even hundreds of nodes representing different code files.

[0125] In order to enable LLM to focus on some code files that are most likely to be modified when performing code generation, in one implementation, the file nodes in the first context subgraph can be sorted based on their relevance to the first question text to determine a number of first code files; then, in the reasoning layer of the large language model, target reasoning is performed based on the first code files, the mapping representation and the text representation.

[0126] In this implementation, the relevance ranking can be evaluated and ranked by calculating the representation similarity between each associated node and the first problem text, and the code files represented by the top K associated nodes in the ranking result are classified as the first code files. The K first code files selected by ranking are used as one of the inputs of LLM when performing the target reasoning in the subsequent steps, which can assist LLM to more accurately locate those code files that are critical to solving the code task, thereby improving the efficiency and accuracy of LLM in understanding the code repository and generating a target code text that is more in line with expectations.

[0127] In a specific application scenario, the relevance ranking can also be performed by designing a prompt template containing a thought chain prompt text to instruct the large model to perform step-by-step reasoning from the associated nodes and select the first code file step by step. Fig. 7A An exemplary prompt template containing a thought chain prompt text is provided. In the prompt template, a problem description (in <issue>The first question text wrapped in a label), the name of the code repository (shown as [astropy / astropy] in the figure), the list of files corresponding to the associated nodes contained in the first context subgraph (the "following are the files retrieved from this repository" paragraph in the figure), task instructions (the "task" paragraph marked in gray in the figure), output format requirements (the "output in the following format" paragraph marked in gray in the figure), and other additional requirements, such as precautions, expected / unexpected output examples, etc. It can be seen that in the task instruction paragraph, the idea of ​​thinking chain is applied to describe the sorting task in steps to guide LLM to perform step-by-step reasoning. Fig. 7A Correspondingly, Figure 7B An exemplary result output for the thought chain text prompt template is provided. It can be seen that LLM follows the output format given in the prompt template and the task instructions, analyzes step by step from the given file list, selects the first code file, and outputs the specific reasoning process and the first code file, including: specific analysis information (wrapped with [analysis_start], [analysis_end] tags), and the selected first code file (wrapped with [related file_start], [related file_end] tags). These first code files are input into LLM when LLM performs target reasoning later to assist LLM in paying more attention to these files.

[0128] In addition, when outputting the target code text, LLM will output different types of target code text according to the different purposes of the code task indicated by the first problem text. According to one implementation method, if the target code text contains a target modification to the historical code, the target modification is packaged and submitted. In some code tasks related to code modification (for example, fixing code defects, adding new functions), the target code text generated by LLM needs to be merged (Code Merge) into the source code of the code repository before it can play a role. In this scenario, the generated target code text can be output in the format of GitPatch (Git patch file) (usually generated by the Git format-patch command), and the patch file can be applied to the current code repository through Git Apply to complete the code generation task for the first problem text. In downstream work, the correctness of the patch file can be verified by executing test cases.

[0129] The above is an introduction to the main process of a code generation method based on a code graph provided in an embodiment of this specification. In some embodiments of this specification, a method for constructing a code graph is also provided. The construction method includes at least the following steps: constructing a first abstract syntax tree based on the first source code contained in the code repository. Traversing the first abstract syntax tree, adding each node representing the code element and the directed edges representing the structural relationship between the nodes to the code graph. Performing semantic analysis on the code elements contained in the first source code to obtain a number of first semantic relationships. In the code graph, according to the first semantic relationship, the source node and the target node are connected by a directed edge.

[0130] Specifically, an Abstract Syntax Tree (AST) is a data structure that displays the various code elements and code relationships in a source code file in a tree structure, where each node represents a code element in the source code, such as a module, class declaration, function definition, etc.; the connection between nodes reflects the grammatical hierarchy and code relationship between code elements, such as function calls, variable references, etc.

[0131] By performing AST analysis on the code files in the code repository, we can obtain the first abstract syntax tree corresponding to each module and package. We traverse the first abstract syntax tree, add each node representing the code element to the code graph, and add the connection representing the structural relationship between these nodes as a directed edge to the code graph. The code graph is recorded as:

[0132] G=(V,E) (1)

[0133] Among them, V is a set of nodes representing code elements in the code graph, and E is a set of directed edges representing code relationships between nodes in the code graph.

[0134] According to a specific implementation, after performing the above AST traversal operation, the node set and directed edge set in the code graph can be shown as:

[0135] V = {repo}∪V mod ∪V file ∪V class ∪V func ∪V field (2)

[0136] E=E contains ∪E imports ∪E extends (3)

[0137] Among them, {repo} represents the current code repository, V mod Indicates module / library, V file Indicates the code file, V class Represents class, V func Represents function definition, V field Indicates a field / variable. E contains Represents the directory hierarchy relationship between source code files in the code repository, E imports Represents the import relationship between modules, classes, and methods in the source code file. extends Represents the inheritance relationship between classes.

[0138] For example, assume that the current code repository repo contains a code module mod, which contains a source code file file, which defines a class c that inherits from class d. Correspondingly, in the code diagram, it can be represented as:

[0139]

[0140] After building a code graph by analyzing AST, the semantic relationship between each node in the code graph can be analyzed based on the source code (the semantic analysis), the code relationship between code elements (the first semantic relationship) can be parsed, and added to the code graph as a directed edge.

[0141] According to a specific implementation, after performing the above semantic analysis, the directed edge set in the code graph can be updated as follows:

[0142] E=E contains ∪E imports ∪E extends ∪E calls ∪E uses (5)

[0143] Among them, E calls Represents the calling relationship between code elements, E uses Represents a reference / usage relationship to a code element.

[0144] For example, suppose function m1 calls function m2, and function m2 uses variable field f, which is also used by function m3. Correspondingly, in the code graph, it can be represented as:

[0145]

[0146] By executing the above steps, AST analysis and semantic analysis are performed on each source code file in the code repository, and a code graph corresponding to the code repository can be gradually constructed, in which each node represents a code element and the directed edges represent the code relationship between the code elements.

[0147] According to one or more embodiments, a code generation method based on a code graph is described in detail above. By using the above method provided in the embodiments of this specification, in the process of LLM executing the code generation task, based on the problem text, the code graph representing the code repository can be analyzed, and the context subgraph related to the problem text can be extracted and input into LLM, thereby improving the accuracy of LLM in understanding the code structure and correctly generating the target code text for the problem text.

[0148] In this specification, the word "first" in terms such as the first question text and the first model, as well as the corresponding "second", "third" (if any), etc. in the text, are merely for the convenience of distinction and description and do not have any limiting meaning.

[0149] The foregoing describes certain embodiments of the present specification, and other embodiments are within the scope of the appended claims. In some cases, the actions or steps described in the claims may be performed in an order different from that in the embodiments, and the desired results may still be achieved. In addition, the processes depicted in the accompanying drawings do not necessarily have to be performed in the specific order or sequential order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0150] Figure 8 Schematic diagram of a code generation device based on a code graph according to an embodiment of this specification. The device 800 is deployed in a computing device, which can be implemented by any device, equipment, platform, device cluster, etc. with computing and processing capabilities. Figure 2 The method embodiment shown corresponds to the embodiment shown in FIG. 800 . The device 800 includes:

[0151] The receiving module 801 is configured to receive a first question text for a code in a code repository.

[0152] The retrieval module 802 is configured to perform relevance retrieval in a pre-constructed code graph according to the first question text to determine a first context subgraph, wherein the code graph includes nodes representing code elements in the code repository and directed edges representing code relationships between nodes.

[0153] The first encoding module 803 is configured to perform graph representation processing on the first context subgraph to obtain a mapping representation of each node mapped to the embedding space of the large language model.

[0154] The second encoding module 804 is configured to encode the first question text in the embedding layer of the large language model to obtain a text representation.

[0155] The reasoning module 805 is configured to perform target reasoning based on the mapping representation and the text representation in the reasoning layer of the large language model to obtain a target code text for the first question text.

[0156] According to another embodiment of the present specification, there is also provided a computer program product, including a computer program / instruction, which is executed by a processor to implement the aforementioned combination Figure 2 The steps of the method.

[0157] According to another embodiment, the present specification further provides a computing device, including a memory and a processor, wherein the memory stores executable code, and when the processor executes the executable code, the aforementioned combination is implemented. Figure 2 The steps of the method.

[0158] Those skilled in the art should be aware that in one or more of the above examples, the functions described in the embodiments of the present invention may be implemented using hardware, software, firmware, or any combination thereof. When implemented using software, these functions may be stored in a computer-readable medium or transmitted as one or more instructions or codes on a computer-readable medium.

[0159] The specific implementation methods described above further describe the purpose, technical solutions and beneficial effects of the embodiments of the present invention in detail. It should be understood that the above description is only a specific implementation method of the embodiments of the present invention and is not intended to limit the scope of protection of the present invention. Any modification, equivalent replacement, improvement, etc. made on the basis of the technical solution of the present invention shall be included in the scope of protection of the present invention.< / issue> < / issue> < / issue>

Claims

1. A code generation method based on a code graph, comprising: Receive a first question text for the code in the code repository; According to the first question text, a relevance search is performed in a pre-constructed code graph to determine a first context subgraph, wherein the code graph includes nodes representing code elements in the code repository and directed edges representing code relationships between nodes; Performing graph representation processing on the first context subgraph to obtain a mapping representation of each node mapped to the embedding space of the large language model; In the embedding layer of the large language model, encoding the first question text to obtain a text representation; In the reasoning layer of the large language model, target reasoning is performed based on the mapping representation and the text representation to obtain a target code text for the first question text.

2. The method according to claim 1, wherein: The code elements include at least one of the following: code module, code file, class definition, method declaration, variable declaration; the code relationship includes at least one of the following: hierarchical relationship, import relationship, reference relationship, inheritance relationship, and call relationship.

3. The method according to claim 1, wherein: The first problem text includes one of the following: code location, error repair, new function addition, and dependency update.

4. The method according to claim 1, wherein: The determining of the first context subgraph comprises: Based on the several associated nodes obtained by the correlation retrieval, a target search is performed, wherein the target search includes, for any associated node, searching along the directed edge in the code graph within a preset hop count range to obtain several extended nodes; The first context subgraph is constructed according to the plurality of associated nodes and their corresponding extended nodes.

5. The method according to claim 1, wherein: The relevance retrieval in the pre-built code graph includes: Extracting a plurality of first identifiers from the first question text, wherein the first identifiers include code element names and / or function description words; In the code graph, based on text fuzzy matching, a number of associated nodes are retrieved, and the text similarity between the code element name of the associated node and the first identifier is higher than a preset threshold.

6. The method according to claim 1, wherein: The relevance retrieval in the pre-built code graph includes: Inputting the first question text into a first model for inference to obtain a first extended text, wherein the first extended text includes a function description text of a code element related to the first question text; Inputting the first extended text and the code graph into a second model for encoding, to obtain a first code corresponding to the first extended text and a semantic code corresponding to the source code of each node in the code graph; In the code graph, a plurality of associated nodes are retrieved, and the representation similarity between the semantic codes corresponding to the associated nodes and the first code is higher than a preset threshold.

7. The method according to claim 6, wherein: The second model is a CGE-Large model.

8. The method according to claim 1, further comprising: For the file nodes in the first context subgraph, sorting the file nodes based on their relevance to the first question text, determining a number of first code files; The target reasoning includes: performing target reasoning based on the first code file, the mapping representation and the text representation in the reasoning layer of the large language model.

9. The method according to claim 8, wherein: The determining of a plurality of first code files comprises: The file nodes and thought chain prompt text in the first context subgraph are input into the third model for reasoning to select the first code files; the thought chain prompt text is used to instruct the third model to perform reasoning in multiple steps to gradually determine the code files corresponding to the K file nodes with the highest relevance to the first question text.

10. The method according to claim 1, wherein: The performing graph representation processing on the first context subgraph includes: Performing characterization processing on the code text content contained in each node in the first context subgraph to obtain the corresponding coding characterization of each node; Each encoded representation is input into a mapping network for mapping processing to obtain the mapping representation.

11. The method according to claim 10, wherein: The code graph includes a first node, and the target code text content included in the first node is a long text; the characterization processing of the code text content included in each node in the first context subgraph includes: Performing text segmentation on the target code text content to obtain multiple text blocks; Performing characterization processing on each text block to obtain a text block encoding characterization corresponding to each text block; Based on the encoding representations of each text block, the encoding representation of the first node is determined.

12. The method according to claim 1, wherein: The target reasoning includes: Using the large language model, the adjacency matrix of the code graph is used as an attention mask to perform attention-based encoding processing.

13. The method according to claim 1, wherein: The method further comprises: If the target code text contains target modifications to the historical code, the target modifications are packaged and submitted.

14. The method according to claim 1, wherein: The code relationship includes a structural relationship and a semantic relationship, and the code graph is constructed by the following steps: Constructing a first abstract syntax tree according to the first source code included in the code repository; Traversing the first abstract syntax tree, and adding each node representing a code element and a directed edge representing a structural relationship between nodes therein to the code graph; Performing semantic analysis on code elements included in the first source code to obtain a plurality of first semantic relationships; In the code graph, according to the first semantic relationship, the source node and the target node are connected through a directed edge.

15. A code generation device based on a code graph, the device comprising: A receiving module is configured to receive a first question text for a code in a code repository; A retrieval module is configured to perform relevance retrieval in a pre-constructed code graph according to the first question text to determine a first context subgraph, wherein the code graph includes nodes representing code elements in the code repository and directed edges representing code relationships between nodes; A first encoding module is configured to perform graph representation processing on the first context subgraph to obtain a mapping representation of each node mapped to a large language model embedding space; A second encoding module is configured to encode the first question text in the embedding layer of the large language model to obtain a text representation; The reasoning module is configured to perform target reasoning based on the mapping representation and the text representation in the reasoning layer of the large language model to obtain a target code text for the first question text.

16. A computer program product, comprising a computer program / instruction, which, when executed by a processor, implements the steps of the method according to any one of claims 1 to 14.

17. A computing device comprising a memory and a processor, characterized in that: The memory stores executable codes, and when the processor executes the executable codes, the method according to any one of claims 1 to 14 is implemented.

Images