Code completion method, apparatus and electronic device
By constructing a vector database and parsing project code files to generate a second vector for the abstract syntax tree, the problem of inaccurate code generation during code completion is solved, achieving more efficient code generation.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- CHONGQING CHANGAN AUTOMOBILE CO LTD
- Filing Date
- 2026-02-24
- Publication Date
- 2026-05-29
AI Technical Summary
In existing technologies, the generated complete code during code completion is inaccurate, leading to low development efficiency.
By constructing a vector database, project code files are parsed into abstract syntax trees and second vectors are generated. These vectors are then used for code completion, improving the accuracy of code block division and the completeness of the code.
It improves the accuracy of the generated completed code, reduces the amount of code developers need to write manually, and increases development efficiency.
Smart Images
Figure CN122111422A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of programming technology, and more specifically to a code completion method, apparatus, and electronic device. Background Technology
[0002] In the field of software development, traditional manual coding has long been dominant. Developers need to rely on their proficiency in programming languages to manually write code line by line to implement software functions. This process not only requires developers to memorize various syntax rules, data structures, and algorithmic logic, but also requires them to invest a lot of time and energy in code debugging and optimization, which limits software development efficiency and often results in long project cycles.
[0003] To address this issue, existing technologies allow developers to write parts of the code, which is then completed by a computer system, thus reducing the workload for developers.
[0004] However, in the existing technology, during the code completion process, the sample code usually needs to be manually divided into segments to obtain code blocks. Then, the split code blocks and the code to be completed are used as the original input to generate the complete code. In this way, due to the inaccurate code splitting, the generated complete code is further inaccurate. Summary of the Invention
[0005] This application provides a code completion method, apparatus, and electronic device to solve the problem that the generated complete code is inaccurate during the code completion process.
[0006] In a first aspect, this application provides a code completion method, comprising: obtaining the code to be completed; searching for a target vector matching a first vector from a vector database, wherein the first vector is a vector obtained by converting the code to be completed, and the vector database stores multiple second vectors, each of which is obtained by converting according to an abstract syntax tree, the abstract syntax tree being obtained by parsing the project code file of the project corresponding to the code to be completed; constructing a target prompt word based on the first vector and the target vector; and inputting the target prompt word into a code completion model to obtain the complete code after the code to be completed.
[0007] As an optional example, before searching for the target vector that matches the first vector from the vector database, the method further includes: obtaining the project code file of the project corresponding to the code to be completed; parsing the project code file with a parser to obtain one or more abstract syntax trees; converting each abstract syntax tree and the code block corresponding to the abstract syntax tree in the project code file into a second vector, and storing the second vector in the vector database.
[0008] In this application, by constructing a vector database, the project code file is parsed into an abstract syntax tree and generated into a second vector with the corresponding code block and stored in the vector database, thereby improving the accuracy of the second vector.
[0009] As an optional example, parsing a project code file by a parser to obtain one or more abstract syntax trees includes: scanning the project code file line by line to identify each basic syntax element in the project code file; and converting multiple consecutive basic syntax elements into tree structures according to syntax rules and in the order of the basic syntax elements, wherein each tree structure is an abstract syntax tree.
[0010] In this application, by identifying the basic syntax elements in the project code file and generating multiple abstract syntax trees in the order of the basic syntax elements, the code in the project code file is divided into code blocks according to the code logic and generated into abstract syntax trees, thereby improving the accuracy of code block division in the project code file and further improving the accuracy of the second vector.
[0011] As an optional example, converting multiple consecutive basic syntax elements into a tree structure according to syntax rules and in the order of basic syntax elements includes: when the code to be completed is function or method level code, converting multiple consecutive basic syntax elements belonging to the same function or method into a tree structure; when the code to be completed is class or component level code, converting multiple consecutive basic syntax elements belonging to the same class or component into a tree structure.
[0012] In this application, when converting the project code file into a tree structure, different strategies are adopted according to the level of the code to be completed: when the code to be completed is function or method level code, multiple consecutive basic syntax elements belonging to the same function or method are converted into a tree structure; when the code to be completed is class or component level code, multiple consecutive basic syntax elements belonging to the same class or component are converted into a tree structure. Thus, the level at which the project code file is generated as an abstract syntax tree is determined according to the level of the code to be completed, making the difference between the second vector generated from the abstract syntax tree and the first vector smaller, and improving the accuracy of the second vector.
[0013] As an optional example, converting multiple consecutive basic syntax elements into a tree structure according to the syntax rules and the order of the basic syntax elements includes: if the number of basic syntax elements in the current tree structure exceeds a preset value, then identifying the state syntax elements and view syntax elements among the basic syntax elements; and separating the node and leaf node of each state syntax element, and the node and leaf node of each view syntax element from the current tree structure into separate tree structures.
[0014] To control the complexity of the abstract syntax tree, this application identifies state syntax elements and view syntax elements if the number of basic syntax elements in the current tree structure exceeds a preset value. Each node and leaf node of a state syntax element, and each node and leaf node of a view syntax element, are extracted from the current tree structure into separate tree structures. This simplifies part of the abstract syntax tree, resulting in an abstract syntax tree that better represents the context of the code to be completed, thus improving the accuracy of the second vector generated from the abstract syntax tree.
[0015] As an optional example, converting each abstract syntax tree (API) and the corresponding code block in the project code file into a second vector includes: converting the code block into a code block vector through an embedding layer; selecting N paths from the API from one random leaf node to another random leaf node, where N is a positive integer; converting each path into a vector; assigning weights to each path based on the number of nodes in each path; and determining the API vector by taking the weighted average of all path vectors and their weights; and concatenating the code block vector and the API vector into a second vector.
[0016] This application uses an Embedding layer to convert code blocks into code block vectors, and obtains an abstract syntax tree vector by selecting random leaf nodes from the abstract syntax tree to form a path and taking a weighted average of the vectors corresponding to the path. Thus, the two can be concatenated to obtain an accurate second vector.
[0017] As an optional example, after storing the second vector in the vector database, the process also includes: when the project code in the project code file changes, parsing the function or method containing the changed code as a code block into an abstract syntax tree; and converting the code block and the abstract syntax tree into a second vector and storing it in the vector database.
[0018] By monitoring the code in the project code file and incrementally updating the vector database with the second vector, a second vector can be generated for both historical and latest code in the project code file. This ensures that the second vectors in the vector database are all related to the code to be completed, thus improving the accuracy of the second vector.
[0019] As an optional example, parsing a project code file with a parser to obtain one or more abstract syntax trees includes: determining the programming language type of the project code file based on its extension or header; and selecting a matching parser to parse the project code file based on the programming language type to obtain one or more abstract syntax trees.
[0020] This application improves the accuracy of generating abstract syntax trees by using different parsers for project code files of different programming languages.
[0021] Secondly, this application provides a code completion device, comprising: an acquisition module for acquiring the code to be completed; a search module for searching a target vector matching a first vector from a vector database, wherein the first vector is a vector obtained by converting the code to be completed, and the vector database stores multiple second vectors, each of which is obtained by converting according to an abstract syntax tree, the abstract syntax tree being obtained by parsing the project code file of the project corresponding to the code to be completed; a construction module for constructing a target prompt word based on the first vector and the target vector; and an input module for inputting the target prompt word into the code completion model to obtain the complete code after the code to be completed.
[0022] Thirdly, this application provides an electronic device, including: at least one communication interface; at least one bus connected to the at least one communication interface; at least one processor connected to the at least one bus; and at least one memory connected to the at least one bus, wherein the processor is configured to: acquire code to be completed; search for a target vector matching a first vector from a vector database, wherein the first vector is a vector obtained by converting the code to be completed, the vector database stores a plurality of second vectors, each second vector being converted according to an abstract syntax tree, the abstract syntax tree being obtained by parsing the project code file of the project corresponding to the code to be completed; construct a target prompt word based on the first vector and the target vector; and input the target prompt word into a code completion model to obtain the complete code after the code to be completed.
[0023] Fourthly, this application also provides a computer storage medium storing computer-executable instructions for executing the code completion method of any of the above claims of this application.
[0024] The beneficial effects of this application are: This application improves the accuracy of the generated second vector by parsing the project code file into an abstract syntax tree, thereby accurately dividing the code according to its logic. This further enhances the accuracy of the completed code. It solves the problem of inaccurate completed code generated in existing technologies. Attached Figure Description
[0025] Figure 1 This is a flowchart of a code completion method according to this application; Figure 2 This is a schematic diagram of an abstract syntax tree according to this application; Figure 3 This is a schematic diagram of another abstract syntax tree in this application; Figure 4 This is a flowchart illustrating the generation of complete code in this application; Figure 5 This is a step sequence diagram of generating complete code according to this application; Figure 6 This is a schematic diagram of a code completion device according to this application; Figure 7 This is a schematic diagram of an electronic device according to this application. Detailed Implementation
[0026] The embodiments of the present invention will be described below with reference to the accompanying drawings and preferred embodiments. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be understood that the preferred embodiments are only for illustrating the present invention and not for limiting the scope of protection of the present invention.
[0027] It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of this application. Therefore, the drawings only show the components related to this application and are not drawn according to the actual number, shape and size of the components in the actual implementation. In the actual implementation, the form, quantity and proportion of each component can be arbitrarily changed, and the layout of the components may also be more complex.
[0028] For ease of description, spatial relative terms may be used in the text to describe the relative position or movement of one element or feature relative to another element or feature, as shown in the figure. These relative terms include, for example, "inside," "outside," "middle," "outer," "below," "below," "above," "front," "back," etc. Such spatial relative terms are intended to include different orientations of the device in use or operation, other than those depicted in the figure. For example, if the device in the figure undergoes a positional flip, orientation change, or change of motion, these directional indications will change accordingly. For instance, an element described as "below other elements or features" or "below other elements or features" will subsequently be oriented "above other elements or features" or "above other elements or features." Therefore, the example term "below" can include both upper and lower orientations. The device may be otherwise oriented (rotated 90 degrees or in other directions), and the spatial relative descriptors used in the text will be interpreted accordingly.
[0029] To address the technical problem of inaccurate generated complete code during the code completion process in existing technologies, this application provides a code completion method that can improve the accuracy of the generated completed code.
[0030] Figure 1 A flowchart of a code completion method provided in this application embodiment includes: S1, retrieve the code to be completed; S2, find the target vector that matches the first vector from the vector database. The first vector is the vector obtained by converting the code to be completed. The vector database stores multiple second vectors. Each second vector is converted according to an abstract syntax tree. The abstract syntax tree is obtained by parsing the project code file of the project corresponding to the code to be completed. S3, construct target prompt words based on the first vector and the target vector; S4. Input the target prompt word into the code completion model to obtain the complete code after the code to be completed.
[0031] The above code completion method can be applied in programming scenarios to assist developers in project development, reduce the amount of code they need to write, and allow the system to automatically complete the code based on the code to be completed. This allows the developer to verify or use the code, thereby improving their development efficiency.
[0032] Once the code to be completed is applied to the project, the project's code file can be obtained based on where the completed code is applied. For example, during the development of a project, a developer might write a portion of code to improve code generation speed. This written code becomes the code to be completed, and the system automatically completes it to obtain the full code. The user then decides whether to apply the full code to the project.
[0033] In this process, developers can input code to be completed. This code can be the name of a function, method, class, or component, or a small amount of code, which can be the initial code or key code. After obtaining the code to be completed, in existing technologies, it is input into a code completion model, which outputs the complete code. However, the complete code output by this strategy is inaccurate. In this application, after obtaining the code to be completed, it can be converted into a first vector. Then, from a second vector stored in a vector database, a target vector with high similarity and strong correlation to the code to be completed is searched. This target vector serves as the context for the code to be completed, supplementing and enhancing its information. The first vector of the code to be completed and the found target vector are combined to construct a target prompt word, which is then input into the code completion model, which outputs the complete code.
[0034] In the above process, the core method to improve the accuracy of the output complete code lies in the second vector in the database. This vector is obtained by parsing the Abstract Syntax Tree (AST) of the project code file corresponding to the code to be completed, and transforming it based on the AST. The process of parsing the project code file to obtain the AST involves identifying the code logic of the project code file. Therefore, the resulting AST naturally divides the code of the project code file according to its logic, and the resulting code blocks are more consistent with the logic and segmentation of the code. Thus, the second vector generated based on the AST is more accurate. The target vector found from the second vector using the first vector of the code to be completed can then serve as the context for the code to be completed, accurately describing the code and providing information that the code to be completed cannot provide. Only by inputting the target prompt words constructed from the first vector and the target vector into the code completion model can the accuracy of the obtained complete code be improved.
[0035] Therefore, this application obtains the code to be completed; searches a vector database for a target vector matching a first vector, where the first vector is a vector obtained by converting the code to be completed; the vector database stores multiple second vectors, each of which is obtained by converting an abstract syntax tree (AST), which is obtained by parsing the project code file of the project corresponding to the code to be completed; constructs target hint words based on the first vector and the target vector; and inputs the target hint words into the code completion model to obtain a scheme for the complete code after completion. Because the project code file is parsed into an AST, the code is accurately divided according to code logic, improving the accuracy of the generated second vectors and further improving the accuracy of the completed code. This solves the problem of inaccurate completed code generated in existing technologies.
[0036] As an optional example, before searching for the target vector matching the first vector from the vector database, the method further includes: obtaining the project code file of the project corresponding to the code to be completed; parsing the project code file with a parser to obtain one or more abstract syntax trees; converting each abstract syntax tree and the code block corresponding to the abstract syntax tree in the project code file into a second vector, and storing the second vector in the vector database.
[0037] In this application, if the target vector needs to be found from the second vector in the vector database based on the first vector of the code to be completed, the vector database needs to be constructed. Constructing the vector database involves generating the code from the project code file as a second vector and storing it in the vector database.
[0038] When building a vector database, the second vector can be generated not only from the project code files of the project containing the code to be completed, but also from project code files of previously completed projects or different projects of different types and business functions. If a sufficient number of project code files are used, the generated second vectors will have broader coverage. Broader coverage allows for the generation of second vectors that encompass various types of code blocks. This method requires the vector database to be built in advance.
[0039] In this application, the project code file of the project corresponding to the code to be completed is used to generate the second vector because the second vector generated from the project code file of the project containing the code to be completed is closer to the code to be completed, the computational cost of generating the second vector in the vector database is smaller, and the second vector has a higher correlation with the code to be completed. This strategy allows for the generation of the corresponding second vector in real-time after obtaining the code to be completed to construct the vector database, or it allows for the pre-generated vector database to be supplemented with the second vector.
[0040] Taking the generation of a second vector from the project code file corresponding to the code to be completed as an example, the project code file is parsed by a parser to obtain one or more abstract syntax trees (ASTs). An abstract syntax tree is a tree-like structure used to analyze code. It records the core features of the code, such as its syntactic structure, logical relationships, variable declarations, and function calls. Computers can quickly identify code and understand its core structure through abstract syntax trees. For example... Figure 2 This is a schematic diagram of an exemplary abstract syntax tree. It should be noted that... Figure 2 The abstract syntax tree is only one representation. Figure 3 This is a diagram illustrating another type of abstract syntax tree. The corresponding code block is: function computeMax(arr: number[]) { 2 / / Sort the array 3const sortedArr = arr.sort((a, b) =>a - b); 4 / / Get the maximum value of the array 5const max = sortedArr[sortedArr.length - 1]; 6return max; 7} This application does not limit the form of the abstract syntax tree representation, as long as it can extract the code logic and important information of the code block.
[0041] When generating an abstract syntax tree (AST) from project code files, the AST divides the project code files into multiple code blocks. Each code block extracts its core code and logic, which is then used to generate the AST. This code block division doesn't involve manually setting any lines of code based on that block's logic; rather, the system automatically divides the code according to its logic. This strategy generates an AST that provides a more reasonable and logically consistent code division for the project code files.
[0042] After generating the abstract syntax tree, converting each abstract syntax tree and the corresponding code block in the project code file into a second vector can improve the accuracy of the generated second vector.
[0043] In this application, by constructing a vector database, the project code file is parsed into an abstract syntax tree and generated into a second vector with the corresponding code block and stored in the vector database, thereby improving the accuracy of the second vector.
[0044] As an optional example, parsing a project code file by a parser to obtain one or more abstract syntax trees includes: scanning the project code file line by line to identify each basic syntax element in the project code file; and converting multiple consecutive basic syntax elements into tree structures according to syntax rules and in the order of the basic syntax elements, wherein each tree structure is an abstract syntax tree.
[0045] In this application, the process of generating an abstract syntax tree from a project code file involves multiple steps. First, it's necessary to identify each basic syntax element in the project code file. Basic syntax elements refer to fundamental lexical units such as keywords, identifiers, constants, and operators. For example, the line `let x = 5;` includes the keyword `let`, the identifier `x`, the operator `=`, the constant `5`, and the separator `;`. The entire project code file is traversed from beginning to end, breaking down all code to obtain the basic syntax elements of each line. Then, following the order of all basic syntax elements, an abstract syntax tree is generated starting from the beginning. When multiple consecutive basic syntax elements generate an abstract syntax tree 1, if subsequent basic syntax elements continue to generate an abstract syntax tree 1, and the code logic is found to be incorrect, an abstract syntax tree 2 is generated. After this traversal, all basic syntax elements of the project code file are generated into multiple abstract syntax trees. The code corresponding to each abstract syntax tree is considered a code block, thus dividing the entire project code file into multiple code blocks in a way that conforms to the code logic.
[0046] During the generation of an abstract syntax tree (AST), in one scenario, all basic syntax elements of the project code file are identified and included in a specific AST. In another scenario, some basic syntax elements, after being identified, contribute little to the generation of the AST, and these elements can be ignored or deleted.
[0047] In this application, by identifying the basic syntax elements in the project code file and generating multiple abstract syntax trees in the order of the basic syntax elements, the code in the project code file is divided into code blocks according to the code logic and generated into abstract syntax trees, thereby improving the accuracy of code block division in the project code file and further improving the accuracy of the second vector.
[0048] As an optional example, converting multiple consecutive basic syntax elements into a tree structure according to syntax rules and in the order of basic syntax elements includes: when the code to be completed is function or method level code, converting multiple consecutive basic syntax elements belonging to the same function or method into a tree structure; when the code to be completed is class or component level code, converting multiple consecutive basic syntax elements belonging to the same class or component into a tree structure.
[0049] In this application, during the process of dividing the project code file into a tree structure, the level of the code blocks in the tree structure can be determined based on the level of the code to be completed. This example matches the case where a second vector is generated after obtaining the code to be completed. The level of the code to be completed refers to whether it is a method, function, class, or component. Depending on the type of code to be completed, a second vector matching the code to be completed is generated. For example, the project code file includes 10 classes, each containing multiple functions and methods. If the code to be completed is a class, the tree structure generated during parsing is a class-level tree structure. However, if the code to be completed is a method, the tree structure generated during parsing is a method-level tree structure. If the code to be completed is a method, and when parsing the code to generate a tree structure, a class-level tree structure is generated, and then a second vector is generated based on the tree structure, then since the second vector includes multiple other methods besides the code to be completed, the difference between the second vector and the first vector is greater than the difference between the second vector and the first vector obtained by generating a method-level tree structure.
[0050] In this application, when converting the project code file into a tree structure, different strategies are adopted according to the level of the code to be completed: when the code to be completed is function or method level code, multiple consecutive basic syntax elements belonging to the same function or method are converted into a tree structure; when the code to be completed is class or component level code, multiple consecutive basic syntax elements belonging to the same class or component are converted into a tree structure. Thus, the level at which the project code file is generated as an abstract syntax tree is determined according to the level of the code to be completed, making the difference between the second vector generated from the abstract syntax tree and the first vector smaller, and improving the accuracy of the second vector.
[0051] As an optional example, converting multiple consecutive basic syntax elements into a tree structure according to the syntax rules and the order of the basic syntax elements includes: if the number of basic syntax elements in the current tree structure exceeds a preset value, then identifying the state syntax elements and view syntax elements among the basic syntax elements; and separating the node and leaf node of each state syntax element, and the node and leaf node of each view syntax element from the current tree structure into separate tree structures.
[0052] In this application, during the process of converting multiple consecutive basic syntax elements into a tree structure according to syntax rules and the order of basic syntax elements, some tree structures become too large due to the excessive content of basic syntax elements. If the size is too large, the generated second vector will contain too much content and cannot accurately reflect the context of the code to be completed. Therefore, this application adds constraints to the generated tree structure for secondary processing. Specifically, if the number of basic syntax elements in the current tree structure exceeds a preset value, it identifies whether the basic syntax elements in the current attribute structure include state syntax elements and view syntax elements. If state syntax elements are included, since state syntax elements correspond to different states, the node containing the state syntax element and its leaf node can be separated into a separate tree structure. Similarly, view syntax elements include multiple syntaxes, so the node containing the view syntax element and its corresponding leaf node can also be separated into a separate tree structure.
[0053] To control the complexity of the abstract syntax tree, this application identifies state syntax elements and view syntax elements if the number of basic syntax elements in the current tree structure exceeds a preset value. Each node and leaf node of a state syntax element, and each node and leaf node of a view syntax element, are extracted from the current tree structure into separate tree structures. This simplifies part of the abstract syntax tree, resulting in an abstract syntax tree that better represents the context of the code to be completed, thus improving the accuracy of the second vector generated from the abstract syntax tree.
[0054] As an optional example, converting each abstract syntax tree (API) and the corresponding code block in the project code file into a second vector includes: converting the code block into a code block vector through an embedding layer; selecting N paths from the API from one random leaf node to another random leaf node, where N is a positive integer; converting each path into a vector; assigning weights to each path based on the number of nodes in each path; and determining the API vector by taking the weighted average of all path vectors and their weights; and concatenating the code block vector and the API vector into a second vector.
[0055] In this application, when converting the abstract syntax tree and the code block corresponding to the abstract syntax tree in the project code file into a second vector, the code block and the abstract syntax tree can be converted into vectors separately, and then the two vectors can be concatenated to obtain the second vector.
[0056] To convert code blocks into vectors, an embedding layer can be used. An embedding layer is a model that can convert unstructured data into numerical vectors. Code blocks can be converted into code block vectors using an embedding layer. The conversion of an abstract syntax tree (AST) is more complex. To obtain accurate code structure information from the AST, random paths can be selected. A random path is a path from one random leaf node to another random leaf node in the AST. This path traverses the root node or leaf nodes, forming a path. In this application, N such paths are obtained, and each path is converted into a vector. Each vector records the path information. The mean of the N vectors is calculated according to assigned weights. The mean vector records the tree structure information of the AST. The weights are determined by the number of leaf nodes in the path; the more leaf nodes, the greater the weight.
[0057] The code block vector and the abstract syntax tree vector can be directly concatenated to obtain a second vector. This second vector is then stored in the database.
[0058] This application uses an Embedding layer to convert code blocks into code block vectors, and obtains an abstract syntax tree vector by selecting random leaf nodes from the abstract syntax tree to form a path and taking a weighted average of the vectors corresponding to the path. Thus, the two can be concatenated to obtain an accurate second vector.
[0059] As an optional example, after storing the second vector in the vector database, the above method also includes: when the project code in the project code file changes, the function or method containing the changed code is treated as a code block and parsed into an abstract syntax tree; the code block and the abstract syntax tree are converted into a second vector and then stored in the vector database.
[0060] In this application, to maintain the real-time nature of the second vector in the vector database, when the project code in the project code file changes, the function or method containing the changed code is treated as a code block and parsed into an abstract syntax tree (AST). The code block and AST are then converted into a second vector and stored in the vector database, enabling incremental updates. The second code already stored in the vector database does not need to be deleted or modified. In other words, if the code in the project code file changes after parsing to generate an AST and then storing the second vector in the vector database, the second vector already stored in the vector database does not need to be deleted or modified. Instead, the newly changed code in the project code file is parsed to obtain an AST, and then the second vector is generated and stored in the vector database.
[0061] By monitoring the code in the project code file and incrementally updating the vector database with the second vector, a second vector can be generated for both historical and latest code in the project code file. This ensures that the second vectors in the vector database are all related to the code to be completed, thus improving the accuracy of the second vector.
[0062] As an optional example, parsing a project code file with a parser to obtain one or more abstract syntax trees includes: determining the programming language type of the project code file based on its extension or header; and selecting a matching parser to parse the project code file based on the programming language type to obtain one or more abstract syntax trees.
[0063] In this application, during the process of parsing the project code file to obtain the abstract syntax tree (API), the programming language type of the project code file can be determined based on its extension or header. For example, the .js extension corresponds to JavaScript, the .py extension to Python, and the .java extension to Java. Based on the programming language type, a matching parser is selected to parse the project code file, resulting in one or more APIs. For JavaScript, the Esprima parser can be used; for Python, the PyParsing parser can be used; and for Java, the JavaParser parser can be used.
[0064] After generating a second vector from the abstract syntax trees parsed by different parsers, there is no need to classify them; they are stored in the same vector database.
[0065] This application improves the accuracy of generating abstract syntax trees by using different parsers for project code files of different programming languages.
[0066] The following example illustrates this. Figure 4 This is a flowchart illustrating the generation of complete code in this application. In order to complete the code to be completed and obtain accurate complete code, this application pre-constructs a vector database, which stores contextual information (second vectors) that is highly relevant to the code to be completed.
[0067] To store the accurate second vector in the vector database, the project code files related to the project to be completed are first obtained. There can be one or more project code files. To obtain the project code files, for each project, a configuration file is created that explicitly lists irrelevant directories or folders within the project. This configuration file will be used throughout the process to guide the program to skip these directories or folders that do not need to be checked. For directories or folders that need to be processed, a recursive method is used to traverse the root directory and all its subdirectories, finding all code files within them and recording detailed information for each file, including but not limited to file path, filename, and modification time. This information is saved in a list for later analysis. Each retained code file is then treated as a project code file for further processing.
[0068] For different project code files, the programming language type can be identified by the file extension or other identifiers (such as the content of the file header). Common programming language extensions include .js (JavaScript), .py (Python), .java (Java), etc. Files of different language types are stored in different lists or collections for easier processing by language. An appropriate parser library is selected (e.g., Esprima for JavaScript, PyParsing for Python, JavaParser for Java, etc.). After reading the content of each file, a parser performs lexical analysis, breaking down the source code of the project code file into indivisible basic syntactic elements (lexical units). Then, syntactic analysis is performed, where the parser checks the arrangement of basic syntactic elements according to syntactic rules (such as indentation rules in Python and bracket matching in Java) and generates the corresponding abstract syntax tree.
[0069] By parsing the abstract syntax tree obtained, important information about the project code files can be extracted, such as file name, file path, language type, function definition, function signature, function call, function content, class definition, constant declaration, variable declaration, and dependencies.
[0070] After generating the abstract syntax tree (Abstract Syntax Tree), to better understand and compare the relationships between different code blocks, the important information of the parsed file (Abstract Syntax Tree) and the code blocks can be converted into vector form, resulting in a second vector. This step can be accomplished by choosing an appropriate embedding model. An embedding model can convert unstructured or discrete data into low-dimensional, dense, numerical vectors, mapping high-dimensional, sparse original data to a low-dimensional space, ensuring that the feature similarity of the vectors matches the actual correlation of the original data. In another example, the Abstract Syntax Tree can be generated by selecting a path between two random leaf nodes, as explained above, and will not be repeated here.
[0071] The generated second vector is inserted into the local vector database by connecting to it. The vector database relies on an indexing algorithm, building its index structure through pre-computed relationships between vectors to accelerate query efficiency and facilitate subsequent retrieval and analysis. Simultaneously, unique identifiers for each file are recorded, such as file path, timestamp, and MD5 hash, to ensure data consistency and traceability.
[0072] When an update to the project code file is detected, only the changed parts, such as the functions or methods containing the changed code, need to be re-parsed and vectorized. Then, the corresponding record in the vector database is updated incrementally, recording the changed file path, timestamp, MD5 hash, and other information. This significantly reduces repetitive work and improves efficiency.
[0073] When searching for a target vector that matches the first vector in the vector database, the cosine similarity algorithm is used to calculate the similarity between the first vector and each second vector in the vector database. The cosine similarity value ranges from [-1, 1], and the closer the value is to 1, the higher the similarity.
[0074] The target prompt word is constructed based on the first vector and the target vector. The target prompt word combines the contextual information of the code to be completed with the relevant code fragments retrieved from the vector database to form a complete prompt message, providing sufficient contextual reference for the code completion model.
[0075] The target prompt word is input into the code completion model to obtain the complete code after it has been completed. The code completion model is based on large language model technology, which can understand the semantics and syntactic structure of the code and generate code completion results that conform to programming standards and logic based on the contextual information in the prompt word. Figure 5 The sequence of each step in the above process is shown.
[0076] The complete code obtained can be given to the developer, who can choose to use it or make adjustments based on the complete code.
[0077] This method fully utilizes the structural and semantic information of existing code in the project, improving the accuracy and relevance of code completion and providing developers with a more intelligent and accurate code completion service.
[0078] The following are embodiments of the apparatus of this application, which can be used to execute the embodiments of the method of this application.
[0079] Figure 6 This is a block diagram of a code completion device provided in an embodiment of this application. This device can be implemented as part or all of an electronic device through software, hardware, or a combination of both. Figure 6 As shown, the device includes: Get module 601, used to retrieve the code to be completed; The lookup module 602 is used to find the target vector that matches the first vector from the vector database. The first vector is the vector obtained by converting the code to be completed. The vector database stores multiple second vectors. Each second vector is converted according to an abstract syntax tree. The abstract syntax tree is obtained by parsing the project code file of the project corresponding to the code to be completed. Construction module 603 is used to construct target prompt words based on the first vector and the target vector; Input module 604 is used to input the target prompt words into the code completion model to obtain the complete code after the code to be completed.
[0080] The aforementioned code completion device can be applied in programming scenarios to assist developers in project development, reduce the amount of code developers need to write, and automatically complete the code based on the code to be completed by the developer, which can then be reviewed or used by the developer, thereby improving the developer's development efficiency.
[0081] Once the code to be completed is applied to the project, the project's code file can be obtained based on where the completed code is applied. For example, during the development of a project, a developer might write a portion of code to improve code generation speed. This written code becomes the code to be completed, and the system automatically completes it to obtain the full code. The user then decides whether to apply the full code to the project.
[0082] In this process, developers can input code to be completed. This code can be the name of a function, method, class, or component, or a small amount of code, which can be the initial code or key code. After obtaining the code to be completed, in existing technologies, it is input into a code completion model, which outputs the complete code. However, the complete code output by this strategy is inaccurate. In this application, after obtaining the code to be completed, it can be converted into a first vector. Then, from a second vector stored in a vector database, a target vector with high similarity and strong correlation to the code to be completed is searched. This target vector serves as the context for the code to be completed, supplementing and enhancing its information. The first vector of the code to be completed and the found target vector are combined to construct a target prompt word, which is then input into the code completion model, which outputs the complete code.
[0083] In the above process, the core method to improve the accuracy of the output complete code lies in the second vector in the database. This vector is obtained by parsing the Abstract Syntax Tree (AST) of the project code file corresponding to the code to be completed, and transforming it based on the AST. The process of parsing the project code file to obtain the AST involves identifying the code logic of the project code file. Therefore, the resulting AST naturally divides the code of the project code file according to its logic, and the resulting code blocks are more consistent with the logic and segmentation of the code. Thus, the second vector generated based on the AST is more accurate. The target vector found from the second vector using the first vector of the code to be completed can then serve as the context for the code to be completed, accurately describing the code and providing information that the code to be completed cannot provide. Only by inputting the target prompt words constructed from the first vector and the target vector into the code completion model can the accuracy of the obtained complete code be improved.
[0084] Therefore, this application obtains the code to be completed; searches a vector database for a target vector matching a first vector, where the first vector is a vector obtained by converting the code to be completed; the vector database stores multiple second vectors, each of which is obtained by converting an abstract syntax tree (AST), which is obtained by parsing the project code file of the project corresponding to the code to be completed; constructs target hint words based on the first vector and the target vector; and inputs the target hint words into the code completion model to obtain a scheme for the complete code after completion. Because the project code file is parsed into an AST, the code is accurately divided according to code logic, improving the accuracy of the generated second vectors and further improving the accuracy of the completed code. This solves the problem of inaccurate completed code generated in existing technologies.
[0085] like Figure 7 As shown in the figure, this application embodiment provides a code completion device, including a processor 111, a communication interface 112, a memory 113, and a communication bus 114, wherein the processor 111, the communication interface 112, and the memory 113 communicate with each other through the communication bus 114. Memory 113 is used to store computer programs; In one embodiment of this application, when the processor 111 executes the program stored in the memory 113, it implements the code completion method provided in any of the foregoing method embodiments, including: obtaining the code to be completed; searching for a target vector matching a first vector from a vector database, wherein the first vector is a vector obtained by converting the code to be completed, the vector database stores multiple second vectors, each of which is obtained by converting according to an abstract syntax tree, the abstract syntax tree being obtained by parsing the project code file of the project corresponding to the code to be completed; constructing a target prompt word based on the first vector and the target vector; and inputting the target prompt word into the code completion model to obtain the complete code after the code to be completed.
[0086] This application also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of the code completion method provided in any of the foregoing method embodiments.
[0087] The device embodiments described above are illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of the solution in this embodiment, depending on actual needs.
[0088] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented using software plus a general-purpose hardware platform, or of course, using hardware. Based on this understanding, the above technical solutions, in essence or the parts that contribute to the related technology, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments, or some parts of the embodiments.
[0089] It should be understood that the terminology used herein is for the purpose of describing particular exemplary embodiments only and is not intended to be limiting. Unless the context clearly indicates otherwise, the singular forms “a,” “an,” and “described” as used herein may also include the plural forms. The terms “comprising,” “including,” “containing,” and “having” are inclusive and therefore indicate the presence of the stated features, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, elements, components, and / or combinations thereof. The method steps, processes, and operations described herein are not construed as requiring them to be performed in a particular order described or illustrated unless the order of performance is explicitly indicated. It should also be understood that additional or alternative steps may be used.
[0090] The above description is merely a specific embodiment of this application, enabling those skilled in the art to understand or implement this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features claimed herein.
Claims
1. A code completion method, characterized in that, include: Get the code to be completed; The target vector that matches the first vector is found in the vector database. The first vector is the vector obtained by converting the code to be completed. The vector database stores multiple second vectors. Each second vector is converted according to an abstract syntax tree. The abstract syntax tree is obtained by parsing the project code file of the project corresponding to the code to be completed. Construct target prompt words based on the first vector and the target vector; The target prompt word is input into the code completion model to obtain the complete code after the code to be completed.
2. The method according to claim 1, characterized in that, Before searching the vector database for a target vector that matches the first vector, the method further includes: Obtain the project code file of the project corresponding to the code to be completed; The project code file is parsed by a parser to obtain one or more of the abstract syntax trees; Each abstract syntax tree and the corresponding code block of the abstract syntax tree in the project code file are converted into a second vector, and the second vector is stored in the vector database.
3. The method according to claim 2, characterized in that, The parser parses the project code file to obtain one or more of the abstract syntax trees, including: Scan the project code file line by line to identify each basic syntax element in the project code file; According to the grammatical rules and in the order of the basic grammatical elements, a series of consecutive basic grammatical elements are converted into a tree structure, wherein each of the tree structures is an abstract syntax tree.
4. The method according to claim 3, characterized in that, According to grammatical rules and in the order of the basic grammatical elements, converting multiple consecutive basic grammatical elements into a tree structure includes: When the code to be completed is function or method level code, multiple consecutive basic syntax elements belonging to the same function or method are converted into a tree structure; When the code to be completed is at the class or component level, multiple consecutive basic syntax elements belonging to the same class or component are converted into a tree structure.
5. The method according to claim 3, characterized in that, According to grammatical rules and in the order of the basic grammatical elements, converting multiple consecutive basic grammatical elements into a tree structure includes: If the number of basic syntax elements in the current tree structure obtained by conversion exceeds the preset value, then the state syntax elements and view syntax elements in the basic syntax elements are identified. Each node containing a state syntax element and its leaf node, along with each node containing a view syntax element and its leaf node, are separated from the current tree structure into individual tree structures.
6. The method according to claim 2, characterized in that, Converting each of the abstract syntax trees and the corresponding code blocks in the project code file into a second vector includes: The code block is converted into a code block vector through the Embedding layer; Select N paths from one random leaf node to another random leaf node from the abstract syntax tree, where N is a positive integer. Convert each path into a vector. Assign weights to each path based on the number of nodes in each path. Take the weighted average of all path vectors and their weights to determine the abstract syntax tree vector. The code block vector and the abstract syntax tree vector are concatenated to form the second vector.
7. The method according to claim 2, characterized in that, After storing the second vector in the vector database, the method further includes: When the project code in the project code file changes, the function or method containing the changed code is treated as a code block and parsed into an abstract syntax tree; The code blocks and abstract syntax trees are converted into second vectors and then stored in the vector database.
8. The method according to any one of claims 2 to 7, characterized in that, The parser parses the project code file to obtain one or more of the abstract syntax trees, including: The programming language type of the project code file is determined based on its extension or header. Based on the programming language type, a matching parser is selected to parse the project code file, resulting in one or more abstract syntax trees.
9. A code completion device, characterized in that, include: The retrieval module is used to retrieve the code to be completed. The search module is used to search for a target vector that matches the first vector from the vector database. The first vector is a vector obtained by converting the code to be completed. The vector database stores multiple second vectors. Each second vector is converted according to an abstract syntax tree. The abstract syntax tree is obtained by parsing the project code file of the project corresponding to the code to be completed. The construction module is used to construct target prompt words based on the first vector and the target vector; The input module is used to input the target prompt word into the code completion model to obtain the complete code after the code to be completed.
10. A code completion device, characterized in that, The system includes at least one communication interface; at least one bus connected to the at least one communication interface; at least one processor connected to the at least one bus; and at least one memory connected to the at least one bus. The processor is configured to: acquire code to be completed; search a vector database for a target vector matching a first vector, wherein the first vector is a vector obtained by converting the code to be completed, and the vector database stores multiple second vectors, each second vector being obtained by converting an abstract syntax tree (AST) based on the project code file of the project corresponding to the code to be completed; construct a target hint word based on the first vector and the target vector; and input the target hint word into a code completion model to obtain the complete code after completion.