Code Search Method, Device and Medium Based on Large Model Agent

Abstract

The present application discloses a code search method, device, and medium based on large model agents, relating to the field of data processing. The method includes: obtaining a first natural language query and a target code library; enhancing the prompt for the first natural language query through the target code library to generate a second natural language query; using a large language model to take the second natural language query as input and output corresponding query code; creating text embeddings for the second natural language query and the query code in the target code library, comparing them based on a multi-stream architecture to obtain target code corresponding to each stream; and combining the target code corresponding to each stream to obtain a code search result. By enhancing the user query through the target code library context, technical details can be automatically supplemented, the polysemy of terms can be alleviated, the search deviation caused by fuzzy input can be reduced, and lexical ambiguity can be dynamically eliminated.

Description

Technical Field

[0001] This application relates to the field of data processing, and specifically to a code search method, device, and medium based on large model agents. Background Art

[0002] The code search task refers to searching in a code library to find code snippets that can solve specific problems. In daily work, programmers often need to perform code search tasks.

[0003] Traditional code search uses keyword matching. Nowadays, it is also possible to learn and predict the semantics behind query code snippets. By training large language models (LLMs) on a large amount of text and code, the large language models have obtained the ability to generate natural language to code, thus retrieving more accurately the code that meets the intention.

[0004] However, today's semantic code search still faces many challenges. The natural language queries provided by users may be ambiguous or require more details. The vocabulary mismatch problem is an example. Different people use different keywords or terms to describe the same concept or function, or use the same keyword to describe different functions, which results in inaccurate final search results. For example, the term "model" can refer to a machine learning model, a database schema, or a software design pattern. This problem may lead to weakened code search results or force users to do additional work to provide more details in their input prompts. Summary of the Invention

[0005] To solve the above problems, this application proposes a code search method based on large model agents, including:

[0006] Obtain a first natural language query and a target code library;

[0007] Enhance the first natural language query through the target code library to generate a second natural language query;

[0008] Use the large language model to take the second natural language query as input and output the corresponding query code;

[0009] In the target code library, create text embeddings for the second natural language query and the query code, and compare them based on a multi-stream architecture to obtain the target code corresponding to each stream;

[0010] Combine the target codes corresponding to each stream to obtain the code search result.

[0011] In one example, the target code library is used to enhance the prompt for the first natural language query to generate a second natural language query, which specifically includes:

[0012] Add a restriction condition for the first natural language query;

[0013] According to the restriction condition, attempt to enhance the implementation method of the first natural language query through the context information of the target code library, and during the enhancement process, restrict the enhancement scope through the restriction condition;

[0014] If the attempt fails, enhance the implementation method of the first natural language query based on the network and / or local knowledge base, and during the enhancement process, restrict the enhancement scope and the website call process through the restriction condition;

[0015] Output the second natural language query.

[0016] In one example, in the target code library, text embeddings are created for the second natural language query and the query code, and based on a multi-stream architecture for comparison, the target code corresponding to each stream is obtained, which specifically includes:

[0017] Group the codes in the target code library to obtain the corresponding class set and independent function set;

[0018] Based on each stream in the multi-stream architecture, select some content from the second natural language query and the query code, and compare it with the elements included in the class set and / or the independent function set through text embedding;

[0019] According to the similarity in the comparison results, match the target code corresponding to each stream.

[0020] In one example, based on each stream in the multi-stream architecture, select some content from the second natural language query and the query code, and compare it with the elements included in the class set and / or the independent function set through text embedding, which specifically includes:

[0021] Based on the stream for direct matching in the multi-stream architecture, select the second natural language query;

[0022] Compare the second natural language query with each element in the independent function set through text embedding to obtain the corresponding comparison results.

[0023] In one example, based on each stream in the multi-stream architecture, select some content from the second natural language query and the query code, and compare it with the elements included in the class set and / or the independent function set through text embedding, which specifically includes:

[0024] Select the query code based on the stream for code matching in the multi-stream architecture;

[0025] Compare the similarity of the query code with each element in the class set and the independent function set through text embedding to obtain the corresponding comparison result.

[0026] In one example, based on each stream in the multi-stream architecture, select some content from the second natural language query and the query code, and compare it with the elements included in the class set and / or the independent function set through text embedding, specifically including:

[0027] Based on the stream for component function matching in the multi-stream architecture, split the query code to obtain multiple component functions;

[0028] Compare the similarity of each component function with each element in the independent function set through text embedding to obtain the corresponding comparison result.

[0029] In one example, according to the similarity in the comparison result, match the target code corresponding to each stream, specifically including:

[0030] For the stream for direct matching, select several elements with the highest similarity corresponding to the independent function set as the target code;

[0031] For the stream for code matching, respectively select several elements with the highest similarity corresponding to each in the class set and the independent function set, and combine them as the target code;

[0032] For the stream for component function matching, select the element with the highest similarity corresponding to each component function and combine them as the target code.

[0033] In one example, combine the target codes corresponding to each stream to obtain the code search result, specifically including:

[0034] Obtain the enhanced content of the second natural language query compared to the first natural language query;

[0035] Based on the enhanced content, determine the weights of the target codes corresponding to each stream;

[0036] Based on the weights, combine the target codes corresponding to each stream to obtain the code search result.

[0037] On the other hand, the present application also proposes a code search device based on a large model agent, including:

[0038] At least one processor; and,

[0039] A memory communicatively connected to the at least one processor; wherein,

[0040] The memory stores instructions executable by the at least one processor, and when the instructions are executed by the at least one processor, the at least one processor is enabled to execute operations such as:

[0041] Obtain a first natural language query and a target code library;

[0042] Enhance and prompt the first natural language query through the target code library to generate a second natural language query;

[0043] Use the second natural language query as input through a large language model and output a corresponding query code;

[0044] In the target code library, create text embeddings for the second natural language query and the query code, compare them based on a multi-stream architecture, and obtain the target code corresponding to each stream;

[0045] Combine the target codes corresponding to each stream to obtain a code search result.

[0046] On the other hand, the present application also proposes a non-volatile computer storage medium storing computer-executable instructions, and the computer-executable instructions are configured to:

[0047] Obtain a first natural language query and a target code library;

[0048] Enhance and prompt the first natural language query through the target code library to generate a second natural language query;

[0049] Use the second natural language query as input through a large language model and output a corresponding query code;

[0050] In the target code library, create text embeddings for the second natural language query and the query code, compare them based on a multi-stream architecture, and obtain the target code corresponding to each stream;

[0051] Combine the target codes corresponding to each stream to obtain a code search result.

[0052] The code search method based on a large model agent proposed by the present application can bring the following beneficial effects:

[0053] 1. By enhancing the user query through the target code library context, technical details can be automatically supplemented, the polysemy of terms can be alleviated, the search deviation caused by fuzzy input can be reduced, and lexical ambiguity can be dynamically eliminated.

[0054] 2. Convert the enhanced natural language query into intermediate code, narrow the semantic gap between natural language and code through semantic matching at the code layer, improve the accuracy of intent mapping, and make semantic alignment more precise.

[0055] 3. The multi-stream architecture simultaneously compares natural language queries and generated code, covering function-level and class-level code snippets, reduces the missed detection rate, and integrates the multi-stream matching results, reduces the limitations of a single strategy, and improves the relevance and reliability of search results. Description of the Drawings

[0056] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation of the present application. In the drawings:

[0057] Figure 1 It is a schematic flowchart of the code search method based on the large model intelligent agent in the embodiment of the present application;

[0058] Figure 2 It is a schematic diagram of the code search device based on the large model intelligent agent in the embodiment of the present application. Detailed Embodiments

[0059] To make the objectives, technical solutions, and advantages of the present application clearer, the technical solutions of the present application will be clearly and completely described below in conjunction with the specific embodiments of the present application and the corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present application without creative efforts shall fall within the protection scope of the present application.

[0060] The following will describe in detail the technical solutions provided by each embodiment of the present application in conjunction with the drawings.

[0061] As Figure 1 shown, the embodiment of the present application provides a code search method based on a large model intelligent agent, including:

[0062] S101: Obtain a first natural language query and a target code library.

[0063] The large model intelligent agent is also called the large language model intelligent agent (LLM Agent). Multiple specialized intelligent agents in the large model intelligent agent cooperate to handle different aspects of tasks. Compared with a single large language model, it has enhanced reasoning and decision-making capabilities. Figure 1 The shown code search method can be implemented based on the corresponding large language model intelligent agent framework.

[0064] The first natural language query is the query content input by the user, and the target code library is the code library where the target code to be queried this time is located. For example, the target code library can be GitHub, which is a web-based code hosting platform and software development collaboration community that contains a large number of code snippets and supports functions such as code hosting, collaborative development, community interaction, and open source sharing.

[0065] Here, the first natural language query is referred to as Q, and the target code library is referred to as D. By contextualizing the first natural language query Q compared to D and enriching the first natural language query Q with additional details, the matching between the user input and the correct code snippet is improved.

[0066] S102: Use the target code library to enhance the prompt for the first natural language query and generate a second natural language query.

[0067] The relevant information in the target code library (for example, it can be the text information in the target code, or code snippets, or text information translated from code snippets, etc.), and the relevant information retrieved from the Internet through the Retrieval-augmented Generation (RAG) technology can be used for enhanced prompting.

[0068] Specifically, for the first natural language query, add restrictive conditions. The restrictive conditions can be described in natural language, and the first natural language query and the restrictive conditions are input into a large language model to generate a more detailed second natural language query.

[0069] Among them, the restrictive conditions mainly include restrictions on the enhancement scope, network connection status, etc., to ensure that there will be no excessive enhancement this time.

[0070] According to the restrictive conditions, try to enhance the implementation method of the first natural language query through the context information of the target code library, and during the enhancement process, limit the enhancement scope through the restrictive conditions.

[0071] Considering that the content in the target code library is highly credible, first try to enhance the implementation method through the target code library. If the attempt fails and no corresponding enhancement method is found, then enhance it through other channels. For example, if the attempt fails, enhance the implementation method of the first natural language query based on the network and / or local knowledge base, and during the enhancement process, limit the enhancement scope and the website call process through the restrictive conditions.

[0072] Finally, the second natural language query can be output.

[0073] The following gives an exemplary description: "Given an input text prompt: [Q]. Add more technical details about certain topics in the following text prompt to the general context of the target code library: [D]. If you cannot find its implementation in the target code library, provide information about its general implementation. Ensure that the information you obtain does not exceed the necessary scope and only provide specific information about the topics present in the input text prompt. Your paragraph will help localize the ideas in the input text prompt in a large repository, so deviating from the topic may lead to inaccuracies in the pipeline. You have a timer, so act quickly, and you can only be called twice, and view at most one website each time you are called."

[0074] Among them, the input Q is the first natural language query, D is the target code library, "more technical details of the topic" corresponds to the enhancement of the implementation method, the general implementation method of the target code library (including the local knowledge base and the network) corresponds to the order of attempting to enhance, "the information does not exceed the necessary scope" represents the limiting condition of the enhancement scope, and "view at most one website each time you are called" represents the limiting condition of the website calling process. In this way, it can be ensured that the enhancement is relevant and focused on the topics present in the query.

[0075] For ease of description, here the second natural language query is called A, and obtaining A using the RAG technology can be described as: .

[0076] S103: Through a large language model, take the second natural language query as input and output the corresponding query code.

[0077] After obtaining the query code obtained by converting the second natural language query through the large language model LLM, the quality of the code can be evaluated. If the quality is qualified, continue with the subsequent steps. If the quality is unqualified, the corresponding query code can be regenerated by replacing with other large language models.

[0078] Here the query code is called C, and its output process can be described as: .

[0079] S104: In the target code library, create text embeddings for the second natural language query and the query code, and compare them based on a multi-stream architecture to obtain the target code corresponding to each stream.

[0080] The multi-stream architecture refers to an architecture method in which a system or model uses multiple parallel streams to operate when processing data or performing tasks. Each stream can independently process a part of the data or perform specific subtasks, and these streams cooperate and interact with each other to finally complete the functions of the entire system.

[0081] Text embedding aims to map words, sentences, or codes in text into vectors in a low-dimensional vector space. These vectors can capture the semantic and syntactic information of the text, enabling the computer to understand the content and meaning of the text by calculating and analyzing these vectors.

[0082] By using text embedding to capture subtle meanings, translate natural language queries into codes, and use an integrated method for multi-stream comparison. The accuracy and relevance of the retrieved code snippets are improved by examining multiple aspects of the query and code context.

[0083] Specifically, the codes in the target code library are grouped to obtain the corresponding class set and independent function set. Here, the class set is called Z, and the independent function set is called Y. The independent function set contains all the independent functions in the code library (functions that do not belong to any class), while the class set contains all the class definitions in the code library (including the methods in the classes).

[0084] At this time, based on each stream in the multi-stream architecture, select some content from the second natural language query and the query code, and compare it with the elements contained in the class set and / or independent function set through text embedding. In each stream, the two sides selected for comparison are different, so the code snippet that best meets the user's expectations can be comprehensively selected in different dimensions.

[0085] According to the similarity in the comparison results, the target code corresponding to each stream is matched. During the comparison process, the cosine similarity between the two sides can be obtained by calculating the cosine distance between the two sides.

[0086] The following provides several exemplary streams. In the actual comparison process, other streams can also be added or modified based on requirements.

[0087] The first stream is based on the stream for direct matching in the multi-stream architecture. It directly matches the natural language with the code for calculation, so the second natural language query is selected.

[0088] The second natural language query is compared with each element in the independent function set through text embedding to obtain the corresponding comparison results.

[0089] Generate an embedding vector for the second natural language query A, as described by the formula: , , where is the embedding vector corresponding to the second natural language query A, is the set of embedding vectors corresponding to the independent function set, is the embedding vector corresponding to each element (i.e., each independent function) in the independent function set.

[0090] Determine the corresponding similarity by calculating the cosine distance as the comparison result: .

[0091] At this time, several elements with the highest similarity corresponding to the independent function set can be selected as the target code. For example, select three elements as the target code and add them to the final set: .

[0092] The second stream is based on the stream for code matching in the multi-stream architecture. After converting natural language to code, matching calculations are performed between the codes, so the query code is selected.

[0093] Compare the similarity between the query code and each element in the class set and the independent function set through text embedding to obtain the corresponding comparison result.

[0094] Similarly, generate an embedding vector for the query code as described by the formula: , , , where is the embedding vector corresponding to the query code C, is the set of embedding vectors corresponding to the class set, is the embedding vector corresponding to each element in the class set (i.e., each class and its contained methods).

[0095] Determine the corresponding similarity by calculating the cosine distance as the comparison result: 、 .

[0096] It is possible to select several elements with the highest similarity corresponding to the class set and the independent function set respectively and combine them as the target code. For example, select three elements with the highest similarity as the target code: 、 .

[0097] The third stream is based on the stream for component function matching in the multi-stream architecture. It does not directly compare natural language or code, but splits the code into multiple component functions and then makes comparisons at a more refined level. Therefore, the query code is split to obtain multiple component functions.

[0098] Compare the similarity between each component function and each element in the independent function set through text embedding to obtain the corresponding comparison result.

[0099] Split the query code C to obtain multiple component functions and generate embedding vectors corresponding to each component function as described by the formula: 、 , where is a component function, is the embedding vector corresponding to the component function.

[0100] Determine the corresponding similarity by calculating the cosine distance as the comparison result: .

[0101] At this time, for the stream used for component function matching, the element with the highest similarity corresponding to each component function can be selected and combined as the target code: .

[0102] S105: Combine the target codes corresponding to each stream to obtain the code search result.

[0103] In each stream, the corresponding target code has been obtained. Here, these target codes can be combined, and the final set obtained can be used as the code search result for this user: .

[0104] In this way, it is ensured that according to the initial input of the first natural language query, code snippets are comprehensively and targeted selected. The creation of the final target set significantly reduces the number of potential code snippets, from a large number of code snippets to about 5 to 15 snippets. And through matching based on code in the multi-stream architecture, the final target set is further processed to identify the most relevant snippets.

[0105] Of course, it is also possible to obtain the enhanced content of the second natural language query compared to the first natural language query, and the enhanced content includes the added text content.

[0106] Based on the enhanced content, determine the weights of the target codes corresponding to each stream. Generally speaking, the more added text content (which can be determined by the text length of the added enhanced content), the more comprehensive the supplement to the user's description, the higher the credibility of the second natural language query. Considering that there may be inaccurate translations during the translation process from natural language to code, at this time, the weight of the stream directly matched based on the second natural language query can be increased, and the weights of other streams can be decreased. That is to say, the longer the text length of the enhanced content, the higher the weight of the stream used for direct matching.

[0107] Based on the weights, combine the target codes corresponding to each stream to obtain the code search result. The higher the weight, the more target codes can be selected from its corresponding target code and added to the code search result. Or, the higher the weight, the higher its ranking in the code search result.

[0108] 1. By enhancing the user query through the target code library context, technical details can be automatically supplemented, the polysemy of terms can be alleviated, search biases caused by fuzzy input can be reduced, and lexical ambiguity can be dynamically eliminated.

[0109] 2. Convert the enhanced natural language query into intermediate code, reduce the semantic gap between natural language and code through semantic matching at the code layer, improve the accuracy of intent mapping, and make semantic alignment more precise.

[0110] 3. The multi-stream architecture simultaneously compares natural language queries and generated code, covering function-level and class-level code fragments, reduces the missed detection rate, and integrates the multi-stream matching results, reduces the limitations of a single strategy, and improves the relevance and reliability of search results.

[0111] As Figure 2 shown, the present application also proposes a code search device based on a large model agent, including:

[0112] At least one processor; and,

[0113] A memory communicatively connected to the at least one processor; wherein,

[0114] The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute as follows:

[0115] Obtain a first natural language query and a target code library;

[0116] Through the target code library, perform enhanced prompting on the first natural language query to generate a second natural language query;

[0117] Using a large language model, take the second natural language query as input and output the corresponding query code;

[0118] In the target code library, create text embeddings for the second natural language query and the query code, compare them based on the multi-stream architecture, and obtain the target code corresponding to each stream;

[0119] Combine the target codes corresponding to each stream to obtain the code search result.

[0120] The present application also proposes a non-volatile computer storage medium storing computer-executable instructions, and the computer-executable instructions are set as:

[0121] Obtain a first natural language query and a target code library;

[0122] Through the target code library, perform enhanced prompting on the first natural language query to generate a second natural language query;

[0123] Using a large language model, take the second natural language query as input and output the corresponding query code;

[0124] In the target code library, create text embeddings for the second natural language query and the query code, and perform comparison based on a multi-stream architecture to obtain the target code corresponding to each stream.

[0125] Combine the target codes corresponding to each stream to obtain the code search result.

[0126] Each embodiment in this application is described in a progressive manner. For the same or similar parts among the embodiments, reference can be made to each other. Each embodiment focuses on the differences from other embodiments. In particular, for the device and medium embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and for the relevant parts, reference can be made to the partial description of the method embodiments.

[0127] The devices and media provided in the embodiments of this application correspond one by one to the methods. Therefore, the devices and media also have beneficial technical effects similar to those of their corresponding methods. Since the beneficial technical effects of the methods have been described in detail above, the beneficial technical effects of the devices and media will not be elaborated here.

[0128] The above are only the embodiments of this application and are not intended to limit this application. For those skilled in the art, various changes and modifications can be made to this application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of this application shall be included within the scope of the claims of this application.

Claims

1. A code search method based on a large model agent, characterized in that: include: Obtaining a first natural language query and a target code base; By using the target code library, enhancing the first natural language query with prompts to generate a second natural language query; Using the large language model, taking the second natural language query as input, and outputting a corresponding query code; In the target code library, create text embeddings for the second natural language query and the query code, compare based on a multi-stream architecture, and obtain the target code corresponding to each stream; The target codes corresponding to each stream are combined to obtain the code search result.

2. The code search method based on large model agent according to claim 1 is characterized in that: By using the target code library, enhancing the first natural language query to generate a second natural language query specifically includes: Adding restriction conditions to the first natural language query; According to the restriction condition, attempt to enhance the first natural language query implementation manner through the context information of the target code library, and during the enhancement process, limit the enhancement scope through the restriction condition; If the attempt fails, the implementation method of the first natural language query is enhanced based on the network and / or local knowledge base, and during the enhancement process, the enhancement scope and the website calling process are limited by the restriction conditions; Output a second natural language query.

3. The code search method based on large model agent according to claim 1 is characterized in that: In the target code library, text embedding is created for the second natural language query and the query code, and comparison is performed based on a multi-stream architecture to obtain a target code corresponding to each stream, specifically including: Grouping the codes in the target code library to obtain corresponding class sets and independent function sets; Based on each stream in the multi-stream architecture, select a portion of the second natural language query and the query code, and compare them with the elements included in the class set and / or the independent function set through text embedding; According to the similarity in the comparison results, the target code corresponding to each stream is matched.

4. The code search method based on large model agent according to claim 3 is characterized in that: Based on each stream in the multi-stream architecture, selecting a portion of the content in the second natural language query and the query code and comparing the content with the elements included in the class set and / or the independent function set through text embedding, specifically includes: Selecting the second natural language query based on the stream for direct matching in the multi-stream architecture; The second natural language query is compared with each element in the independent function set for similarity through text embedding to obtain a corresponding comparison result.

5. The code search method based on large model agent according to claim 3 is characterized in that: Based on each stream in the multi-stream architecture, selecting a portion of the content in the second natural language query and the query code and comparing the content with the elements included in the class set and / or the independent function set through text embedding, specifically includes: Selecting the query code based on a stream for code matching in a multi-stream architecture; The query code is compared with each element in the class set and the independent function set through text embedding to obtain a corresponding comparison result.

6. The code search method based on large model agent according to claim 3 is characterized in that: Based on each stream in the multi-stream architecture, selecting a portion of the content in the second natural language query and the query code and comparing the content with the elements included in the class set and / or the independent function set through text embedding, specifically includes: Based on the stream for component function matching in the multi-stream architecture, split the query code to obtain multiple component functions; Each component function is compared with each element in the independent function set by text embedding to obtain a corresponding comparison result.

7. The code search method based on a large model agent according to any one of claims 4 to 6, characterized in that: According to the similarity in the comparison results, the target code corresponding to each stream is matched, including: For the flow used for direct matching, select several elements with the highest similarity corresponding to the independent function set as target codes; For the flow used for code matching, several elements with the highest similarity are selected from the class set and the independent function set, and the combination is used as the target code; For the stream used for component function matching, the elements with the highest similarity corresponding to each component function are selected and combined as the target code.

8. The code search method based on large model agent according to claim 7 is characterized in that: Combine the target codes corresponding to each stream to obtain the code search results, including: Obtaining enhanced content of the second natural language query compared to the first natural language query; Based on the enhanced content, determining the weight of the target code corresponding to each stream; Based on the weights, the target codes corresponding to the various flows are combined to obtain a code search result.

9. A code search device based on a large model agent, characterized in that: include: at least one processor; as well as, a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform, for example: Obtaining a first natural language query and a target code base; By using the target code library, enhancing the first natural language query with prompts to generate a second natural language query; Using the large language model, taking the second natural language query as input, and outputting a corresponding query code; In the target code library, create text embeddings for the second natural language query and the query code, compare based on a multi-stream architecture, and obtain the target code corresponding to each stream; The target codes corresponding to each stream are combined to obtain the code search result.

10. A non-volatile computer storage medium storing computer executable instructions, characterized in that: The computer executable instructions are configured to: Obtaining a first natural language query and a target code base; By using the target code library, enhancing the first natural language query with prompts to generate a second natural language query; Using the large language model, taking the second natural language query as input, and outputting a corresponding query code; In the target code library, create text embeddings for the second natural language query and the query code, compare based on a multi-stream architecture, and obtain the target code corresponding to each stream; The target codes corresponding to each stream are combined to obtain the code search result.

Images