Retrieval code reordering method and device based on natural language

Abstract

The invention provides a retrieval code reordering method and device based on a natural language, and the method comprises the steps: obtaining a plurality of candidate code blocks corresponding to a natural language question of a user, and calculating the word frequency of each candidate code block in the plurality of candidate code blocks; obtaining a plurality of keywords obtained by rewriting based on the user natural language question, and calculating the keyword frequency of each keyword in the plurality of keywords; calculating based on the word frequency and the keyword frequency to obtain a first word frequency inverse document frequency and a second word frequency inverse document frequency of each keyword in the plurality of keywords; and performing calculation based on the first word frequency inverse document frequency, the second word frequency inverse document frequency and the logarithm of the keyword length to obtain a weighted score of the corresponding keyword, and performing reordering based on the weighted score to obtain an ordering result. Through the method provided by the invention, the performance of the retrieval enhancement generation system is further improved.

Description

Technical Field

[0001] The present invention relates to the technical field of data processing, and in particular to a natural language-based retrieval code reordering method and device. Background Art

[0002] In traditional RAG (Retrieval-Augmented Generation) systems, the vector library->ReRank model is often used to complete the retrieval enhancement process. However, this solution is affected by the effects of the embedding model and the rerank model, resulting in unsatisfactory rerank results.

[0003] In the code question-and-answer scenario, how to make the sorting of retrieved results more in line with user needs is a technical problem that needs to be solved at present. Summary of the invention

[0004] The present invention provides a natural language-based retrieval code reordering method and device to solve the defects in the prior art.

[0005] The present invention provides a natural language-based retrieval code reordering method, comprising the following steps: Acquire multiple candidate code blocks corresponding to the user's natural language question, and calculate the word frequency of each candidate code block in the multiple candidate code blocks; wherein the word frequency is: the number of times each word appears in the corresponding candidate code block; Acquire multiple keywords rewritten based on the user's natural language question, and calculate the keyword frequency of each keyword in the multiple keywords; wherein the keyword frequency is: the number of times each keyword appears in the user's natural language question; Calculating based on the word frequency and the keyword frequency, obtaining a first word frequency inverse document frequency and a second word frequency inverse document frequency of each keyword in the multiple keywords; wherein the first word frequency inverse document frequency represents: the word frequency inverse document frequency of the keyword in the multiple candidate code blocks, and the second word frequency inverse document frequency represents: the word frequency inverse document frequency of the keyword in the user's natural language question; Based on the first term frequency inverse document frequency, the second term frequency inverse document frequency and the logarithm of the keyword length, a weighted score of the corresponding keyword is obtained, and re-sorting is performed based on the weighted score to obtain a sorting result.

[0006] According to a natural language-based retrieval code reordering method provided by the present invention, the step of obtaining multiple candidate code blocks corresponding to a user's natural language question and calculating the word frequency of each candidate code block in the multiple candidate code blocks includes: Acquire multiple candidate code blocks corresponding to the user's natural language question, and perform word segmentation processing on each of the multiple candidate code blocks using a regular expression; For each candidate code block in the multiple candidate code blocks, the number of times each word appears in the corresponding candidate code block is calculated to obtain the word frequency of each candidate code block in the multiple candidate code blocks.

[0007] According to a natural language-based retrieval code reordering method provided by the present invention, the calculation is performed based on the word frequency and the keyword frequency to obtain a first word frequency inverse document frequency and a second word frequency inverse document frequency of each keyword in the multiple keywords, including: For each keyword, multiply the inverse document frequency of the keyword by the word frequency of the corresponding word to obtain the first word frequency inverse document frequency of the keyword; For each keyword, the inverse document frequency of the keyword and the keyword term frequency of the keyword are multiplied to obtain a second term frequency inverse document frequency of the keyword.

[0008] According to a natural language-based retrieval code reordering method provided by the present invention, the weighted score of the corresponding keyword is obtained by calculating based on the first term frequency inverse document frequency, the second term frequency inverse document frequency and the logarithm of the keyword length, including: For each keyword, calculate the logarithm of the keyword length; The first term frequency inverse document frequency, the second term frequency inverse document frequency and the logarithm of the keyword length are multiplied to obtain a weighted score for the keyword.

[0009] According to a natural language-based retrieval code reordering method provided by the present invention, before obtaining multiple candidate code blocks corresponding to the user's natural language question and calculating the word frequency of each candidate code block in the multiple candidate code blocks, the method also includes: Obtain user's natural language questions; Based on the user's natural language question, a search is performed in the target vector library to obtain the multiple candidate code blocks corresponding to the user's natural language question.

[0010] According to a natural language-based retrieval code reordering method provided by the present invention, after obtaining the user's natural language question, the method further includes: The user's natural language question is rewritten based on the target vector library through a large language model to obtain the multiple keywords corresponding to the user's natural language question.

[0011] The present invention also provides a natural language-based retrieval code reordering device, comprising the following modules: A first acquisition module is used to acquire multiple candidate code blocks corresponding to the user's natural language question, and calculate the word frequency of each candidate code block in the multiple candidate code blocks; wherein the word frequency is: the number of times each word appears in the corresponding candidate code block; A second acquisition module is used to acquire multiple keywords rewritten based on the user's natural language question, and calculate the keyword frequency of each keyword in the multiple keywords; wherein the keyword frequency is: the number of times each keyword appears in the user's natural language question; A calculation module, configured to calculate based on the word frequency and the keyword frequency to obtain a first word frequency inverse document frequency and a second word frequency inverse document frequency of each keyword in the multiple keywords; wherein the first word frequency inverse document frequency represents: the word frequency inverse document frequency of the keyword in the multiple candidate code blocks, and the second word frequency inverse document frequency represents: the word frequency inverse document frequency of the keyword in the user's natural language question; The reordering module is used to calculate based on the first term frequency inverse document frequency, the second term frequency inverse document frequency and the logarithm of the keyword length to obtain a weighted score of the corresponding keyword, and reorder based on the weighted score to obtain a ranking result.

[0012] According to a natural language-based retrieval code reordering device provided by the present invention, the first acquisition module is specifically used to: Acquire multiple candidate code blocks corresponding to the user's natural language question, and perform word segmentation processing on each of the multiple candidate code blocks using a regular expression; For each candidate code block in the multiple candidate code blocks, the number of times each word appears in the corresponding candidate code block is calculated to obtain the word frequency of each candidate code block in the multiple candidate code blocks.

[0013] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the natural language-based retrieval code reordering method described above is implemented.

[0014] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the computer program implements any of the natural language-based retrieval code reordering methods described above.

[0015] The present invention also provides a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the computer program implements any of the above-mentioned natural language-based retrieval code reordering methods.

[0016] The present invention provides a natural language-based retrieval code reordering method and device, which obtains multiple candidate code blocks corresponding to user natural language questions and calculates the word frequency of each candidate code block in the multiple candidate code blocks; wherein the word frequency is: the number of times each word appears in the corresponding candidate code block; obtains multiple keywords rewritten based on the user natural language question, and calculates the keyword frequency of each keyword in the multiple keywords; wherein the keyword frequency is: the number of times each keyword appears in the user natural language question; based on the word frequency and the keyword frequency, a first word frequency inverse document frequency and a second word frequency inverse document frequency of each keyword in the multiple keywords are calculated; wherein the first word frequency inverse document frequency represents: the word frequency inverse document frequency of the keyword in the multiple candidate code blocks, and the second word frequency inverse document frequency represents: the word frequency inverse document frequency of the keyword in the user natural language question; based on the first word frequency inverse document frequency, the second word frequency inverse document frequency and the logarithm of the keyword length, a weighted score of the corresponding keyword is obtained, and reordering is performed based on the weighted score to obtain a sorting result. It can be seen that the present invention further improves the performance of the retrieval enhancement generation system by adding a step of calculating the inverse document frequency of the term frequency, re-ranking according to the calculated similarity, and weighting the similarity of each term frequency inverse document frequency according to specific rules. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0018] Figure 1 This is one of the flow charts of the natural language-based retrieval code reordering method provided by the present invention.

[0019] Figure 2 This is the second flowchart diagram of the natural language-based retrieval code reordering method provided by the present invention.

[0020] Figure 3 This is the third flowchart diagram of the natural language-based retrieval code reordering method provided by the present invention.

[0021] Figure 4It is a schematic diagram of the structure of the natural language-based retrieval code reordering device provided by the present invention.

[0022] Figure 5 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0023] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0024] Before describing the method of the present invention in detail, the terms involved in the present invention are first explained schematically.

[0025] TF-IDF (Term Frequency-Inverse Document Frequency) is a method used to measure the importance of a word in a document. It is very commonly used in fields such as text analysis, information retrieval, and natural language processing. In simple terms, TF-IDF can help us find the "key" and "unique" words in an article, rather than those common but meaningless words.

[0026] ReRank is a technique used to "secondarily sort" search results. After entering a keyword in the search engine, such as "cat feeding methods", the search engine will give a bunch of results based on the keyword, but the order of these results may not be ideal, and some important information may be ranked at the back. At this time, ReRank can be used to make the order of the results more in line with user needs.

[0027] Combine the following Figure 1-Figure 5 A natural language-based retrieval code reordering method and device of the present invention are described.

[0028] Figure 1 is one of the flow charts of the natural language based retrieval code reordering method provided by the present invention, such as Figure 1 As shown, the method includes the following: Step 100: Obtain multiple candidate code blocks corresponding to the user's natural language question, and calculate the word frequency of each candidate code block in the multiple candidate code blocks; wherein the word frequency is: the number of times each word appears in the corresponding candidate code block.

[0029] It should be noted that in the code question and answer scenario, there may be valid information provided by multiple data sources. This embodiment combines the information from multiple data sources and re-sorts them to filter out the most important information to enhance the effect of the entire RAG system. This embodiment can be applied to the code repository question and answer scenario.

[0030] Before step 100, the method further includes: Step S110: Obtain the user's natural language question.

[0031] Step S120: searching in a target vector library based on the user's natural language question to obtain the multiple candidate code blocks corresponding to the user's natural language question.

[0032] Specifically, the user's natural language question is used as a query to search in the target vector library. The target vector library may contain a large number of pre-processed code blocks and their corresponding vector representations. During the retrieval process, the similarity between the user's question and the code block vector in the library is calculated to find the best matching candidate code block. For the multiple candidate code blocks retrieved, the word frequency of each word in each code block is calculated for subsequent calculations.

[0033] Furthermore, after obtaining multiple candidate code blocks, the search results can be preliminarily sorted using conventional sorting algorithms (such as BM25, cosine similarity, etc.). This step is not necessary, but it can improve the efficiency of subsequent TF-IDF ReRank.

[0034] Step S130: rewrite the user's natural language question through a large language model based on the target vector library to obtain the multiple keywords corresponding to the user's natural language question.

[0035] Specifically, large language models (such as GPT, BERT, etc.) are used to rewrite or parse the user's natural language questions. The purpose of rewriting is to extract the core elements of the question and generate more accurate or easier to search keywords. Keywords can be used to further filter or sort candidate code blocks in subsequent steps. For multiple keywords obtained by rewriting, the word frequency of each keyword is calculated for subsequent calculations.

[0036] Step 200: Obtain multiple keywords rewritten based on the user's natural language question, and calculate the keyword frequency of each keyword in the multiple keywords; wherein the keyword frequency is: the number of times each keyword appears in the user's natural language question.

[0037] Step 300: Calculate based on the word frequency and the keyword frequency to obtain a first word frequency inverse document frequency and a second word frequency inverse document frequency for each of the multiple keywords; wherein the first word frequency inverse document frequency represents: the word frequency inverse document frequency of the keyword in the multiple candidate code blocks, and the second word frequency inverse document frequency represents: the word frequency inverse document frequency of the keyword in the user's natural language question.

[0038] Specifically, based on the word frequency and keyword frequency of the candidate code block, the term frequency inverse document frequency (TF-IDF) of each keyword in two different contexts (the candidate code block set and the user question rewriting result) is calculated. Here, we call it the first term frequency inverse document frequency (TF-IDF1, for the candidate code block set) and the second term frequency inverse document frequency (TF-IDF2, for the user question rewriting result).

[0039] For each keyword, we need to find its occurrence in all candidate code blocks and calculate its inverse document frequency (IDF). Then, for each candidate code block containing the keyword, we multiply the word frequency (TF1) of the keyword in the code block by the inverse document frequency (IDF) of the keyword to get the first word frequency inverse document frequency (TF-IDF1) of the keyword in the code block.

[0040] For each keyword, its appearance in the user's question or rewritten result also needs to be considered. Specifically, the keyword's keyword frequency (TF2, i.e., the number of times the keyword appears in the user's question or rewritten result) is multiplied by the keyword's inverse document frequency (IDF) to obtain the second keyword frequency inverse document frequency (TF-IDF2) of the keyword in the user's question or rewritten result.

[0041] In one embodiment, how to calculate the term frequency inverse document frequency (TF-IDF) of each keyword in two different contexts (a set of candidate code blocks and a user question rewriting result), namely, the first term frequency inverse document frequency (TF-IDF1) and the second term frequency inverse document frequency (TF-IDF2) is described.

[0042] Assume that the number of candidate code blocks N=3, and the set of candidate code blocks is: code_block1: def add(a, b): return a + b; code_block2: def subtract(a, b): return a – b; code_block3: def multiply(a, b): return a * b.

[0043] The user's natural language question is: How do I subtract two numbers in Python? The keywords rewritten based on the user's natural language question are: ['def', 'subtract', 'numbers', 'Python'].

[0044] 1) Calculate the inverse document frequency (IDF) of each keyword.

[0045] The IDF calculation formula is: , where DF(t) represents the number of documents containing keyword t (i.e., the number of candidate code blocks).

[0046] Calculate the IDF of 4 keywords respectively: 2) Calculate the first term frequency inverse document frequency (TF-IDF1) for each keyword in each candidate code block.

[0047] ①The word segmentation result of candidate code block 1 is: ['def', 'add', 'a', 'b', 'return', 'a', '+', 'b'].

[0048] Calculate the TF-IDF1 of each keyword corresponding to the candidate code block 1: def:TF1 = 1 / 8, TF-IDF1 =1 / 8× subtract:TF1 = 0, TF-IDF1 = 0 numbers: TF1 = 0, TF-IDF1 = 0 Python: TF1 = 0, TF-IDF1 = 0 ②The word segmentation result of candidate code block 2 is: ['def', 'subtract', 'a', 'b', 'return', 'a', '-', 'b'].

[0049] Calculate the TF-IDF1 of each keyword corresponding to candidate code block 2: def:TF1 = 1 / 8, TF-IDF1 =1 / 8× subtract: TF1 = 1 / 8, TF-IDF1 = 1 / 8× numbers: TF1 = 0, TF-IDF1 = 0 Python: TF1 = 0, TF-IDF1 = 0 ③The word segmentation result of candidate code block 3 is: ['def', 'multiply', 'a', 'b', 'return', 'a', '*', 'b'].

[0050] Calculate the TF-IDF1 of each keyword corresponding to the candidate code block 3: def:TF1 = 1 / 8, TF-IDF1 =1 / 8× subtract:TF1 = 0, TF-IDF1 = 0 numbers: TF1 = 0, TF-IDF1 = 0 Python: TF1 = 0, TF-IDF1 = 0 3) Calculate the second term frequency inverse document frequency (TF-IDF2) for each keyword in the user's natural language question.

[0051] def: TF2 = 0, TF-IDF2 = 0 subtract: TF2 = 1, TF-IDF2 =1× numbers: TF2 = 1, TF-IDF2 =1×log3 Python: TF2 = 1, TF-IDF2 =1×log3 Step 400: Calculate based on the first term frequency inverse document frequency, the second term frequency inverse document frequency and the logarithm of the keyword length to obtain a weighted score for the corresponding keyword, and re-sort based on the weighted score to obtain a sorting result.

[0052] It should be noted that the logarithm of the keyword length is introduced to consider the impact of the keyword length in the weighted score, thereby preventing long keywords from automatically obtaining higher weights due to the inclusion of more words.

[0053] Specifically, the weighted score of each keyword is calculated by combining the first term frequency inverse document frequency, the second term frequency inverse document frequency, and the logarithm of the keyword length (used to adjust the weight difference between long keywords and short keywords). Then, the candidate code blocks are re-ranked according to these weighted scores.

[0054] Weighted score weighted_score(t) = TF-IDF1(t, d) * TF-IDF2(t) * log(length(t)). The calculation of the weighted score of each keyword in each candidate code block is described in combination with the above embodiment.

[0055] For candidate code block 1: weighted_score('def') = TF-IDF1 * TF-IDF2 * log3=0 weighted_score(' subtract ') = TF-IDF1 * TF-IDF2 * log8=0 weighted_score(' numbers ') = TF-IDF1 * TF-IDF2 * log7=0 weighted_score(' Python ') = TF-IDF1 * TF-IDF2 * log6=0 The total weighted score of candidate code block 1 is 0.

[0056] For candidate code block 2: weighted_score('def') = TF-IDF1 * TF-IDF2 * log3=0 weighted_score(' subtract ') = TF-IDF1 * TF-IDF2 * log8= 1 / 8× * * log8≈0.0386 weighted_score(' numbers ') = TF-IDF1 * TF-IDF2 * log7=0 weighted_score(' Python ') = TF-IDF1 * TF-IDF2 * log6=0 The total weighted score of candidate code block 2 is 0.0386.

[0057] For candidate code block 3: weighted_score('def') = TF-IDF1 * TF-IDF2 * log3=0 weighted_score(' subtract ') = TF-IDF1 * TF-IDF2 * log8=0 weighted_score(' numbers ') = TF-IDF1 * TF-IDF2 * log7=0 weighted_score(' Python ') = TF-IDF1 * TF-IDF2 * log6=0 The total weighted score of candidate code block 3 is 0.

[0058] The result of reordering the candidate code blocks according to the above weighted scores is: candidate code block 2, candidate code block 1, candidate code block 3.

[0059] By comprehensively considering multiple statistics between candidate code blocks and user questions (including word frequency, keyword frequency, TF-IDF, and keyword length logarithm), the candidate code blocks are effectively re-ranked. This method helps to improve the accuracy and relevance of code retrieval, thereby providing users with better code recommendation services.

[0060] The above is a description of the steps of the natural language-based retrieval code reordering method provided by the present invention. It can be seen from the description of the above steps that according to the natural language-based retrieval code re-ranking method provided by the present invention, multiple candidate code blocks corresponding to the user's natural language question are obtained, and the word frequency of each candidate code block in the multiple candidate code blocks is calculated; wherein the word frequency is: the number of times each word appears in the corresponding candidate code block; multiple keywords rewritten based on the user's natural language question are obtained, and the keyword frequency of each keyword in the multiple keywords is calculated; wherein the keyword frequency is: the number of times each keyword appears in the user's natural language question; based on the word frequency and the keyword frequency, a first word frequency inverse document frequency and a second word frequency inverse document frequency of each keyword in the multiple keywords are calculated; wherein the first word frequency inverse document frequency represents: the word frequency inverse document frequency of the keyword in the multiple candidate code blocks, and the second word frequency inverse document frequency represents: the word frequency inverse document frequency of the keyword in the user's natural language question; based on the first word frequency inverse document frequency, the second word frequency inverse document frequency and the logarithm of the keyword length, a weighted score of the corresponding keyword is obtained, and re-ranking is performed based on the weighted score to obtain a ranking result. It can be seen that the present invention further improves the performance of the retrieval enhancement generation system by adding a step of calculating the inverse document frequency of the term frequency, re-ranking according to the calculated similarity, and weighting the similarity of each term frequency inverse document frequency according to specific rules.

[0061] Based on the above embodiments, in this embodiment, Figure 2FIG. 2 is a flowchart diagram of the natural language-based retrieval code reordering method provided by the present invention. Figure 2 As shown, step 100 obtains multiple candidate code blocks corresponding to the user's natural language question, and calculates the word frequency of each candidate code block in the multiple candidate code blocks, including: Step 210: Acquire multiple candidate code blocks corresponding to the user's natural language question, and perform word segmentation processing on each of the multiple candidate code blocks using a regular expression.

[0062] Specifically, for each candidate code block, a word segmentation process is performed using a regular expression. A regular expression can be used to match, find, and replace specific patterns in a text. In the present embodiment, a regular expression is used to identify word boundaries in a code block, thereby segmenting the code block into separate words. Since the code usually contains many special characters and identifiers, the word segmentation process may need to be customized for the characteristics of the code. For example, the naming rules of the identifier, the processing of the annotation, and the parsing of the code structure may need to be considered.

[0063] Step 220: For each candidate code block in the multiple candidate code blocks, calculate the number of times each word appears in the corresponding candidate code block to obtain the word frequency of each candidate code block in the multiple candidate code blocks.

[0064] Specifically, for each candidate code block after word segmentation, count the number of times each word appears in the corresponding code block. This usually involves traversing the list of words in the code block and using a hash table or dictionary to record the number of times each word appears. Ultimately, each candidate code block will get a word frequency statistics, that is, a mapping relationship between words and their number of occurrences.

[0065] The natural language-based retrieval code reordering method provided in this embodiment performs word segmentation processing on candidate code blocks through regular expressions and calculates the number of occurrences of each word in the corresponding code block, so as to obtain word frequency statistics information of each candidate code block for subsequent reordering of candidate code blocks.

[0066] Based on the above embodiments, in this embodiment, Figure 3 FIG. 3 is a flowchart diagram of a natural language-based retrieval code reordering method provided by the present invention. Figure 3 As shown, step 300 performs calculation based on the word frequency and the keyword frequency to obtain a first word frequency inverse document frequency and a second word frequency inverse document frequency of each keyword in the multiple keywords, including: Step 310: For each keyword, multiply the inverse document frequency of the keyword and the word frequency of the corresponding word to obtain the first word frequency inverse document frequency of the keyword.

[0067] Specifically, for each keyword, we need to find its occurrence in all candidate code blocks and calculate its inverse document frequency (IDF). Then, for each candidate code block containing the keyword, we multiply the word frequency (TF) of the keyword in the code block by the inverse document frequency (IDF) to get the first word frequency inverse document frequency (TF-IDF1) of the keyword in the code block.

[0068] Step 320: For each keyword, multiply the inverse document frequency of the keyword by the keyword term frequency of the keyword to obtain a second term frequency inverse document frequency of the keyword.

[0069] Specifically, for each keyword, its appearance in the user's question or rewritten result needs to be considered. Specifically, the keyword frequency (i.e., the number of times the keyword appears in the user's question or rewritten result) is multiplied by the inverse document frequency (IDF) to obtain the second word frequency inverse document frequency (TF-IDF2) of the keyword.

[0070] By calculating the first term frequency inverse document frequency (TF-IDF1) and the second term frequency inverse document frequency (TF-IDF2) of each keyword, we can get two statistics that reflect the importance of keywords in different contexts (candidate code blocks and user questions). These statistics can be used for subsequent weighted score calculation and re-ranking of candidate code blocks.

[0071] Continue to see Figure 3 Step 400 calculates based on the first term frequency inverse document frequency, the second term frequency inverse document frequency and the logarithm of the keyword length to obtain a weighted score of the corresponding keyword, including: Step 330: For each keyword, calculate the logarithm of the keyword length.

[0072] Specifically, for each keyword, the logarithm of its length is first calculated. The keyword length can refer to the number of characters in the keyword. The use of the logarithm is to introduce the factor of keyword length in the weighted score, while avoiding weight imbalance caused by excessive length differences.

[0073] Step 340: multiply the first term frequency inverse document frequency, the second term frequency inverse document frequency and the logarithm of the keyword length to obtain a weighted score for the keyword.

[0074] Specifically, the weighted score of each keyword is obtained by multiplying the first term frequency inverse document frequency (TF-IDF1), the second term frequency inverse document frequency (TF-IDF2) and the logarithm of the keyword length. This weighted score comprehensively considers the occurrence of the keyword in the candidate code block, the occurrence in the user question, and the length of the keyword itself. For details, please refer to the following formula.

[0075] Weighted score weighted_score(t) = TF-IDF1(t, d) * TF-IDF2(t) * log(length(t)) Among them, TF-IDF1(t, d) represents the first term frequency inverse document frequency of keyword t in candidate code block d, TF-IDF2(t) represents the second term frequency inverse document frequency of keyword t, and log(length(t)) represents the logarithm of the length of keyword t.

[0076] The natural language-based retrieval code reranking method provided in this embodiment adds a TF-IDF calculation step after the rerank model gives the result, reranks according to the calculated similarity, and weights each TF-IDF similarity according to specific rules, thereby further improving the performance of the RAG system.

[0077] The natural language-based retrieval code reordering device provided by the present invention is described below. The natural language-based retrieval code reordering device described below and the natural language-based retrieval code reordering method described above can refer to each other.

[0078] Figure 4 is a schematic diagram of the structure of the natural language based retrieval code reordering device provided by the present invention, such as Figure 4 As shown, the retrieval code reordering device based on natural language provided by the present invention includes: The first acquisition module 401 is used to acquire multiple candidate code blocks corresponding to the user's natural language question, and calculate the word frequency of each candidate code block in the multiple candidate code blocks; wherein the word frequency is: the number of times each word appears in the corresponding candidate code block; The second acquisition module 402 is used to acquire multiple keywords rewritten based on the user's natural language question, and calculate the keyword frequency of each keyword in the multiple keywords; wherein the keyword frequency is: the number of times each keyword appears in the user's natural language question; A calculation module 403 is used to calculate based on the word frequency and the keyword frequency to obtain a first word frequency inverse document frequency and a second word frequency inverse document frequency of each keyword in the multiple keywords; wherein the first word frequency inverse document frequency represents: the word frequency inverse document frequency of the keyword in the multiple candidate code blocks, and the second word frequency inverse document frequency represents: the word frequency inverse document frequency of the keyword in the user's natural language question; The reordering module 404 is used to calculate based on the first term frequency inverse document frequency, the second term frequency inverse document frequency and the logarithm of the keyword length to obtain weighted scores of corresponding keywords, and reorder based on the weighted scores to obtain a ranking result.

[0079] The retrieval code reordering device based on natural language provided by the present invention obtains multiple candidate code blocks corresponding to the user's natural language question, and calculates the word frequency of each candidate code block in the multiple candidate code blocks; wherein the word frequency is: the number of times each word appears in the corresponding candidate code block; obtains multiple keywords rewritten based on the user's natural language question, and calculates the keyword frequency of each keyword in the multiple keywords; wherein the keyword frequency is: the number of times each keyword appears in the user's natural language question; based on the word frequency and the keyword frequency, a first word frequency inverse document frequency and a second word frequency inverse document frequency of each keyword in the multiple keywords are calculated; wherein the first word frequency inverse document frequency represents: the word frequency inverse document frequency of the keyword in the multiple candidate code blocks, and the second word frequency inverse document frequency represents: the word frequency inverse document frequency of the keyword in the user's natural language question; based on the first word frequency inverse document frequency, the second word frequency inverse document frequency and the logarithm of the keyword length, a weighted score of the corresponding keyword is obtained, and reordering is performed based on the weighted score to obtain a sorting result. It can be seen that the present invention further improves the performance of the retrieval enhancement generation system by adding a step of calculating the inverse document frequency of the term frequency, re-ranking according to the calculated similarity, and weighting the similarity of each term frequency inverse document frequency according to specific rules.

[0080] Based on the above embodiment, in this embodiment, the first acquisition module 401 is specifically used to: Acquire multiple candidate code blocks corresponding to the user's natural language question, and perform word segmentation processing on each of the multiple candidate code blocks using a regular expression; For each candidate code block in the multiple candidate code blocks, the number of times each word appears in the corresponding candidate code block is calculated to obtain the word frequency of each candidate code block in the multiple candidate code blocks.

[0081] Based on the above embodiments, in this embodiment, the calculation module 403 is specifically used for: For each keyword, multiply the inverse document frequency of the keyword by the word frequency of the corresponding word to obtain the first word frequency inverse document frequency of the keyword; For each keyword, the inverse document frequency of the keyword and the keyword term frequency of the keyword are multiplied to obtain a second term frequency inverse document frequency of the keyword.

[0082] Based on the above embodiment, in this embodiment, the reordering module 404 is specifically used for: For each keyword, calculate the logarithm of the keyword length; The first term frequency inverse document frequency, the second term frequency inverse document frequency and the logarithm of the keyword length are multiplied to obtain a weighted score for the keyword.

[0083] Based on the above embodiment, in this embodiment, the device further includes a retrieval module, which is specifically used to: Before obtaining a plurality of candidate code blocks corresponding to the user's natural language question and calculating the word frequency of each candidate code block in the plurality of candidate code blocks, obtaining the user's natural language question; Based on the user's natural language question, a search is performed in the target vector library to obtain the multiple candidate code blocks corresponding to the user's natural language question.

[0084] Based on the above embodiment, in this embodiment, the device further includes a rewriting module, which is specifically used to: After obtaining the user's natural language question, the user's natural language question is rewritten through a large language model based on the target vector library to obtain the multiple keywords corresponding to the user's natural language question.

[0085] Figure 5 An example of a physical structure diagram of an electronic device is shown in FIG. Figure 5 As shown, the electronic device may be a robot or other electronic device, and the electronic device may include: a processor 510, a communications interface 520, a memory 530, and a communication bus 540, wherein the processor 510, the communications interface 520, and the memory 530 communicate with each other through the communication bus 540. The processor 510 may call the logic instructions in the memory 530 to execute the natural language-based retrieval code reordering method, including: Acquire multiple candidate code blocks corresponding to the user's natural language question, and calculate the word frequency of each candidate code block in the multiple candidate code blocks; wherein the word frequency is: the number of times each word appears in the corresponding candidate code block; Acquire multiple keywords rewritten based on the user's natural language question, and calculate the keyword frequency of each keyword in the multiple keywords; wherein the keyword frequency is: the number of times each keyword appears in the user's natural language question; Calculating based on the word frequency and the keyword frequency, obtaining a first word frequency inverse document frequency and a second word frequency inverse document frequency of each keyword in the multiple keywords; wherein the first word frequency inverse document frequency represents: the word frequency inverse document frequency of the keyword in the multiple candidate code blocks, and the second word frequency inverse document frequency represents: the word frequency inverse document frequency of the keyword in the user's natural language question; Based on the first term frequency inverse document frequency, the second term frequency inverse document frequency and the logarithm of the keyword length, a weighted score of the corresponding keyword is obtained, and re-sorting is performed based on the weighted score to obtain a sorting result.

[0086] In addition, the logic instructions in the above-mentioned memory 530 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0087] On the other hand, the present invention further provides a computer program product, the computer program product comprising a computer program, the computer program can be stored on a non-transitory computer-readable storage medium, and when the computer program is executed by a processor, the computer can execute the natural language-based retrieval code reordering method provided by the above methods, including: Acquire multiple candidate code blocks corresponding to the user's natural language question, and calculate the word frequency of each candidate code block in the multiple candidate code blocks; wherein the word frequency is: the number of times each word appears in the corresponding candidate code block; Acquire multiple keywords rewritten based on the user's natural language question, and calculate the keyword frequency of each keyword in the multiple keywords; wherein the keyword frequency is: the number of times each keyword appears in the user's natural language question; Calculating based on the word frequency and the keyword frequency, obtaining a first word frequency inverse document frequency and a second word frequency inverse document frequency of each keyword in the multiple keywords; wherein the first word frequency inverse document frequency represents: the word frequency inverse document frequency of the keyword in the multiple candidate code blocks, and the second word frequency inverse document frequency represents: the word frequency inverse document frequency of the keyword in the user's natural language question; Based on the first term frequency inverse document frequency, the second term frequency inverse document frequency and the logarithm of the keyword length, a weighted score of the corresponding keyword is obtained, and re-sorting is performed based on the weighted score to obtain a sorting result.

[0088] In another aspect, the present invention further provides a non-transitory computer-readable storage medium having a computer program stored thereon, which is implemented when the computer program is executed by a processor to perform the natural language-based retrieval code reordering method provided by the above methods, including: Acquire multiple candidate code blocks corresponding to the user's natural language question, and calculate the word frequency of each candidate code block in the multiple candidate code blocks; wherein the word frequency is: the number of times each word appears in the corresponding candidate code block; Acquire multiple keywords rewritten based on the user's natural language question, and calculate the keyword frequency of each keyword in the multiple keywords; wherein the keyword frequency is: the number of times each keyword appears in the user's natural language question; Calculating based on the word frequency and the keyword frequency, obtaining a first word frequency inverse document frequency and a second word frequency inverse document frequency of each keyword in the multiple keywords; wherein the first word frequency inverse document frequency represents: the word frequency inverse document frequency of the keyword in the multiple candidate code blocks, and the second word frequency inverse document frequency represents: the word frequency inverse document frequency of the keyword in the user's natural language question; Based on the first term frequency inverse document frequency, the second term frequency inverse document frequency and the logarithm of the keyword length, a weighted score of the corresponding keyword is obtained, and re-sorting is performed based on the weighted score to obtain a sorting result.

[0089] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Those of ordinary skill in the art may understand and implement it without creative effort.

[0090] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0091] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A natural language based retrieval code reordering method, characterized in that: include: Acquire multiple candidate code blocks corresponding to the user's natural language question, and calculate the word frequency of each candidate code block in the multiple candidate code blocks; wherein the word frequency is: the number of times each word appears in the corresponding candidate code block; Acquire multiple keywords rewritten based on the user's natural language question, and calculate the keyword frequency of each keyword in the multiple keywords; wherein the keyword frequency is: the number of times each keyword appears in the user's natural language question; Calculating based on the word frequency and the keyword frequency, obtaining a first word frequency inverse document frequency and a second word frequency inverse document frequency of each keyword in the multiple keywords; wherein the first word frequency inverse document frequency represents: the word frequency inverse document frequency of the keyword in the multiple candidate code blocks, and the second word frequency inverse document frequency represents: the word frequency inverse document frequency of the keyword in the user's natural language question; Based on the first term frequency inverse document frequency, the second term frequency inverse document frequency and the logarithm of the keyword length, a weighted score of the corresponding keyword is obtained, and re-sorting is performed based on the weighted score to obtain a sorting result.

2. The natural language based retrieval code reordering method according to claim 1, characterized in that: The step of obtaining a plurality of candidate code blocks corresponding to the user's natural language question and calculating the word frequency of each of the plurality of candidate code blocks comprises: Acquire multiple candidate code blocks corresponding to the user's natural language question, and perform word segmentation processing on each of the multiple candidate code blocks using a regular expression; For each candidate code block in the multiple candidate code blocks, the number of times each word appears in the corresponding candidate code block is calculated to obtain the word frequency of each candidate code block in the multiple candidate code blocks.

3. The natural language based retrieval code reordering method according to claim 1, characterized in that: The calculation based on the word frequency and the keyword frequency to obtain a first word frequency inverse document frequency and a second word frequency inverse document frequency of each keyword in the multiple keywords includes: For each keyword, multiply the inverse document frequency of the keyword by the word frequency of the corresponding word to obtain the first word frequency inverse document frequency of the keyword; For each keyword, the inverse document frequency of the keyword and the keyword term frequency of the keyword are multiplied to obtain a second term frequency inverse document frequency of the keyword.

4. The natural language based retrieval code reordering method according to claim 1, characterized in that: The calculating based on the first term frequency inverse document frequency, the second term frequency inverse document frequency and the logarithm of the keyword length to obtain a weighted score of the corresponding keyword includes: For each keyword, calculate the logarithm of the keyword length; The first term frequency inverse document frequency, the second term frequency inverse document frequency and the logarithm of the keyword length are multiplied to obtain a weighted score for the keyword.

5. The method for reordering retrieval codes based on natural language according to claim 1, characterized in that: Before obtaining a plurality of candidate code blocks corresponding to the user's natural language question and calculating the word frequency of each of the plurality of candidate code blocks, the method further includes: Obtain user's natural language questions; Based on the user's natural language question, a search is performed in the target vector library to obtain the multiple candidate code blocks corresponding to the user's natural language question.

6. The natural language based retrieval code reordering method according to claim 5, characterized in that: After obtaining the user's natural language question, the method further includes: The user's natural language question is rewritten based on the target vector library through a large language model to obtain the multiple keywords corresponding to the user's natural language question.

7. A retrieval code reordering device based on natural language, characterized in that: include: A first acquisition module is used to acquire multiple candidate code blocks corresponding to the user's natural language question, and calculate the word frequency of each candidate code block in the multiple candidate code blocks; wherein the word frequency is: the number of times each word appears in the corresponding candidate code block; A second acquisition module is used to acquire multiple keywords rewritten based on the user's natural language question, and calculate the keyword frequency of each keyword in the multiple keywords; wherein the keyword frequency is: the number of times each keyword appears in the user's natural language question; A calculation module, configured to calculate based on the word frequency and the keyword frequency to obtain a first word frequency inverse document frequency and a second word frequency inverse document frequency of each keyword in the multiple keywords; wherein the first word frequency inverse document frequency represents: the word frequency inverse document frequency of the keyword in the multiple candidate code blocks, and the second word frequency inverse document frequency represents: the word frequency inverse document frequency of the keyword in the user's natural language question; The reordering module is used to calculate based on the first term frequency inverse document frequency, the second term frequency inverse document frequency and the logarithm of the keyword length to obtain a weighted score of the corresponding keyword, and reorder based on the weighted score to obtain a ranking result.

8. The natural language based retrieval code reordering device according to claim 7, characterized in that: The first acquisition module is specifically used to: Acquire multiple candidate code blocks corresponding to the user's natural language question, and perform word segmentation processing on each of the multiple candidate code blocks using a regular expression; For each candidate code block in the multiple candidate code blocks, the number of times each word appears in the corresponding candidate code block is calculated to obtain the word frequency of each candidate code block in the multiple candidate code blocks.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the natural language-based retrieval code reordering method according to any one of claims 1 to 6 is implemented.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the natural language-based retrieval code reordering method according to any one of claims 1 to 6 is implemented.

Images