Search enhancement code generation method and system using dual-view embedding vectors

By combining dual-view embedding vectors and the BM25 algorithm, the difficulty of information retrieval with similar semantics but different lexicalities in existing technologies is solved, improving the accuracy and generalization ability of code generation, especially for automated code generation in complex command-line scenarios.

CN120523451BActive Publication Date: 2025-10-24UNIV OF SCI & TECH OF CHINA
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202511018026.3
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2025-07-23
Publication Date
2025-10-24
Estimated Expiration
2045-07-23

AI Technical Summary

Technical Problem

Existing code generation models struggle to generalize to unknown functional or system environments when faced with unfamiliar commands, poorly documented functions, or rare usage patterns. Furthermore, traditional lexical retrieval tools cannot effectively capture semantically similar but lexically different information.

Method used

A retrieval enhancement method using dual-view embedding vectors is employed to generate complementary embeddings of the original text and the cleaned text. This method combines the BM25 algorithm and semantic similarity scoring to integrate lexical and semantic matching, thereby improving retrieval accuracy.

Benefits of technology

It improves the accuracy and generalization ability of code generation, especially in terms of command-level and token-level accuracy metrics, thereby enhancing retrieval and generation quality.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN120523451B_ABST
    Figure CN120523451B_ABST
Patent Text Reader

Abstract

The application relates to the field of artificial intelligence and natural language processing technology, and discloses a retrieval enhanced code generation method and system using double-view embedding vectors; the method comprises the following steps: generating two complementary embedding vectors for each Bash document in a knowledge base; calculating the double-view semantic similarity scores of the embedding of a user query and each Bash document in the knowledge base, and simultaneously calculating the BM25 lexical similarity scores of the Bash documents based on a BM25 algorithm; fusing the double-view semantic similarity scores and the BM25 lexical similarity scores, and ranking the Bash documents according to the final scores obtained; and inputting the N Bash documents with the highest final scores and the user query into a large language model to generate target Bash code. The application enhances the retrieval stage by using a double embedding strategy, and combines the BM25 algorithm and semantic similarity, so that more accurate and context-related information is provided for the generator, and the accuracy of the generated code can be improved.
Need to check novelty before this filing date? Find Prior Art

Description

TECHNICAL FIELD

[0001] The application relates to the field of artificial intelligence and natural language processing, and particularly relates to a retrieval-augmented code generation method and system using double-view embedding vectors. BACKGROUND

[0002] Code generation based on natural language (NL) instructions is a core task in software engineering and programming assistance. The rise of large-scale pre-trained language models (PLMs) has made significant progress in techniques for translating user intent into executable code. However, when faced with unfamiliar commands, poorly documented functions, or rare usage patterns, code generation models in the prior art often encounter difficulties, especially in the field of Bash code generation. Code generation models in the prior art are usually trained on static corpora, and without external support, it is difficult to generalize to unknown functions or system environments.

[0003] Retrieval-augmented generation (RAG) as a code generation framework can retrieve relevant external documents and adjust the model on these documents during reasoning. Retrieval-augmented generation allows the model to integrate the latest, task-specific information on demand. For example, when retrieval-augmented generation is used to generate Bash code, its concise documents can significantly improve the quality of code generation.

[0004] Retrieval-augmented generation (RAG) technology in code generation usually relies on traditional lexical retrievers, such as the BM25 algorithm, which prioritizes matching exact tokens. While these traditional lexical retrievers are effective in matching direct terminology, they often fail to retrieve relevant documents when the query's vocabulary and phrases are significantly different from the knowledge base, thereby severely limiting the ability to capture semantically similar but lexically different information. The above challenges have promoted the development of semantic retrieval techniques based on dense embedding models. Semantic retrieval addresses this limitation by mapping queries and code knowledge to a shared vector space, enabling the system to capture subtle semantic relationships and retrieve information based on underlying meanings, and generate enhanced code using various context-dependent knowledge. Dense embedding retrieval conceptually bridges the semantic gap between user queries and the most relevant documents, successfully connecting relevant concepts despite differences in language and structure.

[0005] Despite these advances, lexical and semantic retrieval methods still face a core limitation: they tend to treat documents as a monolithic block of text, failing to account for the different roles played by different types of content. For example, in Bash command documents, some paragraphs provide high-level descriptions of command behavior, while others list detailed parameter usage enclosed in placeholder symbols (e.g. <filename> , <pkgname>). These segments contribute differently to downstream generation: descriptive text supports intent understanding, while placeholders guide syntax and structure. However, existing methods typically collapse all content into one embedding, discarding this important structural distinction. SUMMARY

[0006] To solve the above technical problems, the present application provides a retrieval enhanced code generation method and system using dual-view embedding vectors. The present application proposes a hybrid retrieval mechanism that combines the strengths of lexical and semantic matching. The present application introduces dual-view embeddings, computing two embedding representations for each document: one from the original document and another from the document obtained by cleaning placeholder patterns. When querying, the present application uses a weighted combination of two similarity scores to score documents, achieving finer-grained matching through surface-level phrases and deeper semantic alignment.

[0007] To solve the above technical problems, the present application adopts the following technical solutions:

[0008] In a first aspect, the present application provides a retrieval enhanced code generation method using dual-view embedding vectors, comprising:

[0009] generating two complementary embedding vectors for each Bash document in the knowledge base, including the embedding of the original text view and the embedding of the cleaned text view after removing placeholders;

[0010] calculating the dual-view semantic similarity scores of the user query embedding and each Bash document in the knowledge base, and calculating the BM25 lexical similarity scores of the Bash documents based on the BM25 algorithm; the dual-view semantic similarity scores are obtained by combining the similarity of the user query embedding and the original text view embedding and the similarity of the user query embedding and the cleaned text view embedding;

[0011] fusing the dual-view semantic similarity scores and the BM25 lexical similarity scores, and ranking the Bash documents according to the final scores obtained;

[0012] inputting the top N Bash documents with the highest final scores and the user query into a large language model to generate target Bash code.

[0013] In one embodiment, the two complementary embedding vectors generated for each Bash document in the knowledge base include the embedding of the original text view and the embedding of the cleaned text view after removing placeholders, specifically comprising:

[0014] For a given Bash document d, the original view and the cleaned view ; the original view is an unmodified document containing placeholders; the cleaned view is a document obtained by cleaning the placeholder patterns;

[0015] are input into an encoder to obtain an embedding of the original text view and an embedding of the cleaned text view, respectively. and are input into an encoder to obtain an embedding of the original text view and an embedding of the cleaned text view, respectively.

[0016] In one embodiment, the inputting and into an encoder to obtain an embedding of the original text view and an embedding of the cleaned text view, respectively, specifically includes:

[0017] ;

[0018] ;

[0019] is a pre-trained dense encoder from the Nomic-Embed model, is an embedding of the original text view, is an embedding of the cleaned text view.

[0020] In one embodiment, the calculating the bi-view semantic similarity score of the user query and each Bash document in the knowledge base specifically includes:

[0021] calculating an embedding of the user query q based on a pre-trained dense encoder of the Nomic-Embed model;

[0022] calculating a cosine similarity between the embedding of the user query and the embedding of the original text view : ;

[0023] calculating a cosine similarity between the embedding of the user query and the embedding of the cleaned text view : ; denotes calculating the cosine similarity;

[0024] the bi-view semantic similarity score , is a proportionality coefficient.

[0025] In one embodiment, the fusing the bi-view semantic similarity score and the BM25 lexical similarity score, and ranking the Bash documents according to the obtained final score specifically includes:

[0026] ;

[0027] is the final score, is the BM25 lexical similarity score of Bash document, is the final score of semantic similarity, is the proportion coefficient, is the embedding of user query, is the i-th Bash document.

[0028] In a second aspect, the present application provides a retrieval enhanced code generation system using dual-view embedding vectors, comprising:

[0029] an embedding calculation module, which generates two complementary embedding vectors for each Bash document in the knowledge base, including the embedding of the original text view and the embedding of the cleaned text view after removing placeholders;

[0030] a similarity calculation module, which calculates the dual-view semantic similarity scores of the embedding of the user query and each Bash document in the knowledge base, and simultaneously calculates the BM25 lexical similarity scores of the Bash documents based on the BM25 algorithm; the dual-view semantic similarity scores are obtained by combining the similarity of the embedding of the user query and the embedding of the original text view, and the similarity of the embedding of the user query and the embedding of the cleaned text view;

[0031] a ranking module, which fuses the dual-view semantic similarity scores and the BM25 lexical similarity scores, and ranks the Bash documents according to the final scores obtained;

[0032] a code generation module, which inputs the top N Bash documents with the highest final scores and the user query into a large language model to generate target Bash code.

[0033] In one embodiment, the two complementary embedding vectors for each Bash document in the knowledge base, including the embedding of the original text view and the embedding of the cleaned text view after removing placeholders, specifically include:

[0034] for a given Bash document d, the original view and the cleaned view are constructed respectively; the original view is the unmodified document containing placeholders; the cleaned view is the document obtained by cleaning the placeholder pattern;

[0035] are input into the encoder respectively to obtain the embedding of the original text view and the embedding of the cleaned text view.

[0036] In one embodiment, the input of and into the encoder to obtain the embedding of the original text view and the embedding of the cleaned text view specifically includes:

[0037] ​​ ;

[0038] ;

[0039] is a pre-trained dense encoder from Nomic-Embed model, is an embedding of the raw text view, is an embedding of the cleaned text view.

[0040] In one embodiment, the computing the bi-view semantic similarity score of the embedding of the user query and each Bash document in the knowledge base comprises:

[0041] computing the embedding of the user query based on the pre-trained dense encoder of the Nomic-Embed model;

[0042] computing the cosine similarity of the embedding of the user query and the embedding of the raw text view : ;

[0043] computing the cosine similarity of the embedding of the user query and the embedding of the cleaned text view : ; denotes the computing the cosine similarity;

[0044] the bi-view semantic similarity score , is a proportionality coefficient.

[0045] In one embodiment, the fusing the bi-view semantic similarity score and the BM25 lexical similarity score, ranking the Bash documents according to the final score obtained comprises:

[0046] ;

[0047] is the final score, is the BM25 lexical similarity score of the Bash document, is the final score of the semantic similarity, is a proportionality coefficient, is the embedding of the user query, is the i-th Bash document.

[0048] The method and system of the present application correspond to each other, and the specific technical solutions of the method are also applicable to the system.

[0049] Compared with the prior art, the present application has the beneficial technical effects that:

[0050] The application proposes a hybrid retrieval enhancement framework tailored for natural language for Bash code generation. The application enhances the retrieval stage by using a dual embedding strategy and combines the BM25 algorithm and semantic similarity to provide more accurate and context-related information for the generator. Experiments show that the performance of the hybrid retrieval enhancement framework of the application is always better than that of existing methods such as DocPrompting, especially in challenging indicators such as command-level accuracy, exact match and token-level F1, the performance of the application is more outstanding.

[0051] The analysis of the application shows that retrieval quality plays a key role in downstream generation, and richer retrieval signals are more beneficial to large language models. Although additional semantic processing will bring moderate runtime overhead, in applications where correctness is the key indicator, the trade-off between correctness and additional overhead needs to be made. BRIEF DESCRIPTION OF DRAWINGS

[0052] Figure 1 The method flowchart in the embodiment of the application;

[0053] Figure 2 The schematic diagram of the dual-view retrieval framework in the embodiment of the application;

[0054] Figure 3 The code generation schematic diagram based on the hybrid retrieval enhancement framework in the embodiment of the application;

[0055] Figure 4 The schematic diagram of the hybrid retrieval enhancement framework in the embodiment of the application. DETAILED DESCRIPTION

[0056] A preferred embodiment of the application will be described in detail below with reference to the accompanying drawings.

[0057] As shown in Figure 1 and Figure 3 , a retrieval enhancement code generation method using dual-view embedding vectors includes the following steps:

[0058] S1, generating two complementary embedding vectors for each Bash document in the knowledge base, including embedding of the original text view and embedding of the cleaned text view after removing placeholders;

[0059] S2, calculating the dual-view semantic similarity scores of the embedding of the user query and each Bash document in the knowledge base, and calculating the BM25 lexical similarity scores of the Bash documents based on the BM25 algorithm; the dual-view semantic similarity scores are obtained by combining the similarity of the embedding of the user query and the embedding of the original text view and the similarity of the embedding of the user query and the embedding of the cleaned text view;

[0060] S3, fuse the dual-view semantic similarity score and the BM25 lexical similarity score, rank the Bash documents by the final score;

[0061] S4, input the N Bash documents with the highest final scores into the large language model to generate the target Bash code.

[0062] In view of the limitations of the existing retrieval enhancement generation framework in Bash code generation, such as decoupling of semantic and structural information of documents, and separation of lexical and semantic retrieval, the present application proposes a hybrid retrieval enhancement framework based on in-depth analysis of natural language semantics and parameter mark structure characteristics in command description. In the dense semantic retrieval, a dual-view embedding strategy is adopted to improve the accuracy and robustness of retrieval, and thus improve the quality and generalization ability of code generation, so as to better meet the demand of automatic code generation in complex command line scenarios.

[0063] As shown in Figure 4 The hybrid retrieval enhancement framework (BashRAG++) proposed by the present application integrates sparse and dense retrieval signals into a unified retrieval pipeline, thereby improving the retrieval quality of code generation. Specifically, for each document, the present application constructs a dual-view embedding: one from the original text, and the other from a placeholder-removed version emphasizing structure and description. The two views capture the complementary semantics of the same document and are aggregated through a dual-view retrieval framework. In order to further improve the robustness, the generated dual-view semantic similarity score is combined with the traditional BM25 lexical similarity score through a hybrid scoring mechanism. This design enables the retriever to benefit from both precise term matching and semantic understanding, ultimately providing the generator with more relevant and diverse contextual information and improving the quality of generated Bash code.

[0064] 1. Dual-view embedding retrieval:

[0065] In order to better capture the semantic richness of Bash documents, the present application proposes a dual-view embedding retrieval strategy. Bash manual pages often contain placeholders, such as <pkgname>Or <command> , can hinder direct semantic matching with natural language queries. However, simply removing these placeholders entirely can result in the loss of valuable structural or contextual cues. To balance these perspectives, the invention generates two complementary embeddings for each document and combines their retrieval signals.

[0066] Dual-view embedding construction: Given a document d, the invention constructs two textual views:

[0067] 1) Original view , the unmodified document containing placeholders.

[0068] 2) Cleaned view , the document obtained by cleaning the placeholder patterns.

[0069] Placeholder cleaning example. For example, in the cleaned view, a pattern such as "apt install <pkgname>such as "apt install" from a paragraph like "apt install foo bar baz" to remove the possibly misleading symbol placeholders while preserving the executable intent. Another example is "kill <pid>It becomes "kill", emphasizing the action.

[0070] Each view is encoded into a respective embedding using the same encoder:

[0071] ; (1)

[0072] is the embedding of the original text view and the embedding of the cleaned text view ; d is the document for which the embedding is needed, including and .

[0073] is the encoder. In the preferred embodiment, the present invention uses a pre-trained dense encoder from the Nomic-Embed model, which has shown strong performance on semantic retrieval benchmarks. During retrieval, the encoder is kept frozen to isolate the impact of the retrieval strategy proposed by the present invention.

[0074] Similarly, for a user query, the present invention computes its embedding q: the user query is input into the encoder to obtain the embedding of the user query.

[0075] The cosine similarity between the embedding of the user query and the embedding of the text view is computed:

[0076] ; (2)

[0077] s is the cosine similarity obtained, including the cosine similarity between the embedding of the user query and the embedding of the original text view and the cosine similarity between the embedding of the user query and the embedding of the cleaned text view .

[0078] The two cosine similarities are combined by weighted average to compute the dual-view semantic similarity score:

[0079] ; (3)

[0080] where is the proportionality coefficient, in the preferred embodiment λ = 0.5, giving equal weight to the two views. This design allows the retrieval system to benefit from both the structural fidelity of the original view and the semantic clarity of the cleaned view.

[0081] The dual-view retrieval framework of the present invention is shown in Figure 2 .

[0082] 2. Hybrid scoring using the BM25 algorithm:

[0083] While the dual-view embedding strategy effectively improves semantic retrieval, traditional sparse retrieval methods like the BM25 algorithm remain competitive in code-related tasks as they are robust to rare tokens, proper nouns, and command-specific syntax. To further improve retrieval accuracy, the invention employs a hybrid scoring mechanism that integrates sparse (BM25) and dense (dual-view embedding) retrieval signals.

[0084] Given a user query, the invention retrieves a set of candidate documents }. For each document, the invention computes:

[0085] 1) BM25 lexical similarity score ;

[0086] 2) dual-view semantic similarity score.

[0087] The final score is computed as:

[0088] ; (4)

[0089] is the final score, is the BM25 lexical similarity score for the Bash document, is the final score for the semantic similarity, is the proportionality coefficient, is the embedding for the user query, is the i-th Bash document.

[0090] It was found through practical verification that setting β = 0.05 can achieve a good balance between exact keyword matching and semantic alignment. A higher β value will lead to fragile results that rely mainly on literal overlap, while over-reliance on dual-view embedding may miss key command tokens, which is particularly evident in short or highly symbolic queries. This hybrid design allows the system to take full advantage of surface-level token overlap (through BM25) and deeper semantic representations (through dual-view embedding). The above technical solution is particularly useful in the Bash domain, as commands in the Bash domain often contain both rare keywords and semantically meaningful options or descriptions. By introducing dual-view embedding and a hybrid scoring pipeline, the hybrid retrieval enhancement framework of the invention effectively combines semantic understanding with exact token matching.

[0091] Figure 4 The architecture of the hybrid retrieval-augmentation framework proposed by the present application is shown. The hybrid retrieval-augmentation framework improves retrieval accuracy by integrating BM25 techniques and hybrid scoring of dual-view dense embedding without additional retriever training. The retrieved documents are then fed into a frozen-parameter large language model for generation, which can be a decoder-only structure or an encoder-decoder structure, making the framework plug-and-play between different model series.

[0092] The method of the present application is model-agnostic and introduces negligible computational overhead (11 milliseconds per query). Experiments on a TLDR (Too Long; Didn't Read) style Bash code generation dataset show that all Top-K metrics of the present application are improved compared to the retrieval-augmentation scheme using the BM25 algorithm alone. For example, the Top-1 retrieval accuracy is improved from 32.2% to 37.1%, and the Top-10 retrieval accuracy is improved from 58.4% to 64.1%.

[0093] The present application conducts experiments on the TLDR Bash dataset introduced in the DocPrompting technology, which is a benchmark specifically for natural language to Bash code generation. The dataset contains 9187 natural language (NL) intent and Bash command pairs, covering 1879 unique Bash commands, including commonly used commands (such as cat, tar) and less common utilities (such as faketime, toilet). To evaluate model generalization, the training set, development set, and test set are constructed using disjoint sets of command names. The relevant document corpus contains over 400,000 document passages from the Bash manual maintained by the TLDR community. Each document passage usually describes a concept or usage (e.g., parameter flags or command examples).

[0094] To evaluate the experimental results, the present application sets two baselines, namely no retrieval and retrieval using the DocPrompting technology (specifically using the BM25 algorithm), to conduct comparative experiments with the hybrid retrieval-augmentation framework (BashRAG++) proposed by the present application. The code generation results are shown in Table 1.

[0095] Table 1: Code generation results

[0096]

[0097] To better understand the effectiveness of each component in the retrieval pipeline, the present invention conducts an ablation study by enabling or disabling different signals (BM25 algorithm, raw text view embedding, and clean text view embedding). In Table 2, BM25 denotes retrieval using only the BM25 algorithm, Raw Embedding denotes retrieval using only the raw text view embedding, Clean Embedding denotes retrieval using only the clean text view embedding, Raw+Clean denotes retrieval using both the raw text view embedding and the clean text view embedding, BM25+Raw denotes retrieval using both the BM25 algorithm and the raw text view embedding, and BM25+Clean denotes retrieval using both the BM25 algorithm and the clean text view embedding.

[0098] As shown in Table 2, among all Top-K metrics, the semantic retrieval effect with the clean text view embedding (e.g., Top-1 accuracy: 19.2%) is always better than that with the raw text view embedding (e.g., Top-1 accuracy: 17.3%). This verifies the assumption of the present invention that text cleaning (removing irrelevant tokens or noisy symbols) enhances the ability of the semantic model to match the intent and the manual content. In the combination scheme without the lexical basis (Raw+Clean), it does not produce significant improvement than using the clean text view embedding alone, and even slightly lower than it. This shows that the information in the raw text view embedding and the clean text view embedding is partially redundant, and simply aggregating both without using the lexical structure can introduce noise. On the other hand, combining the BM25 algorithm with the raw text view embedding or the clean text view embedding always outperforms any single mode. For example, the combination scheme of BM25+Clean achieves a Top-1 accuracy of 36.6%, which is significantly better than the BM25 scheme (32.2%) or the Clean scheme (19.2%). This shows that there is a strong complementarity between the lexical and semantic retrieval signals, where the BM25 algorithm provides high-precision term matching, and the embedding captures potential intent-level similarity.

[0099] Table 2: Retrieval effect under different conditions

[0100]

[0101] The hybrid retrieval augmentation framework (BashRAG++) of the present invention integrates the BM25 algorithm with the raw text view embedding and the clean text view embedding, achieving the highest or second-highest accuracy on all metrics (e.g., 37.1% Top-1, 50.1% Top-3). This confirms that the dual-view embeddings can reinforce each other when taking the lexical cues as an appropriate basis, resulting in a more robust and accurate retrieval pipeline.

[0102] It should be understood that although the steps in the flowcharts of the drawings are shown in a sequence as indicated by arrows, the steps are not necessarily executed in the order as indicated by the arrows. Unless otherwise specifically noted, the execution of the steps is not strictly limited in sequence, and the steps can be executed in other sequences. Moreover, at least some of the steps in the flowcharts of the drawings can include multiple steps or multiple stages, which are not necessarily executed at the same time, but can be executed at different times, and the execution sequence of the steps or stages is not necessarily sequential, but can be round-robin or alternating with at least some of the other steps or the steps or stages in the other steps.

[0103] Based on the description of the method embodiments, the application further provides a system. The system can be a software (application), module, component, server, client, etc. using the method described in the embodiments of the present application and combined with necessary implementation hardware. Based on the same innovative concept, the apparatus in one or more embodiments provided by the embodiments of the present application is described as follows. Since the implementation scheme of the apparatus for solving the problem is similar to the method, the implementation of the specific apparatus in the embodiments of the present application can be referred to the implementation of the foregoing method, and the repeated parts will not be described herein. The term "module" or "module" used below is a combination of software and / or hardware that can realize a predetermined function. Although the apparatus described in the following embodiments is preferably realized in software, the realization of hardware or a combination of software and hardware is also possible and conceived.

[0104] A retrieval enhanced code generation system using dual-view embedding vectors, comprising:

[0105] An embedding calculation module, generating two complementary embedding vectors for each Bash document in the knowledge base, including embedding of the original text view and embedding of the cleaned text view after removing placeholders;

[0106] A similarity calculation module, calculating the dual-view semantic similarity scores of the embedding of the user query and each Bash document in the knowledge base, and simultaneously calculating the BM25 lexical similarity scores of the Bash documents based on the BM25 algorithm; the dual-view semantic similarity scores are obtained by combining the similarity of the embedding of the user query and the embedding of the original text view and the similarity of the embedding of the user query and the embedding of the cleaned text view;

[0107] A ranking module, fusing the dual-view semantic similarity scores and the BM25 lexical similarity scores, and ranking the Bash documents according to the final scores obtained;

[0108] The code generation module inputs the N Bash documents with the highest final scores and the user query into the large language model to generate target Bash code.

[0109] The technical features of the above embodiments can be combined in any manner. To make the description concise, not all possible combinations of the technical features in the above embodiments are described, but as long as the combinations of the technical features do not exist contradictions, they should be considered as falling within the scope of the present disclosure.

[0110] It is apparent for those skilled in the art that the present application is not limited to the details of the foregoing exemplary embodiments, and the present application can be carried out in other concrete forms without departing from the spirit or basic characteristics of the present application. Accordingly, the embodiments are to be considered in all respects as illustrative and not restrictive, the scope of the present application being indicated by the appended claims rather than by the foregoing description, and all changes which come within the meaning and range of equivalency of the claims are therefore intended to be embraced therein and their intended scope is to be understood to be not limited by any restraining of the appended claims or of the foregoing description, but rather only by whatever limitations are advised by the prior art.

[0111] Furthermore, it should be understood that, although the present specification is described in terms of embodiments, not every embodiment contains only one independent technical solution, and the present specification is described in this manner only for the sake of clarity, and those skilled in the art should consider the specification as a whole, and the technical solutions in each embodiment can also be combined appropriately to form other embodiments that those skilled in the art can understand.< / pid> < / pkgname> < / pkgname> < / pkgname> < / filename>

Claims

1. A retrieval augmentation code generation method using a dual-view embedding vector, characterized by, The method comprises the following steps: Two complementary embedding vectors are generated for each Bash document in the knowledge base, including an embedding of the raw text view and an embedding of the cleaned text view after removing placeholders, specifically including: for a given Bash document d, raw view and cleaned view are constructed, respectively; the raw view is the unmodified document containing placeholders; the cleaned view is the document obtained by cleaning the placeholder pattern; and and are input to an encoder, respectively, to obtain the embedding of the raw text view and the embedding of the cleaned text view: ; ; is a pretrained dense encoder from the Nomic-Embed model, is the embedding of the raw text view, is the embedding of the cleaned text view; calculating a double-view semantic similarity score of an embedding of a user query and each Bash document in a knowledge base, and simultaneously calculating a BM25 lexical similarity score of the Bash document based on a BM25 algorithm; the double-view semantic similarity score is obtained by combining the similarity of the embedding of the user query and the embedding of an original text view and the similarity of the embedding of the user query and the embedding of a cleaned text view; fusing the double-view semantic similarity score and the BM25 lexical similarity score, and ranking the Bash documents according to the final score obtained; inputting the N Bash documents with the highest final scores into a large language model together with the user query to generate target Bash code.

2. The retrieval enhanced code generation method using dual view embedding vectors according to claim 1, wherein, The method for calculating the double-view semantic similarity score of the embedding of the user query and each Bash document in the knowledge base comprises the following steps: calculating the embedding q of the user query based on a pre-training dense encoder of an Nomic-Embed model; calculating a cosine similarity of an embedding of a user query and an embedding of a raw text view : ; Computing cosine similarity of an embedding of a user query and an embedding of a cleaned text view : ; denotes computing cosine similarity; Dual view semantic similarity score , is a proportionality coefficient.

3. The method for generating retrieval-enhanced code using dual-view embedding vectors according to claim 1, wherein: The method for fusing the double-view semantic similarity score and the BM25 lexical similarity score, and ranking the Bash documents according to the final score obtained comprises the following steps: ; is the final score, is the BM25 lexical similarity score for the Bash document, is the final score for semantic similarity, is the scaling factor, is the embedding for the user query, is the i-th Bash document.

4. A retrieval augmentation code generation system using dual-view embedding vectors, characterized by, The method comprises the following steps: embedding the Bash documents in the knowledge base, including embedding of the original text view and embedding of the cleaned text view after removing the placeholders, specifically including: for a given Bash document d, constructing an original view and a cleaned view ; the original view is the unmodified document containing placeholders; the cleaned view is the document obtained by cleaning the placeholder pattern; inputting and to the encoder respectively to obtain the embedding of the original text view and the embedding of the cleaned text view: ; ; is a pre-trained dense encoder from the Nomic-Embed model, is the embedding of the original text view, is the embedding of the cleaned text view; a similarity calculation module is configured to calculate a double-view semantic similarity score of an embedding of a user query and each Bash document in a knowledge base, and simultaneously calculate a BM25 lexical similarity score of the Bash document based on a BM25 algorithm; the double-view semantic similarity score is obtained by combining the similarity of the embedding of the user query and the embedding of an original text view and the similarity of the embedding of the user query and the embedding of a cleaned text view; a ranking module is configured to fuse the double-view semantic similarity score and the BM25 lexical similarity score, and rank the Bash documents according to the final score obtained; a code generation module is configured to input the N Bash documents with the highest final scores into a large language model together with the user query to generate target Bash code.

5. The retrieval augmented code generation system using dual-view embedding vectors of claim 4, wherein, The method for calculating the double-view semantic similarity score of the embedding of the user query and each Bash document in the knowledge base comprises the following steps: calculating the embedding q of the user query based on a pre-training dense encoder of an Nomic-Embed model; calculating a cosine similarity of an embedding of a user query and an embedding of a raw text view : ; Computing cosine similarity of an embedding of a user query and an embedding of a cleaned text view : ; denotes computing cosine similarity; Dual view semantic similarity score , is a proportionality coefficient.

6. The retrieval augmented code generation system using dual-view embedding vectors of claim 4, wherein, The method for fusing the double-view semantic similarity score and the BM25 lexical similarity score, and ranking the Bash documents according to the final score obtained comprises the following steps: ; is the final score, is the BM25 lexical similarity score of the Bash document, is the final score of semantic similarity, is the proportionality coefficient, is the embedding for the user query, is the i-th Bash document.

Citation Information

Patent Citations

  • Three-friction stepper for juxtaposedly pushing double piezoelectrics and scanning probe microscope thereof

    CN101521195A

  • Encoder Using Machine-Trained Term Frequency Weighting Factors that Produces a Dense Embedding Vector

    US20200311542A1