Code retrieval method and computing device

Abstract

The embodiment of the application provides a code retrieval method and a computing device, the method comprises the following steps: obtaining a query question input by a user; performing multi-dimensional information extraction processing on the query question to obtain a query information group; wherein the query information group comprises query information corresponding to multiple retrieval dimensions; obtaining a preset code vector database; wherein the preset code vector database comprises multiple code segments and retrieval information groups associated with the code segments, and the retrieval information group comprises multiple retrieval information corresponding to multiple retrieval dimensions; for each code segment, calculating a comprehensive similarity result of the code segment according to the associated retrieval information group and the query information group; and determining a target code segment of the query question according to the comprehensive similarity result, which overcomes the limitations of traditional single vector retrieval, realizes more accurate code semantic matching, and effectively improves the accuracy of code retrieval.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to a code retrieval method and computing device. Background Technology

[0002] With the application of large language models in the field of software development, code question answering systems based on Retrieval-Augmented Generation (RAG) have become important tools to assist development.

[0003] In related technologies, users typically use natural language to describe functional requirements. The code RAG system uses the original code as the sole source of knowledge and stores it in a vectorized form. When it receives a user's natural language question, the code RAG system matches the natural language question with the vectorized original code and provides the target code corresponding to the natural language question based on the matching result.

[0004] However, the code retrieval methods in related technologies cannot achieve precise alignment between natural language queries and code logic, resulting in low accuracy in code retrieval. Summary of the Invention

[0005] This application provides a code retrieval method and computing device that can improve the accuracy of code retrieval.

[0006] In a first aspect, embodiments of this application provide a code retrieval method, including:

[0007] Obtain the query question input by the user;

[0008] The query question is subjected to multi-dimensional information extraction processing to obtain a query information group; wherein, the query information group includes query information corresponding to multiple retrieval dimensions;

[0009] Obtain a preset code vector database; wherein, the preset code vector database includes multiple code snippets and search information groups associated with the code snippets, and the search information groups include multiple search information corresponding to the multiple search dimensions;

[0010] For each code segment, a comprehensive similarity result is calculated based on the associated retrieval information group and the query information group.

[0011] Based on the comprehensive similarity results, the target code segment for the query question is determined.

[0012] This application provides a multi-dimensional matching code retrieval method. When a user asks a question using natural language, information is extracted from the query from multiple retrieval dimensions to obtain query information corresponding to these dimensions. The similarity of code fragments is calculated by combining this stored retrieval information. Each retrieval dimension captures similarity information from different perspectives, achieving matching between the query and the code fragment across multiple dimensions. This enables precise alignment between natural language queries and code logic. This method overcomes the limitations of traditional single-vector retrieval, achieving more accurate code semantic matching and effectively improving the accuracy of code retrieval.

[0013] In one possible implementation, calculating the comprehensive similarity result of each code segment based on the associated retrieval information group and the query information group includes:

[0014] For each of the search dimensions, the similarity between the search information associated with the code fragment and the query information is calculated to obtain multiple similarities corresponding to multiple search dimensions;

[0015] The multiple similarities are fused to obtain the comprehensive similarity result.

[0016] In this embodiment, when calculating the similarity between code snippets and query questions, the similarity between the search information and the query information is calculated separately for each search dimension. The results from each search dimension are then fused to obtain a comprehensive similarity result. This method of calculating separately and then fusing achieves accurate matching across each search dimension and enables collaborative decision-making based on multi-dimensional information. It ensures that the matching results for each search dimension are fully considered, avoiding the omission of information from a single dimension. Furthermore, through fusion processing, code snippets that match across multiple dimensions simultaneously receive higher comprehensive scores, effectively improving the accuracy and relevance of the search results.

[0017] In one possible implementation, the fusion process of the multiple similarities to obtain the comprehensive similarity result includes:

[0018] The query question is processed to identify the query intent, so as to obtain the query intent features corresponding to each of the search dimensions;

[0019] Calculate the proportion of the query intent feature corresponding to each retrieval dimension among all query intent features;

[0020] The fusion weight of the retrieval dimension is determined based on the proportion of the aforementioned features.

[0021] Based on the fusion weights, a weighted sum of the similarities is calculated to obtain the comprehensive similarity result.

[0022] In this embodiment, when performing similarity fusion across multiple search dimensions, query intent is identified based on the query question to analyze user query intent preferences and then configure different fusion weights for different search dimensions. Specifically, the query intent features corresponding to each search dimension are obtained, the feature proportion of each dimension among all features is calculated, the fusion weight of each search dimension is dynamically determined based on the feature proportion, and finally, a weighted sum of similarities is calculated based on the fusion weights to obtain the comprehensive similarity result. This scheme can dynamically adjust the weights of each dimension according to the actual query situation, achieving adaptive adjustment of search weights. Customized weight configurations are provided for different query needs, further improving the accuracy and flexibility of code retrieval.

[0023] In one possible implementation, the retrieval dimensions include code dimensions and natural language dimensions; the natural language dimensions include functional semantic dimensions and / or query text dimensions.

[0024] This application provides a possible retrieval dimension classification method to adapt to the needs of multi-dimensional and precise code retrieval. Specifically, the retrieval dimensions include code dimensions and natural language dimensions, where the natural language dimension can include at least one of functional semantic dimensions and query text dimensions. Here, the code dimension is used to capture the syntactic structure features of the code, the functional semantic dimension is used to capture the functional intent features of the code, and the query text dimension is used to support keyword full-text retrieval. By defining these dimensions, the syntactic structure information of the code itself is combined with the functional semantic information of the code. The introduction of the functional semantic dimension effectively bridges the gap between code form and functional intent. At the same time, the introduction of the query text dimension supports keyword full-text retrieval, achieving a complementary advantage between vector retrieval and keyword retrieval, realizing a dual match between semantic and code requirements, and further improving the accuracy of code retrieval.

[0025] In one possible implementation, the multi-dimensional information extraction process performed on the query question to obtain a query information group includes:

[0026] The query problem is subjected to code feature extraction processing to obtain code query features, and the code query features are vectorized to obtain code query vectors corresponding to the code dimension.

[0027] If the natural language dimension includes the functional semantic dimension, then the query question is subjected to functional semantic extraction processing to obtain functional semantic query features, and the functional semantic query features are vectorized to obtain functional semantic query vectors corresponding to the functional semantic dimension.

[0028] The code query vector, the functional semantic query vector, and / or the query question are determined as the query information in the query information group.

[0029] Here, this embodiment of the application addresses the query question by extracting different query information based on different retrieval dimensions. First, based on the code dimension, code feature extraction and vectorization are performed to obtain a code query vector that accurately matches the code dimension. Second, for the natural language dimension, if the natural language dimension includes the query text dimension, the query text is directly used as the query information for matching; if the natural language dimension includes the functional semantic dimension, functional semantic features are extracted and vectorized to obtain a functional semantic query vector. This scheme achieves multi-dimensional decomposition and vectorization of query information. Through the above method, the user's natural language query is mapped to the same multi-dimensional vector space as the database, achieving similar matching. The code query vector is used to match code structure, the functional semantic query vector is used to match functional intent, and the original query question can be used for subsequent full-text search, realizing the synergy between vector retrieval and keyword retrieval, achieving comprehensive and accurate code retrieval across multiple retrieval dimensions.

[0030] In one possible implementation, prior to obtaining the preset code vector database, the method further includes:

[0031] Obtain the source code file;

[0032] The source code file is split into multiple code segments;

[0033] Based on the code snippet, generate multiple search information corresponding to the multiple search dimensions;

[0034] The code snippet and the corresponding multiple retrieval information are associated and stored in the preset code vector database.

[0035] Before performing code retrieval, this application's embodiments pre-construct a code vector database for multi-dimensional retrieval using a large number of source code files. Specifically, source code files are obtained, segmented into multiple code fragments, and retrieval information corresponding to multiple retrieval dimensions is generated based on these code fragments. Finally, the code fragments and their corresponding retrieval information are associated and stored in the pre-set code vector database, thereby constructing a multi-dimensional knowledge base. This effectively solves the problem of single-dimensional knowledge bases in existing technologies, establishes the association between code fragments and retrieval information, supports subsequent multi-space retrieval and fusion sorting, provides high-quality, multi-dimensional data support for online retrieval, and ensures the accuracy of code retrieval.

[0036] In one possible implementation, generating multiple search information corresponding to the multiple search dimensions based on the code snippet includes:

[0037] The code snippet is subjected to code embedding processing to generate a code embedding vector corresponding to the code dimension;

[0038] The code snippet is subjected to functional summary processing to generate functional summary text corresponding to the dimension of the query text;

[0039] If the natural language dimension includes the functional semantic dimension, then the functional summary text is subjected to text embedding processing to generate a functional semantic vector corresponding to the functional semantic dimension.

[0040] This application's embodiments generate multi-dimensional retrieval information through a dual-path processing approach. The first path performs code embedding processing, generating a code embedding vector corresponding to the code dimension, capturing the syntactic structure features of the code. The second path performs functional summary processing, generating a functional summary text corresponding to the query text dimension, which can be further processed by text embedding to generate a functional semantic vector corresponding to the functional semantic dimension. Through this method, the code embedding vector captures the syntactic features of the code, while the functional summary text and functional semantic vector capture the functional intent of the code, achieving a multi-dimensional semantic description of the same code fragment. The parallel processing of the two paths improves offline construction efficiency and provides complete vectorized information for subsequent multi-dimensional retrieval. This breaks away from the traditional single representation mode of one code fragment corresponding to one vector, instead adopting a multi-dimensional representation mode where one code fragment corresponds to multiple dimension vectors. Each dimension vector captures code information from a different perspective, collectively constituting a complete semantic description of the code fragment, thus improving the accuracy of code retrieval.

[0041] In one possible implementation, the step of performing functional summary processing on the code fragment to generate functional summary text corresponding to the query text dimension includes:

[0042] Get the preset prompt word template;

[0043] The code snippet is then inserted into the prompt word template to generate a complete prompt word;

[0044] The complete prompt word is input into a preset code comprehension model to generate the functional summary text.

[0045] This application embodiment obtains a preset prompt word template, fills the template with code snippets to generate complete prompt words, and then inputs the complete prompt words into a preset code understanding big model to generate functional summary text. This scheme, through prompt word engineering, ensures that the generated functional summaries have a uniform format and structure, facilitating subsequent vectorization and retrieval. Simultaneously, through explicit instruction constraints, it reduces the randomness and illusions of the big model, improving the accuracy of the summaries. The generated summaries can be used simultaneously for vector retrieval at the functional semantic dimension and full-text retrieval at the query text dimension, achieving dual functionality. Through the above method, accurate matching of query questions can be achieved at the natural language dimension, further improving the accuracy of code retrieval.

[0046] In one possible implementation, after determining the target code segment of the query question based on the comprehensive similarity result, the method further includes:

[0047] Obtain the context information of the target code fragment; wherein, the context information includes multiple elements from the original code, functional summary text, and meta-information;

[0048] The context information is formatted according to a preset assembly template to obtain a structured context; wherein, the structured context is used to prompt a preset answer generation model to generate a query answer for the query question.

[0049] In this embodiment, after determining the target code fragment, the context information of the target code fragment is obtained, and the context information is formatted according to a preset assembly template to obtain a structured context. The structured context may include the original code for code analysis, the functional summary for code understanding, and meta-information for code location, etc., providing sufficient and accurate information support for the large model to generate high-quality answers, effectively avoiding errors or illusions caused by missing context, and improving the accuracy of large model reasoning.

[0050] In one possible implementation, after formatting the context information according to a preset assembly template to obtain a structured context, the method further includes:

[0051] Based on the structured context, answer generation prompts are determined; wherein, the answer generation prompts include multiple elements from system role settings, task descriptions, the structured context, and output format requirements;

[0052] The answer generation prompts are input into the preset answer generation model to determine the query answer based on the output of the preset answer generation model.

[0053] This application's embodiments guide large-scale models to generate professional and accurate answers through system role settings and task descriptions. Output format requirements ensure clear answer structure and complete content, and generation is based on a structured context, avoiding errors or illusions caused by missing context. Ultimately, it provides users with complete answers including a summary of the current implementation, problem and bottleneck analysis, optimization suggestions, and optimized code examples, improving the accuracy and comprehensiveness of large-scale model reasoning and further enhancing the user experience.

[0054] In one possible implementation, prior to obtaining the preset code vector database, the method further includes:

[0055] Obtain historical defect data from the code repository; wherein, the historical defect data includes at least one of the following: defective code snippets, defect type identifiers, severity level tags, defect occurrence time, defect fix commit records, and defect-related code change sets;

[0056] The historical defect data is preprocessed to obtain a defect training sample set; wherein, the preprocessing includes: code segmentation of defect code fragments, extraction of abstract syntax tree features of defect code, and labeling of defect type and severity level;

[0057] The code defect prediction model is trained using the defect training sample set to obtain a pre-defined classification or regression model; wherein the code defect prediction model is used to output a corresponding defect risk score based on the input code fragment.

[0058] In response to obtaining the target code segment among the plurality of code segments, the target code segment is input into the code defect prediction model to generate a defect risk score for the target code segment;

[0059] The defect risk score is encoded into a defect risk vector, and the defect risk vector has a preset vector dimension;

[0060] The defect risk vector is used as the retrieval information of the third retrieval dimension of the target code segment, and is associated and stored with the code vector and function vector of the target code segment.

[0061] This application's embodiments introduce a defect prediction model trained on historical defect data, quantifying the potential risks of code into risk vectors and using them as a new retrieval dimension. This allows the retrieval system to simultaneously consider the code's functional matching degree, structural matching degree, and defect risk level when responding to queries. On the one hand, it can prioritize recommending low-risk, high-reliability code snippets; on the other hand, when users explicitly need to investigate high-risk code, the weights can be adjusted according to risk preferences to quickly locate potentially defective code areas, thereby improving the security and usability of code retrieval results.

[0062] In one possible implementation, the above method further includes:

[0063] Obtain the target code fragment and the first programming language identifier of the target code fragment;

[0064] Obtain the preset code translation model;

[0065] In response to the query question matching the target code snippet, or in response to a cross-language translation request initiated by the user, the target programming language is determined; wherein the target programming language is a programming language specified by the user, or a preferred programming language predicted based on the user's historical behavior, or a preset default programming language;

[0066] The code translation model is invoked to translate the target code fragment from the first programming language into an equivalent code fragment using the target programming language;

[0067] Return the equivalent code snippet to determine the query answer using the equivalent code snippet.

[0068] In one possible implementation, before obtaining the preset code vector database, the above method further includes:

[0069] For each of the plurality of code snippets, the code translation model is invoked to generate an equivalent code snippet in at least one preset programming language;

[0070] The equivalent code fragments are associated with the code fragments and stored in the preset code vector database as retrieval information for cross-language retrieval dimensions.

[0071] This application's embodiments, by introducing a code translation model, can overcome the limitations of programming languages ​​and provide users with cross-language equivalent implementation suggestions. When a user is familiar with language A but the codebase only contains an implementation in language B, the system can automatically translate the language B code into language A code, reducing the user's understanding and migration costs. For development teams with multi-language technology stacks, a unified search entry point can be used to obtain implementation references in multiple languages, improving the cross-language applicability of code reuse. By pre-generating and storing equivalent code snippets in common languages ​​offline, the latency of online translation can be reduced, and the system response speed can be improved, thereby further enhancing the user experience.

[0072] Secondly, embodiments of this application provide a code retrieval device, comprising: a first acquisition module, an extraction module, a second acquisition module, a calculation module, and a determination module, wherein,

[0073] The first acquisition module is used to acquire the query question input by the user;

[0074] The extraction module is used to perform multi-dimensional information extraction processing on the query question to obtain a query information group; wherein, the query information group includes query information corresponding to multiple retrieval dimensions;

[0075] The second acquisition module is used to acquire a preset code vector database; wherein, the preset code vector database includes multiple code segments and a search information group associated with the code segments, and the search information group includes multiple search information corresponding to the multiple search dimensions;

[0076] The calculation module is used to calculate the comprehensive similarity result of each code segment based on the associated retrieval information group and the query information group.

[0077] The determination module is used to determine the target code segment of the query question based on the comprehensive similarity result.

[0078] In one possible implementation, the computing module is used for:

[0079] For each of the search dimensions, the similarity between the search information associated with the code fragment and the query information is calculated to obtain multiple similarities corresponding to multiple search dimensions;

[0080] The multiple similarities are fused to obtain the comprehensive similarity result.

[0081] In one possible implementation, the computing module is specifically used for:

[0082] The query question is processed to identify the query intent, so as to obtain the query intent features corresponding to each of the search dimensions;

[0083] Calculate the proportion of the query intent feature corresponding to each retrieval dimension among all query intent features;

[0084] The fusion weight of the retrieval dimension is determined based on the proportion of the aforementioned features.

[0085] Based on the fusion weights, a weighted sum of the similarities is calculated to obtain the comprehensive similarity result.

[0086] In one possible implementation, the retrieval dimensions include code dimensions and natural language dimensions; the natural language dimensions include functional semantic dimensions and / or query text dimensions.

[0087] In one possible implementation, the extraction module is specifically used for:

[0088] The query problem is subjected to code feature extraction processing to obtain code query features, and the code query features are vectorized to obtain code query vectors corresponding to the code dimension.

[0089] If the natural language dimension includes the functional semantic dimension, then the query question is subjected to functional semantic extraction processing to obtain functional semantic query features, and the functional semantic query features are vectorized to obtain functional semantic query vectors corresponding to the functional semantic dimension.

[0090] The code query vector, the functional semantic query vector, and / or the query question are determined as the query information in the query information group.

[0091] In one possible implementation, before the second acquisition module acquires a preset code vector database, the above-mentioned apparatus further includes:

[0092] The third acquisition module is used to acquire source code files;

[0093] The segmentation module is used to segment the source code file to obtain multiple code fragments;

[0094] The generation module is used to generate multiple search information corresponding to the multiple search dimensions based on the code snippet;

[0095] The storage module is used to associate and store the code fragment and the corresponding multiple retrieval information in the preset code vector database.

[0096] In one possible implementation, the generation module is used for:

[0097] The code snippet is subjected to code embedding processing to generate a code embedding vector corresponding to the code dimension;

[0098] The code snippet is subjected to functional summary processing to generate functional summary text corresponding to the dimension of the query text;

[0099] If the natural language dimension includes the functional semantic dimension, then the functional summary text is subjected to text embedding processing to generate a functional semantic vector corresponding to the functional semantic dimension.

[0100] In one possible implementation, the generation module is specifically used for:

[0101] Get the preset prompt word template;

[0102] The code snippet is then inserted into the prompt word template to generate a complete prompt word;

[0103] The complete prompt word is input into a preset code comprehension model to generate the functional summary text.

[0104] In one possible implementation, after the determining module determines the target code fragment of the query question based on the comprehensive similarity result, the above apparatus further includes a context generation module for:

[0105] Obtain the context information of the target code fragment; wherein, the context information includes multiple elements from the original code, functional summary text, and meta-information;

[0106] The context information is formatted according to a preset assembly template to obtain a structured context; wherein, the structured context is used to prompt a preset answer generation model to generate a query answer for the query question.

[0107] In one possible implementation, after the context generation module formats the context information according to a preset assembly template to obtain a structured context, the above apparatus further includes a second determining module, used for:

[0108] Based on the structured context, answer generation prompts are determined; wherein, the answer generation prompts include multiple elements from system role settings, task descriptions, the structured context, and output format requirements;

[0109] The answer generation prompts are input into the preset answer generation model to determine the query answer based on the output of the preset answer generation model.

[0110] Thirdly, embodiments of this application provide a computing device, which includes a memory and a processor;

[0111] Memory and processor are coupled;

[0112] Memory is used to store program instructions;

[0113] The processor is used to invoke program instructions to cause the computing device to execute the code retrieval methods involved in the first aspect and any possible implementation.

[0114] Fourthly, embodiments of this application provide a computer-readable storage medium storing computer-executable instructions, which, when executed by an electronic device, are used to implement the code retrieval method as described in the first aspect and any possible implementation.

[0115] Fifthly, embodiments of this application provide a computer program product, including a computer program that, when executed by an electronic device, implements the code retrieval method involved in the first aspect and any possible implementation.

[0116] The code retrieval method and computing device provided in this application, when a user initiates a query using natural language, extracts and processes information from multiple retrieval dimensions to obtain query information corresponding to these dimensions. This information is then combined with stored retrieval information corresponding to these dimensions to calculate the similarity of code fragments. Each retrieval dimension captures similarity information from different perspectives, achieving matching between the query and the code fragment across multiple retrieval dimensions. This enables precise alignment between natural language queries and code logic. This method overcomes the limitations of traditional single-vector retrieval, achieving more accurate code semantic matching and effectively improving the accuracy of code retrieval. Attached Figure Description

[0117] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0118] Figure 1 This is a schematic diagram of the architecture of a code retrieval system provided in an embodiment of this application;

[0119] Figure 2 Flowchart of the code retrieval method provided in the embodiments of this application Figure 1 ;

[0120] Figure 3 Flowchart of the code retrieval method provided in the embodiments of this application Figure 2 ;

[0121] Figure 4Flowchart of the code retrieval method provided in the embodiments of this application Figure 3 ;

[0122] Figure 5 A data flow diagram of a multidimensional code retrieval and generation system provided in an embodiment of this application;

[0123] Figure 6 This is a schematic diagram of the structure of a code retrieval device provided in an embodiment of this application;

[0124] Figure 7 This is a schematic diagram of the structure of a computing device provided in an embodiment of this application. Detailed Implementation

[0125] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with those of this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of the embodiments of this application as detailed in the appended claims.

[0126] To facilitate understanding, the technical terms involved in the embodiments of this application will be explained first.

[0127] Code snippet (CS): refers to a small, reusable piece of source code text used to perform a specific function or demonstrate programming logic.

[0128] Vector Database (VDB): A database specifically designed for storing, managing, and efficiently retrieving high-dimensional vector data. It is a core infrastructure for enabling semantic search and similarity matching in Artificial Intelligence (AI).

[0129] Embedding Vector (EV): Converts unstructured data such as text, images, and code into fixed-dimensional numerical vector representations, enabling mathematical calculations and similarity comparisons.

[0130] Fused Retrieval-Augmented Generation (Fused RAG): An advanced retrieval enhancement generation technique that expands user queries into multi-dimensional queries and reorders and integrates search results to provide more comprehensive and accurate answers.

[0131] Existing code retrieval methods suffer from the inability to accurately align natural language queries with code logic, resulting in low accuracy in code retrieval. To address this issue, the code retrieval method and computing device provided in this application extract information from the query from multiple retrieval dimensions, thereby obtaining query information corresponding to these dimensions. This information is then combined with stored retrieval information corresponding to these dimensions to calculate the similarity of code fragments. Finally, by incorporating multi-dimensional retrieval methods, the code retrieval results are determined.

[0132] Optionally, this application addresses the technical problem that code RAG systems rely solely on single-dimensional vectors of the original code for retrieval, resulting in a semantic gap between natural language queries and code implementation. This leads to low retrieval recall, poor matching accuracy, and an inability to accurately locate the code logic required by the user. To address this, a multi-dimensional code retrieval method is provided. This method primarily achieves accurate matching by generating multi-dimensional vector representations of the same code block, establishing associations between multi-dimensional vectors using unique identifiers, and simultaneously matching in multiple vector spaces during retrieval, then fusing and sorting the results before returning them. The aim is to improve code retrieval accuracy by overcoming the limitations of traditional single-vector retrieval through multi-dimensional vector representation, achieving more precise code semantic matching. It also enhances code understanding capabilities by combining code structure information and functional semantic information to provide multi-level code understanding. Finally, it resolves the semantic gap problem by bridging the gap between code form and functional intent through dual embedding.

[0133] The following is combined with Figure 1 The application scenarios involved in the embodiments of this application will be described.

[0134] This section describes the specific application environment architecture or hardware architecture that the code retrieval method depends on. (References) Figure 1 , Figure 1 This is a schematic diagram of a code retrieval system architecture provided in an embodiment of this application. The code retrieval system is a computer device. Figure 1 As shown, the above architecture includes at least one of a data acquisition device 101, a processing device 102, and a display device 103.

[0135] It is understood that the structures illustrated in the embodiments of this application do not constitute a specific limitation on the architecture of the code retrieval system. In other feasible embodiments of this application, the above architecture may include more or fewer components than illustrated, or combine some components, or split some components, or arrange different components, which can be determined according to the actual application scenario and is not limited here. Figure 1 The components shown can be implemented in hardware, software, or a combination of both.

[0136] In the specific implementation process, the data acquisition device 101 may include an input / output interface or a communication interface, and the data acquisition device 101 can be connected to the processing device through the input / output interface or the communication interface.

[0137] The processing device 102 can extract information from the query question from multiple retrieval dimensions to obtain query information corresponding to multiple retrieval dimensions. It then combines the retrieval information corresponding to multiple retrieval dimensions to calculate the similarity of code fragments, and finally combines the multi-dimensional retrieval methods to determine the code retrieval results.

[0138] The display device 103 can also be a touch screen or the screen of a terminal device, used to receive user commands while displaying the above-mentioned content, so as to realize interaction with the user.

[0139] It should be understood that the aforementioned processing device can be implemented by a processor reading instructions from memory and executing those instructions, or it can be implemented by a chip circuit.

[0140] Furthermore, the network architecture and business scenarios described in the embodiments of this application are for the purpose of more clearly illustrating the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions provided in the embodiments of this application. As those skilled in the art will know, with the evolution of network architecture and the emergence of new business scenarios, the technical solutions provided in the embodiments of this application are also applicable to similar technical problems.

[0141] Figure 2 Flowchart of the code retrieval method provided in the embodiments of this application Figure 1 ,like Figure 2 As shown, embodiments of this application provide a code retrieval method, which is described in detail below:

[0142] S201: Obtain the query question input by the user.

[0143] The query here is a natural language type.

[0144] Specifically, users can enter natural language queries, such as: "Optimize the code for V2R2 system to handle file uploads, with code snippets attached".

[0145] Optionally, users can input natural language queries through a preset interactive interface, or input queries by voice through a preset sound acquisition device, or send queries via information transmission from a terminal device.

[0146] Optionally, the query can be preprocessed after it is received.

[0147] Optionally, preprocessing includes at least one of removing redundant whitespace, standardizing formatting, detecting encoding errors, and correcting spelling mistakes. This preprocessing effectively improves the accuracy of subsequent feature extraction and the stability of vector generation.

[0148] Optionally, the user input can be standardized, for example, by removing invalid characters or correcting spelling errors.

[0149] Optionally, for the preprocessed query, identify special identifiers and whether the query contains code snippets. These special identifiers may include system names and business scenarios. Identifying code snippets means detecting whether code blocks are embedded in the query, so that targeted code feature extraction can be performed subsequently.

[0150] S202: Perform multi-dimensional information extraction processing on the query question to obtain the query information group.

[0151] The query information group includes query information corresponding to multiple search dimensions.

[0152] In one possible implementation, the retrieval dimensions include code dimensions and natural language dimensions; the natural language dimensions include functional semantic dimensions and / or query text dimensions.

[0153] The query text dimension refers to the dimension of keyword matching between the original text of the user query and the functional summary text of the code snippet.

[0154] This application provides a possible retrieval dimension classification method to adapt to the needs of multi-dimensional and precise code retrieval. Specifically, the retrieval dimensions include code dimensions and natural language dimensions, where the natural language dimension can include at least one of functional semantic dimensions and query text dimensions. Here, the code dimension is used to capture the syntactic structure features of the code, the functional semantic dimension is used to capture the functional intent features of the code, and the query text dimension is used to support keyword full-text retrieval. By defining these dimensions, the syntactic structure information of the code itself is combined with the functional semantic information of the code. The introduction of the functional semantic dimension effectively bridges the gap between code form and functional intent. At the same time, the introduction of the query text dimension supports keyword full-text retrieval, achieving a complementary advantage between vector retrieval and keyword retrieval, realizing a dual match between semantic and code requirements, and further improving the accuracy of code retrieval.

[0155] Optionally, the search dimensions may also include the defect risk dimension.

[0156] In one possible implementation, the retrieval dimension also includes a defect risk dimension. Specifically, a code defect prediction model is trained by combining historical defect (bug) data. This model can learn the correlation between code features and defect risks, encoding potential defect risks in code snippets as defect risk vectors. When a user queries questions related to code quality, stability, or security, such as "What potential bugs are in this code?" or "How to improve code security?", a search can be performed on the defect risk dimension, prioritizing the recall of code snippets with higher defect risks, thereby helping users identify and fix potential problems in advance. By introducing the defect risk dimension, this embodiment not only achieves matching of function and structure but also enables intelligent evaluation of code quality, further enhancing the practical value of code retrieval.

[0157] Optionally, the search dimensions may also include cross-language dimensions.

[0158] In one possible implementation, the retrieval dimension also includes a cross-language dimension. Specifically, by combining a code translation model, equivalent implementation suggestions for different languages ​​are automatically generated. This code translation model can convert code snippets from one programming language into equivalent implementations in another. When a user query involves cross-language development or code migration, such as "how to implement this Java function in Python" or "convert this code to Go," a search can be performed on the cross-language dimension. The code translation model generates equivalent code in the target language, which is then provided to the user as a search result or supplementary information. By introducing the cross-language dimension, this embodiment of the application can support multi-language development scenarios, help users quickly achieve cross-language code migration, and expand the application scope of code retrieval.

[0159] It should be noted that the above search dimensions can be used individually or in any combination. This application embodiment can flexibly configure the type and number of search dimensions according to the needs of actual application scenarios to achieve customized code retrieval services. By introducing defect risk and cross-language dimensions, this application embodiment further enriches the dimensions of code retrieval, enabling it to meet more complex application needs such as code quality assessment and cross-language development, and improving the intelligence and practicality of the code retrieval system.

[0160] S203: Obtain the preset code vector database.

[0161] The preset code vector database includes multiple code snippets and search information groups associated with the code snippets. The search information groups include multiple search information corresponding to multiple search dimensions.

[0162] Optionally, the code snippets in the preset code vector database and the retrieval information groups associated with the code snippets can be obtained by processing source code files in historical code and / or code repositories.

[0163] S204: For each code fragment, calculate the overall similarity result of the code fragment based on the associated retrieval information group and query information group.

[0164] S205: Based on the comprehensive similarity results, determine the target code segment for the query question.

[0165] Optionally, the target code snippet can be a list of code snippets that includes multiple code snippets, or a code snippet with the highest overall similarity result.

[0166] Optionally, the results can be sorted in descending order of comprehensive similarity, and the Top-K results (where K is any positive integer) can be output.

[0167] In one possible implementation, this application embodiment implements intelligent sorting and truncation. All candidates are sorted in descending order of comprehensive score, and the top 5 (Top-5) or top 10 (Top-10) are selected as the final search results.

[0168] Optionally, the sorting process considers result diversity to avoid all results coming from the same file or class. Diversity optimization can be implemented in ways including but not limited to at least one of the following: file-based deduplication and distribution: when multiple candidate code snippets come from the same source code file, the system retains only the one or two snippets with the highest overall score from that file, replacing the remaining snippets with snippets from other files. This ensures that the final search results cover multiple different code files, providing a wider range of information sources. Code type-based distribution control: when there are too many snippets of a certain type of code (such as functions) in the candidate results, the system appropriately reduces the proportion of similar code snippets and increases code snippets of other types (such as classes and methods), so that the search results cover different code levels. Semantic similarity-based clustering deduplication: semantic clustering analysis is performed on the candidate results. When multiple snippets are highly similar in functional semantics, only the snippet with the highest overall score is retained, ensuring the semantic diversity of the search results.

[0169] This application provides a multi-dimensional matching code retrieval method. When a user asks a question using natural language, information is extracted from the query from multiple retrieval dimensions to obtain query information corresponding to these dimensions. The similarity of code fragments is calculated by combining this stored retrieval information. Each retrieval dimension captures similarity information from different perspectives, achieving matching between the query and the code fragment across multiple dimensions. This enables precise alignment between natural language queries and code logic. This method overcomes the limitations of traditional single-vector retrieval, achieving more accurate code semantic matching and effectively improving the accuracy of code retrieval.

[0170] The target code snippet is used to guide the large model in generating query results corresponding to the query question.

[0171] In one possible implementation, after determining the target code fragment for the query question based on the comprehensive similarity result, the method further includes: obtaining context information of the target code fragment; and formatting the context information according to a preset assembly template to obtain a structured context.

[0172] The contextual information includes multiple components from the original code, functional summary text, and metadata. The structured context is used to prompt a pre-defined answer generation model to generate a query answer.

[0173] Specifically, the original code is the complete source code text of the code snippet, used for subsequent code analysis and optimization suggestion generation. The functional summary text is a text generated by the code understanding model, describing the code's functionality in natural language, used to understand the code's business intent. Meta-information includes the code snippet's file path, line number information, code type (such as function, class, method), etc., used for code location and source tracing. Scoring information includes the similarity score of the code snippet across various search dimensions and the overall similarity score, used for relevance assessment.

[0174] Optionally, after obtaining the structured context, the prompt words are sent to the generative large language model, i.e., the preset answer generation model in this embodiment, to obtain the query answer.

[0175] Optionally, the target code snippet can be implemented in the form of a list of code snippets, such as a list of Top-5 data block identifiers (Chunk_ID). In this embodiment, the original code, functional summary, metadata (file path, line number, code type), and score information of each code snippet can be extracted from the database.

[0176] Here, Chunk_ID is used to identify code snippets.

[0177] Optionally, score information for each unique Chunk_ID in the three retrieval spaces is retained. Code snippets that appear in multiple paths simultaneously are marked as high-frequency candidates, typically core results highly relevant to the query.

[0178] The extracted information is formatted according to a standard template to construct a structured context that includes code snippet number, file location, function description, complete code, and relevance score. Each code snippet forms an independent unit of information.

[0179] In this embodiment, after determining the target code fragment, the context information of the target code fragment is obtained, and the context information is formatted according to a preset assembly template to obtain a structured context. The structured context may include the original code for code analysis, the functional summary for code understanding, and meta-information for code location, etc., providing sufficient and accurate information support for the large model to generate high-quality answers, effectively avoiding errors or illusions caused by missing context, and improving the accuracy of large model reasoning.

[0180] In one possible implementation, after formatting the context information according to a preset assembly template to obtain a structured context, the method further includes: determining answer generation prompts based on the structured context; wherein the answer generation prompts include multiple elements from system role settings, task descriptions, structured context, and output format requirements; and inputting the answer generation prompts into a preset answer generation model to determine the query answer based on the output of the preset answer generation model.

[0181] Optionally, the format of each information unit may include at least one of the following: code snippet number, file location, complete code for functional description, and relevance score.

[0182] Specifically, the code snippet number distinguishes different code snippets for easy reference later. The file location includes the file path and line number for quick user location. The function description is a summary text of the code snippet's function, helping to understand the code's business intent. The complete code is the original code text of the code snippet for analysis by larger models. The relevance score is a comprehensive similarity score between the code snippet and the query question, used to assess their relevance.

[0183] Optionally, embodiments of this application may perform answer post-processing, including at least one of the following: formatting the generated text, extracting and highlighting code blocks, adding file path jump links, organizing content according to hierarchical structure, and marking the source of cited code.

[0184] Optionally, after obtaining the query answer, this embodiment of the application can realize result output and interaction. Specifically, it includes a complete answer display function: pushing the processed answer to the user, including a summary of the current implementation, analysis of problems and bottlenecks, optimization suggestions, optimized code examples, relevant code references, and other complete content to improve the user experience.

[0185] This application's embodiments guide large-scale models to generate professional and accurate answers through system role settings and task descriptions. Output format requirements ensure clear answer structure and complete content, and generation is based on a structured context, avoiding errors or illusions caused by missing context. Ultimately, it provides users with complete answers including a summary of the current implementation, problem and bottleneck analysis, optimization suggestions, and optimized code examples, improving the accuracy and comprehensiveness of large-scale model reasoning and further enhancing the user experience.

[0186] Optionally, embodiments of this application can achieve flexible fusion matching based on diverse search dimensions; correspondingly, Figure 3 Flowchart of the code retrieval method provided in the embodiments of this application Figure 2 ,like Figure 3 As shown, embodiments of this application provide a code retrieval method, which is described in detail below:

[0187] S301: Obtain the query question input by the user.

[0188] S302: Perform multi-dimensional information extraction processing on the query question to obtain the query information group.

[0189] The query information group includes query information corresponding to multiple search dimensions.

[0190] S303: Obtain the preset code vector database.

[0191] The preset code vector database includes multiple code snippets and search information groups associated with the code snippets. The search information groups include multiple search information corresponding to multiple search dimensions.

[0192] S304: For each retrieval dimension, calculate the similarity between the retrieval information associated with the code fragment and the query information to obtain multiple similarities corresponding to multiple retrieval dimensions.

[0193] In one possible implementation, embodiments of this application can perform parallel searches simultaneously in three search spaces:

[0194] Search Space 1: Functional Semantic Retrieval. Functional semantic descriptions are input into a Text Embedding (TE) model to generate a 1536-dimensional functional query vector. Cosine similarity retrieval is then performed within the functional vector index space, returning the Top-70 candidates with the highest functional matching degree. The search results emphasize the semantic matching between the functional description and the query intent.

[0195] Search Space Two: Code Structure Search. Code feature keywords are input into a bimodal coding model spanning programming languages ​​and natural language to generate a 768-dimensional code query vector. Cosine similarity search is then performed within the code vector index space, returning the Top-70 candidates with the most similar code structures. The search results emphasize matching code syntax structure and API call patterns.

[0196] Search Space 3: Keyword Text Search: Extract keywords (file upload, V2R2, optimization, asynchronous, performance, etc.) from the query, perform a Best Matching 25 (BM25) full-text search on the functional summary field of the metadata database, and return the Top-70 candidates with the highest keyword matching degree. The search results focus on precise matching of text keywords.

[0197] It should be noted that the three retrieval spaces mentioned above are independent of each other and execute in parallel without blocking each other, which can make full use of computing resources and improve retrieval efficiency.

[0198] After searching in the three retrieval spaces, similarity calculations are performed, including: calculating the cosine similarity between the query vector and all function vectors, calculating the similarity between the query text and function text using BM25, and calculating the cosine similarity between the code query vector and all code embedding vectors in the query.

[0199] It is understood that the above similarity calculation method is only illustrative, and other methods can be used to implement similarity calculation in the embodiments of this application.

[0200] S305: Merge multiple similarities to obtain a comprehensive similarity result.

[0201] In one possible implementation, multiple similarities are fused to obtain a comprehensive similarity result, including: performing query intent recognition processing on the query question to obtain query intent features corresponding to each retrieval dimension; calculating the feature proportion of each retrieval dimension's query intent features among all query intent features; determining the fusion weight of the retrieval dimension based on the feature proportion; and calculating the weighted sum of similarities based on the fusion weight to obtain a comprehensive similarity result.

[0202] Specifically, query intent features can be obtained by decomposing the query question into multiple dimensions. Optionally, the query intent features here can be the query information in step S302 or the features corresponding to the query information.

[0203] Optionally, query intent recognition processing is performed on the query question, including: functional semantic extraction processing and code feature extraction processing.

[0204] In one possible implementation, functional semantic extraction involves identifying the core verbs and business scenarios in the query question as functional semantic features. For example, for the query "optimize the code for handling file uploads in the V2R2 system", the core verb is "optimize", the business scenario is "file upload", and the functional semantic features can be quantified as functional semantic scores or feature vectors.

[0205] In one possible implementation, code feature extraction processing involves identifying the application programming interface (API) names (e.g., "upload", "file", or "multipart"), function naming patterns (e.g., "upload_file" or "handle_upload"), technology stack features (e.g., file stream processing or asynchronous processing), and performance optimization points (e.g., chunked upload or memory management) involved in the query question. Based on the identification results, a list of code feature keywords and patterns is generated as code features.

[0206] Alternatively, functional semantic extraction and code feature extraction can be achieved through keyword recognition, deep learning model-based methods, rule and template-based methods, knowledge graph-based methods, large language model-based prompting methods, or hybrid strategy-based methods.

[0207] Based on the extraction results above, the query intent features corresponding to each retrieval dimension can be determined. For example, the query intent features corresponding to the functional semantic dimension can be represented as the strength of functional semantic features or keyword hit rate; the query intent features corresponding to the code structure dimension can be represented as the number or importance of code features; and the query intent features corresponding to the query text dimension can be represented as the hit rate of general keywords.

[0208] Optionally, the feature percentage can be determined by calculating the weight ratio of the query intent feature corresponding to each retrieval dimension among all query intent features. For example, assuming that the functional semantic feature score is 30%, the code feature score is 40%, and the text feature score is 30% in the query, then the feature percentage of the functional semantic dimension is 30%, the feature percentage of the code dimension is 40%, and the feature percentage of the natural language dimension is 30%.

[0209] Specifically, the fusion weights of the retrieval dimensions are determined based on the feature proportions. One approach is to directly use the feature proportions as the fusion weights for each dimension, i.e., functional semantic weight α=0.3, code structure weight β=0.4, and query text dimension weight γ=0.3. Another approach is to adjust the feature proportions according to a preset mapping rule and then use them as the fusion weights. For example, the functional semantic weight could be set to α=0.4, the code structure weight to β=0.3, and the query text dimension weight to γ=0.3, in order to balance the contribution of each dimension to the retrieval results.

[0210] Finally, based on the fusion weights, a weighted sum of similarities is calculated to obtain the overall similarity result.

[0211] Specifically, the overall similarity is calculated as follows: α * functional semantic similarity + β * code structure similarity + γ * natural language dimension similarity.

[0212] In this embodiment, when performing similarity fusion across multiple search dimensions, query intent is identified based on the query question to analyze user query intent preferences and then configure different fusion weights for different search dimensions. Specifically, the query intent features corresponding to each search dimension are obtained, the feature proportion of each dimension among all features is calculated, the fusion weight of each search dimension is dynamically determined based on the feature proportion, and finally, a weighted sum of similarities is calculated based on the fusion weights to obtain the comprehensive similarity result. This scheme can dynamically adjust the weights of each dimension according to the actual query situation, achieving adaptive adjustment of search weights. Customized weight configurations are provided for different query needs, further improving the accuracy and flexibility of code retrieval.

[0213] S306: Based on the comprehensive similarity results, determine the target code segment for the query question.

[0214] The implementation methods of steps S301 and S201 are the same, and the implementation methods of steps S306 and S205 are the same, which will not be described in detail here.

[0215] In this embodiment, when calculating the similarity between code snippets and query questions, the similarity between the search information and the query information is calculated separately for each search dimension. The results from each search dimension are then fused to obtain a comprehensive similarity result. This method of calculating separately and then fusing achieves accurate matching across each search dimension and enables collaborative decision-making based on multi-dimensional information. It ensures that the matching results for each search dimension are fully considered, avoiding the omission of information from a single dimension. Furthermore, through fusion processing, code snippets that match across multiple dimensions simultaneously receive higher comprehensive scores, effectively improving the accuracy and relevance of the search results.

[0216] The following detailed explanation of step S202 (step S302) is provided with reference to specific embodiments:

[0217] This application describes a code retrieval method in conjunction with specific retrieval dimensions. Optionally, the retrieval dimensions include code dimensions and natural language dimensions; the natural language dimensions include functional semantic dimensions and / or query text dimensions.

[0218] In one possible implementation, the query question undergoes multidimensional information extraction processing to obtain a query information group, including:

[0219] The query question undergoes code feature extraction to obtain code query features, which are then vectorized to obtain code query vectors corresponding to the code dimension. If the natural language dimension includes a functional semantic dimension, the query question undergoes functional semantic extraction to obtain functional semantic query features, which are then vectorized to obtain functional semantic query vectors corresponding to the functional semantic dimension. The code query vectors, functional semantic query vectors, and / or the query question are then identified as query information in the query information group.

[0220] Alternatively, the extraction of code query features can be achieved through keyword recognition, deep learning model-based methods, rule and template-based methods, knowledge graph-based methods, large language model-based hint methods, or hybrid strategy-based methods.

[0221] Optionally, the code query feature can be the code fragment feature in the above embodiments, and the functional semantic query feature can be the functional semantic feature in the above embodiments.

[0222] Optionally, the functional semantic description vector is obtained by calling the text embedding model to calculate the embedding vector of the functional semantic query features.

[0223] Optionally, the code query vector is obtained by calling a bimodal coding model that crosses programming languages ​​and natural language to calculate the embedding vector of the code query features. It is understood that any model capable of achieving bidirectional natural language understanding can implement the functionality of the embodiments of this application.

[0224] Optionally, the above model can be used directly or after being fine-tuned according to specific application scenarios.

[0225] Alternatively, functional semantic extraction and code feature extraction can be achieved through keyword recognition, deep learning model-based methods, rule and template-based methods, knowledge graph-based methods, large language model-based prompting methods, or hybrid strategy-based methods.

[0226] Here, this embodiment of the application addresses the query question by extracting different query information based on different retrieval dimensions. First, based on the code dimension, code feature extraction and vectorization are performed to obtain a code query vector that accurately matches the code dimension. Second, for the natural language dimension, if the natural language dimension includes the query text dimension, the query text is directly used as the query information for matching; if the natural language dimension includes the functional semantic dimension, functional semantic features are extracted and vectorized to obtain a functional semantic query vector. This scheme achieves multi-dimensional decomposition and vectorization of query information. Through the above method, the user's natural language query is mapped to the same multi-dimensional vector space as the database, achieving similar matching. The code query vector is used to match code structure, the functional semantic query vector is used to match functional intent, and the original query question can be used for subsequent full-text search, realizing the synergy between vector retrieval and keyword retrieval, achieving comprehensive and accurate code retrieval across multiple retrieval dimensions.

[0227] In one possible implementation, the embodiments of this application pre-construct a preset code vector database, and accordingly, Figure 4 Flowchart of the code retrieval method provided in the embodiments of this application Figure 3 ,like Figure 4 As shown, embodiments of this application provide a code retrieval method, which is described in detail below:

[0228] S401: Obtain the query question input by the user.

[0229] S402: Perform multi-dimensional information extraction processing on the query question to obtain the query information group.

[0230] The query information group includes query information corresponding to multiple search dimensions.

[0231] S403: Get source code file.

[0232] Optionally, the source code file can be the source code file corresponding to historical code, such as historical project code accumulated in the enterprise code repository; or it can be the source code file in the code repository currently under development, such as the trunk branch or a specific version of the code.

[0233] Optionally, a file filtering step can be included when obtaining source code files. For example, file filtering can be performed using a file filter.

[0234] Specifically, source code files are filtered according to preset configuration rules, removing non-target files and retaining only the core business code files that need to be processed. Non-target files include, but are not limited to, at least one of the following: test files, configuration files, or third-party dependency files. Optionally, binary files and large resource files (files exceeding a preset size threshold) can also be excluded. By filtering files, the amount of data processed subsequently can be effectively reduced, and irrelevant files can be avoided from being included in the knowledge base, thereby improving system processing efficiency and retrieval accuracy.

[0235] Optionally, after obtaining the source code file, a language recognition step can also be included. Specifically, the programming language type of the source code file (such as Python, Java, JavaScript, Go, etc.) is identified, and the corresponding parser and processing strategy are selected based on the language type. Different programming languages ​​have different syntax structures; using an adapted abstract syntax tree parser for different languages ​​can improve the accuracy of code segmentation.

[0236] S404: Split the source code file to obtain multiple code snippets.

[0237] Optionally, intelligent segmentation is performed based on an Abstract Syntax Tree (AST) to identify natural logical boundaries such as functions, classes, and methods. A globally unique identifier (Chunk_ID) is assigned to each code segment to ensure the semantic integrity of each segment after segmentation.

[0238] S405: Generate multiple search information corresponding to multiple search dimensions based on the code snippet.

[0239] It is understood that steps S403-S404 can be executed before or after steps S401 or S402. This application embodiment does not impose specific restrictions on this, as long as steps S403-S404 are executed before step S406.

[0240] In one possible implementation, multiple retrieval information corresponding to multiple retrieval dimensions are generated based on the code snippet, including: performing code embedding processing on the code snippet to generate a code embedding vector corresponding to the code dimension; performing functional summary processing on the code snippet to generate a functional summary text corresponding to the query text dimension; and if the natural language dimension includes a functional semantic dimension, performing text embedding processing on the functional summary text to generate a functional semantic vector corresponding to the functional semantic dimension.

[0241] It should be noted that the two processing paths mentioned above can be executed in parallel without blocking each other, thereby improving offline processing efficiency.

[0242] Optionally, code embedding processing can employ pre-trained code-specific embedding models to convert code snippets into fixed-dimensional vector representations. These models have been pre-trained on large-scale code data and are capable of capturing the syntactic structure and semantic features of the code. In practical applications, pre-trained models can be loaded directly or fine-tuned and optimized according to the characteristics of a specific codebase.

[0243] Optionally, the functional summary processing can use a large language model combined with preset prompt word templates to generate functional summary text in natural language form.

[0244] Optionally, text embedding processing can employ a general text embedding model that maps input text into fixed-dimensional semantic feature vectors. For example, a high semantic similarity sentence vector model based on a Siamese network architecture can be used, which directly measures the semantic relevance between text pairs through a parameter-sharing dual-tower structure; or, a general text embedding foundation model pre-trained on a large-scale multi-domain corpus can be used, which can output dense vector representations containing rich semantic features for texts of arbitrary length.

[0245] This application's embodiments generate multi-dimensional retrieval information through a dual-path processing approach. The first path performs code embedding processing, generating a code embedding vector corresponding to the code dimension, capturing the syntactic structure features of the code. The second path performs functional summary processing, generating a functional summary text corresponding to the query text dimension, which can be further processed by text embedding to generate a functional semantic vector corresponding to the functional semantic dimension. Through this method, the code embedding vector captures the syntactic features of the code, while the functional summary text and functional semantic vector capture the functional intent of the code, achieving a multi-dimensional semantic description of the same code fragment. The parallel processing of the two paths improves offline construction efficiency and provides complete vectorized information for subsequent multi-dimensional retrieval. This breaks away from the traditional single representation mode of one code fragment corresponding to one vector, instead adopting a multi-dimensional representation mode where one code fragment corresponds to multiple dimension vectors. Each dimension vector captures code information from a different perspective, collectively constituting a complete semantic description of the code fragment, thus improving the accuracy of code retrieval.

[0246] In one possible implementation, the code snippet is subjected to functional summary processing to generate functional summary text corresponding to the query text dimension, including: obtaining a preset prompt word template; filling the prompt word template with the code snippet to generate a complete prompt word; and inputting the complete prompt word into a preset code understanding big model to generate functional summary text.

[0247] Optionally, the preset prompt templates can be customized based on the programming language type, code snippet type (function, class, method), or business scenario. For example, for Python functions, the prompt could be designed as "Please describe the functionality of the following Python function in natural language, including input parameters, return value, and core logic:\n{code}"; for Java classes, the prompt could be designed as "Please describe the responsibilities and main methods of the following Java class:\n{code}".

[0248] Optionally, after generating the functional summary text, the summary text can also be length-controlled, deduplicated, and formatted to ensure that the summary quality meets the subsequent retrieval requirements.

[0249] This application embodiment obtains a preset prompt word template, fills the template with code snippets to generate complete prompt words, and then inputs the complete prompt words into a preset code understanding big model to generate functional summary text. This scheme, through prompt word engineering, ensures that the generated functional summaries have a uniform format and structure, facilitating subsequent vectorization and retrieval. Simultaneously, through explicit instruction constraints, it reduces the randomness and illusions of the big model, improving the accuracy of the summaries. The generated summaries can be used simultaneously for vector retrieval at the functional semantic dimension and full-text retrieval at the query text dimension, achieving dual functionality. Through the above method, accurate matching of query questions can be achieved at the natural language dimension, further improving the accuracy of code retrieval.

[0250] In one possible implementation, two parallel processing paths are executed simultaneously on the same code snippet to generate a multidimensional vector representation.

[0251] Path 1: Code embedding processing: After standardizing the segmented data (including at least one of functions, classes or methods) (removing extra whitespace and unifying the encoding format), use the embedding model to generate code embedding vectors of fixed dimensions.

[0252] Path 2: Functional summary processing: After standardizing the segmented data (including at least one of functions, classes or methods), add code understanding prompts and use the code understanding big model to generate functional description text in natural language form.

[0253] Dual-path collaboration mechanism: The two paths described above execute in parallel without blocking each other, improving processing efficiency. Path 1 captures the structural features of the code, while Path 2 captures the functional semantics of the code. The two paths are merged and associated through a unique identifier.

[0254] S406: Associate and store the code snippet and index information in the preset code vector database.

[0255] Optionally, multiple code embedding vectors, functional semantic vectors, original code, and functional summary texts of the same code fragment can be associated with a complete record through Chunk_ID, establishing a one-to-many mapping relationship to support subsequent multi-space retrieval.

[0256] Optionally, the code embedding vectors of all code snippets are stored, supporting similarity retrieval in the code vector space; the functional semantic vectors of all code snippets are stored, supporting similarity retrieval in the functional semantic space. The original content and functional summary of the code snippets are also stored, supporting keyword-based similarity retrieval.

[0257] S407: Obtain the preset code vector database.

[0258] The preset code vector database includes multiple code snippets and search information groups associated with the code snippets. The search information groups include multiple search information corresponding to multiple search dimensions.

[0259] S408: For each code segment, calculate the overall similarity result of the code segment based on the associated retrieval information group and query information group.

[0260] S409: Based on the comprehensive similarity results, determine the target code segment for the query question.

[0261] The implementation of steps S408-S409 can refer to steps S304-S306 in the above embodiments.

[0262] Before performing code retrieval, this application's embodiments pre-construct a code vector database for multi-dimensional retrieval using a large number of source code files. Specifically, source code files are obtained, segmented into multiple code fragments, and retrieval information corresponding to multiple retrieval dimensions is generated based on these code fragments. Finally, the code fragments and their corresponding retrieval information are associated and stored in the pre-set code vector database, thereby constructing a multi-dimensional knowledge base. This effectively solves the problem of single-dimensional knowledge bases in existing technologies, establishes the association between code fragments and retrieval information, supports subsequent multi-space retrieval and fusion sorting, provides high-quality, multi-dimensional data support for online retrieval, and ensures the accuracy of code retrieval.

[0263] To improve the accuracy of code retrieval, this application breaks away from the traditional single representation mode of one code fragment corresponding to one vector in RAG systems, and adopts a multi-dimensional representation mode where one code fragment corresponds to multiple dimension vectors. Each dimension vector captures code information from a different perspective, collectively forming a complete semantic description of the code fragment. The core of this semantic transformation-based multi-dimensional fusion retrieval scheme lies in: offline, actively converting code information into a functional description in natural language form and generating corresponding vectors; online, allowing users' natural language queries to be directly semantically matched with the functional description vectors, thereby bypassing the semantic gap between code and natural language and achieving accurate code retrieval.

[0264] The offline stage refers to the process of associating and storing code fragments and corresponding multiple retrieval information in a preset code vector database, as described in the above embodiments; that is, the stage of building a multidimensional vector knowledge base offline. The online stage refers to the specific process of code retrieval, that is, after obtaining the user query, the target code fragment is finally determined through multidimensional information extraction, multi-space parallel retrieval, fusion sorting.

[0265] Through the coordinated operation of the offline and online phases described above, this embodiment of the application models the syntactic structure and functional semantics of the code separately, enabling natural language queries to be directly matched with the functional semantic vector of the code. This effectively solves the problems of low recall and poor matching accuracy caused by the semantic gap in traditional single-vector retrieval, and improves the accuracy and practicality of code retrieval.

[0266] Optionally, embodiments of this application cover a collaborative working mechanism between code parsing, dual-path vector generation, multi-dimensional fusion retrieval, and intelligent generation. It can intelligently generate multi-dimensional vector representations through parallel dual-path processing and perform multi-space fusion matching during the retrieval phase to improve the accuracy and recall of code retrieval. In one possible implementation, a multi-dimensional code retrieval and generation system provided by embodiments of this application may include the following units:

[0267] Syntax Tree Parsing Unit: This unit performs abstract syntax tree parsing on the source code file, identifying logical structures such as functions, classes, and methods in the code, providing structured information for subsequent intelligent segmentation. It outputs the abstract syntax tree structure object of the file and a list of code unit boundary information (including the type, name, start line number, and end line number of each code unit).

[0268] Intelligent code segmentation unit: Based on the syntax tree parsing results, this unit intelligently segments the code according to natural logical boundaries such as functions, classes, and methods, ensuring that each code snippet is a semantically complete logical unit. It outputs a collection of code snippets, each containing metadata such as code text content, snippet type identifier, starting line number, ending line number, and the file path to which it belongs.

[0269] Identifier Allocation Unit: This unit assigns a globally unique identifier (Chunk_ID) to each code snippet. This identifier is the core link in the multidimensional vector association mechanism. Output: A set of code snippets with unique identifiers.

[0270] Code Preprocessing Unit: This unit performs standardized preprocessing on code snippets, including format normalization, encoding standardization, and length control, providing high-quality input for subsequent embedding computation and summarization generation. Output: Standardized code text.

[0271] Code Embedding Unit: This unit invokes a dedicated code embedding model to convert the standardized code text into a high-dimensional vector representation, capturing the structural features and syntactic patterns of the code. It outputs code embedding vectors.

[0272] Cue Word Construction Unit: This unit constructs standardized cue words for the calling code to understand the large model, guiding the large model to generate high-quality code function summaries. It outputs the complete cue word text.

[0273] Functional Summary Generation Unit: This unit calls the code to understand the large model and generates a natural language functional description of the code snippets based on the constructed prompts. It outputs a functional summary text of the code.

[0274] Summary Embedding Unit: This unit calls a general text embedding model to convert the functional summary text in natural language form into a functional semantic vector, placing it in the same semantic space as the user query. Output: Functional embedding vector.

[0275] Multidimensional Vector Storage Unit: This unit associates multiple dimensions of information (code embedding vector, functional semantic vector, original code, and functional summary) of the same code fragment using unique identifiers, establishing a one-to-many mapping relationship. It outputs the associated data structure, supporting subsequent multi-space retrieval.

[0276] Code / Functional Semantic Vector and Text Retrieval Unit: This unit performs similarity retrieval in the functional semantic vector index space, calculates the cosine similarity between the code fragment vector (i.e., the code query vector) in the query and all code vectors (i.e., the code embedding vector), calculates the cosine similarity between the query vector and all functional semantic vectors, calculates the BM-25 similarity between the query text and all functional summaries, and finally returns the most similar Top-N candidate results.

[0277] It should be noted that, in the embodiments of this application, both code embedding vector and code query vector can be described as code vectors, which refer to fixed-dimensional vector representations obtained by transforming code fragments through a code-specific embedding model, used to characterize the syntactic structural features of the code fragments. The different expressions are merely to emphasize their attributes as embedding vectors or their role as vectors in the retrieval space in different contexts; both are the same concept in the technical solution.

[0278] Accordingly, Figure 5 This application provides a data flow diagram of a multidimensional code retrieval and generation system, which, through an embodiment of the present application, provides a data flow diagram of a multidimensional code retrieval and generation system. Figure 1 The architecture implementation of the code retrieval system in China, such as Figure 5 As shown, the code retrieval system of this application embodiment mainly includes two major processes: knowledge base construction and online retrieval generation.

[0279] Knowledge base construction process: After splitting and preprocessing the code files, functional summary text is generated through the code understanding model. Code embedding vectors and functional summary embedding vectors are generated by embedding the large model. The code fragments, functional summary texts and multi-dimensional vectors are associated and stored to build an enhanced retrieval knowledge base.

[0280] Online retrieval generation process: The prompt words of the user's natural language question are decomposed, and the requirement description vector, code snippet vector and requirement description text are calculated respectively. Based on the enhanced retrieval knowledge base, a hybrid retrieval and ranking are performed, and finally the model generation results are output through the generation of a large model.

[0281] Figure 6 This is a schematic diagram of the structure of a code retrieval device provided in an embodiment of this application. The code retrieval device 60 includes: a first acquisition module 601, an extraction module 602, a second acquisition module 603, a calculation module 604, and a determination module 605, wherein...

[0282] The first acquisition module is used to acquire the query question input by the user;

[0283] The extraction module is used to perform multi-dimensional information extraction processing on the query question to obtain a query information group; wherein, the query information group includes query information corresponding to multiple search dimensions;

[0284] The second acquisition module is used to acquire a preset code vector database; wherein, the preset code vector database includes multiple code fragments and search information groups associated with the code fragments, and the search information groups include multiple search information corresponding to multiple search dimensions;

[0285] The calculation module is used to calculate the comprehensive similarity result of each code segment based on the associated retrieval information group and query information group;

[0286] The determination module is used to determine the target code segment for the query question based on the comprehensive similarity results.

[0287] In one possible implementation, the computing module is used for:

[0288] For each retrieval dimension, the similarity between the retrieval information associated with the code fragment and the query information is calculated to obtain multiple similarities corresponding to multiple retrieval dimensions;

[0289] Multiple similarity scores are fused to obtain a comprehensive similarity result.

[0290] In one possible implementation, the computing module is specifically used for:

[0291] The query questions are processed to identify query intent, so as to obtain the query intent features corresponding to each retrieval dimension;

[0292] Calculate the proportion of the query intent feature corresponding to each retrieval dimension among all query intent features;

[0293] The fusion weights of the retrieval dimensions are determined based on the feature proportions.

[0294] Based on the fusion weights, a weighted sum of similarities is calculated to obtain the overall similarity result.

[0295] In one possible implementation, the retrieval dimensions include code dimensions and natural language dimensions; the natural language dimensions include functional semantic dimensions and / or query text dimensions.

[0296] In one possible implementation, the extraction module is specifically used for:

[0297] The query problem is processed by code feature extraction to obtain code query features, and the code query features are vectorized to obtain code query vectors corresponding to the code dimension.

[0298] If the natural language dimension includes the functional semantic dimension, then the query question is processed by functional semantic extraction to obtain functional semantic query features, and the functional semantic query features are vectorized to obtain functional semantic query vectors corresponding to the functional semantic dimension.

[0299] The code query vector, along with the functional semantic query vector and / or query question, are identified as the query information in the query information group.

[0300] In one possible implementation, before the second acquisition module acquires a preset code vector database, the above-mentioned apparatus further includes:

[0301] The third acquisition module is used to acquire source code files;

[0302] The splitting module is used to split the source code file into multiple code snippets;

[0303] The generation module is used to generate multiple search results corresponding to multiple search dimensions based on code snippets;

[0304] The storage module is used to associate and store code snippets and their corresponding multiple retrieval information to a preset code vector database.

[0305] In one possible implementation, the generation module is used for:

[0306] Code snippets are processed for code embedding to generate code embedding vectors corresponding to the code dimension;

[0307] Perform functional summary processing on the code snippets to generate functional summary text corresponding to the query text dimensions;

[0308] If the natural language dimension includes the functional semantic dimension, then the functional summary text is subjected to text embedding processing to generate a functional semantic vector corresponding to the functional semantic dimension.

[0309] In one possible implementation, the generation module is specifically used for:

[0310] Get the preset prompt word template;

[0311] Fill the code snippet into the prompt template to generate the complete prompt;

[0312] Input the complete prompt words into the preset code comprehension model to generate a functional summary text.

[0313] In one possible implementation, after the determining module determines the target code fragment of the query question based on the comprehensive similarity result, the above apparatus further includes a context generation module for:

[0314] Obtain the context information of the target code snippet; the context information includes multiple elements from the original code, functional summary text, and meta-information.

[0315] Based on the preset assembly template, the context information is formatted to obtain a structured context; the structured context is used to prompt the preset answer generation model to generate the query answer for the query question.

[0316] In one possible implementation, after the context generation module formats the context information according to a preset assembly template to obtain a structured context, the above apparatus further includes a second determining module, used for:

[0317] Based on the structured context, determine the answer generation prompts; these prompts include multiple elements from system role settings, task descriptions, structured context, and output format requirements.

[0318] Input the answer generation prompts into the preset answer generation model, and determine the query answer based on the output of the preset answer generation model.

[0319] Figure 7 This is a schematic diagram of the structure of a computing device provided in an embodiment of this application.

[0320] like Figure 7 As shown, the computing device 70 may include a processor 71 and a memory 72, wherein the processor 71 and the memory 72 are coupled and can communicate with each other; for example, the processor 71 and the memory 72 communicate through a communication bus 73, the memory 72 is used to store computer execution instructions, and the processor 71 is used to call the computer execution instructions in the memory to execute the code retrieval method shown in the above method embodiment.

[0321] Optionally, the computing device 70 may also include a communication interface, which may include a transmitter and / or a receiver.

[0322] Optionally, the aforementioned processor can be a central processing unit (CPU), a graphics processing unit (GPU), other general-purpose processors, a digital signal processor (DSP), or an application-specific integrated circuit (ASIC), etc. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in the embodiments of this application can be directly implemented by a hardware processor, or implemented by a combination of hardware and software modules within the processor.

[0323] This application provides a computer-readable storage medium storing computer program instructions; when the program instructions are executed by an electronic device or a computing device, the electronic device or computing device performs the above-described code retrieval method.

[0324] This application provides a computer program product, which includes computer program instructions. When the computer program instructions are executed, they cause a computing electronic device or computing device to perform the above-described code retrieval method.

[0325] All or part of the steps in the above method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a readable memory. When the program is executed, it performs the steps of the above method embodiments; and the aforementioned memory (storage medium) includes: read-only memory (ROM), RAM, flash memory, hard disk, solid-state drive, magnetic tape, floppy disk, optical disk, and any combination thereof.

[0326] This application describes embodiments of methods, apparatus (systems), and computer program products according to embodiments of this application with reference to flowchart illustrations and / or block diagrams. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processing unit of a general-purpose computer, special-purpose computer, embedded processor, or other programmable terminal device to produce a machine, such that the instructions, which execute via the processing unit of the computer or other programmable terminal device, generate instructions for implementing the flowchart illustrations. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0327] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable terminal device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0328] These computer program instructions can also be loaded onto a computer or other programmable terminal device, causing a series of operational steps to be performed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable device for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0329] Obviously, those skilled in the art can make various modifications and variations to the embodiments of this application without departing from the spirit and scope of the embodiments of this application. Therefore, if these modifications and variations to the embodiments of this application fall within the scope of the claims of this application and their equivalents, the embodiments of this application are also intended to include these modifications and variations.

[0330] In the embodiments of this application, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. The term "or" and its variations can mean "and / or." In the embodiments of this application, the terms "first," "second," etc., are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. In the embodiments of this application, "multiple" refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, and B alone. The character " / " generally indicates that the preceding and following related objects have an "or" relationship.

[0331] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the invention disclosed in the specification and in practice. The embodiments of this application are intended to cover any variations, uses, or adaptations of the embodiments of this application that follow the general principles of the embodiments of this application and include common knowledge or customary technical means in the art not disclosed in the embodiments of this application.

Claims

1. A code retrieval method, characterized in that, include: Obtain the query question input by the user; The query question is subjected to multi-dimensional information extraction processing to obtain a query information group; wherein, the query information group includes query information corresponding to multiple retrieval dimensions; Obtain a preset code vector database; wherein, the preset code vector database includes multiple code snippets and search information groups associated with the code snippets, and the search information groups include multiple search information corresponding to the multiple search dimensions; For each code segment, a comprehensive similarity result is calculated based on the associated retrieval information group and the query information group. Based on the comprehensive similarity results, the target code segment for the query question is determined.

2. The method according to claim 1, characterized in that, For each code segment, the comprehensive similarity result is calculated based on the associated retrieval information group and the query information group, including: For each of the search dimensions, the similarity between the search information associated with the code fragment and the query information is calculated to obtain multiple similarities corresponding to multiple search dimensions; The multiple similarities are fused to obtain the comprehensive similarity result.

3. The method according to claim 2, characterized in that, The process of fusing the multiple similarities to obtain the comprehensive similarity result includes: The query question is processed to identify the query intent, so as to obtain the query intent features corresponding to each of the search dimensions; Calculate the proportion of the query intent feature corresponding to each retrieval dimension among all query intent features; The fusion weight of the retrieval dimension is determined based on the proportion of the aforementioned features. Based on the fusion weights, a weighted sum of the similarities is calculated to obtain the comprehensive similarity result.

4. The method according to any one of claims 1 to 3, characterized in that, The retrieval dimensions include code dimensions and natural language dimensions; the natural language dimensions include functional semantic dimensions and / or query text dimensions.

5. The method according to claim 4, characterized in that, The process of extracting multidimensional information from the query question to obtain a query information group includes: The query problem is subjected to code feature extraction processing to obtain code query features, and the code query features are vectorized to obtain code query vectors corresponding to the code dimension. If the natural language dimension includes the functional semantic dimension, then the query question is subjected to functional semantic extraction processing to obtain functional semantic query features, and the functional semantic query features are vectorized to obtain functional semantic query vectors corresponding to the functional semantic dimension. The code query vector, the functional semantic query vector, and / or the query question are determined as the query information in the query information group.

6. The method according to claim 4, characterized in that, Before obtaining the preset code vector database, the following is also included: Obtain the source code file; The source code file is split into multiple code segments; Each code segment is processed in parallel using two paths to generate multiple retrieval information corresponding to multiple retrieval dimensions; wherein, the retrieval information includes code vectors corresponding to the code dimension generated after code embedding processing of the code segment, and function vectors corresponding to the natural language dimension generated after function summary processing and embedding processing of the code segment. The code snippet and the corresponding multiple retrieval information are associated and stored in the preset code vector database.

7. The method according to claim 6, characterized in that, The step of generating multiple search information corresponding to the multiple search dimensions based on the code fragment includes: The code snippet is subjected to code embedding processing to generate a code embedding vector corresponding to the code dimension; The code snippet is subjected to functional summary processing to generate functional summary text corresponding to the dimension of the query text; If the natural language dimension includes the functional semantic dimension, then the functional summary text is subjected to text embedding processing to generate a functional semantic vector corresponding to the functional semantic dimension.

8. The method according to claim 7, characterized in that, The step of performing functional summary processing on the code fragment to generate functional summary text corresponding to the query text dimension includes: Get the preset prompt word template; The code snippet is then inserted into the prompt word template to generate a complete prompt word; The complete prompt word is input into a preset code comprehension model to generate the functional summary text.

9. The method according to any one of claims 6 to 8, characterized in that, After determining the target code fragment for the query question based on the comprehensive similarity result, the method further includes: Obtain the context information of the target code fragment; wherein, the context information includes multiple elements from the original code, functional summary text, and meta-information; The context information is formatted according to a preset assembly template to obtain a structured context; wherein, the structured context is used to prompt a preset answer generation model to generate a query answer for the query question.

10. The method according to claim 9, characterized in that, After formatting the context information according to a preset assembly template to obtain a structured context, the method further includes: Based on the structured context, answer generation prompts are determined; wherein, the answer generation prompts include multiple elements from system role settings, task descriptions, the structured context, and output format requirements; The answer generation prompts are input into the preset answer generation model to determine the query answer based on the output of the preset answer generation model.

11. A computing device, characterized in that, The computing device includes a memory and a processor; The memory and the processor are coupled; The memory is used to store program instructions; The processor is used to invoke the program instructions to cause the computing device to perform the method as described in any one of claims 1-10.

Images