Code retrieval method and device, equipment and medium

Abstract

The invention relates to the technical field of intelligent decision, and discloses a code retrieval method and device, equipment and a medium, and the method comprises the steps: obtaining and extracting a query intention of a code query text sent by a target user; extracting a search word in the code query text according to the query intention, and performing vocabulary expansion on the search word through a vocabulary expansion rule to obtain an expanded vocabulary; retrieving a first code snippet and a second code snippet corresponding to each expanded vocabulary from the source code file according to multiple retrieval types; collecting the first code snippets and the second code snippets into retrieval results according to retrieval types, and calculating weight values of the retrieval results of different retrieval types in the retrieval results of the source code file; and combining the retrieval results of different retrieval types according to the weight values to obtain a target code retrieval result. According to the method, the retrieval results are merged according to the weights, high-weight core results are reserved preferentially, and the comprehensiveness of the retrieval results is improved while the accuracy of the retrieval results is guaranteed.

Description

Technical Field

[0001] This invention relates to the field of intelligent decision-making technology, and in particular to code retrieval methods, devices, equipment, and media. Background Technology

[0002] As software development becomes increasingly complex and codebases grow larger, developers face unprecedented challenges in understanding, debugging, and maintaining the code.

[0003] Traditional code retrieval methods often rely on keyword matching or simple text search. However, these methods cannot effectively understand the semantic structure of the code, and they are also relatively simplistic, which leads to low accuracy in the search results.

[0004] Therefore, in the face of the ever-increasing demand for code retrieval, current code retrieval methods urgently need to be improved to address the problem of insufficient retrieval accuracy of existing methods. Summary of the Invention

[0005] This invention provides a code retrieval method, apparatus, device, and medium, which mainly solves the problem of insufficient accuracy of retrieval results during code retrieval.

[0006] Firstly, a code retrieval method is provided, including: Obtain the code query text sent by the target user, and extract the query intent of the code query text; Based on the query intent, the search terms in the code query text are extracted, and the search terms are expanded using preset vocabulary expansion rules to obtain expanded vocabulary; According to various preset search types, the first and second code segments corresponding to each expanded term are retrieved from the preset source code files. The first code fragment and the second code fragment are aggregated into the search results of the source code file according to the search type, and the weight value of the search results of each different search type in the search results of the source code file is calculated. The search results for each different search type are merged according to the weight value to obtain the target code search results.

[0007] Secondly, a code retrieval device is provided, comprising: The query intent extraction module is used to obtain the code query text sent by the target user and extract the query intent of the code query text; The lexical expansion module is used to extract lexical terms from the code query text according to the query intent, and expand the lexical terms according to the preset lexical expansion rules to obtain expanded vocabulary. The fragment content retrieval module is used to retrieve the first and second code fragments corresponding to each expanded term from the preset source code files according to various preset retrieval types. The weight value calculation module is used to aggregate the first code fragment and the second code fragment into the search results of the source code file according to the search type, and calculate the weight value of the search results of each different search type in the search results of the source code file. The search result merging module is used to merge the search results of each different search type according to the weight value to obtain the target code search results.

[0008] Thirdly, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the code retrieval method described above.

[0009] Fourthly, a computer-readable storage medium is provided, which stores a computer program that, when executed by a processor, implements the steps of the code retrieval method described above.

[0010] In the above-mentioned code retrieval method, apparatus, computer equipment, and storage medium, the following steps are taken: First, by acquiring the code query text and extracting the query intent, the core user's search needs are accurately captured, avoiding deviations in search direction due to merely matching surface text. Second, by extracting and expanding the vocabulary, the core search terms are focused on based on intent, and vocabulary expansion covers synonyms / variants (such as stems, plurals, and tense variations), improving search coverage and reducing missed detections. Third, by retrieving corresponding results according to multiple search types (such as first code fragment, second code fragment), the search adapts to different search scenarios, taking into account both code structure and text semantics, ensuring the comprehensiveness of search results. Fourth, by aggregating results and calculating the weight of each search type, the value of results of different search types is quantified (e.g., the weight of the first code fragment is higher than that of the text content), avoiding indiscriminate merging that dilutes core results and improving the rationality of results. Fifth, by merging search results according to weight, high-weight core results are prioritized for retention, balancing the comprehensiveness and accuracy of the search. Attached Figure Description

[0011] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the description of the embodiments of the present invention will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0012] Figure 1 This is a schematic diagram of an application environment for a code retrieval method according to an embodiment of the present invention; Figure 2 This is a flowchart illustrating a code retrieval method according to an embodiment of the present invention; Figure 3 This is a schematic diagram of a code retrieval device according to an embodiment of the present invention; Figure 4 This is a schematic diagram of the structure of a computer device according to one embodiment of the present invention; Figure 5 This is another structural schematic diagram of a computer device according to one embodiment of the present invention.

[0013] The objectives, features, and advantages of this invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0014] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0015] The code retrieval method provided in this embodiment of the invention can be applied to, for example... Figure 1 In this application environment, the client communicates with the server via a network. The server first analyzes the user's query intent and search terms, retrieves relevant first code snippets and text content from the source code files based on the search terms, calculates the weight value of each type, obtains the target code retrieval results, and feeds the target code retrieval results back to the client. The client can be, but is not limited to, various personal computers, laptops, smartphones, tablets, and portable wearable devices. The server can be implemented using a standalone server or a server cluster consisting of multiple servers. The invention will be described in detail below through specific embodiments.

[0016] Please see Figure 2 As shown, Figure 2 A flowchart illustrating a code retrieval method provided in an embodiment of the present invention includes the following steps: S1. Obtain the code query text sent by the target user and extract the query intent of the code query text.

[0017] In this embodiment of the invention, the code query text can be text entered by the user in a preset dialog box, or it can be a code block at the cursor position in a preset code editing area.

[0018] In this invention, the code retrieval-related text content input by the user can be received through interactive entry points such as IDE plugin search boxes and web search interfaces. Then, the text is transmitted to the backend in a standardized string form through technical means such as command line parameter parsing. Finally, leading and trailing spaces and invalid symbols are filtered out and the encoding format is unified to obtain the code query text.

[0019] In this embodiment of the invention, extracting the query intent of the code query text includes: Identify the programming language name, operation action, and target object in the code query text; The text syntax of the code query text is identified based on the programming language name, the operation action, and the target object; The text syntax is mapped to a dimension mapping vector according to a preset dimension, and the programming language name, the operation action and the target object are converted into encoding vectors respectively. The dimension mapping vector and the converted encoding vector are then fused into a feature vector. Each intent in the preset code query intent library is converted into an intent vector; Calculate the similarity between the feature vector and the intent vector; The intent vectors with similarity greater than a preset similarity threshold are aggregated into a query intent vector, and the query intent corresponding to the query intent vector is identified.

[0020] In this embodiment of the invention, during the process of matching the preset stop word list with the code query text, the original code query text is first split into individual words by spaces and punctuation marks (word segmentation). The word list after splitting is traversed, and each word is compared with the preset stop word list one by one. Words that belong to the stop word list are filtered out, the matched stop words are removed, and the remaining valid words are concatenated into a coherent text, and the code query text after word removal is output.

[0021] In this invention, the text syntax is mapped to a dimension mapping vector according to a preset dimension, and the programming language name, the operation action, and the target object are respectively converted into encoding vectors. The dimension mapping vector and the converted encoding vector are fused into a feature vector, which can be converted and encoded by a pre-trained language model, or each intent in the preset code query intent library can be converted into an intent vector. This technology is existing technology and will not be described in detail here.

[0022] Furthermore, predefined element word rules are used. Element words include programming language names, operation actions, and target objects. For example, from "Python reads an Excel file," Python is identified, and the Excel file is read. A predefined basic syntax structure for code queries is used (the core is the subject-verb-object logic of "programming language + operation action + target object"). The extracted core element words are substituted into the basic syntax structure to analyze semantic relationships and output the analysis text syntax. Based on the analysis text syntax (core element word combinations), the core element words for code queries are compared one by one with element word combinations in the predefined code query intent library. If multiple intents are matched (such as simultaneously matching "function implementation" and "code example"), the highest priority intent is selected according to a predefined priority (such as function implementation > code example > filename retrieval), and the final query intent is output.

[0023] For example, the grammatical structure of the core element words is: Python (tool), read (action), Excel file (object), output [programming language: Python] execute [operation action: read] operation, the target is [target object: Excel file]. At the same time, it can also be a grammatical structure presented in pure code form.

[0024] In this embodiment of the invention, by identifying core element words such as programming language name, operation action and target object, redundant information is filtered out from the code query text, the key dimensions of the search are accurately locked, and the system is ensured to focus on the user's real needs. By analyzing the syntax of the code query text, the logical relationship between core element words is clarified, and the system avoids misunderstanding the semantic errors caused by keyword stuffing.

[0025] S2. Extract the search terms from the code query text according to the query intent, and expand the search terms according to the preset vocabulary expansion rules to obtain expanded vocabulary.

[0026] In this embodiment of the invention, the search term can be a keyword within the code query text.

[0027] In this embodiment of the invention, extracting the search terms from the code query text according to the query intent includes: The search term suggestion text is constructed based on the preset search suggestion term template and the query intent; The code query text is retrieved based on the search term hint text to obtain the initial search terms; The initial search terms are deduplicated to obtain the search terms.

[0028] In this embodiment of the invention, the preset search term prompt template can be a predefined, structured text framework used to standardize the way large language models (LLMs) generate search terms, including role settings, inputs, outputs, and constraints.

[0029] Furthermore, the user's code query text and the identified query intent are filled into a preset search term suggestion template to form a complete search term suggestion text for the current query. The search term suggestion text and the code query text are sent together to the LLM. Based on the pre-trained code knowledge base, the LLM understands the instructions in the suggestion text and analyzes the semantics of the code query text to generate a list of 3-5 keywords, i.e., preliminary search terms.

[0030] Furthermore, regarding the initial search term judgment, if a certain word appears multiple times in the initial search terms, only one instance of that word is retained. Each word in the deduplicated search term set is compared with the original code query text, and the number of times each word appears in the query text is counted. Based on the counted frequency, the words in the deduplicated search term set are sorted in descending order, with the words that appear more frequently being ranked higher.

[0031] For example, the preset search suggestion template is: Please generate 3 most relevant code search terms based on the following user query and intent. User query: {user_query}. Query intent: {query_intent}. Please use a programming language, technical concept, or API name; The populated suggestion text is: Please generate 3 most relevant code search terms based on the following user query and intent. User query: How to implement code to read an Excel file using Python. Query intent: Functionality retrieval. Please use a programming language, technical concept, or API name. Initial search terms generated by LLM: ["pandas", "csv", "matplotlib", "data visualization", "chart generation"].

[0032] In this embodiment of the invention, a preset stemming algorithm is invoked to extract stems from the search terms. Each word in the search terms is traversed, and according to the algorithm's preset word form restoration rules (such as removing inflectional suffixes like -ing, -ed, and -s), words with different grammatical forms are uniformly restored to their core stems (i.e., the basic forms of the words). Then, based on the extracted core stems and combined with a preset vocabulary morphological change rule library (covering common grammatical variant rules such as plural, past tense, present participle, and third-person singular), various legal variant forms corresponding to the stem are generated in reverse. Finally, the core stems and all generated variant forms are integrated to form an expanded vocabulary set containing the original stems and multiple morphological variants, completing the transformation from search terms to expanded vocabulary.

[0033] S3. Retrieve the first and second code segments corresponding to each expanded term from the preset source code files according to various preset search types.

[0034] In this embodiment of the invention, retrieving the first and second code segments corresponding to each expanded term from a preset source code file according to multiple preset retrieval types includes: Identify the syntax tree of the preset source code file, divide the source code file into multiple file blocks according to the syntax tree, and search each of the expanded terms in the source code file according to multiple preset search types to obtain the file blocks corresponding to each expanded term under different search types. Use the code in the file blocks as the first code segment corresponding to each expanded term. The expanded vocabulary is converted into code vocabulary corresponding to the code in the source code file; Identify the location information of the code terms in the source code file; Based on the location information, the preset range context information corresponding to the code words is extracted from the source code file as a second code segment.

[0035] In this embodiment of the invention, during the retrieval of the first code fragment, the syntax parser of the corresponding programming language is invoked to perform lexical and syntactic analysis on the source code, generating a hierarchical abstract syntax tree with syntactic components as nodes; the generated syntax tree is traversed, and the nodes of the syntax tree are traversed according to a preset granularity, extracting the code range and content corresponding to the nodes, and generating structured file blocks with unique identifiers and metadata; all expanded vocabulary is traversed, and matched in the source code according to multiple preset search types, and the corresponding file blocks are located through line number mapping, and after deduplication, an association set of "expanded vocabulary-search type-file block" is formed; the complete code of the file blocks in the association set is extracted, and metadata such as expanded vocabulary and search type are bound, and the first code fragment set corresponding to each expanded vocabulary under different search types is organized.

[0036] Furthermore, during the second code fragment retrieval process, based on the programming language of the preset source code file and the preset vocabulary mapping rules (such as a synonym list), the expanded vocabulary is converted into actual, grammatically correct code vocabulary that exists in the source code. The content of the target source code file is traversed, and the converted code vocabulary is matched one by one to find all positions containing the code vocabulary. The preset context truncation range is determined (such as the N lines before and M lines after the line where the code vocabulary is located, or a fixed number of lines centered on the line where the vocabulary is located). Then, based on the obtained location information (file path, line number), the corresponding position in the source code file is located, and the complete context information surrounding the code vocabulary is extracted according to the preset range as the second code fragment.

[0037] In one embodiment, the contextual text content corresponding to the expanded vocabulary in the source code file can also be extracted as a second code segment. The second code segment is then segmented according to a preset word segmentation rule to obtain vocabulary units. An inverted index (vocabulary directory) is constructed based on the vocabulary units, and maintenance rules are configured for the inverted index to generate a full-text index system.

[0038] Specifically, the formatting and comments are first removed from the source code file, leaving only the plain text content of the code. The plain text content of the code is then split according to English words / identifiers, without splitting programming language keywords (such as public / return) and API names (such as String), and associated with a pre-defined programming language vocabulary. Similar terms are linked (such as mapping "login" to "login", and "thread" to "thread") to improve recall and generate vocabulary units.

[0039] Furthermore, using the SQLiteFTS5 full-text search engine, a mapping relationship of "vocabulary → list of documents containing the vocabulary" is established for each vocabulary unit. When configuring index maintenance rules for each vocabulary unit, incremental updates are included: only updating the index for newly added / modified code files without rebuilding the full index. Parameter configurations include: setting the minimum vocabulary length (e.g., ignoring words with 1 character) and a stop word list (filtering meaningless words). The inverted index is deployed to the SQLiteFTS5 engine to form a full-text index system. The full-text index system is used to perform text content retrieval on the source code files to obtain the second code fragment results.

[0040] In one embodiment, filename result retrieval can also be performed. First, the original path of the source code file is obtained, the distance between each path in the original path and a preset query path is calculated, and paths with a distance less than a preset distance threshold are aggregated into preliminary candidate paths. The weight of each path in the preliminary candidate paths is calculated, and the preliminary candidate paths are sorted in descending order according to the weight. A filename index system is constructed based on the sorted paths, and the filename index system is used to retrieve filenames of the source code file to obtain the code filename results.

[0041] Specifically, the system traverses the entire project's file system, collects the original paths of all source code files (such as C:\project.java), calculates the edit distance between the preset query path (the path / filename entered by the user) and each path in the original path according to the preset distance calculation rules, sets a distance threshold (for example, a threshold of 2), and filters out all original paths whose calculated distance is less than the threshold, and summarizes them into preliminary candidate paths.

[0042] Furthermore, the initial candidate paths are precisely filtered using regular expressions. If the preset query path contains wildcards (e.g., ...), the path is further filtered. .java) or regular expressions (e.g., User). (Service) The system uses a regular expression engine to match the paths in the initial candidate paths, and only retains the paths that match the regular expression rules, that is, the set of matched paths.

[0043] Furthermore, for each path in the matching path set, the weight calculation engine is activated to calculate the path weight from three aspects: file type, access path, and project structure weight. For example, core source code files such as .java / .py / .cpp are given high scores, while configuration / documentation files such as .yml / .md / .txt are given low scores. The access history of the files is checked (frequently opened files are given extra points), the modification time (most recently modified files are given extra points), and the path location is checked. Files in the root directory or core modules (such as src / main / ) are given extra points. The above scores are summarized to obtain the final weight score for each path.

[0044] Furthermore, the sorted paths are associated with their corresponding normalized paths, projects, and weight scores, and saved as a filename index that can be quickly queried and sorted. The filename index is then used to retrieve filenames from source code files to obtain the filename retrieval results.

[0045] In this embodiment of the invention, the reference code snippet index and the full-text index search the source code file based on the search terms (or the code block where the user's cursor is located in the preset code editing area) to obtain the first code snippet and the second code snippet results, reducing the time spent manually browsing the source code and facilitating quick reference.

[0046] In this embodiment of the invention, the step of searching the source code file according to the expanded vocabulary and the syntax tree to determine the first code segment corresponding to the expanded vocabulary and the search type includes: Identify the matching type of the expanded vocabulary, compare the expanded vocabulary and the nodes in the syntax tree dimension by dimension according to the matching type, and gather the nodes that match the matching type into candidate nodes; Identify the grammatical structure boundaries of the syntax tree; Based on the grammatical structure boundary, the candidate nodes are deduplicated to generate the first code segment boundary; Based on the boundaries of the first code fragment, extract multiple expanded terms and first code fragments corresponding to the search type from the source code file.

[0047] In this embodiment of the invention, a Tree-sitter parser (syntax parsing tool) is first used to read the source code file to be processed (such as .java / .py / .cpp files). The Tree-sitter parses the source code line by line according to the syntax rules of the corresponding programming language, generating an abstract syntax tree (AST). By recursively traversing all nodes of the AST, the information such as the type (e.g., function, class), position (line number), and content (e.g., function name, class name) of each node is extracted and summarized into a set of syntax tree nodes.

[0048] Furthermore, based on the type and association of syntax tree nodes, the natural boundaries of the code are automatically identified; for example, function boundaries: from the beginning line of the function definition to the end line (such as lines 2-3 of the login function), class boundaries: from the class keyword to the curly braces at the end of the class (lines 1-4), and variable scope: the scope of the function / class in which the variable is declared.

[0049] Furthermore, based on the identified boundaries, the corresponding text content, i.e. the first code segment, is extracted from the source code. At the same time, the core context information of each segment is recorded, including the file to which it belongs: the source code file path corresponding to the segment; the line number: the starting line number and the ending line number of the segment; and the scope: the parent structure to which the segment belongs (e.g., the login function belongs to the UserService class).

[0050] Furthermore, tags (semantic tags) are automatically generated for each extracted first code snippet. Tag types include: function type: such as "login function" "utility function"; parameter type: such as String / int / List. <user>Additional tags: Programming language (Java), fragment type (class / function), etc.

[0051] Furthermore, the database storage structure is designed, including a main table and a tag table. The main table stores basic information about the first code fragment, with fields including: file path, starting line number, ending line number, fragment content, programming language, fragment type (class / function), unique ID, priority, etc. The tag table stores multiple semantic tags for each fragment (one-to-many relationship, one fragment can correspond to multiple tags) by linking the unique ID of the main table through a foreign key.

[0052] Furthermore, the "basic information + semantic tags" of all first code fragments are stored in the aforementioned database table, and a query index is established for fields such as tags, file paths, and line numbers, ultimately forming a "first code fragment index system". The first code fragments are obtained by searching the source code files through this system.

[0053] In this embodiment of the invention, the vocabulary and syntax tree nodes are expanded by comparing and matching each dimension according to the matching type, and the matching nodes are accurately located to improve the accuracy of the first code fragment retrieval; the candidate nodes are deduplicated based on the grammatical structure boundary and fragment boundaries are generated to avoid duplicate fragments and ensure the uniqueness and completeness of the retrieval results; the first code fragment is extracted based on the boundary, which can accurately lock the effective code range in the source code and output structured, high-quality first code fragment results.

[0054] S4. The first code fragment and the second code fragment are aggregated into the search results of the source code file according to the search type, and the weight value of the search results of each different search type in the search results of the source code file is calculated.

[0055] In this embodiment of the invention, the first code fragment (such as a complete function or class) output by the first code fragment indexing system, which has syntactic integrity and reusability, and the scattered contextual text fragments (i.e., the second code fragments) generated by the full-text indexing system around the search keywords are classified according to a preset search type (such as functional search), and then integrated according to the corresponding source code files, finally forming a structured search result with a single source code file as the unit and clearly classified into two types of fragments according to the search type.

[0056] In this embodiment of the invention, calculating the weight value of the search results for each different search type within the search results of the source code file includes: Select any one of the standard search terms from the expanded vocabulary as the target search term; Identify the number of times the target search term appears in the source code file, and identify the total number of words in the source code file; Calculate the ratio of the occurrence frequency to the total number of words, and use the ratio as the importance weight of the target search term; The importance weights of all the target search terms are concatenated into a search term vector, and the importance weights corresponding to all the target search terms in the first code segment are concatenated into a first weight vector; Calculate the first similarity between the search word vector and the first weight vector, and use the first similarity as the first preliminary weight value of the first code segment; Concatenate the importance weights corresponding to all target search terms in the second code snippet into a second weight vector; Calculate the second similarity between the search word vector and the second weight vector, and use the second similarity as the second preliminary weight value of the second code segment; The first preliminary weight value is linearly calculated by performing a linear operation with the preset weight of the first code segment to obtain the weight value of the first code segment; The second preliminary weight value is linearly calculated by combining it with the preset weight of the second code segment to obtain the weight value of the second code segment.

[0057] In one embodiment, there may be multiple source code files. When there are multiple source code files, firstly, any standard search term from the expanded vocabulary is selected as the target search term. The first occurrence count of the target search term in each file of the source code file is identified, and the total number of first words in the files corresponding to the target search term in the source code file is identified. The first ratio of the first occurrence count to the total number of first words is calculated. The second occurrence count of the target search term in the source code file is identified, and the total number of second words in the source code file is identified. The second ratio of the second occurrence count to the total number of second words is calculated. Then, the weight value of the first code segment and the weight value of the second code segment are calculated through similarity. The steps are the same as those for a single source code file, and will not be repeated here.

[0058] Furthermore, the weight values ​​of filename retrieval results can be calculated. First, the file type and user access frequency of the filename results are identified. Type weights are assigned to the filename results based on the file type, and frequency weights are assigned to the filename results based on the user access frequency. The type weights and frequency weights are then linearly calculated to obtain the preliminary weight values ​​of the filename results. Finally, the preliminary filename result scores are linearly calculated with the preset filename weights to obtain the final weight values ​​of the filename results.

[0059] The following examples use source code files as examples. For instance, expanding the vocabulary: ["open", "read", "csv"] (to find code that reads CSV files); default source code files: 2 files in total, with the following file information: Filename: data_read.py; File type: .py; User access frequency: High (5 times per week); Content (simplified): def read_csv(): f = open("data.csv") content = f.read() return content; Total vocabulary: 28; Filename: test.py; File type: .py; User access frequency: low (once a week); Content (simplified): # Test code print("test") f = open("test.txt"); Total vocabulary: 15; Calculate the first ratio: For data_read.py: the first occurrence of open (within this file): 1 time; the total number of words in the first word of this file: 28; the first ratio = 1 / 28 ≈ 0.0357; For test.py: the first occurrence of "open" is 1 time; the total number of words in the first word of the file is 15; the first ratio = 1 / 15 ≈ 0.0667; Calculate the second ratio (full file dimension): The second occurrence of "open" in all files: 1 (data_read.py) + 1 (test.py) = 2 times; the total number of second words in all files: 28 + 15 = 43; the second ratio = 2 / 43 ≈ 0.0465; Calculate the importance weight of the target search term: Assume the linear operation rule: first ratio 0.4 + second ratio 0.6; The weight of `open` in `data_read.py` is approximately 0.03570.4 + 0.04650.6, which is approximately 0.0143 + 0.0279 = 0.0422. The weight of `open` in `test.py` is approximately 0.06670.4 + 0.04650.6, which is approximately 0.0267 + 0.0279 = 0.0546. Calculate the weight of `read`: `data_read.py` weight: 0.0714; `test.py` weight: 0; Calculate the weights of the CSV files: data_read.py weight: 0.0536; test.py weight: 0; Search term vectors (the average weight of all search terms, representing the search intent): (0.0422+0.0714+0.0536) / 3 ≈ 0.0557; The first weight vector in data_read.py (weights of all search terms in that file): [0.0422, 0.0714, 0.0536] → Mean ≈ 0.0557; The first weight vector in test.py: [0.0546, 0, 0] → Mean ≈ 0.0182; Calculate the first similarity (retrieval word vector vs. first weight vector). Using cosine similarity (simplified to the ratio of vector means): The initial similarity score in data_read.py is 0.0557 / 0.0557 = 1, therefore the initial weight value is 1. The first similarity of test.py = 0.0182 / 0.0557 ≈ 0.327 → Initial weight value = 0.327; Calculate the second weight vector and the second similarity: The second weight vector is logically consistent with the first weight vector: The second similarity in data_read.py = 1 → initial weight value = 1; The second similarity score of test.py is 0.327, so the initial weight value is 0.327. Calculate the initial weight value of the filename result: Rule: Type weight 0.5 + Frequency weight 0.5 data_read.py: Type weight = 0.8 (.py file), frequency weight = 0.9 (high access); Preliminary weight value = 0.80.5 + 0.90.5 = 0.85; test.py: Type weight = 0.8, frequency weight = 0.3 (low access); Preliminary weight value = 0.8 + 0.5 + 0.3 = 0.55; final weights in data_read.py: Weight of the first code snippet: 10.6 = 0.6; Text content weight: 10.2 = 0.2; Filename weight: 0.85 0.2 = 0.17; final weights in test.py: Weight of the first code snippet: 0.3270.6 ≈ 0.196; Text content weight: 0.3270.2 ≈ 0.065; Filename weight: 0.55 0.2 = 0.11; In this embodiment of the invention, the importance weight is quantified based on the frequency of occurrence of search terms, making the weight calculation more objective and quantifiable, and avoiding subjective assignment bias; through vector similarity matching, the relevance between search terms and the first code snippet / text content is accurately associated, improving the accuracy of weight evaluation; and by combining preset weights with linear operations, the algorithm quantification results and business rules are taken into account, making the final weight more in line with actual search needs.

[0060] S5. Merge the search results for each different search type according to the weight value to obtain the target code search results.

[0061] In this embodiment of the invention, merging the search results for each different search type according to the weight value to obtain the target code search result includes: Select any one of the first code snippets as the target code snippet; Calculate the code similarity between the target code segment and each of the other code segments within the first code segment; The code segments with a similarity less than a preset code similarity threshold are aggregated into a result of segments to be retained; According to a preset priority rule, one of the results of the segments to be retained is retained, and the retained result and the segments whose code similarity is greater than a preset code similarity threshold are used together as the retained code segment results; Based on the weight value corresponding to the retained code fragment result and the weight value corresponding to the second code fragment, the search results of each different search type are sorted in descending order to obtain the sorted results; The sorted results are used as the target code retrieval results.

[0062] In this embodiment of the invention, all first code fragment results are extracted from the retrieval results. One (optionally the first one) is selected as the target first code fragment according to a preset rule. The code text of the target fragment and the fragment to be compared is preprocessed. A two-dimensional DP array is constructed based on the code text of the target fragment and the fragment to be compared, where dp[i][j] represents the minimum edit distance between the first i characters of the target string and the first j characters of the string to be compared. The edit distance is converted into a similarity value of 0 to 1. For all first code fragments, the similarity with the target fragment is calculated one by one, and the mapping relationship of "first code fragment + similarity" is stored. When the similarity is less than a preset threshold (indicating that the two code fragments are not duplicated), one of them is retained according to the priority rule. If the similarity is greater than or equal to the threshold (considered as duplicate), only the result with the highest similarity is retained.

[0063] For example, set a preset threshold: for example, set it to 0.8 (similarity ≥ 0.8 is considered a duplicate, < 0.8 is considered a non-duplicate); iterate through the similarity results: check the similarity between each first code snippet and the target one by one; if the similarity is ≥ the threshold: it is determined to be a duplicate, and only the snippet with the highest similarity is kept; if the similarity is < the threshold: select to keep the target snippet or the current snippet according to the preset priority rules (such as "higher search score", "longer code length", "line number earlier"); finally, there are no duplicate first code snippets in the "Keep First Code Snippet Results" list.

[0064] In one embodiment, the search results for filenames can also be deduplicated. First, an empty hash table (dictionary) is created to record the combination "file path:starting line number". The key is the string representing this combination, and the value can be a boolean value (such as True) to indicate that it already exists. The filename results (containing file path and starting line number) in the search results are read one by one. For each filename result, "file path:starting line number" is concatenated as a unique key (e.g., / project / utils.py:45). If the key is not in the hash table, the key is stored in the hash table, and the filename result is added to the "retain filename results" list. If the key is already in the hash table, the filename result is skipped (duplicate removal). Finally, the "retain filename results" list is the set of deduplicated filenames.

[0065] Finally, the deduplicated filename results, the results of retaining the first code fragment, and the results of retaining the second code fragment are combined to obtain the target code retrieval results.

[0066] In this embodiment of the invention, redundant results can be eliminated through deduplication, forming a clean set of candidate results; the weight value of the search results is calculated based on the frequency of occurrence of search terms in the code query text, and the search results are sorted according to the weight value, thereby improving the relevance, accuracy and recall of the search results, and ensuring that the retrieved code resources better meet the actual needs of users.

[0067] As can be seen, in the above scheme, for the business of constructing target code retrieval results, the code query text sent by the target user is obtained, and the query intent of the code query text is extracted; according to the query intent, the search terms in the code query text are extracted, and the search terms are expanded according to the preset vocabulary expansion rules to obtain expanded vocabulary; according to multiple preset search types, the first code segment and the second code segment corresponding to each expanded vocabulary are retrieved from the preset source code file; the first code segment and the second code segment are aggregated into the retrieval results of the source code file according to the search type, and the weight value of the retrieval results of each different search type in the retrieval results of the source code file is calculated; according to the weight value, the retrieval results of each different search type are merged to obtain the target code retrieval results. By merging the retrieval results according to weight, the core results with high weight are retained first, which improves the comprehensiveness of the retrieval results while ensuring the accuracy of the retrieval results.

[0068] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0069] In one embodiment, a code retrieval device is provided, which corresponds one-to-one with a code retrieval method described in the above embodiments. For example... Figure 3 As shown, this code retrieval device includes a query intent extraction module 101, a search term extraction and vocabulary expansion module 102, a fragment content retrieval module 103, a weight value calculation module 104, and a search result merging module 105. Detailed descriptions of each functional module are as follows: The query intent extraction module 101 is used to obtain the code query text sent by the target user and extract the query intent of the code query text; The retrieval term extraction and vocabulary expansion module 102 is used to extract the retrieval terms in the code query text according to the query intent, and expand the retrieval terms according to the preset vocabulary expansion rules to obtain expanded vocabulary; The fragment content retrieval module 103 is used to retrieve the first code fragment and the second code fragment corresponding to each expanded term from the preset source code file according to a variety of preset retrieval types. The weight value calculation module 104 is used to aggregate the first code fragment and the second code fragment into the search results of the source code file according to the search type, and calculate the weight value of the search results of each different search type in the search results of the source code file. The retrieval result merging module 105 is used to merge the retrieval results of each different retrieval type according to the weight value to obtain the target code retrieval result.

[0070] In one embodiment, the query intent extraction module 101, when extracting the query intent of the code query text, is used to: Identify the programming language name, operation action, and target object in the code query text; The text syntax of the code query text is identified based on the programming language name, the operation action, and the target object; The text syntax is mapped to a dimension mapping vector according to a preset dimension, and the programming language name, the operation action and the target object are converted into encoding vectors respectively. The dimension mapping vector and the converted encoding vector are then fused into a feature vector. Each intent in the preset code query intent library is converted into an intent vector; Calculate the similarity between the feature vector and the intent vector; The intent vectors with similarity greater than a preset similarity threshold are aggregated into a query intent vector, and the query intent corresponding to the query intent vector is identified.

[0071] In one embodiment, when the retrieval term extraction and vocabulary expansion module 102 extracts retrieval terms from the code query text according to the query intent, it is used to: The search term suggestion text is constructed based on the preset search suggestion term template and the query intent; The code query text is retrieved based on the search term hint text to obtain the initial search terms; The initial search terms are deduplicated to obtain the search terms.

[0072] In one embodiment, when the fragment content retrieval module 103 retrieves the first code fragment and the second code fragment corresponding to each expanded term from a preset source code file according to multiple preset retrieval types, it is used to: Identify the syntax tree of the preset source code file, divide the source code file into multiple file blocks according to the syntax tree, and search each of the expanded terms in the source code file according to multiple preset search types to obtain the file blocks corresponding to each expanded term under different search types. Use the code in the file blocks as the first code segment corresponding to each expanded term. The expanded vocabulary is converted into code vocabulary corresponding to the code in the source code file; Identify the location information of the code terms in the source code file; Based on the location information, the preset range context information corresponding to the code words is extracted from the source code file as a second code segment.

[0073] In one embodiment, when the fragment content retrieval module 103 retrieves the source code file according to the expanded vocabulary and the syntax tree, and determines the first code fragment corresponding to the expanded vocabulary and the retrieval type, it is used to: Identify the matching type of the expanded vocabulary, compare the expanded vocabulary and the nodes in the syntax tree dimension by dimension according to the matching type, and gather the nodes that match the matching type into candidate nodes based on the comparison results; Identify the grammatical structure boundaries of the syntax tree; Based on the grammatical structure boundary, the candidate nodes are deduplicated to generate the first code segment boundary; Based on the boundaries of the first code fragment, extract multiple expanded terms and first code fragments corresponding to the search type from the source code file.

[0074] In one embodiment, the weight value calculation module 104, when calculating the weight value of the search results for each different search type within the search results of the source code file, is used to: Select any one of the standard search terms from the expanded vocabulary as the target search term; Identify the number of times the target search term appears in the source code file, and identify the total number of words in the source code file; Calculate the ratio of the occurrence frequency to the total number of words, and use the ratio as the importance weight of the target search term; The importance weights of all the target search terms are concatenated into a search term vector, and the importance weights corresponding to all the target search terms in the first code segment are concatenated into a first weight vector; Calculate the first similarity between the search word vector and the first weight vector, and use the first similarity as the first preliminary weight value of the first code segment; Concatenate the importance weights corresponding to all target search terms in the second code snippet into a second weight vector; Calculate the second similarity between the search word vector and the second weight vector, and use the second similarity as the second preliminary weight value of the second code segment; The first preliminary weight value is linearly calculated by performing a linear operation with the preset weight of the first code segment to obtain the weight value of the first code segment; The second preliminary weight value is linearly calculated by combining it with the preset weight of the second code segment to obtain the weight value of the second code segment.

[0075] In one embodiment, the retrieval result merging module 105, when merging the retrieval results of each different retrieval type according to the weight value to obtain the target code retrieval result, is used to: Select any one of the first code snippets as the target code snippet; Calculate the code similarity between the target code segment and each of the other code segments within the first code segment; The code segments with a similarity less than a preset code similarity threshold are grouped into a result of segments to be retained; According to a preset priority rule, one of the results of the segments to be retained is retained, and the retained result and the segments whose code similarity is greater than a preset code similarity threshold are used together as the retained code segment results; Based on the weight value corresponding to the retained code fragment result and the weight value corresponding to the second code fragment, the search results of each different search type are sorted in descending order to obtain the sorted results; The sorted results are used as the target code retrieval results.

[0076] This invention provides a code retrieval device for a business that aims to build target code retrieval results. It acquires code query text sent by a target user and extracts the query intent from the query text. Based on the query intent, it extracts search terms from the code query text and expands these terms using preset vocabulary expansion rules to obtain expanded vocabulary. It then retrieves a first code segment and a second code segment corresponding to each expanded vocabulary from a preset source code file according to multiple preset search types. The first and second code segments are then aggregated into retrieval results for the source code file according to the search types. The weight value of each different search type's retrieval results within the source code file's retrieval results is calculated. Finally, the retrieval results for each different search type are merged according to the weight value to obtain the target code retrieval results. By merging retrieval results by weight, high-weight core results are prioritized, ensuring both the accuracy and comprehensiveness of the retrieval results.

[0077] For specific limitations regarding a code retrieval device, please refer to the limitations regarding a code retrieval method above, which will not be repeated here. Each module in the aforementioned code retrieval device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device in hardware form, or stored in the memory of a computer device in software form, so that the processor can call and execute the operations corresponding to each module.

[0078] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 4 As shown, the computer device includes a processor, memory, network interface, and database connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile and / or volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and database. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The network interface is used to communicate with external clients via a network connection. When the computer program is executed by the processor, it implements the functions or steps of a code retrieval method on the server side.

[0079] In one embodiment, a computer device is provided, which may be a client, and its internal structure diagram may be as follows: Figure 5 As shown, the computer device includes a processor, memory, network interface, display screen, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The network interface is used to communicate with an external server via a network connection. When the computer program is executed by the processor, it implements the functions or steps of a code retrieval method on the client side.

[0080] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to perform the following steps: Obtain the code query text sent by the target user, and extract the query intent of the code query text; Extract the search terms from the code query text according to the query intent, and expand the search terms according to the preset vocabulary expansion rules to obtain expanded vocabulary; According to various preset search types, the first and second code segments corresponding to each expanded term are retrieved from the preset source code files. The first code fragment and the second code fragment are aggregated into the search results of the source code file according to the search type, and the weight value of the search results of each different search type in the search results of the source code file is calculated. The search results for each different search type are merged according to the weight value to obtain the target code search results.

[0081] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, the computer program performing the following steps when executed by a processor: Obtain the code query text sent by the target user, and extract the query intent of the code query text; Extract the search terms from the code query text according to the query intent, and expand the search terms according to the preset vocabulary expansion rules to obtain expanded vocabulary; According to various preset search types, the first and second code segments corresponding to each expanded term are retrieved from the preset source code files. The first code fragment and the second code fragment are aggregated into the search results of the source code file according to the search type, and the weight value of the search results of each different search type in the search results of the source code file is calculated. The search results for each different search type are merged according to the weight value to obtain the target code search results.

[0082] It should be noted that the functions or steps that can be implemented by the computer-readable storage medium or computer device described above can be referred to the relevant descriptions on the server side and client side in the foregoing method embodiments. To avoid repetition, they will not be described one by one here.

[0083] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0084] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is used as an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0085] The above-described embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit them. If any software tools or components other than those of our company appear in the embodiments, they are merely illustrative examples and do not represent actual use. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included within the protection scope of the present invention.< / user>

Claims

1. A code retrieval method, characterized in that, include: Obtain the code query text sent by the target user, and extract the query intent of the code query text; Based on the query intent, the search terms in the code query text are extracted, and the search terms are expanded using preset vocabulary expansion rules to obtain expanded vocabulary; According to various preset search types, the first and second code segments corresponding to each expanded term are retrieved from the preset source code files. The first code fragment and the second code fragment are aggregated into the search results of the source code file according to the search type, and the weight value of the search results of each different search type in the search results of the source code file is calculated. The search results for each different search type are merged according to the weight value to obtain the target code search results.

2. The code retrieval method as described in claim 1, characterized in that, The extraction of the query intent from the code query text includes: Identify the programming language name, operation action, and target object in the code query text; The text syntax of the code query text is identified based on the programming language name, the operation action, and the target object; The text syntax is mapped to a dimension mapping vector according to a preset dimension, and the programming language name, the operation action and the target object are converted into encoding vectors respectively. The dimension mapping vector and the converted encoding vector are then fused into a feature vector. Each intent in the preset code query intent library is converted into an intent vector; Calculate the similarity between the feature vector and the intent vector; The intent vectors with similarity greater than a preset similarity threshold are aggregated into a query intent vector, and the query intent corresponding to the query intent vector is identified.

3. The code retrieval method as described in claim 1, characterized in that, The step of extracting search terms from the code query text based on the query intent includes: The search term suggestion text is constructed based on the preset search suggestion term template and the query intent; The code query text is retrieved based on the search term hint text to obtain the initial search terms; The initial search terms are deduplicated to obtain the search terms.

4. The code retrieval method as described in claim 1, characterized in that, The step of retrieving the first and second code segments corresponding to each expanded term from a preset source code file according to multiple preset retrieval types includes: Identify the syntax tree of the preset source code file, divide the source code file into multiple file blocks according to the syntax tree, and search each of the expanded terms in the source code file according to multiple preset search types to obtain the file blocks corresponding to each expanded term under different search types. Use the code in the file blocks as the first code segment corresponding to each expanded term. The expanded vocabulary is converted into code vocabulary corresponding to the code in the source code file; Identify the location information of the code terms in the source code file; Based on the location information, the preset range context information corresponding to the code words is extracted from the source code file as a second code segment.

5. The code retrieval method as described in claim 4, characterized in that, The step of searching the source code file according to the expanded vocabulary and the syntax tree to determine the first code segment corresponding to the expanded vocabulary and the search type includes: Identify the matching type of the expanded vocabulary, compare the expanded vocabulary and the nodes in the syntax tree dimension by dimension according to the matching type, and gather the nodes that match the matching type into candidate nodes based on the comparison results; Identify the grammatical structure boundaries of the syntax tree; Based on the grammatical structure boundary, the candidate nodes are deduplicated to generate the first code segment boundary; Based on the boundaries of the first code fragment, extract multiple expanded terms and first code fragments corresponding to the search type from the source code file.

6. The code retrieval method as described in claim 1, characterized in that, The calculation of the weight value of the search results for each different search type within the search results of the source code file includes: Select any one of the standard search terms from the expanded vocabulary as the target search term; Identify the number of times the target search term appears in the source code file, and identify the total number of words in the source code file; Calculate the ratio of the occurrence frequency to the total number of words, and use the ratio as the importance weight of the target search term; The importance weights of all the target search terms are concatenated into a search term vector, and the importance weights corresponding to all the target search terms in the first code segment are concatenated into a first weight vector; Calculate the first similarity between the search word vector and the first weight vector, and use the first similarity as the first preliminary weight value of the first code segment; Concatenate the importance weights corresponding to all target search terms in the second code snippet into a second weight vector; Calculate the second similarity between the search word vector and the second weight vector, and use the second similarity as the second preliminary weight value of the second code segment; The first preliminary weight value is linearly calculated by performing a linear operation with the preset weight of the first code segment to obtain the weight value of the first code segment; The second preliminary weight value is linearly calculated by combining it with the preset weight of the second code segment to obtain the weight value of the second code segment.

7. The code retrieval method as described in claim 1, characterized in that, The step of merging the search results for each different search type according to the weight value to obtain the target code search results includes: Select any one of the first code snippets as the target code snippet; Calculate the code similarity between the target code segment and each of the other code segments within the first code segment; The code segments with a similarity less than a preset code similarity threshold are grouped into a result of segments to be retained; According to a preset priority rule, one of the results of the segments to be retained is retained, and the retained result and the segments whose code similarity is greater than a preset code similarity threshold are used together as the retained code segment results; Based on the weight value corresponding to the retained code fragment result and the weight value corresponding to the second code fragment, the search results of each different search type are sorted in descending order to obtain the sorted results; The sorted results are used as the target code retrieval results.

8. A code retrieval device, characterized in that, include: The query intent extraction module is used to obtain the code query text sent by the target user and extract the query intent of the code query text; The retrieval term extraction and vocabulary expansion module is used to extract retrieval terms from the code query text according to the query intent, and expand the retrieval terms according to preset vocabulary expansion rules to obtain expanded vocabulary; The fragment content retrieval module is used to retrieve the first and second code fragments corresponding to each expanded term from the preset source code files according to various preset retrieval types. The weight value calculation module is used to aggregate the first code fragment and the second code fragment into the search results of the source code file according to the search type, and calculate the weight value of the search results of each different search type in the search results of the source code file. The search result merging module is used to merge the search results of each different search type according to the weight value to obtain the target code search results.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the code retrieval method as described in any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the code retrieval method as described in any one of claims 1 to 7.

Images