A large model-based code base question and answer method and device

Abstract

The application discloses a code base question and answer method and device based on a large model, and relates to the technical field of software engineering. The method comprises the following steps: performing word segmentation on code files to obtain word units, counting the number of word units in each code file, initializing the directory depth of the code files, determining the hierarchical relationship of the code files, generating a code file set, inputting the code file set into a large language model to generate a natural language description set, inputting the code file set and the natural language description set into the large language model to obtain code explanation, segmenting the code explanation and converting the code explanation into semantic vectors, converting a user's query statement into a query vector, matching the query vector with the semantic vectors to obtain a matching result, and feeding back the matching result to a user interface after integration. The method solves the problems of low accuracy and low efficiency of the existing code base retrieval method, realizes a more intelligent code retrieval method, and improves the accuracy of the large language model question and answer.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of software engineering, and in particular to a code base question and answer method and device based on a large model. BACKGROUND

[0002] With the continuous development of artificial intelligence, more and more fields of question and answer retrieval systems have joined large language models to achieve more efficient and intelligent retrieval to meet the diverse needs of customers.

[0003] However, there are still limitations when retrieving and asking questions in a large and complex code base. Traditional code base retrieval often relies on keyword matching, pattern matching, manual positioning and other methods. These methods lack context understanding and cross-file semantic relationship analysis, resulting in inaccurate retrieval results.

[0004] In addition, the input / output length of the large language model is limited, and the existing technology usually truncates or completes when facing a large number of files in the code base, which causes certain data loss or data redundancy. Moreover, there is a difference between the understanding of natural language and the understanding of code by the large language model. When generating answers, it is difficult to accurately complete the generation of answers based on the existing knowledge base, and manual intervention is required. Therefore, the existing code base retrieval and question and answer method cannot meet the needs of users for efficient question and answer in the code base. SUMMARY

[0005] The embodiments of the present application provide a code base question and answer method and device based on a large model, which solves the problem of low accuracy and efficiency of the existing code base retrieval method. The functionality and technology stack of each code set are described using natural language processing methods, and the code content is reasonably segmented according to the segmentation strategy. The intelligent agent is designed to further retrieve the problems that are not matched by the large language model, improving the retrieval efficiency and accuracy of the code.

[0006] In a first aspect, the embodiments of the present application provide a code base question and answer method based on a large model, comprising: tokenizing code files in a code base to obtain word units, and counting the number of word units in each code file; initializing the directory depth of the code file according to the number of word units, determining the hierarchical relationship of the code file according to the directory depth, and generating a code file set; inputting the code file set into a large language model to generate a natural language description set; inputting the code file set and the natural language description set into the large language model to obtain code interpretation; segmenting the code interpretation and converting it into a semantic vector, and storing it in a vector database; converting a user's query statement into a query vector, and performing matching search on the query vector and the semantic vector in the vector database to obtain a matching result; integrating the matching result and feeding it back to the user interface.

[0007] In one possible implementation, the step of segmenting code files in the code library to obtain word units includes: traversing the code files; performing word segmentation on the Chinese and English text in the code files respectively to obtain segmentation units, and assigning IDs to the segmentation units; establishing a vocabulary based on the IDs of the segmentation units, and obtaining the word units using a data compression algorithm; and obtaining the IDs of the word units based on the vocabulary and mapping rules.

[0008] In one possible implementation, initializing the directory depth of the code file based on the number of word units includes: calculating the directory depth of each file in the code file based on the file path, and arranging them to determine the processing order; wherein, if the directory depths are the same, the files are processed in ascending order based on the number of word units in each code file.

[0009] In one possible implementation, after initializing the directory depth of the code file according to the number of word units, the step includes: truncating the word units to a predefined maximum word unit limit according to the number of word units.

[0010] In one possible implementation, segmenting the code interpretation includes: setting a segment length; segmenting the code interpretation according to the segment length to obtain code segments, and storing the content of the current code segment of a preset length into the next code segment.

[0011] In one possible implementation, the conversion to semantic vectors and storage in a vector database includes: vectorizing code segments to obtain the semantic vectors; and creating an index based on the semantic vectors and storing it in the vector database.

[0012] In one possible implementation, the step of matching the query vector with the semantic vector in the vector database to obtain matching results includes: the matching results include direct matching results and interactive matching results; the direct matching results are obtained by directly deriving the code explanations of all query statements in the vector database based on the large language model; the interactive matching results are obtained by using an agent to search in the vector database and matching based on the vector similarity between the searched semantic vector and the query vector.

[0013] In a second aspect, the embodiments of the present application provide a code base question and answer device based on a large model, comprising: a data processing module, configured to tokenize code files in a code base to obtain word units, and count the number of word units in each code file; initialize the directory depth of the code file according to the number of word units, determine the hierarchical relationship of the code file according to the directory depth, and generate a code file set; input the code file set into a large language model to generate a natural language description set; input the code file set and the natural language description set into the large language model to obtain code interpretation; a vectorization processing module, configured to segment and convert the code interpretation into semantic vectors, and store the semantic vectors in a vector database; a matching module, configured to convert a query statement of a user into a query vector, and perform matching search on the query vector and the semantic vectors in the vector database to obtain a matching result; and a feedback module, configured to integrate the matching result, and feed back to a user interface a code base question and answer device based on a large model, comprising: a data processing module, configured to tokenize code files in a code base to obtain word units, and count the number of word units in each code file; initialize the directory depth of the code file according to the number of word units, determine the hierarchical relationship of the code file according to the directory depth, and generate a code file set; input the code file set into a large language model to generate a natural language description set; input the code file set and the natural language description set into the large language model to obtain code interpretation; a vectorization processing module, configured to segment and convert the code interpretation into semantic vectors, and store the semantic vectors in a vector database; a matching module, configured to convert a query statement of a user into a query vector, and perform matching search on the query vector and the semantic vectors in the vector database to obtain a matching result; and a feedback module, configured to integrate the matching result, and feed back to a user interface.

[0014] In a third aspect, the embodiments of the present application provide a device for a code base question and answer method based on a large model, the device comprising: a processor; a memory for storing processor executable instructions; and the processor executes the executable instructions to implement the method as described in the first aspect or any possible implementation manner of the first aspect.

[0015] In a fourth aspect, the embodiments of the present application provide a non-volatile computer readable storage medium, comprising a computer program or instructions for storing, when the computer program or instructions are executed, causing the method as described in the first aspect or any possible implementation manner of the first aspect to be implemented.

[0016] The one or more technical solutions provided in the embodiments of the present application have at least the following technical effects or advantages:

[0017] The embodiment of the application can help to capture key elements in the code by disassembling the code file into smaller word units, thereby more accurately reflecting the function and semantics of the code; by initializing the directory depth of the code file, the logical association and hierarchical structure between the codes can be clearly shown; by inputting the code file set and the natural language description set into the large language model, the original information of the code and the auxiliary description of the natural language can be fully utilized, so that the large language model can more comprehensively and accurately understand the function and intention of the code, thereby generating more accurate and detailed code explanation; through the vector database, efficient query can be realized, and then the matching result can be quickly returned, thereby improving the user experience. The problem of low accuracy and efficiency of the existing code library retrieval method is solved, thereby improving the understanding ability of the large language model for the code file without human intervention, thereby minimizing the concept understanding error in the code library, and realizing accurate and efficient answering to the user query statement, so that the user can obtain the required code explanation information without long waiting. BRIEF DESCRIPTION OF DRAWINGS

[0018] In order to more clearly illustrate the technical solutions of the embodiments of the application, the following will briefly introduce the drawings needed to be used in the embodiments of the application or the prior art description. Obviously, the drawings in the following description are some embodiments of the application, and those skilled in the art can obtain other drawings according to these drawings without creative labor.

[0019] Figure 1 A flowchart of a code library question and answer method based on a large model provided by the embodiment of the application;

[0020] Figure 2 A flowchart of a method for searching and matching results provided by the embodiment of the application;

[0021] Figure 3 A structural schematic diagram of a code library question and answer device based on a large model provided by the embodiment of the application. DETAILED DESCRIPTION

[0022] The technical solutions in the embodiments of the application will be described clearly and completely below with reference to the drawings in the embodiments of the application. Obviously, the described embodiments are part of the embodiments of the application, not all the embodiments. Based on the embodiments in the application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the application.

[0023] The following description of the technology involved in the embodiments of the present application is provided to facilitate understanding, and should be considered merely as exemplary. Therefore, those of ordinary skill in the art should recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present application. Also, for the sake of clarity and conciseness, the following description omits the description of some well-known functions and structures.

[0024] Figure 1 is a flowchart of a codebase question-answering method based on a large model provided by an embodiment of the present application, comprising steps 101 to 106. Figure 1 The execution order shown in the embodiments of the present application is only one execution order, and is not the only execution order of the codebase question-answering method based on a large model. As long as the final result can be achieved, Figure 1 The steps shown can be executed in parallel or in reverse order, as follows.

[0025] Step 101: Tokenize the code files in the codebase to obtain word units, and count the number of word units in each code file. In the embodiments of the present application, the code files are traversed. The Chinese and English texts in the code files are processed for tokenization to obtain segmented units, and the segmented units are assigned IDs. A vocabulary table is established according to the IDs of the segmented units, and a data compression algorithm is used to obtain word units. According to the vocabulary table and the mapping rules, the IDs of the word units are obtained.

[0026] Specifically, the tokenization process uses regularization to uniformly process the English text, converting it to lowercase format and standardizing the spaces and punctuation marks therein. Based on the spaces after standardization, the text is segmented to obtain word-level segmented units.

[0027] A Chinese tokenization algorithm is used, for example, HanLP (a natural language processing toolkit) is used to tokenize Chinese text to identify basic segmented units. HanLP provides interfaces for multiple programming languages such as Python and C++, and has wide language applicability. For new words (words not in the vocabulary), an HMM (Hidden Markov Model) with Chinese character word formation capability is used for segmentation. The obtained segmented units are mapped to a pre-set unique ID according to the mapping rules.

[0028] According to the vocabulary table, the word units are mapped to unique corresponding IDs. The segmented units are iteratively merged using a data compression algorithm until a pre-set upper limit of the number of mergers (set to 10 times) and / or there are no mergable word units.

[0029] Exemplarily, the data compression algorithm uses BPE (Byte Pair Encoding algorithm), and in each iteration, the pair of segmentation units with the highest frequency and adjacent position in the current segmentation unit sequence is identified and merged to convert into a new word unit. The word or character sequence in the text is converted into a more compact representation while retaining sufficient information for subsequent training and processing of the large language model.

[0030] Using the data compression algorithm, the vocabulary is constructed by merging frequently and continuously appearing word unit sequences, and the corresponding ID of the word unit is obtained according to the mapping rule of the vocabulary, so as to convert the user input text into a numerical input that can be understood by the large language model. The number of word units is counted for subsequent processing of the word units in the training process of the large language model. This step can enhance the processing capability of the large language model for rare words or foreign words, and improve the ability of the large language model to process complex text.

[0031] Step 102: Initialize the directory depth of the code file according to the number of word units, determine the hierarchical relationship of the code file according to the directory depth, and generate a code file set. The code file set is input into the large language model to generate a natural language description set.

[0032] In the embodiment of the present application, the directory depth of each file in the code file is calculated according to the file path, and the processing order is determined by arranging it. If the directory depth is the same, the code files are arranged in ascending order according to the number of word units in each code file.

[0033] Specifically, based on the counting and statistics of the number of word units in each code file, the code file processing is performed. The directory depth of each code file in the code library is calculated according to the file path, and is arranged in descending order according to the directory depth, so as to ensure that the code files in the deepest level can be processed first.

[0034] In addition, the word units can also be truncated to a predefined maximum word unit limit according to the number of word units.

[0035] Specifically, if there are multiple code files with the same directory depth in the sorting process, the code files are arranged in ascending order according to the number of word units in each code file to obtain a file set list under the directory depth. The directory depth can be understood as the number of directory separators in the file path.

[0036] Specifically, the file set list after sorting is traversed, and if the number of word units exceeds the predefined max_token (maximum word unit limit), the word units are truncated to ensure that each code file contains at most max_token number of word units.

[0037] According to the directory depth, the hierarchical relationship of the parent file set to which each to-be-processed code file belongs is determined. The parent file set represents a set of files located above the current code file in the file hierarchy and having a smaller directory depth. If a parent file set meeting the condition is found, and the sum of the token counts of the parent file set and the current code file (denoted as a subset file) does not exceed the max_token limit, the token units of the subset file are merged into the parent file set. According to the merging rule, a code file set is generated, wherein the merging rule includes the condition that the code set obtained after each merging of the parent file set and the subset file complies with the max_token condition, so as to simplify the data volume and provide a good foundation for subsequent data processing or storage.

[0038] Exemplarily, the code file set is intelligently summarized and analyzed by using a large language model such as ChatGPT or Qwen to generate a natural language description set. During the processing of the code file set, all code files are traversed, and the semantic analysis of the code in the code files is performed by using a natural language processing method to obtain the core theme and function of the code files. The large language model reads the context content of the code files according to the natural language description set to extract the code content. It is ensured that the large language model can understand the key classes, functions or related code components in the code files, and how the code files interact with each other. The technology stack elements used by each code file are identified, such as routing logic, business logic and data access. Meanwhile, the context relationship and purpose of the code files can be determined through the file name, and the specific position of the current code file relative to the code files in the entire code library can be located through the file name for subsequent path acquisition. The functions implemented by the code file set are described by natural language to generate a natural language description set. The natural language description set includes how the specific functions of the code in the code set are implemented and what impact the set has on the entire project or code library.

[0039] Step 103: inputting the code file set and the natural language description set into the large language model to obtain code explanation.

[0040] Specifically, the natural language description set enables the large language model to better understand the specific functions or roles of each code file. Further, the technology stack elements applied by each code file set, such as routing logic, business logic and data access, are identified.

[0041] The code file set and the natural language description set are input into a large language model to obtain a processed code explanation. The code explanation can completely express the function and implementation of each function or method. For example, the function of the int_to_string function is described as "convert int type data to string type", instead of using vague words such as "this function".

[0042] The code explanation enables the large language model to efficiently and accurately understand the overall architecture of the project, thereby better using and managing the code library.

[0043] Step 104: Segment the code explanation and convert it into a semantic vector and store it in a vector database. In the embodiment of the present application, the segment length is set. The code explanation is segmented according to the segment length to obtain a code paragraph, and the content of the preset length of the current code paragraph is stored in the next code paragraph. The code paragraph is vectorized to obtain a semantic vector. The semantic vector is indexed and stored in the vector database.

[0044] Specifically, the segmentation processing includes setting the segment length to 200 words. In the embodiment of the present application, each code paragraph contains 10% redundant content at the end of the previous code paragraph, which can effectively promote the large language model to understand the context and will not excessively increase the data burden.

[0045] According to the BCEmbedding (Bilingual and Cross-lingual Semantic Representation Algorithm Model Library), the code paragraph is vectorized to obtain a semantic vector. By using the Embedding Model, the text content of the code explanation can be converted into an accurate and high-dimensional semantic vector, as follows:

[0046] .

[0047] In the formula, represents the semantic vector of the text content of the current th code paragraph, represents the encoding function of the Embedding Model, represents the text of the th code paragraph.

[0048] The semantic index is established, and the vectorized semantic vector is saved to the Milvus (an open source vector database) vector database, so as to more quickly perform subsequent search and retrieval tasks.

[0049] Specifically, the index includes a primary key index and a vector index. The primary key index is a unique ID identifier assigned to each semantic vector. The vector index is established using IVF FLAT (a vector similarity algorithm based on divide-and-conquer method), such as a 768-dimensional semantic vector value, [0.1, 0.2,..., 0.768].

[0050] The vector database further includes attributes of the semantic vectors, such as a file name, a file path, and an original string corresponding to the semantic vector (i.e., a code paragraph text).

[0051] Step 105: converting the user's query statement into a query vector, and performing a matching search on the query vector and the semantic vectors in the vector database to obtain a matching result. In the embodiments of the present application, the matching result includes a direct matching result and an interactive matching result. The direct matching result is obtained according to the large language model and is the code explanation corresponding to the query statement in the vector database. The interactive matching result is obtained according to the vector similarity between the semantic vectors searched in the vector database and the query vector.

[0052] Specifically, if the answer to all query statements is obtained according to the large language model, the direct matching result is obtained. If the direct matching result cannot be obtained according to the query statement using the large language model, a matching search is performed in the vector database, including: converting the user's query statement into a query vector, calling an Agent (intelligent agent) to perform a vector search in the vector database, matching according to a similarity calculation method, and outputting the answer result of the Agent. According to the answer result of the Agent, the code file corresponding to the path of the semantic vector is determined, and the interactive matching result is obtained.

[0053] The similarity calculation method is specifically as follows:

[0054] .

[0055] In the formula, denotes the cosine similarity, denotes the Euclidean norm of the vector, denotes the query vector, denotes the i-th semantic vector in the vector database.

[0056] The cosine similarity has a value range of [-1, 1]. If two query vectors and a semantic vector are exactly the same, the cosine similarity is 1. If they are completely opposite, the cosine similarity is -1. If they are perpendicular to each other, indicating that there is no correlation between them, the cosine similarity is 0.

[0057] Exemplarily, when the user input query is sent to the large language model, the large language model processes it into a query vector, analyzes the problem, and understands its meaning. Exemplarily, the user input query is: "How to implement string to integer conversion in code?", if the answer can be directly found by the large language model, "Answer" is used as a prefix instruction, and the direct matching result is output, that is, "Answer: In the code library, the int() function is usually used to convert a string to an integer."

[0058] As shown in Figure 2 If the large language model cannot directly answer all the questions input by the user and needs to obtain more information to form an answer, the agent will be called to search for interactive matching results in the code library file, including: matching search on semantic vectors until interactive matching results are output. If the matching search reaches the preset number of cycles (for example, 10 times), and no corresponding results are found within the preset number of cycles, it is returned that no corresponding results are found.

[0059] Exemplarily, the large language model will use "Action" as a prefix instruction to output the function and parameters called by the agent. "Action: retrieve_code(‘How to implement string to integer conversion in code?’)”. After receiving the instruction, the agent will perform a search operation in the code library, and return the most relevant semantic vector, i.e. file and function, according to the similarity.

[0060] Specifically, a file path and function definition corresponding to the returned semantic vector are returned, and the get_code_by_path (a function method for returning code paths) method can retrieve and return the content of the corresponding file. Then read the file content according to the absolute path. This allows the user to directly obtain the content of a single file without traversing the entire directory, such as " / path / to / code.py".

[0061] Further, if the entire vector database needs to be obtained every time a query is made, the number of word units may quickly increase. Through the get_function_define (a function obtaining scheme) method, the agent can obtain the definition of the function according to the file name and function name, reducing the additional computational load and avoiding increasing the search time.

[0062] Step 106: Integrate the matching results and feed back to the user interface. Specifically, after receiving the information returned by the agent, the large language model integrates it into a more specific and easier-to-understand answer for the user, i.e. interactive matching results, such as "Answer: In the file / path / to / code.py, an int() function is defined, which converts a string to an integer."

[0063] Although the present application provides method operation steps as described in the embodiments or flowcharts, more or less operation steps can be included based on routine or non-creative labor. The order of steps listed in the embodiments is only one of the many step execution orders, and does not represent the only execution order. In actual device or client product execution, the method order shown in the embodiments or the drawings can be executed in sequence or in parallel (for example, in a parallel processor or multi-thread processing environment).

[0064] As shown in Figure 3 The embodiments of the present application also provide a code base question and answer device 300 based on a large model. The device comprises a data processing module 301, a vectorization processing module 302, a matching module 303 and a feedback module 304, specifically as follows.

[0065] The data processing module 301 is used to divide the code files in the code base into word units, and count the number of word units in each code file. The directory depth of the code file is initialized according to the number of word units, the hierarchical relationship of the code file is determined according to the directory depth, and a code file set is generated. The code file set is input into a large language model to generate a natural language description set. The code file set and the natural language description set are input into the large language model to obtain code explanation.

[0066] The vectorization processing module 302 is used to segment and convert the code explanation into semantic vectors, and store them in a vector database.

[0067] The matching module 303 is used to convert the user's query statement into a query vector, and search for a matching result by matching the query vector with the semantic vector in the vector database.

[0068] The feedback module 304 is used to integrate the matching result and feed it back to the user interface.

[0069] Some of the modules in the device described in the present application can be described in the general context of computer-executable instructions, such as program modules, executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, classes, etc. that perform particular tasks or implement particular abstract data types. The present application can also be practiced in a distributed computing environment, in which tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media, including storage devices.

[0070] The apparatuses or modules illustrated in the above application examples can be implemented by computer chips or entities, or by products with certain functions. For the convenience of description, the above apparatuses are described as various modules with functions. In the implementation of the application examples, the functions of the modules can be implemented in one or more software and / or hardware. Of course, the modules with certain functions can also be implemented by a combination of multiple sub-modules or sub-units.

[0071] The methods, apparatuses or modules described in the present application can be implemented in a computer readable program code in any appropriate manner, for example, the controller can take the form of, for example, a microprocessor or a processor and a computer readable medium storing computer readable program code (such as software or firmware) executable by the (micro) processor, logic gates, switches, application specific integrated circuits (Application Specific Integrated Circuit; abbreviated as: ASIC), programmable logic controllers and embedded microcontrollers, examples of the controller include but are not limited to the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20 and Silicone Labs C8051F320, the memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art also know that in addition to implementing the controller in a pure computer readable program code, the same function can also be implemented by logically programming the method steps in the form of logic gates, switches, application specific integrated circuits, programmable logic controllers and embedded microcontrollers. Therefore, such a controller can be considered as a hardware component, and the devices included therein for implementing various functions can also be considered as structures within the hardware component. Alternatively, the devices for implementing various functions can also be considered as both software modules implementing the method and structures within the hardware component.

[0072] The embodiments of the present application also provide a device for executing the large model-based code base question and answer method, the device comprising: a processor; a memory for storing processor executable instructions; and the processor executes the executable instructions to implement the method as described in the embodiments of the present application.

[0073] The embodiments of the present application also provide a non-volatile computer readable storage medium having a computer program or instructions stored thereon, when the computer program or instructions are executed, the method as described in the embodiments of the present application is implemented.

[0074] In addition, the functional modules in each of the embodiments of the present application can be integrated in one processing module, or each module can exist independently, or two or more modules can be integrated in one module.

[0075] The storage medium includes, but is not limited to, a random access memory (RAM), a read-only memory (ROM), a cache, a hard disk drive (HDD), or a memory card. The storage medium can be used to store computer program instructions.

[0076] From the above description of the embodiments, those skilled in the art can clearly understand that the present application can be implemented by means of software and the necessary hardware. Based on such an understanding, the technical solutions of the present application can be embodied in the form of a software product or can be embodied in the form of data migration during implementation. The computer software product can be stored in a storage medium, such as a ROM / RAM, a magnetic disk, or an optical disk, and includes a number of instructions for causing a computer device (which can be a personal computer, a mobile terminal, a server, or a network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0077] The various embodiments in the specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on the differences from other embodiments. The whole or part of the present application can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld devices or portable devices, tablet devices, mobile communication terminals, multi-processor systems, microprocessor-based systems, programmable electronic devices, network PCs, small computers, large computers, distributed computing environments including any of the above systems or devices, and the like.

[0078] The above embodiments are only used to illustrate the technical solutions of the present application, and not to limit the present application; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacements to some or all of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the present application.

Claims

1. A codebase question-answering method based on a large model, characterized in that, include: The code files in the code library are segmented into word units, and the number of word units in each code file is counted. The directory depth of the code file is initialized based on the number of word units, the hierarchical relationship of the code file is determined based on the directory depth, and a set of code files is generated. The set of code files is then input into a large language model to generate a set of natural language descriptions. The hierarchy of the parent file set to which each code file to be processed belongs is determined based on the directory depth. The parent file set represents a set of files above the current code file in the file hierarchy with a smaller directory depth. If a parent file set that meets the conditions is found, and the sum of the word unit counts of the parent file set and the current code file does not exceed the maximum word unit limit `max_token`, then the word units of the current code file are merged and input into the parent file set. A code file set is generated according to the merging rules, which include ensuring that the code set obtained after merging the parent file set and the current code file always adheres to the maximum word unit limit `max_token`. The code file set and the natural language description set are input into the large language model to obtain code interpretation; The code is interpreted, segmented, and converted into semantic vectors, which are then stored in a vector database. The user's query statement is converted into a query vector, and the query vector is matched against the semantic vector in the vector database to obtain matching results. These matching results include direct matching results and interactive matching results. If the large language model yields answers to all query statements, a direct matching result is obtained. If the large language model cannot yield direct matching results based on the query statement, a matching search is performed in the vector database, including: converting the user's query statement into a query vector and invoking the agent; performing a vectorized search in the vector database, matching according to a similarity calculation method, and outputting the agent's answer; and determining the code file under the path corresponding to the semantic vector based on the agent's answer to obtain interactive matching results. The matching results are integrated and then fed back to the user interface.

2. The method according to claim 1, characterized in that, The step of segmenting code files in the code library to obtain word units includes: Traverse the code file; The Chinese and English texts in the code file are segmented into word segments to obtain segmentation units, and IDs are assigned to the segmentation units. A vocabulary is built based on the ID of the segmentation unit, and the word unit is obtained using a data compression algorithm; The ID of the word unit is obtained based on the vocabulary and mapping rules.

3. The method according to claim 1, characterized in that, The process of initializing the directory depth of the code file based on the number of word units includes: The directory depth of each file in the code file is calculated based on the file path, and they are arranged to determine the processing order; wherein, if the directory depths are the same, they are processed in ascending order based on the number of word units in each code file.

4. The method according to claim 1, characterized in that, After initializing the directory depth of the code file according to the number of word units, the process includes: The word units are truncated to a predefined maximum word unit limit based on the number of word units.

5. The method according to claim 1, characterized in that, The segmentation of the code interpretation includes: Set the segment length; The code interpretation is segmented according to the segment length to obtain code segments, and the content of the current code segment with a preset length is stored in the next code segment.

6. The method according to claim 5, characterized in that, The conversion into semantic vectors, stored in a vector database, includes: The code segment is vectorized to obtain the semantic vector; An index is created based on the semantic vector and stored in a vector database.

7. A codebase question-answering device based on a large model, characterized in that, include: The data processing module is used to segment code files in the code library to obtain word units and count the number of word units in each code file; The directory depth of the code file is initialized based on the number of word units. The hierarchical relationship of the code file is determined based on the directory depth, and a code file set is generated. This code file set is then input into a large language model to generate a natural language description set. The hierarchical relationship of the parent file set to which each code file to be processed belongs is determined based on the directory depth. Here, the parent file set represents a set located above the current code file in the file hierarchy and having a smaller directory depth. If a parent file set that meets the conditions is found, and the sum of the word unit counts of the parent file set and the current code file does not exceed the maximum word unit limit `max_token`, then the word units of the current code file are merged and input into the parent file set. A code file set is generated according to the merging rules, whereby the merging rules include ensuring that the code set obtained after merging the parent file set and the current code file always follows the maximum word unit limit `max_token`. The code file set and the natural language description set are input into the large language model to obtain code interpretation; A vectorization processing module is used to interpret and segment the code and convert it into semantic vectors, which are then stored in a vector database. The matching module is used to convert the user's query statement into a query vector, and then perform a matching search on the semantic vector in the vector database to obtain matching results. The matching results include direct matching results and interactive matching results. If the large language model yields answers to all query statements, a direct matching result is obtained. If the large language model cannot yield a direct matching result based on the query statement, a matching search is performed in the vector database, including: converting the user's query statement into a query vector and invoking the agent; performing a vectorized search in the vector database, matching according to a similarity calculation method, and outputting the agent's answer; and determining the code file under the path corresponding to the semantic vector based on the agent's answer to obtain interactive matching results. The feedback module is used to integrate the matching results and feed them back to the user interface.

8. An apparatus for performing a codebase question-answering method based on a large model, characterized in that, include: processor; Memory used to store processor-executable instructions; When the processor executes the executable instructions, it implements the method as described in any one of claims 1 to 6.

9. A non-volatile computer-readable storage medium, characterized in that, Includes storage of computer programs or instructions that, when executed, cause the method as described in any one of claims 1 to 6 to be implemented.

Images