Index building method, device, equipment and storage medium

Abstract

The present disclosure provides an index construction method, apparatus, device and storage medium, which performs syntax tree parsing on a code to be stored based on grammatical features, performs code extraction according to the code structure obtained by the parsing, divides the code to be stored into code blocks corresponding to each code component, and then generates index vectors corresponding to the code blocks, completes the index construction of the code in the database, performs syntax tree parsing on the code to be stored, and divides the code to be stored into code blocks according to the code structure, thereby improving the correlation between the index vector and the code to be stored, thereby improving the accuracy of human-computer interaction based on retrieval enhancement.

Description

Technical Field

[0001] The present disclosure relates to artificial intelligence technology, natural language processing technology, and in particular to an index construction method, device, equipment and storage medium. Background Art

[0002] Retrieval-Augmented Generation (RAG) technology is a natural language processing (NLP) technology that combines information retrieval and generative models, aiming to improve the accuracy and richness of the results generated by generative models. In traditional generative models (such as GPT-3), the generation process depends on the model's own parameters and trained data, while the RAG model combines the generation task with the retrieval mechanism of the external knowledge base, which can dynamically retrieve relevant information from the external knowledge base and integrate the retrieved information into the generated results, thereby improving the accuracy and coverage of the generated results. The application of RAG technology mainly involves two stages, namely the retrieval stage and the generation stage. In the retrieval stage, given a user input (query), the retrieval module extracts information related to the user input from the external knowledge base (such as a search engine or a specific database); in the generation stage, the generative model combines the retrieved information, user input and context to generate the corresponding answer. Summary of the invention

[0003] The embodiments of the present disclosure provide an index construction method, apparatus, device and storage medium to improve the relevance of index vectors to stored codes, thereby improving the accuracy of human-computer interaction based on retrieval enhancement.

[0004] One aspect of an embodiment of the present disclosure provides an index construction method, including:

[0005] Performing syntax tree parsing on the code to be stored based on the syntax features to obtain a code structure of the code to be stored, wherein the code structure is used to indicate code components and the start and end positions of each code component in the code to be stored;

[0006] Extracting the code to be stored based on the code structure to obtain code blocks corresponding to the target code components;

[0007] Performing vectorization processing on the code block to generate an index vector corresponding to the code block;

[0008] The code block and the index vector are associated and stored in a database.

[0009] Optionally, the code components include at least one of a class declaration, a variable declaration, a method definition, a business logic, and a comment, wherein the class declaration is used to create a class and declare the attributes and methods of the class, the variable declaration is used to create a variable and declare the name and type of the variable, the method definition is used to indicate the name and method body of the method, the business logic includes the calling code of the method, and the comment is used to explain the code in a natural language;

[0010] The extracting the code to be stored based on the code structure to obtain the code blocks corresponding to the target code components includes:

[0011] Performing a first code extraction on the code to be stored based on the code structure to obtain a first code block corresponding to a first target code component, wherein the first target code component includes at least one of the class declaration, the variable declaration, the method definition, and the comment;

[0012] Performing a second code extraction on the code to be stored based on the code structure, obtaining a second code block corresponding to a second target code component, wherein the second target code component includes at least one of the class declaration, the variable declaration, and the business logic, wherein the business logic in the second code block is a calling code of a method corresponding to the method definition in the first code block;

[0013] The vectorizing the code block to generate an index vector corresponding to the code block includes:

[0014] Performing vectorization processing on the first code block to generate a first index vector corresponding to the first code block;

[0015] Vectorization is performed on the second code block to generate a second index vector corresponding to the second code block.

[0016] Optionally, the performing a second code extraction on the code to be stored based on the code structure to obtain a second code block corresponding to a second target code component includes:

[0017] Determining the number of methods in the code to be stored based on the code structure, where the number of methods is used to represent the number of methods defined in the code to be stored;

[0018] When the number of methods is greater than 1, a second code extraction is performed based on the start and end positions of each method in the code to be stored to obtain the second code block corresponding to each method, wherein the second code block includes the class declaration, the variable declaration, and a calling code corresponding to a method in the business logic.

[0019] Optionally, the associating and storing the code block and the index vector in a database includes:

[0020] The vector set of the code to be stored and the code set are associated and stored in the database, the vector set includes the first index vector and the second index vector, and the code set includes the first code block, the second code block and the code to be stored.

[0021] Optionally, the code set further includes code block attribute information, and the code block attribute information includes at least one of a start line of the code block, an end line of the code block, and a type of the code block.

[0022] Optionally, the method further includes:

[0023] In response to detecting a question input operation, obtaining question information input by the question input operation;

[0024] Extracting features from the question information to obtain a retrieval vector;

[0025] Based on the similarity between the search vector and the index vector in the database, determine a target index vector, wherein the target index vector includes at least one of n vectors with the highest similarity and m vectors with similarity higher than a similarity threshold, where n and m are positive integers;

[0026] The code block corresponding to the target index vector is input into a language model, and the answer information corresponding to the question information is generated by the language model.

[0027] Another aspect of the embodiments of the present disclosure provides an index building device, including:

[0028] A parsing module, used for performing syntax tree parsing on the code to be stored based on the syntax features to obtain a code structure of the code to be stored, wherein the code structure is used to indicate code components and the starting and ending positions of each code component in the code to be stored;

[0029] An acquisition module, used for extracting the code to be stored based on the code structure, and acquiring code blocks corresponding to the target code components;

[0030] A generating module, used for performing vectorization processing on the code block to generate an index vector corresponding to the code block;

[0031] The storage module is used to associate the code block with the index vector and store it in a database.

[0032] Another aspect of the present disclosure provides an electronic device, including:

[0033] Memory for storing computer programs;

[0034] The processor is used to execute the computer program stored in the memory, and when the computer program is executed, the method described in the above aspects is implemented.

[0035] Another aspect of the embodiments of the present disclosure provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the method described in the above aspects is implemented.

[0036] Another aspect of the embodiments of the present disclosure provides a computer program, including computer program instructions, which implement the methods described in the above aspects when executed by a processor.

[0037] Based on the embodiments of the present disclosure, a syntax tree is parsed for the code to be stored based on grammatical features, code extraction is performed according to the code structure obtained by the parsing, the code to be stored is divided into code blocks corresponding to each code component, and then an index vector corresponding to the code block is generated to complete the index construction of the code in the database. By performing a syntax tree parsing on the code to be stored and dividing the code to be stored according to the code structure to obtain code blocks, compared with the traditional text segmentation method, it can not only process long codes, but also maintain the semantic coherence of the codes and avoid the truncation of the code semantics, thereby improving the correlation between the index vector and the code to be stored, and then improving the accuracy of human-computer interaction based on retrieval enhancement.

[0038] The technical solution of the present disclosure is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] The accompanying drawings, which constitute a part of the specification, illustrate embodiments of the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0040] The present disclosure may be more clearly understood from the following detailed description with reference to the accompanying drawings, in which:

[0041] Figure 1 A flowchart of an embodiment of the index building method disclosed herein;

[0042] Figure 2 A flowchart of another embodiment of the index building method disclosed herein;

[0043] Figure 3 A schematic diagram showing the storage of index vectors associated with code blocks according to an embodiment of the present disclosure;

[0044] Figure 4 This is a schematic diagram of the structure of an embodiment of the index-based construction device disclosed in the present invention;

[0045] Figure 5 This is a schematic diagram of the structure of another embodiment of the index-based construction device disclosed in the present invention;

[0046] Figure 6 The figure is a schematic diagram of the structure of an application embodiment of the electronic device disclosed in the present invention. DETAILED DESCRIPTION

[0047] Various exemplary embodiments of the present disclosure will now be described in detail with reference to the accompanying drawings. It should be noted that the relative arrangement of components and steps, numerical expressions and numerical values ​​set forth in these embodiments do not limit the scope of the present disclosure unless otherwise specifically stated.

[0048] Those skilled in the art can understand that the terms "first" and "second" in the embodiments of the present disclosure are only used to distinguish different steps, devices or modules, etc., and neither represent any specific technical meaning nor indicate the necessary logical order between them.

[0049] It should also be understood that in the embodiments of the present disclosure, “plurality” may refer to two or more than two, and “at least one” may refer to one, two, or more than two.

[0050] It should also be understood that any component, data or structure mentioned in the embodiments of the present disclosure can generally be understood as one or more, unless explicitly limited or otherwise indicated in the context.

[0051] In addition, the term "and / or" in the present disclosure is only a description of the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in the present disclosure generally indicates that the associated objects before and after are in an "or" relationship.

[0052] It should also be understood that the description of the various embodiments in the present disclosure focuses on the differences between the various embodiments, and the same or similar aspects thereof can be referenced to each other, and for the sake of brevity, they will not be described one by one.

[0053] At the same time, it should be understood that for the convenience of description, the sizes of the various parts shown in the drawings are not drawn according to the actual proportional relationship.

[0054] The following description of at least one exemplary embodiment is merely illustrative in nature and is in no way intended to limit the present disclosure, its application, or uses.

[0055] Technologies, methods, and equipment known to ordinary technicians in the relevant art may not be discussed in detail, but where appropriate, the technologies, methods, and equipment should be considered part of the specification.

[0056] It should be noted that like reference numerals and letters refer to similar items in the following figures, and therefore, once an item is defined in one figure, it need not be further discussed in subsequent figures.

[0057] In the related technology, after the relevant text is retrieved, the long text will first be segmented to generate index vectors corresponding to each text block, and the segmented text segments and the corresponding index vectors will be associated and stored in a vector library so as to generate a model for vector retrieval to obtain the required text.

[0058] At present, the main method for segmenting long texts is to segment the text based on a fixed text length or by text paragraphs. However, when the text includes code, since the code usually contains more complex business logic and reference relationships, the above text processing methods based on fixed text length or by text paragraphs will cause the code semantics to be truncated, breaking the code coherence, resulting in the index vector corresponding to the generated text block being unable to accurately express the semantics of the original text, inaccurate text retrieval and sorting, and poor human-computer interaction.

[0059] Figure 1 A flowchart of an index building method provided by an exemplary embodiment of the present disclosure. The index building method of the embodiment of the present disclosure can be implemented by an electronic device with a text processing function.

[0060] like Figure 1 As shown, the method comprises the following steps:

[0061] Step 101 , performing syntax tree analysis on the code to be stored based on the syntax features corresponding to the code to be stored, and obtaining the code structure of the code to be stored.

[0062] The syntax tree is an abstract representation of the syntax structure of the source code, which shows the syntax structure of the programming language in a tree form, and each node on the tree represents a structure in the source code. Different programming languages ​​have different syntax features and code structure characteristics. For example, in the Java programming language, the definition of a class and the implementation of a class need to be put together, while in the C++ programming language, the definition of a class and the implementation of a class need to be written separately, and the specific statements of the class definition are also different. Optionally, the target programming language corresponding to the code to be stored can be identified, and based on the syntax features of the target programming language, the syntax tree of the code to be stored can be parsed to obtain the code structure of the code to be stored.

[0063] The code structure is used to indicate the code components included in the code to be stored and the starting and ending positions of each code component in the code to be stored. For example, the code components may include but are not limited to: reference relationships, class declarations, variable declarations, method definitions, etc. Based on the code structure, the starting line and the ending line of each code component in the code to be stored can be determined to facilitate subsequent code division.

[0064] Step 102: extract the code to be stored based on the code structure to obtain the code blocks corresponding to the target code components.

[0065] For codes with long text content, directly storing them in the database may cause the subsequent generated model to be unable to process the long text. Therefore, it is necessary to split the code to be stored first and store the segmented code blocks in the database so that the generated model can process them in segments.

[0066] Optionally, the code segmentation method can be predetermined, and after the code structure is obtained, the code to be stored is extracted based on the code structure and the code segmentation method to obtain the code blocks corresponding to the target code components. For example, considering that class declarations, variable declarations, and method definitions can reflect the main architecture of the code to be stored, and the business logic is to show the specific method call process and specific details, the partial code corresponding to the class declaration, variable declaration, and method definition can be extracted to obtain a code block, and the partial code corresponding to the business logic can be extracted separately to obtain another code block. Since the code segmentation is performed according to the start and end positions of each component indicated by the code structure, it can be ensured that the basic code components will not be truncated. Therefore, based on the code structure, code extraction with one component as the smallest division unit can ensure the coherence of each component and avoid the truncation of the code semantics. The target code components can be set according to actual needs. The embodiments of the present disclosure do not limit the specific code division and combination methods.

[0067] Step 103: perform vectorization processing on the code block to generate an index vector corresponding to the code block.

[0068] In a possible implementation, each code block may be embedded, and a low-dimensional dense vector, namely, an index vector, may be used to represent the corresponding code block.

[0069] Optionally, an embedding layer of a large language model (LLM) may be used to vectorize each code block to obtain an index vector output by the embedding layer.

[0070] Step 104: store the code block and the index vector in association with each other in a database, and generate association relationship information between the code block and the index vector.

[0071] The association information is used to indicate the code block corresponding to the index vector in the database. For example, each code block in the database corresponds to a unique code block identifier, and the code block identifier corresponding to each index vector can be recorded in the association information, so that the code block query can be performed based on the association information.

[0072] Optionally, the index vectors and code blocks in the database correspond one to one. When question information input by the user is received, a search is performed based on the question information, and the retrieved code is processed as above and stored in the database. Then, by comparing the similarity between the feature vector corresponding to the question information and each index vector in the database, the code block corresponding to the index vector with higher similarity is selected and input into a large language model, and the reply information is output through the large language model.

[0073] Based on the embodiments of the present disclosure, a syntax tree is parsed for the code to be stored based on grammatical features, code extraction is performed according to the code structure obtained by the parsing, the code to be stored is divided into code blocks corresponding to each code component, and then an index vector corresponding to the code block is generated to complete the index construction of the code in the database. By performing a syntax tree parsing on the code to be stored and dividing the code to be stored according to the code structure to obtain code blocks, compared with the traditional text segmentation method, it can not only process long codes, but also maintain the semantic coherence of the codes and avoid the truncation of the code semantics, thereby improving the correlation between the index vector and the code to be stored, and then improving the accuracy of human-computer interaction based on retrieval enhancement.

[0074] In a possible implementation, the code components include at least one of class declaration, variable declaration, method definition, business logic and annotation. Among them, class declaration is used to create a class and declare the attributes and methods of the class, variable declaration is used to create a variable and declare the name and type of the variable, method definition is used to indicate the name and method body of the method, business logic includes the calling code of the method, and annotation is used to explain the code in natural language. In addition, other components may be included, such as reference relationships, etc. Figure 2 A flowchart of an index building method provided by another exemplary embodiment of the present disclosure, the method comprising the following steps:

[0075] Step 201 , performing syntax tree parsing on the code to be stored based on the syntax features corresponding to the code to be stored, and obtaining the code structure of the code to be stored.

[0076] The specific implementation of step 201 may refer to the above step 101, and will not be described in detail in this embodiment of the present disclosure.

[0077] Step 202: Perform a first code extraction on the code to be stored based on the code structure to obtain a first code block corresponding to the first target code component.

[0078] The first target code component includes at least one of a class declaration, a variable declaration, a method definition and a comment.

[0079] Since the business logic in the code to be stored occupies a relatively long space, and users' questions about coding usually involve the main structure of the code and are less concerned with specific method call details, the class declaration, variable declaration, method definition and comments can be extracted separately and placed in the same code block, namely the first code block. The content of the first code block can be used to answer major programming questions.

[0080] Step 203: extract the second code from the code to be stored based on the code structure, and obtain a second code block corresponding to the second target code component.

[0081] The second target code component includes at least one of a class declaration, a variable declaration, and a business logic. The business logic in the second code block is the calling code of the method corresponding to the method definition in the first code block. That is, the first code block contains code contents related to the main structure of the code, such as class declaration, variable declaration, method definition, and comments, which can be used to answer users' programming questions about the code structure, while the second code block contains specific method calling process codes, which can be used to answer users' questions about method calling.

[0082] The business logic includes the method calling code, which can show the specific process of the method calling, so it can be extracted separately to form a new code block to answer the user's questions about the details of the method calling. Optionally, the second target code component also includes class declarations and variable declarations, which are integrated with the business logic in the same code block to ensure the integrity of the code logic and semantics.

[0083] Optionally, when the code to be stored involves multiple methods, code extraction may be performed for a single method so that the method corresponds to the second code block one by one. Step 203 may include the following steps:

[0084] Step 203a, determining the number of methods in the code to be stored based on the code structure, where the number of methods is used to represent the number of methods defined in the code to be stored.

[0085] Step 203b: When the number of methods is greater than 1, a second code is extracted based on the start and end positions of each method in the code to be stored to obtain a second code block corresponding to each method.

[0086] The second code block includes a class declaration, a variable declaration, and a calling code corresponding to a method in the business logic.

[0087] Illustratively, the starting and ending positions (including the starting code line and the ending code line) of each method in the code to be stored can be determined based on the code structure. When the number of methods is greater than 1, for example, the business logic part of the code to be stored includes the calling code of 5 methods, then the business logic part can be divided into 5 code blocks based on the starting and ending positions of each method calling code, each code block contains the calling code of 1 method, and a class declaration and a variable declaration are added in each code block, thereby obtaining 5 second code blocks.

[0088] In another possible implementation, the calling codes corresponding to another preset number of methods can also be placed in the same second code block. For example, every two methods can be extracted into a code block and class declarations and variable declarations can be added. The embodiment of the present disclosure does not limit the specific extraction method of the second code block.

[0089] Step 204: perform vectorization processing on the first code block to generate a first index vector corresponding to the first code block.

[0090] Step 205: perform vectorization processing on the second code block to generate a second index vector corresponding to the second code block.

[0091] After the code extraction obtains the first code block and the second code block, vectorization processing is performed on the first code block and the second code block respectively to obtain a first index vector corresponding to the first code block and a second index vector corresponding to the second code block.

[0092] It is worth mentioning that there is no strict execution order for the above steps 204 and 205.

[0093] Step 206: associate the vector set of the code to be stored with the code set and store them in a database, and generate association relationship information.

[0094] Among them, the vector set includes a first index vector and a second index vector, the code set includes a first code block, a second code block and a code to be stored, and the association relationship information includes first association relationship information and second association relationship information, the first association relationship information is used to indicate the first code block corresponding to the first index vector in the database, and the second association relationship information is used to indicate the second code block corresponding to the second index vector in the database.

[0095] By storing the first code block, the second code block and the original text of the code to be stored in the code collection respectively, when the programming question input by the user only involves the code architecture and does not care about the business logic details, the relevant first code block can be provided for question answering; when the programming question input by the user involves the method calling process, the corresponding second code block or the complete code to be stored can be provided for question answering.

[0096] Optional, such as Figure 3 As shown, the code set may also include code block attribute information, which includes at least one of the starting line of the code block, the ending line of the code block, and the type of the code block. Through the code block attribute information, the position of each code block in the original code to be stored can be located.

[0097] Based on the embodiments of the present disclosure, code extraction is performed based on the code structure of the code to be stored to obtain two types of code blocks, namely, a first code block including code backbone information and a second code block including specific business logic, which can be used to answer users' programming questions about code architecture and programming questions about specific method calling processes, respectively, and can ensure the integrity of code semantics, thereby improving the accuracy and efficiency of human-computer interaction.

[0098] In a possible implementation, after the index vector is generated, the index vector may be used for retrieval enhancement to obtain a code for generating reply information. The index construction method provided in the embodiment of the present disclosure further includes the following steps:

[0099] Step 1: In response to detecting a question input operation, obtaining question information input by the question input operation.

[0100] In a possible implementation, a user's question input operation may be received through an intelligent question-answering dialog box in an application or webpage to obtain question information input by the question input operation. The question information may include at least one of the following information types: text information, image information, and video information.

[0101] Illustratively, a user inputs the text "What properties and methods are included in entity class A?" through the smart question and answer dialog box, and the text content can be determined as question information input for the question input operation.

[0102] Step 2: Extract features from question information to obtain retrieval vectors.

[0103] Optionally, a retrieval vector may be obtained by identifying keywords in the question information and performing vectorization processing on the keywords.

[0104] Step three, based on the similarity between the search vector and the index vector in the database, determine the target index vector, which includes at least one of the n vectors with the highest similarity and the m vectors with similarity higher than the similarity threshold, where n and m are positive integers.

[0105] Step 4: determine the target code block corresponding to the target index vector based on the association relationship information.

[0106] Step 5: Input the target code block into the language model, and generate answer information corresponding to the question information through the language model.

[0107] Optionally, the code block identifier corresponding to each index vector may be recorded in the association relationship information, so that the target code block may be acquired according to the code block identifier corresponding to the target index vector in the association relationship information.

[0108] Illustratively, the language model may adopt a large language model such as a bidirectional encoder model (Bidirectional Encoder Representations from Transformers, BERT), a generative pre-trained model (Generative Pre-trained Transformer, GPT). The reply information output by the language model may include at least one of the forms of reply text, reply image, and reply video. After obtaining the reply information output by the language model, the reply information may be displayed through the dialogue interface of the application or web page.

[0109] Figure 4 The structure block diagram of an index building device provided by an exemplary embodiment of the present disclosure is shown. The index building device includes:

[0110] A parsing module 401, configured to perform syntax tree parsing on the code to be stored based on the syntax features corresponding to the code to be stored, and obtain a code structure of the code to be stored, wherein the code structure is used to indicate the code components included in the code to be stored and the start and end positions of each code component in the code to be stored;

[0111] An acquisition module 402 is used to extract the code to be stored based on the code structure parsed by the parsing module 401, and obtain code blocks corresponding to the target code components;

[0112] A generating module 403 is used to perform vectorization processing on the code block obtained by the obtaining module 402 to generate an index vector corresponding to the code block;

[0113] The storage module 404 is used to associate and store the code block obtained by the obtaining module 402 and the index vector generated by the generating module 403 in a database, and generate association relationship information between the code block and the index vector, where the association relationship information is used to indicate the code block corresponding to the index vector in the database.

[0114] Optionally, in a possible implementation, the code components include at least one of a class declaration, a variable declaration, a method definition, a business logic, and a comment, wherein the class declaration is used to create a class and declare the attributes and methods of the class, the variable declaration is used to create a variable and declare the name and type of the variable, the method definition is used to indicate the name and method body of the method, the business logic includes the calling code of the method, and the comment is used to explain the code in a natural language;

[0115] The acquisition module 402 may also be used for:

[0116] Performing a first code extraction on the code to be stored based on the code structure to obtain a first code block corresponding to a first target code component, where the first target code component includes at least one of a class declaration, a variable declaration, a method definition, and a comment;

[0117] Performing a second code extraction on the code to be stored based on the code structure, obtaining a second code block corresponding to a second target code component, the second target code component including at least one of a class declaration, a variable declaration, and a business logic, wherein the business logic in the second code block is a calling code of a method corresponding to a method definition in the first code block;

[0118] The above generation module 403 can also be used for:

[0119] Performing vectorization processing on the first code block to generate a first index vector corresponding to the first code block;

[0120] The second code block is vectorized to generate a second index vector corresponding to the second code block.

[0121] Optionally, in a possible implementation manner, the acquisition module 402 may also be used to:

[0122] Determine the number of methods in the code to be stored based on the code structure, where the number of methods is used to represent the number of methods defined in the code to be stored;

[0123] When the number of methods is greater than 1, a second code extraction is performed based on the start and end positions of each method in the code to be stored to obtain a second code block corresponding to each method, which includes a class declaration, a variable declaration, and a calling code corresponding to a method in the business logic.

[0124] Optionally, in a possible implementation manner, the storage module 404 may also be used for:

[0125] A vector set and a code set of codes to be stored are associated and stored in a database, and association relationship information is generated, wherein the vector set includes a first index vector and a second index vector, the code set includes a first code block, a second code block and the code to be stored, and the association relationship information includes first association relationship information and second association relationship information, the first association relationship information is used to indicate a first code block corresponding to the first index vector in the database, and the second association relationship information is used to indicate a second code block corresponding to the second index vector in the database.

[0126] Optionally, in a possible implementation, the code set further includes code block attribute information, where the code block attribute information includes at least one of a start line of the code block, an end line of the code block, and a type of the code block.

[0127] Optionally, in a possible implementation manner, as Figure 5 As shown, the index building device may also include:

[0128] The information acquisition module 501 is used to acquire the question information input by the question input operation in response to detecting the question input operation;

[0129] The feature extraction module 502 is used to extract features from the question information obtained by the information acquisition module 501 to obtain a retrieval vector;

[0130] A retrieval module 503 is used to determine a target index vector based on the similarity between the retrieval vector extracted by the feature extraction module 502 and the index vector in the database, where the target index vector includes at least one of n vectors with the highest similarity and m vectors with similarity higher than a similarity threshold, where n and m are positive integers;

[0131] A determination module 504, configured to determine a target code block corresponding to the target index vector retrieved by the retrieval module 503 based on the association relationship information;

[0132] The reply module 505 is used to input the target code block into the language model, and generate reply information corresponding to the question information through the language model.

[0133] Each embodiment in this specification is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The parts of the same, similar or corresponding contents between the various embodiments can be referred to each other. Since the method, device, system, and equipment embodiments are basically corresponding, the relevant parts can be referred to the description of the corresponding parts. The methods, devices, systems, and equipment of the embodiments of the present disclosure also correspond to each other in specific implementation methods and beneficial technical effects. The relevant contents can be referred to each other and will not be repeated.

[0134] In addition, an embodiment of the present disclosure further provides an electronic device, including:

[0135] Memory for storing computer programs;

[0136] The processor is used to execute the computer program stored in the memory, and when the computer program is executed, the retrieval-enhanced interaction method described in any of the above embodiments of the present disclosure is implemented.

[0137] Figure 6 This is a schematic diagram of the structure of an application embodiment of the electronic device disclosed in the present invention. Figure 6The electronic device according to the embodiment of the present disclosure is described. The electronic device may be any one or both of the first device and the second device, or a stand-alone device independent of them, and the stand-alone device may communicate with the first device and the second device to receive the collected input signals from them.

[0138] like Figure 6 As shown, the electronic device includes one or more processors and memory.

[0139] The processor may be a central processing unit (CPU) or other forms of processing units having data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device to perform desired functions.

[0140] The memory may include one or more computer program products, and the computer program product may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may include, for example, a random access memory (RAM) and / or a cache memory (cache), etc. The non-volatile memory may include, for example, a read-only memory (ROM), a hard disk, a flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor may execute the program instructions to implement the interactive method based on retrieval enhancement and / or other desired functions of the various embodiments of the present disclosure described above.

[0141] In one example, the electronic device may further include: an input device and an output device, and these components are interconnected via a bus system and / or other forms of connection mechanisms (not shown).

[0142] In addition, the input device may also include, for example, a keyboard, a mouse, and the like.

[0143] The output device can output various information to the outside, including the determined distance information, direction information, etc. The output device can include, for example, a display, a speaker, a printer, a communication network and a remote output device connected thereto, and the like.

[0144] Of course, to simplify, Figure 6 Only some of the components related to the present disclosure in the electronic device are shown, and components such as a bus, an input / output interface, etc. are omitted. In addition, according to specific application situations, the electronic device may further include any other appropriate components.

[0145] In addition to the above-mentioned methods and devices, an embodiment of the present disclosure may also be a computer program product, which includes computer program instructions, which, when executed by a processor, enable the processor to execute the steps of the retrieval-enhanced interaction method according to various embodiments of the present disclosure described in the above part of this specification.

[0146] The computer program product may be written in any combination of one or more programming languages ​​to write program code for performing the operations of the disclosed embodiments, including object-oriented programming languages ​​such as Java, C++, etc., and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a separate software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0147] In addition, an embodiment of the present disclosure may also be a computer-readable storage medium having computer program instructions stored thereon, which, when executed by a processor, enable the processor to execute the steps of the retrieval-enhanced interaction method according to various embodiments of the present disclosure described in the above part of this specification.

[0148] The computer readable storage medium can adopt any combination of one or more readable media. The readable medium can be a readable signal medium or a readable storage medium. The readable storage medium can include, for example, but is not limited to, a system, device or device of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination of the above. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0149] A person of ordinary skill in the art can understand that all or part of the steps of implementing the above method embodiment can be completed by hardware related to program instructions, and the aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps of the above method embodiment; and the aforementioned storage medium includes: ROM, RAM, magnetic disk or optical disk, etc., various media that can store program codes.

[0150] The basic principles of the present disclosure are described above in conjunction with specific embodiments. However, it should be noted that the advantages, strengths, effects, etc. mentioned in the present disclosure are only examples and not limitations, and it cannot be considered that these advantages, strengths, effects, etc. are required by each embodiment of the present disclosure. In addition, the specific details disclosed above are only for the purpose of illustration and ease of understanding, and are not limitations. The above details do not limit the present disclosure to the necessity of adopting the above specific details to be implemented.

[0151] Each embodiment in this specification is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the system embodiment, since it basically corresponds to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0152] The block diagrams of the devices, apparatuses, equipment, and systems involved in this disclosure are only illustrative examples and are not intended to require or imply that they must be connected, arranged, and configured in the manner shown in the block diagrams. As will be appreciated by those skilled in the art, these devices, apparatuses, equipment, and systems can be connected, arranged, and configured in any manner. Words such as "including," "comprising," "having," and the like are open words, referring to "including but not limited to," and can be used interchangeably therewith. The words "or" and "and" used herein refer to the words "and / or," and can be used interchangeably therewith, unless the context clearly indicates otherwise. The word "such as" used herein refers to the phrase "such as but not limited to," and can be used interchangeably therewith.

[0153] The method and apparatus of the present disclosure may be implemented in many ways. For example, the method and apparatus of the present disclosure may be implemented by software, hardware, firmware, or any combination of software, hardware, and firmware. The above order of steps for the method is for illustration only, and the steps of the method of the present disclosure are not limited to the order specifically described above, unless otherwise specifically stated. In addition, in some embodiments, the present disclosure may also be implemented as a program recorded in a recording medium, which includes machine-readable instructions for implementing the method according to the present disclosure. Therefore, the present disclosure also covers a recording medium storing a program for executing the method according to the present disclosure.

[0154] It should also be noted that in the apparatus, device and method of the present disclosure, each component or each step can be decomposed and / or recombined. Such decomposition and / or recombination should be regarded as equivalent solutions of the present disclosure.

[0155] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use the present disclosure. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects without departing from the scope of the present disclosure. Therefore, the present disclosure is not intended to be limited to the aspects shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

[0156] The above description has been given for the purpose of illustration and description. In addition, this description is not intended to limit the embodiments of the present disclosure to the forms disclosed herein. Although multiple example aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, changes, additions and sub-combinations thereof.

Claims

1. An index construction method, characterized in that: include: Performing syntax tree parsing on the code to be stored based on the syntax features corresponding to the code to be stored to obtain a code structure of the code to be stored, wherein the code structure is used to indicate the code components included in the code to be stored and the starting and ending positions of each code component in the code to be stored; Extracting the code to be stored based on the code structure to obtain code blocks corresponding to the target code components; Performing vectorization processing on the code block to generate an index vector corresponding to the code block; The code block and the index vector are associated and stored in a database, and association relationship information between the code block and the index vector is generated, where the association relationship information is used to indicate the code block corresponding to the index vector in the database.

2. The method according to claim 1, characterized in that The code components include at least one of a class declaration, a variable declaration, a method definition, a business logic, and a comment. The class declaration is used to create a class and declare the attributes and methods of the class. The variable declaration is used to create a variable and declare the name and type of the variable. The method definition is used to indicate the name and method body of the method. The business logic includes the calling code of the method. The comment is used to explain the code in a natural language. The extracting the code to be stored based on the code structure to obtain the code blocks corresponding to the target code components includes: Performing a first code extraction on the code to be stored based on the code structure to obtain a first code block corresponding to a first target code component, wherein the first target code component includes at least one of the class declaration, the variable declaration, the method definition, and the comment; Performing a second code extraction on the code to be stored based on the code structure, obtaining a second code block corresponding to a second target code component, wherein the second target code component includes at least one of the class declaration, the variable declaration, and the business logic, wherein the business logic in the second code block is a calling code of a method corresponding to the method definition in the first code block; The vectorizing the code block to generate an index vector corresponding to the code block includes: Performing vectorization processing on the first code block to generate a first index vector corresponding to the first code block; Vectorization is performed on the second code block to generate a second index vector corresponding to the second code block.

3. The method according to claim 2, characterized in that The step of performing a second code extraction on the code to be stored based on the code structure to obtain a second code block corresponding to a second target code component includes: Determining the number of methods in the code to be stored based on the code structure, where the number of methods is used to represent the number of methods defined in the code to be stored; When the number of methods is greater than 1, a second code extraction is performed based on the start and end positions of each method in the code to be stored to obtain the second code block corresponding to each method, wherein the second code block includes the class declaration, the variable declaration, and a calling code corresponding to a method in the business logic.

4. The method according to claim 2, characterized in that: The step of associating and storing the code block and the index vector in a database, and generating association relationship information between the code block and the index vector, includes: The vector set and the code set of the code to be stored are associated and stored in the database, and the association relationship information is generated, the vector set includes the first index vector and the second index vector, the code set includes the first code block, the second code block and the code to be stored, and the association relationship information includes first association relationship information and second association relationship information, the first association relationship information is used to indicate the first code block corresponding to the first index vector in the database, and the second association relationship information is used to indicate the second code block corresponding to the second index vector in the database.

5. The method according to claim 4, characterized in that The code set also includes code block attribute information, and the code block attribute information includes at least one of a start line of the code block, an end line of the code block, and a type of the code block.

6. The method according to any one of claims 1 to 5, characterized in that: The method further comprises: In response to detecting a question input operation, obtaining question information input by the question input operation; Extracting features from the question information to obtain a retrieval vector; Based on the similarity between the search vector and the index vector in the database, determine a target index vector, wherein the target index vector includes at least one of n vectors with the highest similarity and m vectors with similarity higher than a similarity threshold, where n and m are positive integers; Determine the target code block corresponding to the target index vector based on the association relationship information; The target code block is input into a language model, and answer information corresponding to the question information is generated by the language model.

7. An index building device, characterized in that: include: A parsing module, used for performing syntax tree parsing on the code to be stored based on the syntax features corresponding to the code to be stored, and obtaining a code structure of the code to be stored, wherein the code structure is used to indicate the code components included in the code to be stored and the starting and ending positions of each code component in the code to be stored; An acquisition module, used for extracting the code to be stored based on the code structure, and acquiring code blocks corresponding to the target code components; A generating module, used for performing vectorization processing on the code block to generate an index vector corresponding to the code block; A storage module is used to associate and store the code block and the index vector in a database, and generate association relationship information between the code block and the index vector, wherein the association relationship information is used to indicate the code block corresponding to the index vector in the database.

8. An electronic device, characterized in that: include: Memory for storing computer programs; A processor, configured to execute a computer program stored in the memory, and when the computer program is executed, implement the method described in any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method described in any one of claims 1 to 6 is implemented.

10. A computer program comprising computer program instructions, characterized in that When the computer program instructions are executed by a processor, the method described in any one of claims 1 to 6 is implemented.

Images