Retrieval enhancement generation-based retrieval method, product, equipment and medium
By identifying entities in the initial query information and using the knowledge graph for expansion and multi-layer semantic matching, the problem of low retrieval accuracy of the search enhancement generative model is solved, and higher retrieval accuracy and recall rate are achieved, adapting to the knowledge base requirements in different fields. .
Patent Information
- Application Number
- CN202411481538.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-23
- Publication Date
- 2024-11-22
AI Technical Summary
The existing search enhancement generation model has poor retrieval accuracy and inaccurate search results. Especially when processing large-scale data and semantic differences are large, it is difficult to meet the needs of high-quality text generation.
By obtaining the initial query information input by the user, identifying entities and expanding them based on the knowledge graph, combining keyword matching and semantic matching, a final match score is generated, and the target text block collection is filtered out, and the final answer to the initial query information is generated.
It improves the search accuracy and recall rate of related documents, can better understand the user's query intention and text content, adapt to the professional knowledge base in different fields, and improves the system's adaptability and the accuracy of query comprehension.
Smart Images

Figure CN119003795A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of natural language processing technology, and in particular to a retrieval method, product, device and medium based on retrieval enhancement generation. Background Art
[0002] In recent years, Retrieval-Augmented Generation (RAG) technology has emerged in the field of text generation as a method that combines information retrieval and generative models. The RAG model makes up for the lack of factual basis in the pure generative model by first retrieving relevant documents and then generating text. The accuracy of the retrieval stage is crucial to the quality of the final generated results.
[0003] At present, traditional retrieval algorithms mainly rely on keyword matching, while deep learning models focus more on matching at the semantic level. However, each of them has certain limitations; for example, keyword matching cannot understand semantic differences, resulting in low relevance of retrieval results for complex queries, making it difficult to meet the needs of high-quality text generation. Deep learning models have limited capabilities when processing large-scale data, and may not be able to accurately capture the differences when there is a large semantic difference between the query and the document, resulting in reduced retrieval accuracy; at the same time, the computational complexity is high and it relies heavily on the amount of data.
[0004] In view of the above problems, how to solve the poor retrieval accuracy and inaccurate retrieval results of the current retrieval enhancement generation model is an urgent problem to be solved by technicians in this field. Summary of the invention
[0005] The purpose of the present invention is to provide a retrieval method, product, device and medium based on retrieval enhancement generation to solve the problem of poor retrieval accuracy and inaccurate retrieval results of the current retrieval enhancement generation model.
[0006] In order to solve the above technical problems, the present invention provides a retrieval method based on retrieval enhancement generation, comprising:
[0007] Obtaining initial query information input by a user and identifying entities in the initial query information;
[0008] Expanding entities in the initial query information based on the knowledge graph to generate expanded query information and corresponding related knowledge graph information; wherein the related knowledge graph information at least includes other entities linked to the entity matching, entity relationships between entities, and concepts related to the initial query information;
[0009] Based on the extended query information and the relevant knowledge graph information, keyword matching and semantic matching are performed on each text block to obtain the keyword matching score and semantic matching score corresponding to each text block;
[0010] The keyword matching scores and the corresponding semantic matching scores are merged to obtain the final matching scores of each text block;
[0011] A target text block set is screened out from each text block according to the final matching score corresponding to each text block, and a final answer corresponding to the initial query information is generated according to the target text block set.
[0012] On the one hand, obtaining initial query information input by a user and identifying entities in the initial query information includes:
[0013] Preprocessing the initial query information to generate a vocabulary sequence corresponding to the initial query information;
[0014] Identify entities in a word sequence based on natural language processing techniques.
[0015] On the other hand, the preprocessing of the initial query information includes:
[0016] Use a word segmentation tool to split the initial query information into multiple words;
[0017] Initialize each word and remove meaningless words from each word;
[0018] Keywords are extracted based on each of the initialized words to generate a word sequence.
[0019] On the other hand, the entity in the initial query information is expanded based on the knowledge graph to generate the expanded query information and the corresponding related knowledge graph information, including:
[0020] Match the entities in the initial query information with the nodes in the knowledge graph to determine other entities linked to the entity matching and the entity relationships between the entities;
[0021] According to the vocabulary sequence, entities in the initial query information, other entities linked to the entity matching, and entity relationships between entities, the extended query information and related knowledge graph information are generated.
[0022] On the other hand, the keyword matching and semantic matching of each text block are performed based on the extended query information and the relevant knowledge graph information to obtain the keyword matching score and semantic matching score corresponding to each text block, including:
[0023] Based on a preset algorithm, keyword matching is performed on each text block and the extended query information to obtain a keyword matching score between each text block and the extended query information;
[0024] Based on a preset model, semantic matching is performed on each text block and the extended query information respectively to obtain a first semantic matching score between each text block and the extended query information;
[0025] Semantic matching is performed on each text block and the initial query information according to relevant knowledge graph information to obtain a second semantic matching score between each text block and the initial query information.
[0026] On the other hand, the keyword matching of each text block and the extended query information based on a preset algorithm to obtain a keyword matching score between each text block and the extended query information includes:
[0027] Obtaining the words in the extended query information, the frequency of each word in the corresponding text block, the extended query information word frequency and the inverse document frequency;
[0028] According to the words in the extended query information, the frequency of each word in the corresponding text block, the word frequency of the extended query information and the inverse document frequency, the text block and the extended query information are matched with keywords through the best matching algorithm to obtain the keyword matching score between the text block and the extended query information.
[0029] On the other hand, the semantic matching of each text block with the extended query information based on the preset model to obtain a first semantic matching score between each text block and the extended query information includes:
[0030] The extended query information and each text block are vectorized through the embedding model to obtain an extended query vector and a vector of each text block;
[0031] The vector similarity between each text block vector and the extended query vector is calculated respectively to obtain a first semantic matching score between each text block and the extended query information.
[0032] On the other hand, the method of fusing the keyword matching scores and the corresponding semantic matching scores to obtain the final matching scores corresponding to the text blocks includes:
[0033] Obtain weight parameters corresponding to the keyword matching score, the first semantic matching score, and the second semantic matching score respectively;
[0034] The products of the keyword matching score, the first semantic matching score, and the second semantic matching score and the corresponding weight parameters are determined respectively, and the products are added together to obtain a final matching score corresponding to the text block.
[0035] On the other hand, the step of selecting a target text block set from each text block according to the final matching score corresponding to each text block includes:
[0036] Sort each text block in descending order according to the final matching score of each text block;
[0037] Generate a candidate text block set according to a preset number of text blocks with the highest final matching scores among the text blocks;
[0038] The text blocks whose final matching scores are lower than the minimum relevance threshold in the candidate text block set are deleted to obtain the target text block set.
[0039] On the other hand, generating a final answer corresponding to the initial query information according to the target text block set includes:
[0040] splicing the target text block set and the initial query information based on a preset format;
[0041] Select a backend large language model, and select a corresponding small sample example for the backend large language model according to the query type of the initial query information;
[0042] The concatenated target text block set and the initial query information are input into the backend large language model to output the final answer corresponding to the initial query information.
[0043] On the other hand, after generating the final answer corresponding to the initial query information according to the target text block set, the method further includes:
[0044] Conduct a preliminary evaluation of the final answer corresponding to the initial query information to determine whether the final answer is accurate;
[0045] If it is confirmed that the final answer is inaccurate, the structure of the final answer is adjusted according to the matching degree between the final answer and the initial query information, or the text block is reselected and the search is performed again.
[0046] On the other hand, after generating the final answer corresponding to the initial query information according to the target text block set, the method further includes:
[0047] Combine the initial query information, the expanded query information and the final answer to generate retrieval sample information;
[0048] Save the retrieved sample information to the database.
[0049] In order to solve the above technical problem, the present invention also provides a computer program product, including a computer program or instructions, which, when executed by a processor, implements the steps of the above retrieval method based on retrieval enhancement generation.
[0050] In order to solve the above technical problems, the present invention further provides a retrieval device based on retrieval enhancement generation, comprising:
[0051] Memory for storing computer programs;
[0052] A processor is used to implement the steps of the above-mentioned retrieval method based on retrieval enhancement generation when executing the computer program.
[0053] In order to solve the above technical problem, the present invention also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the above-mentioned retrieval method based on retrieval enhancement generation are implemented.
[0054] The retrieval method based on retrieval enhancement generation provided by the present invention obtains the initial query information input by the user and identifies the entities in the initial query information; expands the entities in the initial query information based on the knowledge graph to generate extended query information and corresponding related knowledge graph information; wherein the related knowledge graph information at least includes other entities linked to the entity matching, entity relationships between entities, and concepts related to the initial query information; performs keyword matching and semantic matching on each text block based on the extended query information and the related knowledge graph information to obtain the keyword matching score and semantic matching score corresponding to each text block; fuses each keyword matching score and the corresponding semantic matching score to obtain the final matching score corresponding to each text block; selects a target text block set from each text block according to the final matching score corresponding to each text block, and generates the final answer corresponding to the initial query information according to the target text block set.
[0055] The beneficial effects of the present invention are that when the user inputs the initial query information, by identifying the entities in the initial query information, it can help the system capture the key information and semantic structure in the user's query, thereby improving the accuracy of query understanding; expanding the entities in the initial query information based on the knowledge graph can supplement the implicit background information, so that the system can better handle incomplete or ambiguous queries; and the introduction of the knowledge graph enables the system to use domain-specific structured knowledge to better understand and process queries in professional fields; at the same time, the knowledge graph can be easily expanded or replaced with professional knowledge bases in different fields, improving the adaptability of the system. In the semantic matching process, keyword matching is combined with semantic matching, and multi-layer semantic matching can more comprehensively understand the user's query intention and the content of the text block to be retrieved, significantly improving the retrieval accuracy.
[0056] On the other hand, the present invention specifically pre-processes the initial query information to generate a vocabulary sequence corresponding to the initial query information; identifies entities in the vocabulary sequence based on natural language processing technology, thereby realizing the recognition of entities in the initial query information, so as to facilitate the subsequent expansion and retrieval of the initial query information based on the entities. The entities in the initial query information are matched with the nodes in the knowledge graph to determine other entities linked to the entity matching and the entity relationships between each entity; based on the vocabulary sequence, the entities in the initial query information, the other entities linked to the entity matching and the entity relationships between each entity, the extended query information and the related knowledge graph information are generated, thereby realizing the expansion of the initial query information, being able to supplement the background information implicit in the initial query information, and enabling the system to better handle incomplete or ambiguous queries. By performing keyword matching on each text block and the extended query information based on a preset algorithm, the keyword matching score between each text block and the extended query information is obtained; by performing semantic matching on each text block and the extended query information based on a preset model, the first semantic matching score between each text block and the extended query information is obtained; by performing semantic matching on each text block and the initial query information based on relevant knowledge graph information, the second semantic matching score between each text block and the initial query information is obtained, thereby realizing multi-layer semantic matching of the query, enabling the system to understand the query intent and document content more comprehensively, and to capture correlations at different levels, from literal matching to deep semantic understanding, thus greatly improving the recall rate and accuracy of related documents. The weight parameters corresponding to the keyword matching score, the first semantic matching score and the second semantic matching score are obtained respectively, the product of the keyword matching score, the first semantic matching score and the second semantic matching score and the corresponding weight parameter is determined, and each product is summed up to obtain the final matching score corresponding to the text block, thereby realizing the fusion of multi-layer semantic matching results and improving the retrieval accuracy.
[0057] In addition, the present invention also provides a computer program product, a retrieval device and a medium based on retrieval enhancement generation, with the same effects as above. BRIEF DESCRIPTION OF THE DRAWINGS
[0058] In order to more clearly illustrate the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0059] Figure 1 A flowchart of a retrieval method based on retrieval enhancement generation provided by an embodiment of the present invention;
[0060] Figure 2 A schematic diagram of a retrieval device based on retrieval enhancement generation provided by an embodiment of the present invention;
[0061] Figure 3 A schematic diagram of a retrieval device based on retrieval enhancement generation provided by an embodiment of the present invention. DETAILED DESCRIPTION
[0062] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0063] The core of the present invention is to provide a retrieval method, product, device and medium based on retrieval enhancement generation to solve the problem of poor retrieval accuracy and inaccurate retrieval results of the current retrieval enhancement generation model.
[0064] In order to enable those skilled in the art to better understand the solution of the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings and specific implementation methods.
[0065] At present, the system based on RAG technology mainly adopts a single deep learning retrieval model, which uses a dual-tower structure to vectorize queries and documents and calculates similarities in the vector space to retrieve relevant documents. However, this method has limited processing capabilities for large-scale data. When the semantic difference between the query and the document is large, it may not be able to accurately capture this difference, resulting in reduced retrieval accuracy. In addition, due to the high computational complexity, the real-time performance of the system is poor, making it difficult to meet the needs of efficient processing.
[0066] Another type of method uses traditional information retrieval methods, such as the Term Frequency-Inverse Document Frequency (TF-IDF) algorithm and the Best Matching 25 (BM25) algorithm, to retrieve relevant documents through precise matching of keywords. This type of method works well when processing structured text, has low computational complexity, and can respond to user queries quickly. However, this method only focuses on matching at the lexical level and cannot effectively capture the deep semantic relationship of the text, resulting in low relevance of the retrieval results when processing complex queries, making it difficult to meet users' needs for high-quality text generation. Therefore, in order to solve the above problems, the present invention proposes a retrieval method based on retrieval enhancement generation.
[0067] Figure 1 A flowchart of a search method based on search enhancement generation provided by an embodiment of the present invention. Figure 1 As shown, the method includes:
[0068] S10: Acquire initial query information input by the user, and identify entities in the initial query information.
[0069] Specifically, the initial query information in natural language input by the user is first obtained. It should be noted that this solution aims to retrieve the final answer corresponding to the initial query information based on the pre-selected text block (document). In addition, the initial query information in this embodiment is applicable to all language types.
[0070] Entity recognition is an important task in natural language processing. Its goal is to identify and classify specific types of entities from text, such as names of people, places, names of organizations, dates, times, etc. In this embodiment, it is necessary to identify the entities in the initial query information in order to facilitate subsequent entity-based retrieval. For example, when the user's initial query information is: What are the characteristics of the poems of poet A. The entities identified are: poet A and poems. It should be noted that the specific method of entity recognition is not limited in this embodiment, and it depends on the specific implementation situation.
[0071] S11: Expand entities in the initial query information based on the knowledge graph to generate extended query information and corresponding related knowledge graph information.
[0072] The relevant knowledge graph information at least includes other entities linked to the entity matching, entity relationships between entities, and concepts related to the initial query information.
[0073] The knowledge graph is a structured way of representing knowledge, which consists of entities (nodes) and relationships (edges) between entities. In the RAG system, the knowledge graph can help to better understand the entities in the query information and the relationships between them, thereby improving the quality of retrieval and generation. In this embodiment, the entities in the initial query information are expanded based on the knowledge graph to generate extended query information and corresponding related knowledge graph information. It can be understood that the extended query information adds other entities, related concepts and relationships matched and linked from the knowledge graph compared to the initial query information. For example, when the user's initial query information is: What are the characteristics of the poems of poet A? The entities identified are: poet A and poetry; after query matching in the knowledge graph, some other entities can be linked: {'poet A':{'node': 'poet A', 'neighbors': ['dynasty B', 'poet C']}, 'poetry': {'node': 'poetry', 'neighbors': ['dynasty B', 'literature']}}, and the relationship between entities is extracted: [{'start': 'poet A', 'end': 'poetry', 'path':['poet A', 'dynasty B', 'poetry']}]. In this way, the semantic scope of the initial query information is expanded, and the expanded query information is obtained. The expanded query information can be specifically: the characteristics of poet A's poetry, dynasty B, poet C and literature.
[0074] In addition, the relevant knowledge graph information at least contains other entities linked to the entity matching, that is, the entities identified in the query and successfully matched with the knowledge graph; the entity relationships between entities, that is, the direct and indirect relationships between entities; and other concepts or entities related to the initial query information. The relevant knowledge graph information participates in the subsequent semantic matching.
[0075] It should be noted that the specific process of expanding the entities in the initial query information based on the knowledge graph to generate the expanded query information and the corresponding related knowledge graph information is not limited in this embodiment, and it depends on the specific implementation situation. In addition, the selection method of the knowledge graph is not limited in this embodiment. For example, different knowledge graphs can be used for different scenarios to meet different retrieval needs.
[0076] S12: Based on the extended query information and related knowledge graph information, keyword matching and semantic matching are performed on each text block to obtain a keyword matching score and a semantic matching score corresponding to each text block.
[0077] S13: Fusing each keyword matching score and the corresponding semantic matching score respectively to obtain a final matching score corresponding to each text block.
[0078] Furthermore, after obtaining the extended query information and related knowledge graph information, keyword matching and semantic matching are performed on each text block to be retrieved based on the extended query information and related knowledge graph information, so as to obtain the keyword matching score and semantic matching score corresponding to each text block.
[0079] It should be noted that the specific processes of keyword matching and semantic matching are not limited in this embodiment. For example, keyword matching can be performed using the BM25 algorithm or the TF-IDF algorithm, and semantic matching can be performed by calculating vector cosine similarity, depending on the specific implementation situation.
[0080] Finally, the keyword matching scores and corresponding semantic matching scores of each text block are merged to obtain the final matching scores corresponding to each text block. The final matching scores are used to select appropriate target text blocks, so as to retrieve the final answer to the initial query information based on the target text blocks. In this embodiment, there is no restriction on the process of determining the final matching scores.
[0081] S14: Filter out a target text block set from each text block according to the final matching score corresponding to each text block, and generate a final answer corresponding to the initial query information according to the target text block set.
[0082] After obtaining the final matching scores corresponding to each text block, a target text block set is screened out from each text block based on the final matching scores corresponding to each text block, and a final answer corresponding to the initial query information is generated based on the target text block set.
[0083] It should be noted that in this embodiment, there is no restriction on the method of screening the target text block set, and there is no restriction on the process of generating the final answer, which depends on the specific implementation situation.
[0084] In this embodiment, when the user inputs the initial query information, by identifying the entities in the initial query information, it can help the system capture the key information and semantic structure in the user's query, thereby improving the accuracy of query understanding; expanding the entities in the initial query information based on the knowledge graph can supplement the implicit background information, so that the system can better handle incomplete or ambiguous queries; and the introduction of the knowledge graph enables the system to use domain-specific structured knowledge to better understand and process queries in professional fields; at the same time, the knowledge graph can be easily expanded or replaced with professional knowledge bases in different fields, improving the adaptability of the system. In the semantic matching process, keyword matching is combined with semantic matching, and multi-layer semantic matching can more comprehensively understand the user's query intention and the content of the text block to be retrieved, significantly improving the retrieval accuracy.
[0085] Based on the above embodiments, in some embodiments, obtaining the initial query information input by the user and identifying the entities in the initial query information includes:
[0086] S101: Preprocess the initial query information to generate a vocabulary sequence corresponding to the initial query information;
[0087] S102: Identify entities in the vocabulary sequence based on natural language processing techniques.
[0088] In a specific implementation, in order to identify the entities in the initial query information, it is first necessary to preprocess the initial query information to generate a vocabulary sequence corresponding to the initial query information , which are specific words; it should be noted that the vocabulary sequence is actually another form of the user's initial query information. Further, entities are identified in the vocabulary sequence based on natural language processing techniques.
[0089] For example, in this embodiment, the Named-entity recognition (NER) technique can be specifically used to identify the entities in the initial query information: First, select an NER model, and a pre-trained model such as BERT-NER can be used; further, apply the NER model to the initial query information text, input the user's query into the NER model, and obtain the recognition result; extract the entities and their types from the model output, and finally save the identified entities and their types for subsequent use.
[0090] It should be noted that in this embodiment, there is no limitation on the specific process of preprocessing the initial query information. It can directly perform word segmentation preprocessing, or remove meaningless words after word segmentation. For example, in some embodiments, the initial query information is preprocessed. Specifically, the initial query information is segmented into multiple words through a word segmentation tool, each word is initialized, and meaningless words in each word are removed, such as "de", "le", etc.; finally, keywords are extracted based on each initialized word to generate a vocabulary sequence.
[0091] In this embodiment, the initial query information is preprocessed to generate a vocabulary sequence corresponding to the initial query information; entities are identified in the vocabulary sequence based on natural language processing techniques, realizing the identification of entities in the initial query information, so as to facilitate subsequent expansion and retrieval of the initial query information based on the entities.
[0092] Based on the above embodiments, in some embodiments, expanding the entities in the initial query information based on a knowledge graph to generate an extended query information and corresponding relevant knowledge graph information includes:
[0093] S111: Matching entities in the initial query information with nodes in the knowledge graph to determine other entities linked to the entity matching and entity relationships between the entities;
[0094] S112: Generate extended query information and related knowledge graph information based on the vocabulary sequence, entities in the initial query information, other entities linked to the entity matching, and entity relationships between entities.
[0095] In order to expand the initial query information, in this embodiment, the identified entities are specifically matched with nodes in the knowledge graph, and the potential relationships between entities in the query are analyzed, that is, the entities extracted from the initial query information are For input, For specific entities, find the related concepts and relationships of these entities in the knowledge graph It can be understood that the concepts related to an entity include other entities that are linked to the entity, and the relationship between the entity That is, the entity relationship between entities.
[0096] Further, based on the matching linked entities, queries are performed in the knowledge graph to obtain relevant information. The original initial query information is combined with the additional information obtained from the knowledge graph, specifically based on the word sequence. (i.e. initial query information), entities in the initial query information , other entities linked to entity matching, and entity relationships between entities , generate extended query information and related knowledge graph information. It is understood that the related knowledge graph information includes the following: linked entities, entities identified in the query and successfully matched with the knowledge graph; entity relationships, direct and indirect relationships between entities; related concepts, other concepts or entities related to the initial query information.
[0097] In this embodiment, the entities in the initial query information are matched with the nodes in the knowledge graph to determine other entities linked to the entity match and the entity relationships between the entities; based on the vocabulary sequence, the entities in the initial query information, other entities linked to the entity match and the entity relationships between the entities, the extended query information and related knowledge graph information are generated, thereby expanding the initial query information, supplementing the background information implicit in the initial query information, and enabling the system to better handle incomplete or ambiguous queries.
[0098] On the basis of the above embodiments, in some embodiments, keyword matching and semantic matching are performed on each text block based on the extended query information and the relevant knowledge graph information to obtain the keyword matching score and semantic matching score corresponding to each text block, including:
[0099] S121: performing keyword matching on each text block and the extended query information based on a preset algorithm to obtain a keyword matching score between each text block and the extended query information;
[0100] S122: performing semantic matching on each text block and the extended query information based on a preset model to obtain a first semantic matching score between each text block and the extended query information;
[0101] S123: Perform semantic matching on each text block and the initial query information according to relevant knowledge graph information to obtain a second semantic matching score between each text block and the initial query information.
[0102] In a specific implementation, when performing keyword search, keyword matching is performed on each text block and the extended query information based on a preset algorithm to obtain a keyword matching score between each text block and the extended query information. It should be noted that the preset algorithm is not limited in this embodiment, for example, it can be a BM25 algorithm or a TF-IDF algorithm.
[0103] When performing semantic retrieval, two layers of semantic matching are specifically performed in this embodiment: on the one hand, semantic matching is performed on each text block and the extended query information based on the preset model to obtain a first semantic matching score between each text block and the extended query information; on the other hand, semantic matching is performed on each text block and the initial query information based on the relevant knowledge graph information to obtain a second semantic matching score between each text block and the initial query information. It should be noted that there is no restriction on the preset model in this embodiment. The keyword matching and semantic matching processes are described in detail below in conjunction with specific embodiments:
[0104] (1) Keyword search;
[0105] Specifically, the BM25 algorithm is used to calculate the score between the extended query information and the text block. First, the vocabulary in the extended query information, the frequency of each vocabulary in the corresponding text block, the frequency of the extended query information, and the inverse document frequency are obtained; based on the vocabulary in the extended query information, the frequency of each vocabulary in the corresponding text block, the frequency of the extended query information, and the inverse document frequency, the text block and the extended query information are matched by keywords through the best matching algorithm to obtain the keyword matching score between the text block and the extended query information. The specific formula is as follows:
[0106] ;
[0107] in, is the keyword matching score, For a text block, To expand the query information, To expand the vocabulary in the query information, is the inverse document frequency, For vocabulary In the corresponding text block The word frequency in To expand the query information word frequency, , and It is an adjustable parameter.
[0108] (2) One level of semantic retrieval;
[0109] Specifically, an embedding model is used to vectorize the extended query information and each text block to obtain an extended query vector and each text block vector; the vector similarity between each text block vector and the extended query vector is calculated to obtain the first semantic matching score between each text block and the extended query information. When calculating text similarity, the cosine similarity formula can usually be used, and the formula is as follows:
[0110] ;
[0111] in, is the first semantic matching score, and are the vectors of two text fragments respectively.
[0112] (3) Second-level semantic retrieval;
[0113] Specifically, the relevant knowledge graph information is used to construct the similarity between the initial query information and the text block. The specific formula is as follows:
[0114] ;
[0115] in, is the second semantic matching score, The function integrates relevant knowledge graph information into text representation; For a text block, This is the initial query information.
[0116] In this embodiment, keyword matching is performed on each text block and the extended query information based on a preset algorithm to obtain a keyword matching score between each text block and the extended query information; semantic matching is performed on each text block and the extended query information based on a preset model to obtain a first semantic matching score between each text block and the extended query information; semantic matching is performed on each text block and the initial query information according to relevant knowledge graph information to obtain a second semantic matching score between each text block and the initial query information, thereby achieving multi-layer semantic matching of queries, enabling the system to understand query intent and document content more comprehensively, and to capture correlations at different levels, from literal matching to deep semantic understanding, thereby greatly improving the recall and accuracy of related documents.
[0117] On the basis of the above embodiments, in some embodiments, each keyword matching score and the corresponding semantic matching score are merged to obtain a final matching score corresponding to each text block, including:
[0118] S131: Obtain weight parameters corresponding to the keyword matching score, the first semantic matching score, and the second semantic matching score respectively.
[0119] S132: respectively determining the product of the keyword matching score, the first semantic matching score, and the second semantic matching score with the corresponding weight parameter, and summing up the products to obtain a final matching score corresponding to the text block.
[0120] Specifically, after obtaining the keyword matching score, the first semantic matching score, and the second semantic matching score, the multi-level matching scores need to be fused. The three-level matching scores are fused using a weighted method, as follows:
[0121] ;
[0122] in, is the final matching score, , and is an adjustable weight parameter, .
[0123] In this embodiment, weight parameters corresponding to the keyword matching score, the first semantic matching score and the second semantic matching score are obtained respectively, the product of the keyword matching score, the first semantic matching score and the second semantic matching score and the corresponding weight parameters is determined, and the products are added to obtain the final matching score corresponding to the text block, thereby realizing the fusion of multi-layer semantic matching results and improving the retrieval accuracy.
[0124] On the basis of the above embodiments, in some embodiments, a target text block set is screened out from each text block according to the final matching score corresponding to each text block, including:
[0125] S141: sorting the text blocks in descending order according to the final matching scores of the text blocks;
[0126] S142: generating a candidate text block set according to a preset number of text blocks with the highest final matching scores among the text blocks;
[0127] S143: Deleting text blocks whose final matching scores are lower than the minimum relevance threshold from the candidate text block set to obtain a target text block set.
[0128] Specifically, first, the text blocks are sorted in descending order according to their final matching scores to obtain a sequence of text blocks whose final matching scores are arranged from large to small. Further, a candidate text block set is generated based on a preset number of text blocks with the highest final matching scores in each text block. In this embodiment, there is no restriction on the preset number, for example, the range can be set to 5 to 10. Finally, the text blocks in the candidate text block set whose final matching scores are lower than the minimum relevance threshold are deleted to obtain a target text block set. In this embodiment, there is no restriction on the size of the minimum relevance threshold, which depends on the specific implementation situation. In this way, the screening of the target text block set is achieved.
[0129] Based on the above embodiments, in some embodiments, generating a final answer corresponding to the initial query information according to the target text block set includes:
[0130] S144: splicing the target text block set and the initial query information based on a preset format;
[0131] S145: Selecting a backend large language model, and selecting a corresponding small sample example for the backend large language model according to the query type of the initial query information;
[0132] S146: Input the concatenated target text block set and the initial query information into the backend large language model to output a final answer corresponding to the initial query information.
[0133] In order to retrieve the final answer, in this embodiment, the target text block set and the initial query information are spliced based on a preset format. In this embodiment, there is no restriction on the preset format. Further, a back-end large language model is selected, and a corresponding small sample example (few-shot) is selected for the back-end large language model according to the query type of the initial query information, so that the back-end large language model can adapt to the answer method of the user's query. Finally, the spliced target text block set and the initial query information are input into the back-end large language model, and the back-end large language model outputs the final answer corresponding to the initial query information, thereby realizing the retrieval and answer to the user's initial query information.
[0134] In order to iterate and self-correct the retrieval process, based on the above embodiments, in some embodiments, after generating the final answer corresponding to the initial query information according to the target text block set, the following is further included:
[0135] S15: Perform a preliminary evaluation on the final answer corresponding to the initial query information to determine whether the final answer is accurate; if the final answer is confirmed to be inaccurate, proceed to step S16;
[0136] S16: Adjust the structure of the final answer according to the matching degree between the final answer and the initial query information, or reselect the text block and perform the search again.
[0137] Specifically, after obtaining the final answer, the final answer corresponding to the initial query information can be further preliminarily evaluated to determine whether the final answer is accurate. Specifically, the accuracy and completeness of the final answer content and its relevance to the initial query information can be checked and determined through a built-in scoring function or a rule-based verification method. If the final answer is confirmed to be accurate, the evaluation ends.
[0138] If it is confirmed that the final answer is inaccurate, the structure of the final answer is adjusted according to the degree of match between the final answer and the initial query information, or the text block is reselected and searched again, thereby achieving search optimization and enabling the system to provide more comprehensive and accurate answers.
[0139] Based on the above embodiments, in some embodiments, after generating a final answer corresponding to the initial query information according to the target text block set, the method further includes:
[0140] S17: combining the initial query information, the extended query information and the final answer to generate retrieval sample information;
[0141] S18: Save the retrieved sample information to the database.
[0142] In the specific implementation, in order to improve the response speed and retrieval accuracy of the system, after generating the final answer corresponding to the initial query information, the initial query information, the extended query information and the final answer can be further combined to generate retrieval sample information; the retrieval sample information is saved to the database, so that when there is a new retrieval task in the future, the retrieval sample information in the database can be used as a few-shot, so that the back-end large language model can adapt to the way of answering user queries, thereby improving more accurate answer results.
[0143] In addition, in order to evaluate the performance of the RAG system, appropriate evaluation criteria can be further established, such as precision: measuring the proportion of correct answers in the retrieved documents; recall: measuring the proportion of correct answers in the retrieved documents; F1-score: combining the evaluation criteria of precision and recall, and taking the harmonic mean of the two; generation quality: evaluating whether the generated text is accurate, fluent, logical, etc.
[0144] When evaluating the RAG system, manual evaluation can be used, specifically, the retrieval results and generated texts are manually evaluated to determine their accuracy and quality. Automated tools can also be used to evaluate the retrieval results and generated texts, such as using ROUGE and other indicators to evaluate the quality of the generated text. At the same time, user feedback on the retrieval results and generated texts is collected to understand their satisfaction and improvement directions. Further, based on the evaluation results, the optimization direction of the RAG system can be determined, including data quality improvement: if the retrieval results are inaccurate or the quality of the generated text is not high, it may be due to data quality problems, and data cleaning, entity resolution and other operations are required to improve data quality; retriever improvement: if the retrieval results are inaccurate, it may be due to the problem of the retriever, and query transformation, hybrid retrieval and other operations are required to improve the retrieval accuracy; feedback mechanism optimization: if the quality of the generated text is not high, it may be due to the problem of the feedback mechanism, and rerank, back prompt and other operations are required to improve the generation quality; data enhancement: if the retrieval results are inaccurate or the quality of the generated text is not high, it may be due to insufficient data diversity, and data synthesis and other operations are required to increase data diversity.
[0145] Furthermore, when optimizing the RAG system, you can optimize step by step: perform optimization operations step by step, and make only a small change each time to ensure the controllability of the optimization effect. Furthermore, before performing the optimization operation, perform experimental verification to determine the effect of the optimization operation. During the optimization process, collect user feedback in a timely manner to understand the optimization effect and make adjustments.
[0146] In this way, by establishing effective evaluation criteria, adopting appropriate evaluation methods, and determining optimization directions and strategies, the retrieval accuracy and generation quality of the RAG system can be continuously improved, thereby improving the overall performance.
[0147] In the above-mentioned embodiment, the retrieval method based on retrieval enhancement generation is described in detail. The present invention also provides an embodiment corresponding to the retrieval device based on retrieval enhancement generation.
[0148] Figure 2 A schematic diagram of a search device based on search enhancement generation provided by an embodiment of the present invention. Figure 2 As shown, the device comprises:
[0149] The identification module 10 is used to obtain the initial query information input by the user and identify the entities in the initial query information;
[0150] An expansion module 11 is used to expand the entities in the initial query information based on the knowledge graph to generate the expanded query information and the corresponding related knowledge graph information; wherein the related knowledge graph information at least includes other entities linked to the entity matching, entity relationships between entities, and concepts related to the initial query information;
[0151] A matching module 12 is used to perform keyword matching and semantic matching on each text block based on the extended query information and the relevant knowledge graph information to obtain a keyword matching score and a semantic matching score corresponding to each text block;
[0152] A fusion module 13 is used to fuse each keyword matching score with the corresponding semantic matching score to obtain a final matching score corresponding to each text block;
[0153] The answer module 14 is used to screen out a target text block set from each text block according to the final matching score corresponding to each text block, and generate a final answer corresponding to the initial query information according to the target text block set.
[0154] In some embodiments, the identification module 10 includes:
[0155] A preprocessing submodule, used for preprocessing the initial query information to generate a vocabulary sequence corresponding to the initial query information;
[0156] The recognition submodule is used to recognize entities in the vocabulary sequence based on natural language processing technology.
[0157] In some embodiments, the preprocessing submodule includes:
[0158] The word segmentation submodule is used to segment the initial query information into multiple words through a word segmentation tool;
[0159] The initialization submodule is used to initialize each word and remove meaningless words from each word;
[0160] The extraction submodule is used to extract keywords based on each word after initialization processing to generate a word sequence.
[0161] In some embodiments, the expansion module 11 includes:
[0162] The matching submodule is used to match the entities in the initial query information with the nodes in the knowledge graph to determine other entities linked to the entity matching and the entity relationships between the entities;
[0163] The generation submodule is used to generate extended query information and related knowledge graph information based on the vocabulary sequence, entities in the initial query information, other entities linked to the entity matching, and entity relationships between entities.
[0164] In some embodiments, the matching module 12 includes:
[0165] The keyword matching submodule is used to perform keyword matching on each text block and the extended query information based on a preset algorithm to obtain a keyword matching score between each text block and the extended query information;
[0166] A first semantic matching submodule, configured to perform semantic matching on each text block and the extended query information based on a preset model to obtain a first semantic matching score between each text block and the extended query information;
[0167] The second semantic matching submodule is used to perform semantic matching on each text block and the initial query information according to relevant knowledge graph information to obtain a second semantic matching score between each text block and the initial query information.
[0168] In some embodiments, the keyword matching submodule includes:
[0169] The first acquisition submodule is used to acquire the vocabulary in the extended query information, the frequency of each vocabulary in the corresponding text block, the frequency of the extended query information and the inverse document frequency;
[0170] The first processing submodule is used to perform keyword matching between the text block and the extended query information through the best matching algorithm according to the words in the extended query information, the word frequency of each word in the corresponding text block, the word frequency of the extended query information and the inverse document frequency, so as to obtain the keyword matching score between the text block and the extended query information.
[0171] In some embodiments, the first semantic matching submodule includes:
[0172] The second processing submodule is used to vectorize the extended query information and each text block through the embedding model to obtain an extended query vector and a vector of each text block;
[0173] The third processing submodule is used to calculate the vector similarity between each text block vector and the extended query vector respectively, so as to obtain a first semantic matching score between each text block and the extended query information.
[0174] In some embodiments, the fusion module 13 includes:
[0175] A matching score acquisition submodule, used to respectively acquire weight parameters corresponding to the keyword matching score, the first semantic matching score, and the second semantic matching score;
[0176] The final matching score calculation submodule is used to respectively determine the product of the keyword matching score, the first semantic matching score and the second semantic matching score with the corresponding weight parameter, and add up the products to obtain the final matching score corresponding to the text block.
[0177] In some embodiments, the answer module 14 includes:
[0178] A sorting submodule, used to sort each text block in descending order according to the final matching score of each text block;
[0179] A candidate text block set generation submodule is used to generate a candidate text block set according to a preset number of text blocks with the highest final matching scores among the text blocks;
[0180] The deletion submodule is used to delete the text blocks whose final matching scores are lower than the minimum relevance threshold in the candidate text block set to obtain the target text block set.
[0181] In some embodiments, the answer module 14 includes:
[0182] A splicing submodule, used for splicing a target text block set and initial query information based on a preset format;
[0183] A selection submodule is used to select a backend large language model and select a corresponding small sample example for the backend large language model according to the query type of the initial query information;
[0184] The input submodule is used to input the concatenated target text block set and the initial query information into the backend large language model to output the final answer corresponding to the initial query information.
[0185] In some embodiments, it also includes:
[0186] The evaluation submodule is used to make a preliminary evaluation of the final answer corresponding to the initial query information to determine whether the final answer is accurate; if the final answer is confirmed to be inaccurate, the adjustment submodule is triggered;
[0187] The adjustment submodule is used to adjust the structure of the final answer according to the matching degree between the final answer and the initial query information, or to reselect the text block and retrieve it again.
[0188] In some embodiments, it also includes:
[0189] A combination submodule is used to combine the initial query information, the extended query information and the final answer to generate retrieval sample information;
[0190] The saving submodule is used to save the retrieved sample information to the database.
[0191] Since the embodiments of the apparatus part correspond to the embodiments of the method part, please refer to the description of the embodiments of the method part for the embodiments of the apparatus part, which will not be repeated here.
[0192] In addition, the present invention also provides a computer program product, including a computer program or instructions, which, when executed by a processor, implements the steps of the above-mentioned retrieval method based on retrieval enhancement generation.
[0193] Figure 3 A schematic diagram of a search device based on search enhancement generation provided by an embodiment of the present invention. Figure 3 As shown, the retrieval device generated based on retrieval enhancement includes:
[0194] A memory 20, used for storing computer programs;
[0195] The processor 21 is used to implement the steps of the retrieval method based on retrieval enhancement generation mentioned in the above embodiment when executing the computer program.
[0196] The retrieval device based on retrieval enhancement generation provided in this embodiment may include but is not limited to a smart phone, a tablet computer, a laptop computer or a desktop computer.
[0197] Among them, the processor 21 may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor 21 can be implemented in at least one hardware form of a digital signal processor (DSP), a field programmable gate array (FPGA), and a programmable logic array (PLA). The processor 21 may also include a main processor and a coprocessor. The main processor is a processor for processing data in an awake state, also known as a central processing unit (CPU); the coprocessor is a low-power processor for processing data in a standby state. In some embodiments, the processor 21 may be integrated with a graphics processing unit (GPU), and the GPU is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, the processor 21 may also include an artificial intelligence (AI) processor, which is used to process computing operations related to machine learning.
[0198] The memory 20 may include one or more computer-readable storage media, which may be non-transitory. The memory 20 may also include a high-speed random access memory, and a non-volatile memory, such as one or more disk storage devices, flash memory storage devices. In this embodiment, the memory 20 is at least used to store the following computer program 201, wherein, after the computer program is loaded and executed by the processor 21, it can implement the relevant steps of the retrieval method based on retrieval enhancement generation disclosed in any of the aforementioned embodiments. In addition, the resources stored in the memory 20 may also include an operating system 202 and data 203, etc., and the storage method may be temporary storage or permanent storage. Among them, the operating system 202 may include Windows, Unix, Linux, etc. Data 203 may include but is not limited to data involved in the retrieval method based on retrieval enhancement generation.
[0199] In some embodiments, the retrieval device based on retrieval enhancement generation may also include a display screen 22 , an input and output interface 23 , a communication interface 24 , a power supply 25 , and a communication bus 26 .
[0200] Those skilled in the art will understand that Figure 3 The structure shown in the figure does not constitute a limitation on the retrieval device based on retrieval enhancement generation, and may include more or fewer components than those shown in the figure.
[0201] Finally, the present invention also provides an embodiment corresponding to a computer-readable storage medium. The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps recorded in the above method embodiment are implemented.
[0202] It is understandable that if the method in the above embodiment is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium to execute all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk, etc. Various media that can store program codes.
[0203] The above is a detailed introduction to a retrieval method, product, device and medium based on retrieval enhancement generation provided by the present invention. The various embodiments in the specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same and similar parts between the embodiments can refer to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can refer to the method part description. It should be pointed out that for ordinary technicians in this technical field, without departing from the principle of the present invention, several improvements and modifications can be made to the present invention, and these improvements and modifications also fall within the scope of protection of the present invention.
[0204] It should also be noted that, in this specification, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "comprise", "include" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the statement "comprises a ..." does not exclude the presence of other identical elements in the process, method, article or device including the element.
Claims
1. A retrieval method based on retrieval enhancement generation, characterized in that: include: Obtaining initial query information input by a user and identifying entities in the initial query information; Expanding entities in the initial query information based on the knowledge graph to generate expanded query information and corresponding related knowledge graph information; wherein the related knowledge graph information at least includes other entities linked to the entity matching, entity relationships between entities, and concepts related to the initial query information; Based on the extended query information and the relevant knowledge graph information, keyword matching and semantic matching are performed on each text block to obtain the keyword matching score and semantic matching score corresponding to each text block; The keyword matching scores and the corresponding semantic matching scores are merged to obtain the final matching scores of each text block; A target text block set is screened out from each text block according to the final matching score corresponding to each text block, and a final answer corresponding to the initial query information is generated according to the target text block set.
2. The retrieval method based on retrieval enhancement generation according to claim 1, characterized in that: The obtaining of initial query information input by the user and identifying entities in the initial query information includes: Preprocessing the initial query information to generate a vocabulary sequence corresponding to the initial query information; Identify entities in a word sequence based on natural language processing techniques.
3. The retrieval method based on retrieval enhancement generation according to claim 2, characterized in that: The preprocessing of the initial query information includes: Use a word segmentation tool to split the initial query information into multiple words; Initialize each word and remove meaningless words from each word; Keywords are extracted based on each of the initialized words to generate a word sequence.
4. The retrieval method based on retrieval enhancement generation according to claim 2, characterized in that: The entity in the initial query information is expanded based on the knowledge graph to generate the expanded query information and the corresponding related knowledge graph information, including: Match the entities in the initial query information with the nodes in the knowledge graph to determine other entities linked to the entity matching and the entity relationships between the entities; According to the vocabulary sequence, entities in the initial query information, other entities linked to the entity matching, and entity relationships between entities, the extended query information and related knowledge graph information are generated.
5. The retrieval method based on retrieval enhancement generation according to claim 1, characterized in that: The keyword matching and semantic matching of each text block are performed based on the extended query information and the relevant knowledge graph information to obtain the keyword matching score and semantic matching score corresponding to each text block, including: Based on a preset algorithm, keyword matching is performed on each text block and the extended query information to obtain a keyword matching score between each text block and the extended query information; Based on a preset model, semantic matching is performed on each text block and the extended query information respectively to obtain a first semantic matching score between each text block and the extended query information; Semantic matching is performed on each text block and the initial query information according to relevant knowledge graph information to obtain a second semantic matching score between each text block and the initial query information.
6. The retrieval method based on retrieval enhancement generation according to claim 5, characterized in that: The keyword matching of each text block with the extended query information based on a preset algorithm to obtain a keyword matching score between each text block and the extended query information includes: Obtaining the words in the extended query information, the frequency of each word in the corresponding text block, the extended query information word frequency and the inverse document frequency; According to the words in the extended query information, the frequency of each word in the corresponding text block, the word frequency of the extended query information and the inverse document frequency, the text block and the extended query information are matched with keywords through the best matching algorithm to obtain the keyword matching score between the text block and the extended query information.
7. The retrieval method based on retrieval enhancement generation according to claim 5, characterized in that: The performing semantic matching on each text block and the extended query information based on the preset model to obtain a first semantic matching score between each text block and the extended query information includes: The extended query information and each text block are vectorized through the embedding model to obtain an extended query vector and a vector of each text block; The vector similarity between each text block vector and the extended query vector is calculated respectively to obtain a first semantic matching score between each text block and the extended query information.
8. The retrieval method based on retrieval enhancement generation according to claim 5, characterized in that: The method of fusing the keyword matching scores and the corresponding semantic matching scores to obtain the final matching scores of the text blocks includes: Obtain weight parameters corresponding to the keyword matching score, the first semantic matching score, and the second semantic matching score respectively; The products of the keyword matching score, the first semantic matching score, and the second semantic matching score and the corresponding weight parameters are determined respectively, and the products are added together to obtain a final matching score corresponding to the text block.
9. The retrieval method based on retrieval enhancement generation according to claim 1, characterized in that: The step of selecting a target text block set from each text block according to the final matching score corresponding to each text block includes: Sort each text block in descending order according to the final matching score of each text block; Generate a candidate text block set according to a preset number of text blocks with the highest final matching scores among the text blocks; The text blocks whose final matching scores are lower than the minimum relevance threshold in the candidate text block set are deleted to obtain the target text block set.
10. The retrieval method based on retrieval enhancement generation according to claim 1, characterized in that: Generating a final answer corresponding to the initial query information according to the target text block set includes: splicing the target text block set and the initial query information based on a preset format; Select a backend large language model, and select a corresponding small sample example for the backend large language model according to the query type of the initial query information; The concatenated target text block set and the initial query information are input into the backend large language model to output the final answer corresponding to the initial query information.
11. The retrieval method based on retrieval enhancement generation according to claim 1, characterized in that: After generating the final answer corresponding to the initial query information according to the target text block set, the method further includes: Conduct a preliminary evaluation of the final answer corresponding to the initial query information to determine whether the final answer is accurate; If it is confirmed that the final answer is inaccurate, the structure of the final answer is adjusted according to the matching degree between the final answer and the initial query information, or the text block is reselected and the search is performed again.
12. The retrieval method based on retrieval enhancement generation according to any one of claims 1 to 11, characterized in that: After generating the final answer corresponding to the initial query information according to the target text block set, the method further includes: Combine the initial query information, the expanded query information and the final answer to generate retrieval sample information; Save the retrieved sample information to the database.
13. A computer program product comprising a computer program or instructions, characterized in that When the computer program or instruction is executed by a processor, the steps of the search method based on search enhancement generation described in any one of claims 1 to 12 are implemented.
14. A retrieval device based on retrieval enhancement generation, characterized in that: include: Memory for storing computer programs; A processor, configured to implement the steps of the retrieval method based on retrieval enhancement generation as described in any one of claims 1 to 12 when executing the computer program.
15. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the retrieval method based on retrieval enhancement generation according to any one of claims 1 to 12 are implemented.
Citation Information
Patent Citations
Intelligent retrieval method and system based on exploration and development knowledge graph and storage medium
CN116401350A
Vector knowledge base-based large-model question-answer dialogue method and system and storage medium
CN118484526A
Large model output generation method based on fusion information, medium and equipment
CN118606465A
Archive information retrieval method and device, computer equipment and readable storage medium
CN118708790A
Knowledge retrieval enhancement-based characteristic agricultural product standardized file content generation method
CN118779407A
Cited By
Scientific and technological intelligence information retrieval result reordering method and equipment
CN119988585A
Data search enhanced rearrangement method, system and equipment
CN120067311A
Retrieval enhancement generation method and system oriented to chemical corpus data
CN120104741A
Method and device for enhancing embedded vector features based on semantic weight model
CN120106085A
Information generation method, electronic device, storage medium and program product
CN120492689A
