Information retrieval system and method based on multi-path recall, terminal, medium and product

Abstract

The invention provides an information retrieval system and method based on multi-path recall, a terminal, a medium and a product. The system comprises a data layer used for storing and managing various types of data; the recall layer is used for performing multi-path recall retrieval on the input to-be-retrieved information through a built-in multi-path recall retrieval module based on the data stored in the data layer to obtain a recall result; the fusion layer is used for fusing the recall results by using a pre-configured fusion mode; and the output layer is used for screening the fused result and outputting the screened result as a retrieval result of the information to be retrieved. The multi-path recall retrieval module disclosed by the invention combines the advantages of multiple retrieval modes. In addition, through the multi-path recall module and fusion processing, the accuracy of the retrieval result is improved.

Description

Technical Field

[0001] This application relates to the field of data analysis technology, and particularly to an information retrieval system, method, terminal, medium and product based on multi-channel recall. Background Art

[0002] Traditional information retrieval systems are mainly based on keyword retrieval, which performs poorly in dealing with complex semantics and context information. For example, in a question-and-answer system, the user's question has a certain semantic ambiguity, and it is difficult to accurately find relevant answers simply by keyword matching. The recalled results may deviate greatly from the user's true needs.

[0003] Moreover, current recall methods, such as Retrieval-Augmented Generation (RAG), full-text retrieval, Graph Retrieval-Augmented Generation (GraphRAG), and Agentic Retrieval-Augmented Generation (AgenticRAG), etc., all have limitations. Traditional RAG has limited capabilities in dealing with complex knowledge structures and relationships; full-text retrieval has weak semantic understanding and context awareness capabilities; GraphRAG has high costs in constructing and maintaining graph structures and has poor processing effects on some unstructured data; although AgenticRAG can perform dynamic retrieval according to tasks, its adaptability in multi-task and multi-scenario needs to be further improved.

[0004] With the explosive growth of Internet information, the requirements for information retrieval are becoming more and more diverse and complex. Users not only hope to quickly and accurately find relevant information, but also hope that the system can provide personalized services according to different scenarios and requirements. A single recall method cannot meet the needs of users. Summary of the Invention

[0005] In view of the above-mentioned disadvantages of the prior art, the purpose of this application is to provide an information retrieval system, method, terminal, medium and product based on multi-channel recall, which is used to solve the problem that a single recall method cannot cope with diverse retrieval requirements.

[0006] To achieve the above purpose and other related purposes, the first aspect of this application provides an information retrieval system based on multi-channel recall, including: a data layer for storing and managing various types of data; a recall layer for performing multi-channel recall retrieval on the input information to be retrieved based on the data stored in the data layer through a built-in multi-channel recall retrieval module to obtain recall results; a fusion layer for fusing the recall results using a pre-configured fusion method; and an output layer for screening the fused results and outputting the screened results as the retrieval results of the information to be retrieved.

[0007] In some embodiments of the first aspect of the present application, the multi-channel recall retrieval module is an expandable module; the preset multi-channel recall retrieval module includes: a RAG unit, a full-text retrieval unit, and a GraphRAG unit.

[0008] In some embodiments of the first aspect of the present application, based on the data stored in the data layer, through the built-in multi-channel recall retrieval module, multi-channel recall retrieval is performed on the input information to be retrieved, and a recall result is obtained, including: based on the data stored in the data layer, each unit of the multi-channel recall retrieval module is made to perform retrieval on the input information to be retrieved respectively, and a preliminary recall result of each unit is obtained; the preliminary recall results of each unit are respectively roughly ranked to obtain a final recall result.

[0009] In some embodiments of the first aspect of the present application, the fusion method is configured as any one or a combination of two of the weighted fusion method and the Rerank fusion method.

[0010] In some embodiments of the first aspect of the present application, when the fusion method is configured as a combination of the weighted fusion method and the Rerank fusion method, the method for fusing the recall results includes: setting the weights of each unit in the multi-channel recall retrieval module; based on the set weights, using the weighted fusion method to fuse and rank the recall results to obtain candidate results; inputting the candidate results into the Rerank model for fine ranking to obtain a final fusion result.

[0011] In some embodiments of the first aspect of the present application, multiple types of data include: text data and graph data; among them, the text data includes: document data, article data, question-and-answer pair data, and knowledge base data; the graph data includes: knowledge graph data.

[0012] To achieve the above object and other related objects, the second aspect of the present application provides an information retrieval method based on multi-channel recall, which is applied to an information retrieval system based on multi-channel recall. The information retrieval system includes: a data layer, a recall layer, a fusion layer, and an output layer. The method includes: the recall layer performs multi-channel recall retrieval on the input information to be retrieved through the built-in multi-channel recall retrieval module based on the data stored in the data layer to obtain a recall result; the fusion layer uses a pre-configured fusion method to fuse the recall result; the output layer screens the fused result and outputs the screened result as the retrieval result of the information to be retrieved.

[0013] To achieve the above object and other related objects, the third aspect of the present application provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the information retrieval method based on multi-channel recall is implemented.

[0014] For achieving the above object and other related objects, a fourth aspect of the present application provides a computer program product, which includes computer program code. When the computer program code runs on a computer, the computer is enabled to implement the information retrieval method based on multi-channel recall.

[0015] For achieving the above object and other related objects, a fifth aspect of the present application provides an electronic terminal, including a memory, a processor, and a computer program stored on the memory; the processor executes the computer program to implement the information retrieval method based on multi-channel recall.

[0016] As described above, the information retrieval system, method, terminal, medium, and product based on multi-channel recall of the present application have the following beneficial effects:

[0017] The multi-channel recall retrieval module of the present application combines the advantages of various retrieval methods. Moreover, the present application improves the accuracy of retrieval results through the multi-channel recall module and fusion processing. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 It shows a schematic block diagram of the information retrieval system based on multi-channel recall in an embodiment of the present application.

[0019] Figure 2 It shows a specific structural schematic diagram of the information retrieval system based on multi-channel recall in an embodiment of the present application.

[0020] Figure 3 It shows a schematic flowchart of the information retrieval method based on multi-channel recall in an embodiment of the present application.

[0021] Figure 4 It shows a structural schematic diagram of an electronic terminal in an embodiment of the present application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0022] The following uses specific specific examples to illustrate the embodiments of the present application. Those skilled in the art can easily understand other advantages and effects of the present application from the content disclosed in this specification. The present application can also be implemented or applied through other different specific embodiments, and various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present application. It should be noted that, without conflict, the following embodiments and the features in the embodiments can be combined with each other.

[0023] In the embodiments of the present application, terms such as "first" and "second" are used to distinguish identical or similar items with basically the same functions and effects. Those skilled in the art can understand that terms such as "first" and "second" do not limit the quantity and execution order, and "first", "second", etc. do not necessarily mean different.

[0024] It should be noted that in the embodiments of the present application, words such as "exemplary" or "for example" indicate examples, illustrations or explanations. Any embodiment or design solution described as "exemplary" or "for example" in the present application should not be construed as being more preferred or having more advantages than other embodiments or design solutions. Rather, the use of words such as "exemplary" or "for example" is intended to present relevant concepts in a specific manner.

[0025] In the embodiments of the present application, "at least one" means one or more, and "a plurality" means two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B may represent: A exists alone, A and B exist simultaneously, and B exists alone, where A and B may be singular or plural. The character " / " generally indicates that the associated objects before and after are in an "or" relationship. "At least one (item)" or similar expressions refer to any combination of these items, including any combination of single item (s) or plural item (s). For example, at least one (item) of a, b or c may represent: a, b, c, a - b, a - c, b - c or a - b - c, where a, b, c may be single or multiple.

[0026] For ease of understanding the embodiments of the present application, first in combination with Figure 1 detailed description. Figure 1 FIG. shows a schematic structural diagram of an information retrieval system based on multi - path recall in the embodiments of the present invention. The multi - path recall information retrieval system 1 in this embodiment mainly includes:

[0027] A data layer 11 for storing and managing various types of data;

[0028] A recall layer 12 for performing multi - path recall retrieval on the input information to be retrieved based on the data stored in the data layer through a built - in multi - path recall retrieval module to obtain a recall result;

[0029] A fusion layer 13 for fusing the recall results using a pre - configured fusion method;

[0030] An output layer 14 for screening the fused results and outputting the screened results as the retrieval results of the information to be retrieved.

[0031] It should be understood that the division of layers in the embodiments of this application is illustrative, merely a logical function division, and there may be other division methods in actual implementation. Additionally, in each embodiment of this application, each functional layer can be integrated in a processor, exist separately physically, or two or more modules can be integrated in one layer. The above-integrated layer can be implemented in the form of hardware or in the form of software function modules.

[0032] In one embodiment, the data layer is responsible for storing and managing various types of data, including but not limited to text data, graph data, etc. Text data includes but is not limited to documents, articles, question-and-answer pairs, knowledge bases, etc. Graph data includes but is not limited to knowledge graphs, social networks, etc. Data management includes but is not limited to data deletion, data addition, etc. The data layer adopts a distributed storage system, such as Miluvs, ElasticSearch, NebulaGraph, etc., to improve the storage security and read-write performance of data. It should be understood that the data stored in the data layer serves as the data source for retrieval by the recall layer. The data layer supports multiple types of data sources, including text data, graph data, etc., and can flexibly select and configure data sources according to different application scenarios and requirements.

[0033] In one embodiment, the multi-way recall retrieval module is an expandable module, mainly including: a RAG unit, a full-text retrieval unit, and a GraphRAG unit. It should be noted that each unit of the multi-way recall retrieval module can be developed and maintained independently, and those skilled in the art can add new units to the multi-way recall retrieval module according to requirements, such as image retrieval, voice retrieval, etc., to improve the recall ability of the multi-way recall retrieval module.

[0034] In one embodiment, based on the data stored in the data layer, through the internally provided multi-way recall retrieval module, the input information to be retrieved is subjected to multi-way recall retrieval to obtain a recall result, including: based on the data stored in the data layer, each unit of the multi-way recall retrieval module is made to retrieve the input information to be retrieved respectively to obtain the preliminary recall results of each unit; the preliminary recall results of each unit are respectively subjected to rough ranking to obtain the final recall result.

[0035] Specifically, as Figure 2 shown, the multi-way recall retrieval module mainly includes: a RAG unit, a full-text retrieval unit, and a GraphRAG unit. Based on the data stored in the data layer, the RAG unit, the full-text retrieval unit, and the GraphRAG unit are respectively used to retrieve the input information to be retrieved to obtain the preliminary recall results of the RAG unit, the full-text retrieval unit, and the GraphRAG unit; the preliminary recall results of the RAG unit, the full-text retrieval unit, and the GraphRAG unit are respectively subjected to rough ranking to obtain the final recall result.

[0036] The RAG unit adopts the RAG (Retrieval-Augmented Generation) method. The specific process includes: encoding the input information to be retrieved, converting it into vector data, and then performing similarity matching in the data of the data layer to obtain the document most relevant to the information to be retrieved. The similarity matching can adopt methods such as cosine similarity and Euclidean distance.

[0037] Cosine similarity is a commonly used method to measure the cosine value of the angle between two vectors, which can measure the similarity degree of two vectors in terms of direction. For the vector q corresponding to the input information to be retrieved and the document vector di, the calculation formula of their cosine similarity is referred to as Formula 1 below:

[0038]

[0039] Among them, q·di represents the dot product of the two vectors, and ‖q‖ and ‖di‖ represent the L2 norms of the vectors q and di respectively. The value range of cosine similarity is between [-1, 1], and the closer the value is to 1, the more similar the two vectors are.

[0040] Euclidean distance refers to the straight-line distance between two points in Euclidean space, which can measure the spatial distance between two vectors. For the vector q of the input information to be retrieved and the document vector di, the calculation formula of their Euclidean distance is referred to as Formula 2 below:

[0041]

[0042] Among them, q j and di j represent the j-th components of the vectors q and di respectively. The smaller the value of Euclidean distance, the closer the two vectors are.

[0043] In this embodiment, calculate the similarity score between the vector q of the input information to be retrieved and each document vector in the data layer, then sort the documents according to the similarity score, and input the k documents with the highest similarity score and the information to be retrieved into the large model for inductive summary to generate relevant answers (preliminary recall results).

[0044] The full-text retrieval unit adopts the full-text retrieval method. The main process of the full-text retrieval method includes:

[0045] In the full-text retrieval unit, the inverted index technology is the core foundation, which can efficiently construct an index for text data and provide strong support for subsequent rapid retrieval.

[0046] Inverted index, also known as reverse index. First, word segmentation is performed. Word segmentation is to split a continuous text sequence into individual lexical units. Common word segmentation algorithms include rule-based word segmentation algorithms and statistics-based word segmentation algorithms. Rule-based word segmentation algorithms, such as the maximum matching method, can be further divided into forward maximum matching method, reverse maximum matching method, and bidirectional maximum matching method. Taking the forward maximum matching method as an example, it starts from the left side of the text, uses the longest word length in the dictionary as the window, and searches in the dictionary to see if there is a vocabulary within this window. If there is, it is taken as a word segmentation result, and then the window moves to the right to continue the matching; if not, the window length is reduced by 1 and the search continues. Statistics-based word segmentation algorithms, such as Hidden Markov Model (HMM) and Conditional Random Field (CRF), learn from a large amount of text data, statistically analyze the probability of vocabulary appearance and the context relationship, so as to achieve more accurate word segmentation.

[0047] After the word segmentation operation, stemming and stop word filtering are performed on each vocabulary. Stemming is to restore the vocabulary to its basic form.

[0048] Then, an inverted index table is constructed. The inverted index table mainly consists of two parts: the vocabulary table and the posting list. The vocabulary table records all the unique vocabulary that has appeared, and each vocabulary corresponds to a posting list. The posting list records in which documents the vocabulary appears and the specific location information in the document.

[0049] When the user inputs the information to be retrieved, the module will perform the same preprocessing operations on the information to be retrieved, including word segmentation, stemming, and stop word filtering, and convert the information to be retrieved into a series of keywords. Then, the module searches in the inverted index according to these keywords. For each keyword, the module will find the corresponding posting list in the vocabulary table and obtain the document set containing this keyword.

[0050] After finding the relevant documents, the module needs to sort these documents according to the relevance of the documents. Common document relevance ranking algorithms include TF-IDF (Term Frequency-Inverse Document Frequency) algorithm and BM25 algorithm.

[0051] TF-IDF algorithm: The TF-IDF algorithm comprehensively considers the term frequency (TF) of a vocabulary in a document and the inverse document frequency (IDF). The term frequency (TF) represents the frequency of a vocabulary appearing in a document, and the calculation formula is Formula 3 as follows:

[0052]

[0053] Among them, t represents the information to be retrieved, and d represents the document.

[0054] The inverse document frequency (IDF) represents the general importance of a vocabulary, and the calculation formula is Formula 4 as follows:

[0055]

[0056] where N is the total number of documents, and n t is the number of documents containing the information to be retrieved.

[0057] The TF-IDF value is the product of the term frequency and the inverse document frequency. For document d and query keyword t, the calculation formula for its TF-IDF value is Formula 5 as follows:

[0058] TF-IDF t,d = TF t,d × IDF t ; (Formula 5)

[0059] The relevance score of a document can be obtained by summing the TF-IDF values of all keywords in the query. The higher the score, the stronger the relevance of the document to the query.

[0060] The BM25 algorithm is an improvement of the TF-IDF algorithm, which takes into account the influence of document length on relevance. For document d and query keyword t, the calculation formula for the BM25 score BM25 t,d is Formula 6 as follows:

[0061]

[0062] where k1 and b are both variables. k1 is used to control the saturation degree of TF, and b is used to control the document length normalization strength. L d is the document length, and L avg is the average length of all documents.

[0063] The system will sort the documents according to these relevance scores and take the top N documents with high scores as the preliminary recall results of the full-text retrieval unit. N is any positive integer.

[0064] GraphRAG Unit: The GraphRAG unit adopts GraphRAG. The core innovation of GraphRAG lies in the deep integration of graph computing theory and the RAG framework, achieving non-linear association and reasoning of information through the topological structure of the knowledge graph. Its advantages include: Dynamic graph construction: Entity-relationship extraction: Using LLM with prompt capabilities to extract triples from unstructured text and resolve entity ambiguities. Graph storage optimization: Using an attribute graph database (such as NebulaGraph) to store node attributes (entity types, abstract texts) and edge attributes (relationship strengths, source documents), supporting efficient multi-hop queries. Graph index for accelerating retrieval: Hybrid index strategy: Establishing an inverted index for high-frequency entities and a graph embedding index for community structures simultaneously. Caching hot subgraphs: Pre-generating Embeddings for subgraphs involved in high-frequency queries and caching them to reduce the overhead of real-time traversal calculations. Co-optimization of multi-hop reasoning and community detection: GraphRAG achieves precise context awareness through graph algorithms, breaking through the local retrieval limitations of traditional RAG and solving the global problem recall that traditional rag cannot solve. Application of community detection algorithms: Community partitioning: Using graph community clustering algorithms to divide the graph into semantically tight communities, and generating summary vectors for each community through LLM. Hierarchical community summaries and parallelized queries: The hierarchical summary mechanism of GraphRAG significantly improves the efficiency of complex queries.

[0065] Parallelized query process: Query routing: "Decompose" the input information to be retrieved into sub-queries, distribute them to relevant communities, and use the attention mechanism to dynamically weight the summary contributions of each community to generate the final retrieval context.

[0066] In one embodiment, the rough ranking process includes: respectively ranking the preliminary recall results of the RAG unit, the full-text retrieval unit, and the GraphRAG unit through re-ranking models such as deep learning-based re-ranking models and list-style re-ranking models to obtain the final recall results of the RAG unit, the full-text retrieval unit, and the GraphRAG unit.

[0067] In one embodiment, the fusion method is configured as a weighted fusion method, or a Rerank fusion method, or a combination of the weighted fusion method and the Rerank fusion method.

[0068] It should be noted that in an information retrieval system, the fusion layer plays a crucial role. Its core mission is to deeply fuse and optimize the results of multiple retrievals. Multiple retrievals usually cover various different retrieval strategies such as traditional RAG (Retrieval-Augmented Generation), full-text retrieval, GraphRAG (Graph Retrieval-Augmented Generation), etc. Each strategy is based on its unique algorithms and data structures, searching the knowledge base from different dimensions and perspectives to retrieve information relevant to the user's query. However, these retrieval results often have diversity and differences, and the fusion layer is required to perform effective integration to extract the content that best meets the user's needs.

[0069] Specifically, the weighted fusion method assigns specific weights to each unit, comprehensively considering the retrieval results of each module, thereby achieving the fusion of results. The assignment of weights is not random, but based on the performance evaluation of each unit on historical data, as well as the specific characteristics and requirements of the current query.

[0070] When determining the weights, multiple factors need to be considered. For example, for queries with high requirements for semantic understanding, RAG and GraphRAG may perform better because they can utilize pre-trained language models and context information for semantic matching and reasoning. Therefore, higher weights can be assigned to these two units. On the contrary, for queries with high requirements for keyword matching, the full-text retrieval module may be more applicable because it is based on technologies such as inverted indexes and can quickly and accurately locate documents containing keywords. At this time, a higher weight can be assigned to the full-text retrieval unit. In addition, indicators such as the recall rate, accuracy rate, and F1 value of the retrieval module can also be considered, and machine learning algorithms (such as linear regression, logistic regression, etc.) can be used to dynamically adjust the weights to adapt to different types of queries. The weighted fusion method can be adjusted according to different application scenarios and requirements. Through experiments and evaluations, the weights of each retrieval module can be optimized to improve the retrieval effect of the system.

[0071] Suppose there are n retrieval units, and the retrieval result of each unit is represented by R i and the corresponding weight is i, where For each document d in each retrieval result R i , the comprehensive score S(d) of the document in all retrieval modules can be calculated according to the following formula 7:

[0072]

[0073] where S i (d) represents the score of document d in the i-th retrieval module. Sort all documents according to the comprehensive score S(d), and the higher the score, the more likely the document is the result that best meets the user's needs.

[0074] It should be noted that the weighted fusion method can be adjusted according to different application scenarios and requirements. Through experiments and evaluations, the weights of each unit can be optimized to improve the recall effect of the module.

[0075] The Rerank fusion method uses a Rerank model for fusion. The Rerank model is a ranking model based on deep learning. It can perform more refined evaluation and ranking on the results of multi-channel recall to improve the quality of the final results. The Rerank model usually uses pre-trained language models (such as BERT, GPT, etc.) as the basis. By extracting features and performing representation learning on the recall results, it captures the semantic relationships and context information between documents and queries. In the training stage, the Rerank model uses a large amount of labeled data for supervised learning to learn how to predict the relevance scores of documents based on the features of documents and queries. In the inference stage, the Rerank model re-scores and ranks the results of multi-channel recall, and ranks the most relevant documents in the front.

[0076] To train the Rerank model, a large amount of labeled data needs to be prepared. Each sample contains a query, a set of recall results, and corresponding relevance labels. The training process usually uses a cross-entropy loss function or a ranking loss function (such as Pairwise Ranking Loss, Listwise Ranking Loss, etc.). The parameters of the model are updated through the backpropagation algorithm, enabling the model to accurately predict the relevance scores of documents. To improve the generalization ability and performance of the model, some optimization techniques can be adopted, such as data augmentation, regularization, learning rate adjustment, etc.

[0077] In practical applications, the weighted fusion method and the Rerank model method can be combined to give full play to the advantages of both. First, use the weighted fusion method to perform preliminary fusion and ranking on the results of multi-channel recall to obtain a candidate result list. Then, input the candidate result list into the Rerank model for refined ranking to further improve the quality of the final results. In this way, while ensuring the recall efficiency, the accuracy and relevance of the results can be improved, providing users with higher-quality information services.

[0078] In one embodiment, the output layer ranks the fused results and selects the top M results from the ranked results as the retrieval results of the information to be retrieved (recall TOP-M). M is a positive integer, which can be set by those skilled in the art according to actual needs. The retrieval results can be displayed in ways such as text, charts, visualization, etc. to improve the user experience.

[0079] It should be noted that by adopting the multi-channel recall method, the present invention comprehensively utilizes the advantages of RAG, full-text retrieval, and GraphRAG, which can improve the accuracy and diversity of recall. For different types of queries, the system can select an appropriate recall method according to the specific situation, thereby improving the recall effect. RAG and GraphRAG can perform semantic understanding and context awareness on queries and can handle complex semantic and context information. Full-text retrieval can quickly recall a large number of relevant documents, and GraphRAG can process complex knowledge structures and relationships. Through the integration of multi-channel recall, the semantic understanding and context awareness capabilities of the system can be improved. Although GraphRAG has a high cost in constructing and maintaining the graph structure, by combining with other recall modules, the dependence on the graph structure can be reduced, thereby reducing the construction and maintenance costs. At the same time, the system adopts a distributed storage system and efficient algorithms, which improve the performance and scalability of the system and further reduce the cost. The system adopts a modular design, and each module can be developed and maintained independently. Data sources, recall modules, and fusion strategies can be flexibly selected and configured according to different application scenarios and requirements. At the same time, the system supports adding new recall modules to meet the changing user needs.

[0080] To better illustrate the information retrieval system based on multi-channel recall of the present invention, a specific embodiment is provided below.

[0081] Embodiment 1. A retrieval process.

[0082] First, various types of data are uploaded in the data layer. Then, full-text retrieval fields are constructed to establish an Elasticsearch inverted index; data based on the milvus vector database is constructed and imported; a knowledge graph based on GraphRAG is constructed.

[0083] The information to be retrieved "Method for implementing error correction codes in quantum computers" is input, and the RAG unit returns the top 50 semantically relevant documents;

[0084] The full-text retrieval unit matches the keywords "quantum computer" and "error correction code"; GraphRAG expands along the path of "quantum computing → error correction → surface code".

[0085] A weight of 0.3 is assigned to the RAG unit, a weight of 0.3 is assigned to the full-text retrieval unit, and a weight of 0.4 is assigned to GraphRAG. After score normalization, the results are sorted according to the weighted scores, and finally the top 20 merged results are returned.

[0086] Figure 3 It is a schematic flow chart of the information retrieval method based on multi-channel recall provided by the embodiment of the present application. As Figure 3As shown, the information retrieval method based on multi-channel recall is applied to an information retrieval system based on multi-channel recall. The information retrieval system includes: a data layer, a recall layer, a fusion layer, and an output layer. The method includes:

[0087] Step 31: Based on the data stored in the data layer, the recall layer performs multi-channel recall retrieval on the input information to be retrieved through a built-in multi-channel recall retrieval module, and obtains a recall result.

[0088] Step 32: The fusion layer fuses the recall results using a pre-configured fusion method.

[0089] Step 33: The output layer screens the fused results and outputs the screened results as the retrieval results of the information to be retrieved.

[0090] It should be understood that the specific processes of the steps of the method have been described in detail in the above system embodiments. For the sake of brevity, they will not be repeated here.

[0091] In one embodiment, the multi-channel recall retrieval module is an expandable module; the preset multi-channel recall retrieval module includes: a RAG unit, a full-text retrieval unit, and a GraphRAG unit.

[0092] In one embodiment, based on the data stored in the data layer, performing multi-channel recall retrieval on the input information to be retrieved through a built-in multi-channel recall retrieval module to obtain a recall result includes: based on the data stored in the data layer, enabling each unit of the multi-channel recall retrieval module to perform retrieval on the input information to be retrieved respectively, and obtaining preliminary recall results of each unit; respectively performing rough ranking on the preliminary recall results of each unit to obtain the final recall result.

[0093] In one embodiment, the fusion method is configured as any one or a combination of two of the weighted fusion method and the Rerank fusion method.

[0094] In one embodiment, when the fusion method is configured as a combination of the weighted fusion method and the Rerank fusion method, the method of fusing the recall results includes: setting the weights of each unit in the multi-channel recall retrieval module; based on the set weights, using the weighted fusion method to fuse and rank the recall results to obtain candidate results; inputting the candidate results into the Rerank model for fine ranking to obtain the final fusion result.

[0095] In one embodiment, multiple types of data include: text data and graph data; among them, text data includes: document data, article data, question-and-answer pair data, and knowledge base data; graph data includes: knowledge graph data.

[0096] Figure 4 is a schematic block diagram of an electronic terminal provided by an embodiment of the present application. As Figure 4 shown, the electronic terminal includes: at least one processor 401, a memory 402, at least one network interface 403, and a user interface 405. Each component in the device is coupled together through a bus system 404. It can be understood that the bus system 404 is used to implement the connection and communication between these components. In addition to the data bus, the bus system 404 also includes a power bus, a control bus, and a status signal bus. However, for the sake of clear illustration, in Figure 4 all kinds of buses are labeled as the bus system.

[0097] Among them, the user interface 405 may include a display, a keyboard, a mouse, a trackball, a click gun, a button, a touchpad, or a touch screen, etc.

[0098] It can be understood that the memory 402 may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM, Read Only Memory), a programmable read-only memory (PROM, Programmable Read-Only Memory), which is used as an external cache. By way of example but not limitation, many forms of RAM are available, such as static random access memory (SRAM, StaticRandom Access Memory), synchronous static random access memory (SSRAM, Synchronous StaticRandomAccess Memory). The memory described in the embodiments of the present invention is intended to include but not be limited to these and any other suitable categories of memories.

[0099] The memory 402 in the embodiments of the present invention is used to store various categories of data to support the operation of the electronic terminal 400. Examples of these data include: any executable program for operating on the electronic terminal 400, such as an operating system 4021 and an application program 4022; the operating system 4021 contains various system programs, such as a framework layer, a core library layer, a driver layer, etc., for implementing various basic services and processing hardware-based tasks. The application program 4022 may include various application programs, such as a media player (Media Player), a browser (Browser), etc., for implementing various application services. Implementing the information retrieval method based on multi-way recall provided by the embodiments of the present invention may be included in the application program 4022.

[0100] The method disclosed in the embodiments of the present invention above can be applied to the processor 401 or implemented by the processor 401. The processor 401 may be an integrated circuit chip with signal processing capabilities. During implementation, the steps of the above method can be completed by the integrated logic circuit in hardware or instructions in software form in the processor 401. The above-mentioned processor 401 may be a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The processor 401 can implement or execute the various methods, steps, and logic block diagrams disclosed in the embodiments of the present invention. The general-purpose processor 401 may be a microprocessor or any conventional processor, etc. Combining the steps of the accessory optimization method provided in the embodiments of the present invention can be directly embodied as being executed and completed by a hardware decoding processor, or by a combination of hardware and software modules in the decoding processor. The software module may be located in a storage medium, and this storage medium is located in the memory. The processor reads the information in the memory and combines its hardware to complete the steps of the foregoing method.

[0101] In an exemplary embodiment, the electronic terminal 400 may be an application-specific integrated circuit (ASIC), a DSP, a programmable logic device (PLD), or a complex programmable logic device (CPLD) for executing the foregoing method.

[0102] According to the method provided in the embodiments of the present application, the present application also provides a computer program product, which includes: computer program code, when the computer program code runs on a computer, it causes the computer to execute Figure 3 the information retrieval method based on multi-channel recall shown in the embodiments.

[0103] According to the method provided in the embodiments of the present application, the present application also provides a computer-readable storage medium, which stores program code, when the program code runs on a computer, it causes the computer to execute Figure 3 the information retrieval method based on multi-channel recall shown in the embodiments.

[0104] As used in this specification, the terms "component", "module", "system", etc. are used to represent computer-related entities, hardware, firmware, combinations of hardware and software, software, or software in execution. For example, a component can be, but is not limited to, a process running on a processor, a processor, an object, an executable file, an execution thread, a program, and / or a computer. By way of illustration, both an application running on a computing device and the computing device can be components. One or more components can reside in a process and / or an execution thread, and a component can be located on one computer and / or distributed between two or more computers. Further, these components can execute from various computer-readable media having various data structures stored thereon. A component can communicate, for example, according to signals having one or more data packets (e.g., data from two components interacting with each other from a local system, a distributed system, and / or a network, such as data interacting with other systems via signals over the Internet) via local and / or remote processes.

[0105] Those of ordinary skill in the art will appreciate that the various illustrative logical blocks and steps described in connection with the embodiments disclosed herein can be implemented in electronic hardware, or in a combination of computer software and electronic hardware. Whether such functions are implemented in hardware or software depends upon the particular application and design constraints of the technical solution. Skilled artisans may implement the described functions in different ways for each particular application, but such implementation should not be considered to exceed the scope of this application.

[0106] Those skilled in the art can clearly understand that, for the sake of convenience and brevity of description, the specific working processes of the systems, apparatuses, and units described above can refer to the corresponding processes in the foregoing method embodiments, and will not be described herein again.

[0107] In several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative. For example, the division of units is only a logical function division, and there can be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the couplings or direct couplings or communication connections shown or discussed among each other can be through some interfaces, and the indirect couplings or communication connections of the apparatuses or units can be in electrical, mechanical, or other forms.

[0108] The unit described as a separation component may or may not be physically separated. The component shown as a unit may or may not be a physical unit, that is, it may be located in one place or distributed across multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0109] In addition, in each embodiment of this application, each functional unit can be integrated in a processing unit, or each unit can exist physically alone, or two or more units can be integrated in one unit.

[0110] In the above embodiments, the functions of each functional unit can be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions (programs). When the computer program instructions (programs) are loaded and executed on a computer, the processes or functions according to the embodiments of this application are generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center by wire (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wirelessly (such as infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that the computer can access or a data storage device such as a server or data center that includes one or more integrated available media. The available medium can be a magnetic medium (for example, a floppy disk, a hard disk, a magnetic tape), an optical medium (for example, a high-definition digital video disc (DVD)), or a semiconductor medium (for example, a solid state disk (SSD), etc.).

[0111] If a function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods of various embodiments of this application. The foregoing storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical discs that can store program codes.

[0112] As described above, the above is only the specific implementation manner of this application, but the protection scope of this application is not limited thereto. Any person skilled in the art within the technical scope disclosed by this application can easily think of changes or substitutions, which should all be covered by the protection scope of this application. Therefore, the protection scope of this application should be subject to the protection scope of the claims.

[0113] In summary, this application provides an information retrieval system, method, terminal, medium, and product based on multi-channel recall. The system includes: a data layer for storing and managing various types of data; a recall layer for performing multi-channel recall retrieval on the input information to be retrieved based on the data stored in the data layer through a built-in multi-channel recall retrieval module to obtain recall results; a fusion layer for fusing the recall results using a pre-configured fusion method; and an output layer for screening the fused results and outputting the screened results as the retrieval results of the information to be retrieved. The multi-channel recall retrieval module of this application combines the advantages of various retrieval methods. Moreover, this application improves the accuracy of the retrieval results through the multi-channel recall module and fusion processing. Therefore, this application effectively overcomes various shortcomings in the prior art and has high industrial utilization value.

[0114] The above embodiments merely illustrate the principles and effects of this application and are not used to limit this application. Any person familiar with this technology can modify or change the above embodiments without departing from the spirit and scope of this application. Therefore, all equivalent modifications or changes completed by those with ordinary knowledge in the technical field without departing from the spirit and technical ideas disclosed by this application should still be covered by the claims of this application.

Claims

1. An information retrieval system based on multi-channel recall, characterized in that including: a data layer for storing and managing various types of data; a recall layer for performing multi-way recall retrieval on the input information to be retrieved based on the data stored in the data layer through a built-in multi-way recall retrieval module to obtain a recall result; a fusion layer for fusing the recall result by using a pre-configured fusion method; an output layer for screening the fused result and outputting the screened result as the retrieval result of the information to be retrieved.

2. The information retrieval system based on multi-channel recall according to claim 1, wherein The multi-way recall retrieval module is an expandable module; the preset multi-way recall retrieval module includes: a RAG unit, a full-text retrieval unit, and a GraphRAG unit.

3. The information retrieval system based on multi-channel recall according to claim 2, wherein Performing multi-way recall retrieval on the input information to be retrieved based on the data stored in the data layer through a built-in multi-way recall retrieval module to obtain a recall result, including: Based on the data stored in the data layer, enabling each unit of the multi-way recall retrieval module to respectively retrieve the input information to be retrieved to obtain preliminary recall results of each unit; Performing rough ranking on the preliminary recall results of each unit respectively to obtain a final recall result.

4. The information retrieval system based on multi-channel recall according to claim 1, characterized in that, The fusion method is configured as any one or a combination of two of a weighted fusion method and a Rerank fusion method.

5. The information retrieval system based on multi-channel recall according to claim 4, characterized in that, When the fusion method is configured as a combination of a weighted fusion method and a Rerank fusion method, the method for fusing the recall result includes: Setting the weights of each unit in the multi-way recall retrieval module; Based on the set weights, using the weighted fusion method to fuse and rank the recall result to obtain candidate results; Inputting the candidate results into a Rerank model for fine ranking to obtain a final fused result.

6. The information retrieval system based on multi-channel recall according to claim 1, characterized in that, The various types of data include: text data and graph data; wherein, the text data includes: document data, article data, question-and-answer pair data, and knowledge base data; the graph data includes: knowledge graph data.

7. An information retrieval method based on multi-channel recall, characterized in that, Applied to an information retrieval system based on multi-way recall, the information retrieval system includes: a data layer, a recall layer, a fusion layer, and an output layer, and the method includes: The recall layer performs multi-way recall retrieval on the input information to be retrieved based on the data stored in the data layer through a built-in multi-way recall retrieval module to obtain a recall result; The fusion layer fuses the recall result by using a pre-configured fusion method; The output layer screens the fused result and outputs the screened result as the retrieval result of the information to be retrieved.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, The computer program, when executed by a processor, implements the method according to claim 7.

9. A computer program product, characterized in that, The computer program product includes computer program code, and when the computer program code runs on a computer, the computer is caused to implement the method according to claim 7.

10. An electronic terminal, comprising a memory, a processor, and a computer program stored on the memory, characterized in that, The processor executes the computer program to implement the method according to claim 7.

Images