50 results about "Relevance (information retrieval)" patented technology

The invention provides a full-text retrieval method and system fusing various types of documents, and relates to the technical field of information retrieval, and the method comprises the following steps: obtaining document representation through document content extraction and structure recognition, generating a cross-modal semantic vector by using word embedding and nonlinear transformation, constructing a hierarchical index and a cross-document association graph, and obtaining a full-text retrieval result; the basic correlation score is calculated after the query request is received, and the comprehensive score of the candidate content segments is calculated based on the association graph to determine the optimal retrieval result, so that unified representation and retrieval of heterogeneous documents are realized, the cross-document retrieval precision and relevance are improved, and the processing capability of a retrieval system on complex queries is enhanced.

Techniques for multi-index retrieval in knowledge databases to enhance retrieval augmented generation (RAG) systems. The techniques involve a query processor that receives a query from a RAG agent, compares it to index summaries, and selects target indexes. The processor then searches these indexes, fuses the retrieved content items, and reranks the results before sending them back to the RAG agent. This approach combines query routing, multi-index fusion, and reranking to improve information retrieval for RAG applications. The technique offers several advantages, including enhanced retrieval efficiency through specialized indices, increased relevance and precision of retrieved information, scalability for large datasets and high query volumes, and optimized querying across diverse data sources. The techniques address challenges in managing extensive, distributed datasets and are compatible with existing RAG frameworks, providing a solution for complex information retrieval tasks.

The invention discloses a user participation degree prediction method based on distillation multi-modal retrieval enhancement, and belongs to the technical field of social media data mining and user behavior analysis. According to the method, the correlation of the UGC is evaluated by introducing the self-enhanced distillation module, and the top-K related UGC is selected in combination with the selected retriever, so that the interference of irrelevant information on the related UGC is effectively avoided, and the noise caused by irrelevant artifacts can be filtered out when the related UGC is retrieved, thereby keeping the original feature representation of the related UGC. The heterogeneous graph construction module enhances the interactive representation capability between UGCs through multi-relation modeling, and can more accurately optimize a prediction result in user participation prediction. According to the method, information retrieval between the related UGC and the unrelated UGC can be better balanced, excessive diffusion of unrelated information is avoided, and the performance of the model in user participation degree prediction is improved. Particularly, when complex multi-modal data containing a large number of irrelevant UGCs is processed, the mechanism can effectively enhance the distinction degree of the relevant UGCs, and the prediction accuracy is improved.

Systems and method for generating a response to a query. The method includes using a first large language model (LLM) to generate synthetic information related to a query; generating an amended query based on the synthetic information related to the query; using an information retrieval system to retrieve, from a plurality of chunks, a set of chunks that are relevant to the amended query, wherein each chunk of the plurality of chunks is all or a portion of a document in a corpus of documents; using a second LLM to rank the set of chunks based on a relevance to the query; selecting a subset of chunks from the set of chunks based on the ranking; and using a third LLM to generate a response to the query based on the subset of chunks.

There are provided systems and methods for identifying relevance of documents for automated retrieval models using large language models. An online transaction processor or other service provider may provide computing services and platforms to entities, which may include chatbots, information retrieval systems, question-and-answer systems, and the like. To provide better retrieval model training and refinement, the service provider may generate training data from user interaction logs, which may include user feedback that may be used to determine if documents are relevant to queries, and therefore should be retrieved for answering those queries by automated retrieval models. An LLM may be used as a judge to determine whether chatbot responses reference certain document. If not references, the query may be analyzed to determine whether certain retrieved documents are relevant. Data pairs may be generated for the training data from these processes and used for model refinement.

The invention discloses a self-adaptive multi-feature fusion hybrid retrieval sorting method and system, and belongs to the technical field of information retrieval. The method comprises the steps that after user query is received, a mixed retrieval process and an intention recognition process are executed in parallel; performing multi-dimensional feature extraction on the candidate documents obtained by the mixed retrieval, wherein the multi-dimensional feature extraction comprises semantic correlation features, keyword matching features, document authority features and timeliness features; and according to the identified query type, adaptively selecting a fusion weight, and carrying out weighted fusion on the multi-dimensional feature vector to calculate a final score and sort the final score. According to the method, the problem that weight distribution is rigid in traditional mixed retrieval is solved through an intention self-adaptive dynamic weight mechanism, meanwhile, by introducing multi-dimensional service features, the ranking result not only ensures the correlation, but also meets the quality requirement under a service scene, and the accuracy and practicability of a retrieval system are remarkably improved.

The invention provides an RAG mixed retrieval method and device, and relates to the technical field of information retrieval, and the method comprises the following steps: obtaining a query text and a term library, and calculating to obtain a vector retrieval weight and a keyword retrieval weight according to the query text and the term library; obtaining a document library, and performing vector retrieval on the document library according to the query text and the vector retrieval weight to obtain a first candidate document set; performing keyword retrieval on the document library according to the query text and the keyword retrieval weight to obtain a second candidate document set; merging the first candidate document set and the second candidate document set to obtain a fused document set; and calculating, screening and sorting comprehensive feature scores of the documents in the fused document set to obtain a final document sequence, and inputting the final document sequence into the large language model to obtain an answer text. According to the method, the limitation of a single scoring dimension in professional field retrieval is overcome, and the correlation between the answer text and the query text is improved through multi-dimensional feature joint evaluation.

The invention provides a multi-path information retrieval method and system based on a language model and uncertainty evaluation. The multi-path information retrieval method comprises the following steps: receiving a user query; performing semantic analysis on user query by using a pre-trained language model so as to output a quantitative uncertainty index in an unsupervised manner; dynamically determining a quantity ratio for controlling query generation and a balance parameter for controlling result sorting based on the index; based on the user query and the strategy parameters, generating and executing two groups of differential queries at least comprising exploratory queries and utilizable queries to obtain multiple paths of retrieval results; and combining multiple paths of results, and carrying out dynamic balance reordering between the correlation and diversity of the results according to balance parameters by adopting an ordering mechanism. According to the method, the retrieval strategy can be adaptively adjusted according to the uncertainty of query, and the coverage degree and diversity of results are remarkably improved.

The invention provides an information retrieval method, device and equipment based on large model questions and answers and a medium, and the method comprises the following steps: firstly, receiving a natural language question input by a user through a terminal, and forwarding the natural language question to a semantic analysis engine after authentication of an API (Application Program Interface) gateway; then oral vocabularies are removed through a normalization module, abbreviation is completed, a retrieval category is determined in combination with an intention recognition module, key information is extracted through a slot extraction module, and meanwhile permission-semantic coupling retrieval is completed in combination with a company organization structure; then adopting a multi-path recall mode of vector semantic retrieval and keyword supplementary retrieval to obtain a candidate result, and performing correlation rearrangement and permission filtering to obtain a target retrieval result; and finally, performing fragment positioning on different types of contents, and displaying a result with specific position information at a front end in a preset form. According to the method, the retrieval accuracy and efficiency can be improved, cross-modal unified retrieval is realized, authority security is guaranteed, and efficient and accurate information retrieval requirements in large enterprises are met.

The invention relates to the technical field of artificial intelligence and information retrieval, in particular to a dynamic retrieval strategy self-adaptive adjustment method and system based on reinforcement learning. According to the method, a retrieval process is modeled into a Lukov decision process, and a strategy model is trained to realize autonomous optimization and dynamic adjustment of a retrieval strategy. The method comprises the following steps: defining a state space comprising a query type, context correlation, a historical retrieval effect and knowledge base characteristics; defining an action space including vector retrieval, keyword retrieval and mixed retrieval; constructing a dual-target reward function fusing the retrieval income and the retrieval cost; training a strategy model by utilizing a historical retrieval track; calling the strategy model in real time to dynamically adjust the retrieval mode when the retrieval enhancement generation system runs; and the model is iteratively optimized through continuous feedback. According to the method, the problems of low recall rate and low calculation efficiency of a traditional fixed retrieval strategy under complex query are solved, dynamic balance between retrieval precision and resource consumption is realized, and the method is suitable for intelligent retrieval scenes of multi-field and multi-modal knowledge bases.

The invention discloses a bidirectional information retrieval enhancement generation method for a large language model, and belongs to the technical field of artificial intelligence. In order to overcome the defects that noise is introduced and key evidences are omitted due to the fact that traditional RAG only executes'query-document 'one-way retrieval, a two-way semantic perception retrieval enhancement generation model and a two-stage training framework are constructed, wherein in the first stage, the positive / negative example distance is increased in an embedded space in a contrast learning self-supervision mode; in the second stage, fine-grained correlation discrimination is carried out on query-document bidirectional sentences through supervised dichotomy, and probabilistic correlation scores are output; in the reasoning stage, the bidirectional probabilities are fused according to Bayesian to obtain final relevancy, document reordering is carried out, and plug and play can be achieved without fine adjustment of LLM in the whole process. According to the method, the accuracy and consistency of single-hop and multi-hop questions and answers and fact checking tasks are remarkably improved, and the method has the advantages of light weight and low deployment cost.

The invention discloses a retrieval enhancement generation method and system based on long-term memory and multi-source knowledge iterative fusion. The method comprises the following steps: firstly, after user inquiry is received, triggering long-term memory database retrieval, external knowledge base retrieval and large language model internal knowledge generation in parallel to construct a multi-source candidate knowledge base, and improving result correlation by adopting a self-adaptive retrieval method based on a dynamic top-k value; secondly, inputting user query and multi-source knowledge into the large language model for iterative fusion, and generating a final answer through conflict detection, fusion processing and confidence evaluation; and finally updating the long-term memory database. The method can improve the accuracy and stability of the generated answers in a multi-round interaction and complex task scene, reduces the risk of answer one-sidedness, inference incompleteness or fact deviation, enhances the multi-source knowledge utilization ability and long-term learning ability of the system, and improves the user experience. Therefore, the comprehensive performance and reliability of the system in applications such as intelligent question answering, information retrieval assistance and text generation are obviously expanded.

The application discloses a bidirectional information retrieval enhancement generation method for a large language model, and belongs to the technical field of artificial intelligence. In order to overcome the defects of traditional RAG that only performs one-way retrieval from 'query to document' and introduces noise and omits key evidence, a bidirectional semantic perception retrieval enhancement generation model and a two-stage training framework are constructed. In the first stage, the positive / negative example distance is pulled apart in the embedding space in a contrast learning self-supervised manner. In the second stage, a supervised two-classification is used to finely distinguish the relevance of the query-document bidirectional sentence pair, and an output probability correlation is obtained. In the reasoning stage, the bidirectional probability is fused according to Bayes to obtain the final correlation degree, and the document is reordered. The whole process can realize plug and play without fine-tuning the LLM. The application significantly improves the accuracy and consistency of single-hop, multi-hop question answering and fact checking tasks, and has the advantages of light weight and low deployment cost.

A dense encoder is adapted as a hierarchical corpus encoder in an information retrieval system to use negative samples from sibling nodes in a hierarchical tree of vector embeddings for documents in a corpus. Both the encoder and hierarchical tree are co-trained using a loss function that takes the document hierarchy into account. The hierarchical corpus encoder may be used in both supervised training cases where query-document relevance judgments are present and in zero-shot cases where a query dataset is absent. The hierarchical corpus encoder demonstrates significant performance improvements over a variety of dense encoder and generative retrieval baselines, under both supervised and unsupervised scenarios, thereby establishing the effectiveness of jointly learning a document hierarchy.

The application relates to the technical field of artificial intelligence, in particular to a retrieval method and device based on multiple agents, equipment and a medium. Applied to a financial scene, the application realizes the intellectualization and dynamicization of the retrieval process through a multiple-agent cooperative working mechanism. The retrieval core elements are accurately extracted and a dynamic cognitive graph is updated by analyzing the agents, deep semantic mining and correlation expansion are carried out based on the cognitive seeds of semantic agents, the dependence on keywords in traditional retrieval is effectively broken, and the breadth and depth of semantic understanding are improved. The query statement is optimized by combining the extended semantics and the dynamic cognitive graph of the context agent, so that the query is more in line with the real intention and context of the user. The retrieval agent efficiently retrieves the candidate path in the updated dynamic cognitive graph, the recommendation agent filters the optimal result through cognitive consistency scoring, and the accuracy, relevance and logical coherence of the retrieval result are ensured, and the accuracy of the complex information retrieval task is significantly improved.

A dense encoder is adapted as a hierarchical corpus encoder in an information retrieval system to use negative samples from sibling nodes in a hierarchical tree of vector embeddings for documents in a corpus. Both the encoder and hierarchical tree are co-trained using a loss function that takes the document hierarchy into account. The hierarchical corpus encoder may be used in both supervised training cases where query-document relevance judgments are present and in zero-shot cases where a query dataset is absent. The hierarchical corpus encoder demonstrates significant performance improvements over a variety of dense encoder and generative retrieval baselines, under both supervised and unsupervised scenarios, thereby establishing the effectiveness of jointly learning a document hierarchy.

The application discloses a dynamic intelligent agent retrieval tree-based time-sensitive news retrieval method and system, an electronic device and a storage medium, and belongs to the technical field of information retrieval and artificial intelligence. The technical scheme of the application comprises the following steps: constructing a retrieval tree with a user query as a root node, the tree being expanded in semantics through multi-agent cooperation; obtaining a current news corpus and constructing a representative evaluation agent; based on the evaluation agent, dynamically selecting an optimal sub-tree from the retrieval tree that meets a preset condition; and using the optimal sub-tree to search the full news corpus to obtain a target news document. The application decouples high-cost semantic expansion and high-concurrency online retrieval, reduces query delay and computing cost, and dynamically selects an optimal sub-tree to adapt to changes in news content in real time, thereby improving the timeliness and relevance of the retrieval result.

The application discloses a user participation degree prediction method based on distillation multimodal retrieval enhancement, and belongs to the technical field of social media data mining and user behavior analysis.The application introduces a self-enhanced distillation module to evaluate the relevance of UGC, and selects top-K relevant UGC in combination with a selected retriever, so that irrelevant information is effectively avoided from interfering with relevant UGC, and the noise caused by irrelevant artifacts is filtered out when the relevant UGC is retrieved, so that the original feature representation of the relevant UGC is maintained.A heterogeneous graph construction module enhances the interaction representation capability between UGC through multi-relation modeling, and can more accurately optimize the prediction result in user participation degree prediction.The application can better balance the information retrieval between relevant UGC and irrelevant UGC, avoid excessive spread of irrelevant information, and improve the performance of the model in user participation degree prediction.In particular, when complex multimodal data containing a large amount of irrelevant UGC is processed, the mechanism of the application can effectively enhance the distinguishability of relevant UGC and improve the prediction accuracy.

This invention relates to the field of information retrieval technology, specifically to a retrieval enhancement generation method based on machine learning for second-level retrieval of user data. This method solves the technical problems in existing technologies where structured medical data suffers severe loss of key clinical semantics during vectorization and fixed-length segmentation, and cannot support cross-indicator correlation analysis and low-latency feedback. The method includes: acquiring user queries against a structured user database; determining the semantic loss of at least one candidate data segment corresponding to the user query in the temporal continuity and medical relevance dimensions; using the semantic loss to characterize the degree of clinical semantic loss caused by segmentation of the candidate data segment; determining the completeness of the indicator association between the candidate data segment and the user query based on the semantic loss; and adjusting the retrieval results for the user query based on the indicator association completeness.

This application provides a context optimization method and apparatus for a retrieval-enhanced generation system, relating to the field of information retrieval technology. The method includes: receiving a user query and retrieving a knowledge document set related to the user query through a knowledge base retrieval; dynamically optimizing the knowledge document set by iteratively executing symmetric and asymmetric merging operations using an optimization objective function based on information bottlenecks to generate a compressed context; wherein, the symmetric merging operation is used for document pairs with low relevance scores; and the asymmetric merging operation is used for documents that identify and fuse semantic redundancy based on conditional entropy. The context optimization method and apparatus for a retrieval-enhanced generation system provided by this application, through dynamic intelligent fusion and compression of retrieved documents, can provide larger language models with higher information density and more accurate context at a lower computational cost, thereby obtaining more reliable and complete answers.

The present disclosure discloses a question and answer library-based information retrieval method, a question and answer system and a computing device, wherein the question and answer library contains at least one question and at least one answer corresponding to each question, and the question and answer library-based information retrieval method comprises: in response to a user query, determining a recalled question and a candidate answer subset corresponding to the recalled question from the question and answer library as at least one question and answer pair, wherein the candidate answer subset is from an answer set; inputting the user query and the at least one question and answer pair into a preset matching model for processing to obtain a matching degree of the user query and each question and answer pair; and sorting the question and answer pairs according to the matching degree as a retrieval result. According to the scheme of the present disclosure, the accuracy and relevance of the question and answer can be significantly improved, and the quality of the question and answer can be improved.

This application belongs to the field of information retrieval technology and discloses a multimodal patent semantic retrieval method and system. By establishing relation anchor points, extracting local cross-modal relation representations around the anchor points, and verifying the cross-modal consistency of these relations, reliable cross-modal relations can be identified. Furthermore, technical features / components and successfully aligned cross-modal relations are constructed into a graph structure representation, and a graph matching algorithm is used to calculate similarity. As a result, the overall logic and details of the technical solution can be understood more effectively, and solutions containing only the same technical features but with different relational structures can be distinguished, thereby improving the accuracy and relevance of multimodal patent retrieval.

The application discloses a retrieval enhancement generation method and system based on multi-dimensional reordering, relates to the technical field of information retrieval, and comprises the following steps: S1, constructing a tensor index, a keyword index and a compressed abstract; S2, driving a Qwen large model to perform query expansion and hypothetical answer generation through a double-task query processing template; S3, performing mixed retrieval through a semantic and keyword double-path index, and performing two-stage reordering by using an improved DistilBERT model and the Qwen large model; S4, performing multi-dimensional evaluation through sub-problem decomposition; S5, iteratively generating sub-problem answers according to the sub-problem order; and S6, performing multi-dimensional evaluation correction to generate a final answer. The application overcomes the limitations of traditional retrieval enhancement generation technology in information matching accuracy, context relevance evaluation and answer generation quality, and provides an efficient and accurate solution.

This application relates to the field of information retrieval technology, and in particular to a multimodal image retrieval method, apparatus, device, and medium. First, the retrieval request is subjected to target semantic parsing to obtain a target semantic description. Then, based on the target semantic description, target entities are extracted from the original images, and fine-grained matching at the entity level is performed, effectively overcoming the deficiency of traditional global features being insensitive to local targets. Finally, based on a first similarity reflecting the target entity matching degree and a second similarity reflecting global relevance, the target similarity corresponding to each original image is determined, ensuring that the final target retrieval result highly matches the retrieval intent in both the target and scene dimensions. This fundamentally improves the accuracy of image retrieval.

This application provides a method, apparatus, computer device, storage medium, and computer program product for obtaining online information. The method includes: in response to an online information query request, obtaining query text and corresponding keywords and text vectors; performing parallel retrieval based on the keywords and text vectors to obtain an initial search result set; the initial search result set includes multiple search results; based on document identifiers in each search result, deduplicating the initial search result set to obtain a candidate search result set; determining the relevance score between the online information document and the query text for each candidate search result in the candidate search result set, and based on each relevance score, selecting and returning the target search result from the candidate search result set to respond to the online information query request. This method can improve the accuracy of the response results returned by the information retrieval system, thereby improving the efficiency of users obtaining online information.

The invention relates to the technical field of information retrieval and machine learning, and provides a domain-to-entity adaptive document rearrangement method, which comprises the following steps of: inputting a plurality of discrimination sample pairs into a rearrangement model to calculate comprehensive correlation scores, and sorting to obtain a corresponding document rearrangement result; the rearrangement model comprises the steps of pre-training a rearrangement module to output a semantic basic correlation score; the domain / entity identification module generates domain weight parameters of a plurality of target domains through a classification network and determines corresponding company entities, and the combined adaptive adjustment module performs sparse selection on low-rank adaptive branches corresponding to the plurality of target domains and performs incremental feature modeling on a discrimination sample pair based on a started low-rank matrix; generating a corresponding incremental correlation score; the reordering execution module is used for ordering the comprehensive correlation scores and outputting results, a specific knowledge base or retrieval implementation does not need to be limited, flexible butt joint with an existing retrieval assembly can be achieved, and the method has the advantages of being fast in convergence, low in cost and the like in the aspects of cross-subject migration and entity-level adaptation.

Embodiments of the present application provide a training method of a recall model, an electronic device, a storage medium and a program product, the recall model comprising: a query encoding model and an object encoding model pre-trained through contrastive learning, determining a first embedding representation corresponding to a first query question through the query encoding model; determining a plurality of first object information satisfying a set similarity condition based on the similarity between the first embedding representation and embedding representations corresponding to different object information in an object information library; wherein the embedding representations of different object information are obtained by encoding different object information based on the object encoding model; determining a relevance score between the plurality of first object information and the first query question respectively; determining target object information from the plurality of first object information, wherein the relevance score of the target object information satisfies a set advantage correlation condition; and fine-tuning the query encoding model according to the target object information. Through the scheme, the accuracy of information retrieval can be improved.

The application provides an intelligent retrieval accurate matching method and system based on multi-level semantic decomposition, relates to the technical field of information retrieval, and comprises the following steps: generating a hierarchical semantic representation tree by performing semantic decomposition on a text to be retrieved; constructing a semantic propagation graph and optimizing the same by using an adaptive probability sampling strategy; extracting a semantic feature vector with context dependency relationship fused to establish an index; and analyzing a retrieval request and matching the same with the semantic propagation graph to generate a result. The application improves retrieval accuracy, enhances semantic understanding capability, optimizes retrieval efficiency, and improves the relevance and accuracy of the result.

The keyword matching method, device and electronic equipment across language environments provided by the embodiments of the present application relate to the technical field of information retrieval. First, a source language keyword used for matching a target language text is obtained; then, the source language keyword is processed by word segmentation, and the source language keyword is classified into a short keyword string or a long keyword string; then, when the source language keyword is a short keyword string, keyword cross-language matching is performed through semantic expansion to optimize the missing report problem of the source language keyword exact matching; when the language keyword is a long keyword string, the source language keyword cross-language matching is performed based on a semantic-level fuzzy matching technology, the overall matching degree of the keyword is calculated in combination with the semantic matching value and the overall relevance of the matching segment to the target text, so as to comprehensively consider the overall matching degree and the local matching degree of the source language keyword. The above scheme can adopt different matching strategies based on the classification of the source language keyword, and ensure the accuracy of the matching result.

The invention discloses a retrieval enhancement generation method based on self-reflection and multi-stage reordering, and relates to the technical field of natural language processing and information retrieval. According to the method, firstly, diversified rewriting is conducted on user query, semantic coherent knowledge blocks are constructed based on a domain database, mixed retrieval is conducted, multi-stage reordering and diversity screening are achieved through reciprocal ordering fusion, cross encoder fine arrangement and a maximum boundary correlation algorithm, and finally a self-reflection mechanism is introduced. And evaluating context quality and iteratively optimizing a retrieval result until a threshold value is met, and inputting the retrieval result into the large model to generate an answer. Through the technical innovation, the recall rate, the accuracy rate and the information diversity of retrieval are improved, so that a large language model can generate more accurate and detailed answers with factual basis.