Collection type document selection method and device, equipment and readable storage medium

Abstract

The invention provides a collective document selection method, device and equipment based on information demand identification, a readable storage medium and a computer program product. The collective document selection method comprises the following steps: in response to user query, recalling a plurality of candidate documents from a knowledge base; decomposing the user query based on chain thinking reasoning, and explicitly identifying a plurality of information sub-targets required for answering the user query; on the basis of the plurality of information sub-targets, respectively selecting an unordered document subset from the plurality of candidate documents; the subset of documents is input to a generative model along with the user query to generate a final answer to the user query. According to the method and the device, all information requirements can be covered to the maximum extent through a simple collective document selection normal form, meanwhile, internal redundancy is minimized, and user experience is improved.

Description

Technical Field

[0001] This disclosure relates to the fields of artificial intelligence and natural language processing, and in particular to a method, apparatus, device, readable storage medium, and computer program product for aggregate document selection based on information demand identification. Background Technology

[0002] Retrieval-Augmented Generation (RAG) technology effectively improves the accuracy and reliability of large language models in knowledge-intensive tasks by retrieving relevant information from external knowledge bases as context, and has become a key technology for scenarios such as intelligent question answering and report generation.

[0003] In current mainstream RAG system architectures, the retrieval process commonly employs a "reranking" technique. This paradigm typically involves two stages: First, keyword matching (such as the BM25 algorithm) or dense vector retrieval models (such as bge-large) are used to perform initial recall from a large-scale knowledge base, obtaining a relatively large set of candidate documents (e.g., Top-N, N=20). Subsequently, a more refined reranking model (such as bge-reranker-large) is used to evaluate each of the initially recalled candidate documents, scoring each document based on its independent relevance to the user query, and finally selecting the top K documents with the highest scores (i.e., Top-K, e.g., K=5) as the input context for the generative model.

[0004] However, existing RAG retrieval technologies, which are based on reordering, still have significant shortcomings in terms of the comprehensiveness of the answer, the conciseness of the context, and the intelligence of resource scheduling when dealing with complex and multi-layered information needs. They not only ignore the integrity of the set, but also cause serious interference from redundant information, wasting computing resources. In addition, they cannot adaptively adjust according to the complexity of the question and will introduce more noise, which will reduce the quality of the generated answer. Summary of the Invention

[0005] This disclosure was made to address the aforementioned issues, and its purpose is to provide a set-based document selection method, system, device, and storage medium based on information demand identification. By explicitly identifying information demands and performing set optimization, it ensures that the retrieved documents comprehensively cover all dimensions required to answer questions.

[0006] This disclosure provides a summary portion of the invention to briefly introduce the concepts, which will be described in detail in the detailed description portion that follows. This summary portion is not intended to identify key or essential features of the claimed technical solutions, nor is it intended to limit the scope of the claimed technical solutions.

[0007] To address the aforementioned technical problems, this disclosure provides a method for selecting aggregated documents, employing the following technical solution:

[0008] In response to user queries, retrieve multiple candidate documents from the knowledge base;

[0009] The user query is decomposed based on chain-like reasoning, and multiple information sub-targets required to answer the user query are explicitly identified.

[0010] Based on the multiple information sub-targets, an unordered subset of documents is selected from the multiple candidate documents, wherein the document subsets are configured as a whole to cover all the information sub-targets and reduce, for example, minimize internal redundancy;

[0011] The subset of documents is input together with the user query into the generative model to generate the final answer to the user query;

[0012] The size of the document subset is determined by the number of information sub-targets and the document coverage.

[0013] To address the aforementioned technical problems, this disclosure also provides a collection-based document selection device based on information demand identification, used to retrieve and enhance the generation system to implement the steps of the method described above. The device employs the following technical solution, including:

[0014] The document retrieval module is used to retrieve multiple candidate documents from the knowledge base in response to user queries;

[0015] The decomposition and identification module is used to decompose the user query based on chain-like reasoning and explicitly identify multiple information sub-targets required to answer the user query.

[0016] A document subset selection module is used to select an unordered document subset from the multiple candidate documents based on the multiple information sub-targets, wherein the document subset is configured as a whole to cover all the information sub-targets and reduce, for example, minimize internal redundancy;

[0017] The result generation module is used to input the document subset together with the user query into the generation model to generate the final answer to the user query;

[0018] The size of the document subset is determined by the number of information sub-targets and the document coverage.

[0019] To address the aforementioned technical problems, this disclosure also provides a computer device that employs the following technical solution, including:

[0020] A memory, a processor, and a computer program stored on the memory, characterized in that the processor executes the computer program to implement the steps of the method as described in any of the preceding claims.

[0021] To address the aforementioned technical problems, this disclosure also provides a computer-readable storage medium, employing the technical solutions described below, including:

[0022] It stores a computer program, characterized in that, when the computer program is executed by a processor, it implements the steps of the method as described in any of the preceding claims.

[0023] To address the aforementioned technical problems, this disclosure also provides a computer program product, including a computer program, employing the following technical solution:

[0024] When the computer program is executed by a processor, it implements the steps of the method as described in any of the preceding methods.

[0025] According to the technical solution disclosed in this disclosure, compared with the prior art, this disclosure can maximize the coverage of all information needs and minimize internal redundancy by explicitly identifying information needs and performing set optimization and a simple set-based document selection paradigm. This ensures that the retrieved documents can fully cover all dimensions required to answer questions, has stronger anti-interference and anti-noise capabilities, avoids resource waste or information loss, and improves user experience. Attached Figure Description

[0026] Figure 1 This is an exemplary system architecture diagram to which this disclosure can be applied;

[0027] Figure 2 This is a flowchart of an embodiment of the method according to the present disclosure;

[0028] Figure 3 This is a flowchart of an embodiment of the method according to the present disclosure;

[0029] Figure 4 This is a schematic diagram of one embodiment of the apparatus according to the present disclosure;

[0030] Figure 5 This is a schematic diagram of the structure of a computer device according to an embodiment of the present disclosure.

[0031] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. Throughout the drawings, the same or similar reference numerals denote the same or similar elements. It should be understood that the drawings are schematic, and elements are not necessarily drawn to scale. Detailed Implementation

[0032] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure belongs; the terminology used herein in the specification of the application is for the purpose of describing particular embodiments only and is not intended to be limiting of this disclosure; the terms "comprising" and "having," and any variations thereof, in the specification, claims, and foregoing drawings of this disclosure are intended to cover non-exclusive inclusion. The terms "first," "second," etc., in the specification, claims, or foregoing drawings of this disclosure are used to distinguish different objects and not to describe a particular order.

[0033] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this disclosure. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0034] To enable those skilled in the art to better understand the present disclosure, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below with reference to the accompanying drawings.

[0035] System Structure

[0036] First, the structure of a system according to an embodiment of this disclosure will be described. For example... Figure 1 As shown, system architecture 100 may include, for example, terminal devices 101, 102, 103, and 104, network 105, and server 106. Network 105 is used as a medium to provide a communication link between terminal devices 101, 102, 103, and 104 and server 106.

[0037] In this embodiment, the templated heterogeneous database ETL method runs on electronic devices (e.g., Figure 1The terminal devices 101, 102, 103, or 104 shown can transmit various types of information via network 105. Network 105 can include various connection types, such as wired, wireless communication links, or fiber optic cables. It should be noted that the aforementioned wireless connection methods can include, but are not limited to, 3G / 4G / 5G connections, Wi-Fi connections, Bluetooth connections, WiMAX connections, Zigbee connections, UWB connections, local area networks (“LAN”), wide area networks (“WAN”), the Internet (e.g., the Internet), and end-to-end networks (e.g., ad-hoc end-to-end networks), as well as other currently known or future-developed network connection methods. Network 105 can communicate using any currently known or future-developed network protocol such as HTTP (Hypertext Transfer Protocol), and can interconnect with digital data communication (e.g., communication networks) of any form or medium.

[0038] Users can interact with server 106 via network 105 using terminal devices 101, 102, 103, and 104 to receive or send messages. Various client applications can be installed on terminal devices 101, 102, 103, or 104, such as video streaming and playback applications, web browser applications, shopping applications, search applications, instant messaging tools, email clients, and social media platforms.

[0039] Terminal devices 101, 102, 103, or 104 may be various electronic devices with touch screens and / or web browsing capabilities, including but not limited to smartphones, tablets, e-book readers, MP3 (Motion Picture Experts Compression Standard Audio Layer 3) players, MP4 (Motion Picture Experts Compression Standard Audio Layer 4) players, head-mounted displays, laptops, digital radio receivers, PDAs (Personal Digital Assistants), PMPs (Portable Multimedia Players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and mobile terminals such as digital TVs and desktop computers.

[0040] Server 106 may be, for example, a server that provides various services, such as a backend server that supports the pages displayed or the data transmitted on terminal devices 101, 102, 103 or 104.

[0041] It should be understood that Figure 1 The number of terminal devices, networks, and servers shown is merely illustrative. Depending on implementation needs, any number of terminal devices, networks, and servers can be included.

[0042] Here, the terminal device may implement the methods of the embodiments disclosed herein, either independently or in conjunction with other electronic terminal devices, by running applications in various operating systems.

[0043] Collection-based document selection method

[0044] The aggregate document selection method disclosed herein differs from existing technologies primarily in the following aspects:

[0045] 1. A Paradigm Shift from "Ranking" to "Selection": This disclosure no longer views retrieval as a task of ranking a list of documents, but rather as a set optimization task. The goal is to select an optimal, unordered subset of documents that, as a whole, can most comprehensively and without redundancy satisfy the information needs of the query, actively identifying and eliminating documents with duplicate content to avoid interference from redundant information. In contrast, re-ranking techniques only generate an independent relevance score for each document and then select the Top-K, which fails to guarantee that the selected K documents form a "set" and cannot comprehensively cover all the information dimensions required to answer the question.

[0046] 2. Explicit Information Requirement Identification (IRI): For example, using chain-of-thought (CoT) reasoning to guide a large model (such as an LLM) to decompose the user query, explicitly listing several key information sub-goals necessary to answer the question. For example, "identify a company" can be broken down into finding information on "business strategy," "product claims," ​​and "related controversies."

[0047] 3. Information Needs-Based Set Construction: After identifying information needs, the model iterates through the candidate document pool, selecting the most relevant documents for each information need. The final output is a document set designed to maximize coverage of all information needs while minimizing internal redundancy. The size of this set is dynamic, adapting to the complexity of the problem. It is determined by the number of information needs and document coverage, rather than by a pre-set fixed K value (such as 5 or 10), which would fail to meet the dynamic requirements of the problem's complexity.

[0048] refer to Figure 2 The diagram illustrates a flowchart of an embodiment of the set-based document selection method according to this disclosure. The system employed by the set-based document selection method is, for example, defined as an SBDS (Set-Based Document Selection) system, used to optimize the performance of a retrieval enhancement generation (RAG) system, and includes the following steps:

[0049] S21, in response to a user query, retrieves multiple candidate documents from the knowledge base.

[0050] In one or more embodiments, taking multi-dimensional information retrieval of insurance claims cases as an example, for instance, a user submits a complex claims inquiry to an insurance company's AI customer service system: "My car was flooded in a rainstorm and then rear-ended by another car. Now the engine and rear bumper are damaged. Can I get compensation in this situation?" The system needs to retrieve an optimal set of documents from a massive amount of insurance terms, claims cases, weather reports, and traffic regulations. This set should comprehensively cover all the key information dimensions required to answer this question, such as "natural disaster exemption clauses," "collision accident liability determination," "engine water ingress claims conditions," and "principles for handling superimposed accidents," etc.

[0051] First, for example, an efficient dense vector retrieval model (such as bge-large_zh) can be used to encode the user query and perform a similarity search in the company's knowledge base to recall the top-20 candidate documents most relevant to the query, so as to quickly narrow down the search scope and provide a candidate pool for subsequent refined selection.

[0052] S22 decomposes the user query based on chain-like reasoning and explicitly identifies multiple information sub-goals required to answer the user query, i.e., information needs.

[0053] In one or more embodiments, the user query is decomposed based on chain-like reasoning, explicitly identifying multiple information sub-goals required to answer the user query. This not only decomposes the question but also infers the logical relationships between the sub-goals. Furthermore, during decomposition, the sub-goals are dynamically adjusted by referencing the content of recalled candidate documents, including:

[0054] First, the user query and / or candidate documents are input into a finely tuned large language model. For example, in the above example, the Top-20 candidate document list obtained from the user query is input into the finely tuned "SBDS" model.

[0055] Secondly, the "SBDS" model first executes chain reasoning (CoT) based on a pre-set prompt template, decomposing the problem step by step. Through chain reasoning (CoT) prompts, specific prompt templates guide the model through step-by-step analysis during model fine-tuning and reasoning. Its function is to transform a complex problem with multiple implicit needs into a series of clear and actionable sub-goals, which is a prerequisite for achieving accurate set selection.

[0056] For example, its internal reasoning process (learned during training) will output an analysis similar to the following:

[0057] To answer this question, the following points need to be clarified:

[0058] Does this accident include the natural disaster factor of 'heavy rain', and what are the exclusion or compensation clauses in the policy for such situations?

[0059] Being rear-ended is considered a third-party liability accident, and it is necessary to confirm the division of responsibility and the corresponding claims process.

[0060] Whether the engine damage was directly caused by water ingress is related to whether it falls within the scope of vehicle damage insurance coverage.

[0061] The accident occurred after a flood, so it's necessary to clarify the handling principles for such overlapping accidents: should compensation be processed item by item or based on the primary cause?

[0062] In one or more embodiments, the information sub-targets are further divided into multiple granularities, including main targets (such as "claim eligibility"), sub-targets (such as "natural disaster clauses" and "liability determination"), and fine-grained conditions (such as "whether the engine is flooded within the insurance period"), and information needs of different granularities are matched in document selection through a hierarchical attention mechanism.

[0063] Finally, by explicitly and structurally decomposing the user's question, all necessary information sub-objectives are identified and a structured list of information sub-objectives is output, providing clear guidelines for subsequent accurate selection and avoiding information omissions. Each of these information sub-objectives corresponds to an independent information dimension required to answer the user's query.

[0064] S23, based on multiple information sub-targets, select an unordered subset of documents from multiple candidate documents, wherein the document subset is configured as a whole to cover all information sub-targets and be reduced to minimize internal redundancy. In one or more embodiments, the average number of documents in the unordered document subset is less than 5.

[0065] In one or more embodiments, based on multiple information sub-targets, selecting an unordered subset of documents from multiple candidate documents includes:

[0066] For each identified information sub-target, at least one document that meets a preset relevance score is selected from the candidate documents, such as documents with a relevance score ranking of 80% or higher, or documents with other relevance scores. For example, based on the four information needs identified in the previous step, the SBDS model traverses the candidate document pool and selects the 1-2 most relevant documents with the largest amount of information for each need.

[0067] The documents identified for all information sub-objectives will be merged and deduplicated to form a final document subset. For example, the model will eventually output an unordered set of document IDs, such as [doc_03, doc_15, doc_07, doc_11].

[0068] doc_03: The section on "Natural Disaster Exclusions" in the 2024 version of the vehicle damage insurance terms.

[0069] doc_15: The section on "Rear-end Collision Liability Division" in the "Guidelines for Traffic Accident Liability Determination and Claims Settlement".

[0070] doc_07: "Detailed Rules for Claims under Engine Water Damage Insurance" and similar past cases.

[0071] doc_11: Internal Procedures for Handling Accident Claims with Multiple Causes and Single Result.

[0072] In one or more embodiments, selecting an unordered subset of documents from multiple candidate documents based on multiple information sub-targets further includes:

[0073] The model calculates the overall coverage of a document subset across multiple information sub-targets; based on whether the overall coverage reaches a preset threshold, it dynamically adjusts the size of the document subset. For example, it proactively constructs a complete and non-redundant collection of documents. It doesn't care about the original ranking of documents in the list, only about "which ones to select," ensuring that the final selected documents cover, for example, 100% or 80% of all information needs, while removing documents with duplicate content or only partial relevance. Through set-wise rather than ranking output, the model's final output is a set of document IDs, not an ordered list. Its purpose is to eliminate the constraint of "must select Top-K," allowing the number and content of selections to be entirely driven by information needs, achieving true on-demand selection.

[0074] S24, Input a subset of documents along with the user query into the generative model to generate the final answer to the user query.

[0075] The size of the document subset is determined by the number of information sub-targets and the document coverage. For example, the final document set selected by the SBDS system (averaging about 2.9 documents) is input together with the original user query into a large language model (such as Qwen3-32B or GPT-5), which generates the final, comprehensive, and accurate answer. By utilizing high-quality, highly relevant context, reliable and illusion-free answers are generated, improving user experience and system professionalism.

[0076] In one or more embodiments, the aggregate document selection method based on information demand identification further includes a model training step, comprising:

[0077] The training data is labeled using a teacher model to generate a standard list of information sub-targets and corresponding document subsets for the training query. For example, supervised fine-tuning based on the teacher model involves using a powerful teacher model (such as GPT-5) to label massive amounts of data, generating "ideal" information demand analysis and document selection results. For instance, after the generative model initially generates an answer, if the answer confidence is below a threshold, insufficiently covered information sub-targets are identified, and document selection is re-executed to prioritize supplementing the documents corresponding to the missing sub-targets, iterating until the answer confidence reaches the threshold or the maximum number of iterations is reached.

[0078] Using labeled training data, another large open-source language model is subjected to supervised fine-tuning to obtain a model for performing information demand identification and document subset selection. For example, this data can be used to fine-tune a smaller, more efficient open-source model (Qwen3-8B). The goal is to significantly reduce deployment and inference costs while maintaining performance, making this disclosure practically feasible.

[0079] like Figure 3 As shown, the "aggregate document selection method based on information demand identification (e.g., through an SBDS system)" proposed in this disclosure has the following significant advantages compared with existing mainstream traditional reranking techniques:

[0080] 1. Significantly improved accuracy of generated answers: This invention ensures that the retrieved documents comprehensively cover all dimensions required to answer questions by explicitly identifying information needs and performing set optimization.

[0081] 2. Higher system efficiency and lower cost: This invention selects only about 3 documents as context on average, far fewer than the Top-5 or Top-10 that traditional reordering methods forcibly select. It also has stronger anti-interference and anti-noise capabilities.

[0082] 3. Traditional re-ranking methods tend to select high-scoring documents with similar content, which can easily introduce a lot of redundant information and interfere with the generator, leading to a "the more, the worse" phenomenon. This invention actively removes duplicate and irrelevant documents through set selection, constructing a "refined and comprehensive" context.

[0083] 4. The selection process is more intelligent and adaptive, eliminating the fixed threshold of "Top-K". The number of documents selected is dynamically determined by the complexity of the problem and the amount of information needed. For simple problems, only 1-2 documents may be selected; for complex problems, more will be selected, but the sole goal is always to "satisfy all information needs", avoiding resource waste or information gaps.

[0084] 5. The methodology is universal and scalable. The core idea of ​​this disclosure—"from ranking to selection"—is a paradigm innovation that does not depend on a specific model or dataset. It can be seamlessly integrated into any existing RAG pipeline as a standalone module, replacing traditional reorderers. Furthermore, it can be combined with advanced RAG techniques such as iterative retrieval and query rewriting to further improve system performance.

[0085] This disclosure is applicable, for example, to scenarios that require accurate and comprehensive retrieval of information from massive amounts of documents to support complex question-and-answer, report generation, and decision support, such as intelligent customer service, investment research and analysis, insurance underwriting and claims.

[0086] It should be understood that although the steps in the flowcharts of the accompanying figures are shown sequentially as indicated by the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the accompanying figures may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the sub-steps or stages of other steps.

[0087] Collective document selection device

[0088] To implement the technical solutions in the embodiments of this disclosure, one embodiment provides a collective document selection device. This device, for example, is used to implement the aforementioned collective document selection method. Specifically, it can be applied to various electronic terminal devices, such as... Figure 4 As shown, it includes: document retrieval module 401, decomposition and recognition module 402, document subset selection module 403, and result generation module 404.

[0089] The document retrieval module 401 is used to retrieve multiple candidate documents from the knowledge base in response to a user query. Here, the document retrieval module 401 implements, for example, the various steps and functions in S21 of the above method, which will not be described in detail here.

[0090] The decomposition and identification module 402 is used to decompose the user query based on chain-like reasoning, and explicitly identify multiple information sub-targets required to answer the user query. Here, the decomposition and identification module 402 may implement the various steps and functions in S22 of the method, for example, and will not be described in detail here.

[0091] The document subset selection module 403 is used to select an unordered document subset from the multiple candidate documents based on the multiple information sub-targets. The document subset is configured as a whole to cover all the information sub-targets and minimize internal redundancy. Here, the document subset selection module 403 is, for example, to implement the various steps and functions in S23 of the method, which will not be described in detail here.

[0092] Result generation module 404 is used to input the document subset together with the user query into the generation model to generate a final answer to the user query; wherein, the size of the document subset is determined by the number of information sub-targets and the document coverage. Here, result generation module 404 may implement the various steps and functions in S24 of the method, for example, and will not be described in detail here.

[0093] It should be understood that although each block in the block diagrams of the accompanying drawings may represent a module, a portion of which contains one or more executable instructions for implementing a specified logical function, these modules are not necessarily executed sequentially. The modules and functional units in the device embodiments of this disclosure may be integrated into a single processing module, or each unit may exist physically separately, or two or more modules or functional units may be integrated into a single module. The integrated modules described above may be implemented in hardware or as software functional modules. If the integrated module is implemented as a software functional module and sold or used as an independent product, it may also be stored in a computer-readable storage medium. The aforementioned storage medium may be a read-only memory, a disk, or an optical disk, etc.

[0094] Computer equipment

[0095] The following is for reference. Figure 5 It illustrates a computer device or electronic device (e.g., one suitable for implementing embodiments of the present disclosure) Figure 1 The diagram shows the structure of the terminal device (or server) 500 in this embodiment. The terminal device in this disclosure can be any type of terminal device in the above-described system. Figure 5 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments disclosed herein.

[0096] like Figure 5 As shown, the electronic device 500 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 501 for controlling the overall operation of the electronic device. The processing device may include one or more processors to execute instructions to complete all or part of the steps of the method described above. Furthermore, the processing device 501 may also include one or more modules for handling interactions with other devices.

[0097] Storage device 502 is used to store various types of data. Storage device 502 can be a combination of various types of computer-readable storage media, such as electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatuses, or devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this disclosure, a computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0098] Sensor device 503 is used to sense specified measurand information and convert it into a usable output signal according to a certain rule. It may include one or more sensors. For example, it may include an accelerometer, gyroscope, magnetic sensor, pressure sensor, or temperature sensor, etc., for detecting changes in the on / off state, relative positioning, acceleration / deceleration, temperature, humidity, and light of electronic devices.

[0099] The processing device 501, storage device 502, and sensor device 503 are interconnected via bus 504. Input / output (I / O) interface 505 is also connected to bus 504.

[0100] Multimedia device 506 may include input devices such as touch screen, touchpad, keyboard, mouse, camera, and microphone to receive input signals from the user. These input devices can cooperate with various sensors of the aforementioned sensor device 503 to complete tasks such as gesture input, image recognition input, and distance detection input. Multimedia device 506 may also include output devices such as liquid crystal display (LCD), speaker, and vibrator.

[0101] Power supply unit 507 is used to provide power to various devices in electronic devices, and may include a power management system, one or more power supplies, and components for distributing power to other devices.

[0102] The communication device 508 allows the electronic device 500 to communicate wirelessly or wiredly with other devices to exchange data.

[0103] All of the above devices can also be connected to I / O interface 505 to enable the application of electronic device 500.

[0104] Although the figures show electronic devices with various devices, it should be understood that it is not required to implement or have all of the devices shown. More or fewer devices may be implemented or included alternatively.

[0105] In particular, according to embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device, or installed from a storage device. When the computer program is executed by a processing device, it performs the functions defined in the methods of embodiments of this disclosure.

[0106] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0107] It should be noted that the computer-readable medium described above in this disclosure can be a computer-readable signal medium, a computer-readable storage medium, or any combination thereof. In this disclosure, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can also be any computer-readable medium other than a computer-readable storage medium, which can transmit, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination thereof.

[0108] The aforementioned computer-readable medium may be included in the aforementioned electronic device; or it may exist independently and not assembled into the electronic device.

[0109] Computer program code for performing the operations of this disclosure can be written in one or more programming languages ​​or a combination thereof, including but not limited to object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer can be connected to the user's computer via any type of network, or it can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0110] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0111] The units described in the embodiments of this disclosure can be implemented in software or in hardware. The names of the units are not, in some cases, intended to limit the specific unit.

[0112] The functions described above in this document can be performed at least in part by one or more hardware logic components. For example, exemplary types of hardware logic components that can be used, without limitation, include: field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), system-on-a-chip (SoCs), complex programmable logic devices (CPLDs), and so on.

[0113] According to one or more embodiments of this disclosure, a set-based document selection method based on information demand identification is provided for a retrieval enhancement generation system, employing the following technical solution, including:

[0114] In response to user queries, retrieve multiple candidate documents from the knowledge base;

[0115] The user query is decomposed based on chain-like reasoning, and multiple information sub-targets required to answer the user query are explicitly identified.

[0116] Based on the multiple information sub-targets, an unordered subset of documents is selected from the multiple candidate documents, wherein the document subsets are configured as a whole to cover all the information sub-targets and minimize internal redundancy;

[0117] The subset of documents is input together with the user query into the generative model to generate the final answer to the user query;

[0118] The size of the document subset is determined by the number of information sub-targets and the document coverage.

[0119] According to one or more embodiments of this disclosure, a method for selecting aggregated documents is provided, employing the technical solutions described below, including:

[0120] The chain-like reasoning decomposes the user query, explicitly identifying multiple information sub-targets required to answer the user query, including:

[0121] The user query and / or the candidate document are input into a finely tuned large language model;

[0122] The large language model is guided to reason step by step through a pre-set chain-like thinking prompt template;

[0123] Output a structured list of information sub-targets, where each information sub-target corresponds to an independent information dimension required to answer the user query.

[0124] According to one or more embodiments of this disclosure, a method for selecting aggregated documents is provided, employing the technical solutions described below, including:

[0125] The step of selecting an unordered subset of documents from the multiple candidate documents based on the multiple information sub-targets includes:

[0126] For each identified information sub-target, at least one document that meets a preset relevance is determined from the candidate documents;

[0127] The documents identified for all information sub-targets will be merged and deduplicated to form the final document subset.

[0128] According to one or more embodiments of this disclosure, a method for selecting aggregated documents is provided, employing the technical solutions described below, including:

[0129] The step of selecting an unordered subset of documents from the multiple candidate documents based on the multiple information sub-targets further includes:

[0130] Calculate the overall coverage of the document subset to the multiple information sub-targets;

[0131] The size of the document subset is dynamically adjusted based on whether the overall coverage reaches a preset threshold.

[0132] According to one or more embodiments of this disclosure, a method for selecting aggregated documents is provided, employing the technical solutions described below, including:

[0133] The method also includes a model training step, comprising:

[0134] The teacher model is used to label the training data, generating a standard list of information sub-targets and corresponding document subsets for the training query;

[0135] Using labeled training data, another large open-source language model is fine-tuned under supervision to obtain a model for performing the information demand identification and document subset selection.

[0136] According to one or more embodiments of this disclosure, a set-based document selection method is provided, which adopts the following technical solution, including: the average number of documents in the unordered document subset is less than 5.

[0137] According to one or more embodiments of this disclosure, a collection-type document selection device is provided for implementing the method described in any of the preceding claims, employing the technical solutions described below, including:

[0138] The document retrieval module is used to retrieve multiple candidate documents from the knowledge base in response to user queries;

[0139] The decomposition and identification module is used to decompose the user query based on chain-like reasoning and explicitly identify multiple information sub-targets required to answer the user query.

[0140] The document subset selection module is used to select an unordered document subset from the multiple candidate documents based on the multiple information sub-targets, wherein the document subset is configured as a whole to cover all the information sub-targets and minimize internal redundancy;

[0141] The result generation module is used to input the document subset together with the user query into the generation model to generate the final answer to the user query;

[0142] The size of the document subset is determined by the number of information sub-targets and the document coverage.

[0143] According to one or more embodiments of this disclosure, a computer device is provided, employing the technical solutions described below, including a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of the method as described in any of the preceding claims.

[0144] According to one or more embodiments of the present disclosure, a computer-readable storage medium is provided, which employs the technical solution described below. The computer-readable storage medium stores a computer program thereon, which, when executed by a processor, implements the steps of the method as described in any of the preceding claims.

[0145] According to one or more embodiments of this disclosure, a computer program product is provided, including a computer program that employs the technical solutions described below, including the steps of implementing the method as described in any of the preceding claims when the computer program is executed by a processor.

[0146] The above description is merely a preferred embodiment of this disclosure and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of this disclosure is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described concept. For example, technical solutions formed by substituting the above features with (but not limited to) technical features disclosed in this disclosure that have similar functions.

[0147] Furthermore, while the operations are described in a specific order, this should not be construed as requiring these operations to be performed in the specific order shown or in a sequential order. In certain environments, multitasking and parallel processing may be advantageous. Similarly, while several specific implementation details are included in the above discussion, these should not be construed as limiting the scope of this disclosure. Certain features described in the context of individual embodiments may also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented individually or in any suitable sub-combination in multiple embodiments.

[0148] Although the subject matter has been described using language specific to structural features and / or methodological logic, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. Rather, the specific features and actions described above are merely illustrative examples of implementing the claims.

Claims

1. A set-based document selection method based on information demand identification, used in a retrieval enhancement generation system, characterized in that, include: In response to user queries, retrieve multiple candidate documents from the knowledge base; The user query is decomposed based on chain-like reasoning, and multiple information sub-targets required to answer the user query are explicitly identified. Based on the multiple information sub-targets, an unordered subset of documents is selected from the multiple candidate documents, wherein the document subset is configured as a whole to cover all the information sub-targets and reduce internal redundancy; The subset of documents is input together with the user query into the generative model to generate the final answer to the user query; The size of the document subset is determined by the number of information sub-targets and the document coverage.

2. The collection-based document selection method as described in claim 1, characterized in that, The chain-like reasoning decomposes the user query, explicitly identifying multiple information sub-targets required to answer the user query, including: The user query and / or the candidate document are input into a finely tuned large language model; The large language model is guided to reason step by step through a pre-set chain-like thinking prompt template; Output a structured list of information sub-targets, where each information sub-target corresponds to an independent information dimension required to answer the user query.

3. The collection-based document selection method as described in claim 1, characterized in that, The step of selecting an unordered subset of documents from the multiple candidate documents based on the multiple information sub-targets includes: For each identified information sub-target, at least one document that meets a preset relevance is determined from the candidate documents; The documents identified for all information sub-targets will be merged and deduplicated to form the final document subset.

4. The collection-based document selection method as described in claim 1, characterized in that, The step of selecting an unordered subset of documents from the multiple candidate documents based on the multiple information sub-targets further includes: Calculate the overall coverage of the document subset to the multiple information sub-targets; The size of the document subset is dynamically adjusted based on whether the overall coverage reaches a preset threshold.

5. The collection-based document selection method as described in claim 1, characterized in that, The method also includes a model training step, comprising: The teacher model is used to label the training data, generating a standard list of information sub-targets and corresponding document subsets for the training query; Using the labeled training data, another large open-source language model is fine-tuned under supervision to obtain a model for performing the information demand identification and document subset selection.

6. The collective document selection method as described in claim 1, characterized in that, The average number of documents in the unordered subset of documents is less than 5.

7. A set-based document selection device based on information demand identification, used in a retrieval enhancement generation system to implement the steps of the method as described in any one of claims 1-6, characterized in that, include: The document retrieval module is used to retrieve multiple candidate documents from the knowledge base in response to user queries; The decomposition and identification module is used to decompose the user query based on chain-like reasoning and explicitly identify multiple information sub-targets required to answer the user query. The document subset selection module is used to select an unordered document subset from the multiple candidate documents based on the multiple information sub-targets, wherein the document subset is configured as a whole to cover all the information sub-targets and reduce internal redundancy; The result generation module is used to input the document subset together with the user query into the generation model to generate the final answer to the user query; The size of the document subset is determined by the number of information sub-targets and the document coverage.

8. A computer device, comprising a memory, a processor, and a computer program stored in the memory, characterized in that, The processor executes the computer program to implement the steps of the method as claimed in any one of claims 1-6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method as described in any one of claims 1-6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method as described in any one of claims 1-6.

Images