Output result generation method and system, electronic device, and storage medium

Abstract

The present application discloses a method and system for generating an output result, an electronic device, and a storage medium, and relates to the field of artificial intelligence. The method includes: when an inference request for a target inference application is obtained, determining a target retrieval strategy according to an inference application configuration corresponding to the inference request; when the target retrieval strategy is a single knowledge base retrieval strategy, determining a target knowledge base from multiple knowledge bases, and using the target knowledge base to determine a retrieval result of an inference content corresponding to the inference request; when the target retrieval strategy is a multi-knowledge base retrieval strategy, using multiple knowledge bases to jointly determine a retrieval result corresponding to the inference content; constructing a prompt message queue according to prompt words corresponding to the inference application configuration, historical session information corresponding to the inference request, retrieval results, and inference content; and generating an output result corresponding to the inference request according to the prompt message queue through a target large language model.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence, and in particular to a method and system for generating output results, an electronic device, and a storage medium. Background Art

[0002] With the advancement of artificial intelligence (AI), particularly in natural language processing (NLP), services based on large language models (LLMs) have become an important tool for information retrieval and content generation. However, existing LLM services often rely on overly simplistic knowledge base search strategies for complex queries, failing to effectively address the dynamic retrieval requirements and semantic understanding challenges of multi-knowledge base environments. Traditional methods, such as simple keyword or vector search, often only work within a single knowledge base. Their efficiency and accuracy are significantly compromised when faced with queries that require cross-knowledge base searches. Furthermore, intelligently selecting the most appropriate search strategy (single-base precise search or multi-base parallel search) based on the specific query content remains an unresolved issue. More importantly, the processing and integration of search results, as well as how to effectively transform these results into input for language model generation to generate high-quality content, remain technically unresolved. Therefore, a more intelligent, efficient, and adaptable search strategy and result processing method for multi-knowledge base environments is urgently needed to improve the performance of RAG systems in complex scenarios.

[0003] In related technologies, there is currently no effective solution to the problem of how to choose a strategy for single knowledge base retrieval or parallel retrieval of multiple knowledge bases, and efficiently integrate the retrieval results to generate high-quality output content. Summary of the Invention

[0004] The present application provides a method and system for generating output results, an electronic device, and a storage medium to at least solve the problem of how to select a strategy for single knowledge base retrieval or parallel retrieval of multiple knowledge bases, and efficiently integrate the retrieval results to generate high-quality output content.

[0005] The present application provides a method for generating an output result, including: when an inference request for a target inference application is obtained, determining a target retrieval strategy according to an inference application configuration corresponding to the inference request, wherein the inference application configuration has configuration information corresponding to the target inference application; when the target retrieval strategy is a single knowledge base retrieval strategy, determining a target knowledge base from multiple knowledge bases, and using the target knowledge base to determine a retrieval result of an inference content corresponding to the inference request; when the target retrieval strategy is a multi-knowledge base retrieval strategy, using multiple knowledge bases to jointly determine a retrieval result corresponding to the inference content; constructing a prompt message queue according to prompt words corresponding to the inference application configuration, historical session information corresponding to the inference request, retrieval results, and inference content; and generating an output result corresponding to the inference request according to the prompt message queue through a target large language model.

[0006] The present application also provides a system for generating output results, including: a knowledge base retrieval routing component, which is used to determine the target retrieval strategy according to the inference application configuration corresponding to the inference request when an inference request for a target inference application is obtained, wherein the inference application configuration has configuration information corresponding to the target inference application; when the target retrieval strategy is a single knowledge base retrieval strategy, the target knowledge base is determined from multiple knowledge bases, and the target knowledge base is used to determine the retrieval results of the inference content corresponding to the inference request; when the target retrieval strategy is a multi-knowledge base retrieval strategy, multiple knowledge bases are used to jointly determine the retrieval results corresponding to the inference content; a prompt word construction and context enhancement component, which is used to construct a prompt message queue according to the prompt words corresponding to the inference application configuration, the historical session information corresponding to the inference request, the retrieval results and the inference content; a question and answer execution component, which is used to generate the output result corresponding to the inference request according to the prompt message queue through the target large language model.

[0007] The present application also provides an electronic device, comprising: a memory for storing a computer program; and a processor for implementing the steps of any of the above-mentioned methods for generating an output result when executing the computer program.

[0008] The present application also provides a computer-readable storage medium, in which a computer program is stored, wherein when the computer program is executed by a processor, the steps of any of the above-mentioned methods for generating output results are implemented.

[0009] The present application also provides a computer program product, including a computer program, which implements the steps of any of the above-mentioned methods for generating output results when the computer program is executed by a processor.

[0010] This application addresses the shortcomings of traditional retrieval methods in that retrieval strategies are fixed by determining a target retrieval strategy that matches the inference application configuration. Specifically, when the inference request corresponds to a single knowledge base retrieval strategy, it can select the most appropriate target knowledge base for precise retrieval, avoiding the retrieval of irrelevant knowledge bases and improving retrieval efficiency and targeting. When faced with a multi-knowledge base retrieval strategy, this method can leverage multiple knowledge bases for joint retrieval, ensuring more comprehensive information coverage and overcoming the information blind spots that may exist in a single knowledge base. Furthermore, the retrieval results are combined with historical session information, system prompt words, and the user's current inference content to construct a structured prompt message queue, which serves as the input for the large language model. This multi-information source fusion strategy enriches the model's input context, ensures input consistency and effective model utilization, and thus generates high-quality output results that are more tailored to user needs. That is, by intelligently selecting retrieval strategies, efficiently integrating retrieval results, and optimizing model input, this application provides a more accurate and efficient content generation method, effectively solving the difficult problems of retrieval strategy selection and result processing in a multi-knowledge base environment. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] In order to more clearly illustrate the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0012] Figure 1 This is a hardware structure block diagram of a method for generating an output result according to an embodiment of the present application;

[0013] Figure 2 is a schematic diagram of a RAG search process according to an embodiment of the present application;

[0014] Figure 3 is an architectural diagram of a RAG reasoning system according to an embodiment of the present application;

[0015] Figure 4 is a flowchart of a method for generating an output result according to an embodiment of the present application;

[0016] Figure 5 This is a working principle diagram of a knowledge base retrieval routing component according to an embodiment of the present application;

[0017] Figure 6 This is a structural block diagram of a system for generating output results according to an embodiment of the present application. DETAILED DESCRIPTION

[0018] The following will be combined with the accompanying drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0019] It should be noted that, in the description of this application, the terms "comprises," "includes," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. The terms "first," "second," etc., in this application are used to distinguish similar objects, and are not used to describe a particular order or sequence.

[0020] In order to enable those skilled in the art to better understand the present application, the present application is further described in detail below with reference to the accompanying drawings and specific implementation methods.

[0021] In conjunction with the specific application environment architecture or specific hardware architecture on which the execution of the method for generating the output result depends, the specific application environment architecture or specific hardware architecture is described herein.

[0022] The method embodiments provided in the embodiments of the present application can be executed in a server device or a similar computing device. Taking running on a server device as an example, Figure 1 This is a hardware structure diagram of a method for generating an output result according to an embodiment of the present application. Figure 1 As shown, the server device may include one or more ( Figure 1 Only one is shown) a processor 102 (the processor 102 may include but is not limited to a microprocessor MCU or a programmable logic device FPGA) and a memory 104 for storing data. The server device may also include a transmission device 106 and an input / output device 108 for communication functions. It will be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the above server device. Figure 1 More or fewer components than shown, or with Figure 1 Different configurations shown.

[0023] The memory 104 can be used to store computer programs, for example, software programs and modules of application software, such as the computer program corresponding to the startup method of the operating system in the embodiment of the present application. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, that is, implementing the above-mentioned method. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include a memory remotely located relative to the processor 102, and these remote memories can be connected to the server device via a network. Examples of the above-mentioned network include but are not limited to the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof.

[0024] Transmission device 106 is used to receive or transmit data via a network. A specific example of the aforementioned network may include a wireless network provided by a communication provider of the server device. In one embodiment, transmission device 106 includes a network interface controller (NIC), which can be connected to other network devices via a base station to enable communication with the Internet. In another embodiment, transmission device 106 may be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0025] The following is an explanation of the professional terms that appear in this application:

[0026] RAG (Retrieval-Augmented Generation): An AI method that combines information retrieval and text generation. It first retrieves relevant content from external knowledge sources, then injects the retrieval results and user input into a large language model (LLM) to generate a response. It is widely used in scenarios such as question-answering, dialogue, and content creation. It should be noted that Figure 2 A schematic diagram of a RAG search process is shown.

[0027] LLM (Large Language Model): refers to a large-scale pre-trained language model based on the Transformer architecture, such as DeepSeek, ChatGPT, Claude, Gemini, etc., which has the capabilities of context understanding, language generation, and reasoning. It is the core component for performing RAG generation.

[0028] Prompt: Structured input text used to guide the generation of large language models. It is usually composed of system prompts, user input, historical context, and external knowledge. It is a key factor in the quality of RAG generation.

[0029] Knowledge Base: A collection of semantic content that stores structured or unstructured information. In this application, each knowledge base can independently configure retrieval methods, embedding models, and re-ranking strategies, serving as the basic information source for the RAG retrieval stage.

[0030] ReAct (Reasoning + Acting): A language model interaction paradigm that combines reasoning and action (such as calling tools and selecting paths). It can be used to guide large language models to perform intelligent selection and planning in multi-knowledge base scenarios.

[0031] Function Calling: A mechanism that supports the model to autonomously select and call preset functions or tools based on contextual content. In the RAG system, it is used by the model to select the most appropriate knowledge base or perform retrieval behavior.

[0032] Embedding (vector representation): refers to the process of converting text into a high-dimensional vector form, which is used to calculate semantic similarity. In this application, knowledge documents and user queries are both vectorized using the embedding model to achieve semantic retrieval.

[0033] pgvector: An open-source vector database plug-in based on PostgreSQL, used in this application to perform efficient vector search and hybrid search tasks, supporting parallel query and similarity calculation across multiple knowledge bases.

[0034] Hybrid retrieval refers to a comprehensive search approach that combines both vector search and full-text search techniques. This approach balances semantic matching with keyword precision, improving the recall of relevant content. A common approach involves independently performing both searches and then combining the results using simple strategies (such as score weighting or Reciprocal Rank Fusion (RRF)). However, these fusion strategies are often rigid and lack in-depth consideration of query intent or knowledge base characteristics, nor dynamic adaptability.

[0035] Multi-knowledge base processing: a) Parallel retrieval: This method queries all available knowledge bases simultaneously and aggregates the results. This is simple but inefficient and prone to noise. b) Simple routing: This method directs queries to specific knowledge bases based on predefined rules or metadata tags. This method is more targeted than parallel retrieval, but the rules are rigid and it struggles to handle complex queries. However, this multi-knowledge base processing method has limitations in terms of efficiency, flexibility, and depth of query semantics.

[0036] LLM-based preliminary routing / tool ​​selection: Leveraging the LLM's understanding capabilities to select the appropriate knowledge base (think of it as a "tool" or "function call"). While this improves routing intelligence, existing applications often remain at the "selection of a library" level, lacking deep integration with the details of subsequent hybrid search execution. They typically lack dynamic, intelligent switching between "single-library precise search" and "multi-library parallel search + fusion," and lack systematic designs for unified normalization and structured packaging of post-search results to optimize downstream LLM usage. LLM routing itself may also introduce additional latency.

[0037] Although relevant technologies have explored hybrid retrieval, multi-knowledge base query and preliminary routing using LLM, there is a general lack of an integrated and systematic solution. Specifically, there is a lack of a mechanism that can closely combine advanced hybrid retrieval algorithms with intelligent and dynamic knowledge base access strategies (including precise routing and parallel retrieval). Existing solutions are often discrete, with either simple hybrid retrieval strategies or rigid routing logic or decoupling from retrieval execution, making it difficult to flexibly adjust the overall retrieval process based on query characteristics and model capabilities (such as Function Calling). At the same time, there is also a lack of unified and complete design for how to effectively normalize, sort and contextually encapsulate multi-source results after retrieval to serve the RAG generation link in the best way. These deficiencies limit the retrieval efficiency, accuracy and final generation quality of the RAG system in complex scenarios. This application aims to fill these technical gaps.

[0038] In order to solve the problems existing in the related technologies, such as Figure 3 As shown, this application proposes a RAG reasoning system for reasoning applications, aiming to improve the stability, efficiency, and traceability of RAG reasoning services. The system comprises multiple core modules: 1) an inference request concurrency management component, which manages the resource usage of the inference service by setting a maximum concurrency threshold; 2) an inference application configuration management component, which implements configuration version control and priority binding in multi-turn conversations, ensuring conversation consistency and stability; 3) a system observability and behavior tracking component, which collects and records core operations and performance data during the inference process in real time, supporting fault diagnosis and performance optimization; 4) a session and message management component, which ensures the contextual integrity and interaction status of multi-turn conversations; 5) a knowledge base retrieval routing component, which optimizes the efficiency of knowledge base selection and retrieval result integration; 6) a prompt word construction and context enhancement component, which improves the accuracy of responses in multi-turn conversations; and 7) a knowledge base information-driven LLM question-answering execution component, which efficiently connects prompt word input with large language model question-answering capabilities. By integrating and optimizing these modules, this system significantly improves the reliability and user experience of RAG reasoning applications.

[0039] It should be noted that, in this embodiment, a method for generating an output result is provided, which is applied to the above RAG reasoning system. Figure 4 is a flow chart of a method for generating an output result according to an embodiment of the present application, such as Figure 4 As shown, the method includes the following steps S402-S408:

[0040] Step S402: when an inference request corresponding to a target inference application is obtained, determining a target retrieval strategy according to an inference application configuration corresponding to the inference request, wherein the inference application configuration has configuration information corresponding to the target inference application;

[0041] Step S404: When the target search strategy is a single knowledge base search strategy, a target knowledge base is determined from multiple knowledge bases, and the target knowledge base is used to determine the search results for the inference content corresponding to the inference request; when the target search strategy is a multi-knowledge base search strategy, multiple knowledge bases are used to jointly determine the search results for the inference content;

[0042] It should be noted that the above steps S402 and S404 can be executed by the knowledge base search routing retrieval component.

[0043] Step S406: Constructing a prompt message queue based on the prompt words corresponding to the inference application configuration, the historical session information corresponding to the inference request, the search results, and the inference content;

[0044] Step S408: Generate an output result corresponding to the inference request according to the prompt message queue using the target large language model.

[0045] The above steps address the fixed retrieval strategy limitations of traditional retrieval methods by determining a target retrieval strategy that matches the inference application configuration. Specifically, when the inference request corresponds to a single-knowledge base retrieval strategy, it can select the most appropriate target knowledge base for precise retrieval, avoiding the search of irrelevant knowledge bases and improving retrieval efficiency and targeting. When faced with a multi-knowledge base retrieval strategy, this method can leverage multiple knowledge bases for joint retrieval, ensuring more comprehensive information coverage and overcoming potential information blind spots associated with a single knowledge base. Furthermore, the retrieval results are combined with historical session information, system prompts, and the user's current inference content to construct a structured prompt message queue, which serves as input for the large language model. This multi-information fusion strategy enriches the model's input context, ensures input consistency, and effectively utilizes the model, thereby generating high-quality output results that better meet user needs. This application, through intelligent retrieval strategy selection, efficient integration of retrieval results, and optimized model input, provides a more accurate and efficient content generation method, effectively addressing the challenges of retrieval strategy selection and result processing in a multi-knowledge base environment.

[0046] In an exemplary embodiment, before determining the target retrieval strategy according to the inference application configuration corresponding to the inference request, the method further includes the following steps S11-S13: Step S11: when an inference request corresponding to the target inference application is obtained, determining whether the current number of concurrent inference requests is greater than a preset threshold; Step S12: when the current number of concurrent inference requests is greater than the preset threshold, not responding to the inference request; Step S13: when the current number of concurrent inference requests is less than or equal to the preset threshold, generating a request identifier for the inference request, and tracking and managing the inference request throughout its life cycle based on the request identifier;

[0047] Determining a target retrieval strategy based on an inference application configuration corresponding to an inference request includes: when the current number of concurrent inference requests is less than or equal to a preset threshold, determining a retrieval strategy based on an inference application configuration of a target inference application corresponding to an inference request.

[0048] It should be noted that the above steps S11-S13 can be performed by the inference request concurrency management component. The function of the inference request concurrency management component is described in detail below:

[0049] The inference request concurrency management component provides a concurrent request control mechanism for inference application services. It aims to ensure the stability and responsiveness of large-model inference services in conversational generation scenarios with concurrent access by multiple users. This mechanism sets an application-level maximum concurrent request threshold (the aforementioned preset threshold) to determine the acceptance of each request entering the RAG inference system and generates a unique identifier for tracking and managing the request throughout its lifecycle.

[0050] During the normal conversation generation process, the inference request concurrency management component records the request identifier at the start of the request and automatically triggers resource release logic after generation is complete or data stream output ends. Through a unified recycling mechanism, the inference request concurrency management component removes the request identifier from the current limiting control structure and updates the current active request count, ensuring that the RAG inference system can accept new requests in a timely manner, avoiding long-term resource occupation or concurrency count anomalies.

[0051] Furthermore, the inference request concurrency management component features an anomaly detection and resource recovery mechanism. This automatically cleans up related resource states in the event of anomalies such as request timeouts, interruptions, or execution failures, preventing "hanging" requests from impacting overall service operations. As a core scheduling and protection component within the inference service, this mechanism effectively improves the system's concurrent load capacity and resource utilization efficiency, serving as a critical foundation for ensuring the deployment of highly reliable conversational generation services.

[0052] In an exemplary embodiment, the method further includes: during the creation of the target reasoning application, creating an inference application configuration of the target reasoning application based on the acquired configuration information, wherein the inference application configuration includes: prompt words, an associated knowledge base set, retrieval configuration information, an identifier of the large language model and model parameters of the large language model; the retrieval configuration information includes a retrieval strategy.

[0053] It should be noted that the above steps can be performed by the Reasoning Application Configuration Management component. Specifically, this component features a versioned reasoning application configuration management mechanism for RAG-based knowledge base reasoning applications, ensuring configuration stability and interaction consistency across multiple rounds of conversations. This mechanism, centered around "reasoning application configuration," introduces configuration version binding and configuration priority determination to ensure accurate and consistent model behavior responses when users conduct conversational searches in different scenarios.

[0054] When each inference application is created, the system automatically generates a configuration entity for it, called the "inference application configuration." This configuration includes the following key information: system prompts; associated knowledge bases; vector retrieval and re-ranking models and their parameter configurations; the conversation model used and its specific parameters (such as temperature, top-p, maximum token count, etc.); and other metadata related to inference control.

[0055] Inference application configurations feature version management. Whenever a user updates an inference application's configuration (such as changing the knowledge base, adjusting model parameters, or modifying prompts), the system generates a new configuration version and retains historical versions. This facilitates backtracking, comparison, and debugging, while also serving as a foundation for maintaining semantic consistency across multiple rounds of conversation.

[0056] In an exemplary embodiment, before determining the target retrieval strategy according to the inference application configuration corresponding to the inference request, the method further includes the following steps S21-S23:

[0057] Step S21: If the inference request is the first round of a new session of the target inference application and does not carry an inference application configuration, the latest version of the inference application configuration associated with the target inference application is used as the inference application configuration corresponding to the inference request;

[0058] Step S22: If the inference request is a dialog request for a target session where the target inference application already exists and does not carry an inference application configuration, the inference application configuration corresponding to the target session is used as the inference application configuration corresponding to the inference request;

[0059] Step S23: When the inference request carries an inference application configuration, the carried inference application configuration is used as the inference application configuration corresponding to the inference request.

[0060] It should be noted that the above steps can be performed by the inference application configuration management component. Specifically, when a user initiates a conversation with an inference application, the system adopts a three-tiered priority configuration binding mechanism based on the session status and the user request content:

[0061] First priority: explicit incoming configuration (just-in-time configuration);

[0062] When a user initiates a request in debug mode, they can pass the complete inference application configuration along with the request. The system will then prioritize the immediate configuration for executing the conversation, regardless of any other configurations bound to the current session or application.

[0063] Second priority: session binding configuration (historical consistency);

[0064] If the current conversation is a subsequent round of an existing conversation, the system will use the inference application configuration version bound to the first round of the conversation to ensure the consistency of model behavior and the stability of the knowledge base context throughout the conversation.

[0065] Third priority: the latest configuration of the reasoning application;

[0066] If the current request is the first round of a new session for an inference application and no configuration is explicitly passed in, the system will default to using the latest configuration version currently associated with the inference application as the session execution configuration and bind this configuration version to the session when the session is created.

[0067] This three-tier priority mechanism ensures flexibility while also ensuring the stability and continuity of reasoning services. This is especially true in situations involving dynamic adjustment of the knowledge base, optimization of prompt words, and switching of model reordering strategies, thus avoiding session semantic drift or behavioral anomalies.

[0068] For a better understanding, the following application scenarios are described:

[0069] Official usage scenario: Users initiate a conversation with an inference application through the UI or API, without having to worry about the underlying inference application configuration details. The system will automatically select the appropriate inference application configuration based on the above priority rules, achieving seamless configuration management.

[0070] Debugging and testing scenarios: Developers or advanced users can explicitly pass in a complete set of inference application configurations through debugging mode to quickly verify the impact of different knowledge base combinations, prompt word schemes, or model parameters on generation results, supporting efficient tuning.

[0071] Through the above mechanism, the system realizes the traceable, controllable and scalable management capability of reasoning application configuration. While ensuring the consistency of user experience, it also takes into account debugging flexibility and system governance capabilities. It is one of the key designs for the operational reliability of the RAG reasoning system in the actual production environment.

[0072] In an exemplary embodiment, determining a target retrieval strategy according to an inference application configuration corresponding to an inference request includes the following steps S31-S33:

[0073] Step S31: determining whether the target large language model used by the target reasoning application supports function call reasoning capability or thought chain reasoning capability; and obtaining a retrieval strategy from the reasoning application configuration;

[0074] Step S32: if the target large language model supports function call reasoning capability or thought chain reasoning capability, and the retrieval strategy is a single knowledge base retrieval strategy, determining that the target retrieval strategy is a single knowledge base retrieval strategy;

[0075] Step S33: When the target large language model does not support function call reasoning capability or thought chain reasoning capability, or the retrieval strategy is a multi-knowledge base retrieval strategy, determine that the target retrieval strategy is a multi-knowledge base retrieval strategy.

[0076] It should be noted that the above steps are performed by the knowledge base retrieval routing component. In the process of determining the target retrieval strategy, the knowledge base retrieval routing component first intelligently identifies whether the large language model (LLM) used by the target reasoning application has advanced reasoning capabilities such as function calling or thought chaining. This step is crucial because it determines the selection and execution of subsequent retrieval strategies. If the target LLM supports function calling or thought chaining reasoning, and the reasoning application configuration explicitly specifies a single knowledge base retrieval mode, the system will utilize the LLM's intelligent decision-making capabilities to accurately select a target knowledge base that is most relevant to the current query from multiple preset knowledge bases, performing an efficient and accurate search. This avoids wasted resources and interference from redundant information, improving retrieval efficiency and the relevance of results. Conversely, when the target LLM does not support these advanced reasoning capabilities, or when the reasoning application configuration points to a multi-knowledge base retrieval strategy, the system adopts a parallel retrieval approach, simultaneously retrieving information from all knowledge bases. It then processes the retrieval results through a normalization and fusion ranking mechanism, ensuring that information obtained from multiple sources can be effectively integrated, improving the comprehensiveness and accuracy of retrieval from a more global perspective and providing richer and more comprehensive contextual information for subsequent content generation. This flexible strategy adjustment mechanism not only fully leverages the reasoning advantages of the LLM, but also ensures that the RAG system can operate optimally in different scenarios to meet complex query requirements, while optimizing resource utilization and execution efficiency, significantly improving overall service quality and user experience.

[0077] In an exemplary embodiment, the above-mentioned determination of the target knowledge base from multiple knowledge bases includes: converting each knowledge base in the multiple knowledge bases into a structured tool description object to obtain multiple tool description objects, wherein each tool description object contains a unique identifier and usage description of the corresponding knowledge base; and determining the target knowledge base from the multiple knowledge bases based on the multiple tool description objects and reasoning content through the target large language model.

[0078] It should be noted that the above steps are performed by the knowledge base retrieval routing component. In order to achieve the preferred retrieval of a single knowledge base, this application adopts an innovative tool description object conversion mechanism. Specifically, the system first abstracts each knowledge base into a tool description object with a unique identifier and a clear description of its use. This is intended to enable the target large language model to understand the properties and functions of each knowledge base, which is similar to providing the model with a set of "instructions" for the knowledge base. Subsequently, the model intelligently analyzes and selects the target knowledge base that best matches the query intent based on the user's reasoning content and this set of "instructions". This process effectively simulates the behavior of humans in selecting the most appropriate source of information when solving problems, making retrieval more accurate and efficient. Through the semantic understanding and dynamic decision-making of the model, not only the pertinence of the retrieval is improved, but also the access to irrelevant knowledge bases is reduced, which greatly saves system resources, optimizes the quality of the retrieval results, and lays a solid foundation for the subsequent generation of high-quality output content.

[0079] In an exemplary embodiment, determining the retrieval result of the inference content corresponding to the inference request using the target knowledge base can be achieved through the following steps S41-S42:

[0080] Step S41: Determine the configuration parameters of the target knowledge base, wherein the configuration parameters include: retrieval method, identifier of the embedding model, similarity score threshold, and number of returned fragments;

[0081] Among them, the retrieval method refers to the method used to find document fragments. Common ones include vector retrieval (based on semantic similarity) and full-text retrieval (based on keyword matching). The system selects the most appropriate retrieval method based on the characteristics of the knowledge base.

[0082] Embedding model identifier: Indicates the model type used to construct document vectors. Different embedding models affect the construction method of text vectors and the accuracy of similarity calculations.

[0083] Similarity score threshold: Set a score threshold. Only when the similarity score between the document fragment and the query content reaches or exceeds this threshold will it be included in the search results. It is used to filter out irrelevant information and improve the relevance of the results.

[0084] Number of returned fragments: Specifies the upper limit of the number of document fragments the system returns during retrieval to avoid returning too much irrelevant content. It also controls the input scale of downstream processing and ensures the efficiency and accuracy of model generation.

[0085] Step S42: Based on the inference content, the target knowledge base is searched using the configuration parameters, and the retrieved content is normalized and context-encapsulated to obtain the search results, where the search results include: the retrieved fragment text, the similarity score of the retrieved fragment text, and the position of the retrieved fragment text in the target knowledge base.

[0086] It should be noted that the above steps are performed by the knowledge base retrieval routing component. When determining to use the target knowledge base for retrieval, this application further refines the retrieval operation steps to improve the accuracy of the retrieval and the availability of the results. First, the configuration parameters of the target knowledge base are read, including the retrieval method, the identifier of the embedded model, the similarity score threshold, and the number of returned fragments. These parameters together guide the execution of the retrieval task. Subsequently, based on these precise configurations, the system performs a retrieval on the target knowledge base, efficiently calculates the document similarity through the vector database, filters out high-scoring paragraphs, and performs normalization and context encapsulation to form a structured retrieval result, which includes the retrieved text content, similarity score, and specific location information in the knowledge base. This structured output not only facilitates subsequent processing, but also ensures the integrity and accuracy of the retrieval results, providing a solid foundation for the model to generate high-quality content.

[0087] In an exemplary embodiment, using multiple knowledge bases to jointly determine the search results corresponding to the inference content includes the following steps S51-S52:

[0088] Step S51: determining configuration parameters of each of the multiple knowledge bases, and searching the multiple knowledge bases based on the corresponding configuration parameters to obtain multiple search sub-results, wherein the configuration parameters include: search method, identifier of embedding model, similarity score threshold, and number of returned fragments;

[0089] Optionally, the knowledge base retrieval routing component may search each knowledge base through a different thread.

[0090] Step S52: Fusion sorting and re-screening of multiple search sub-results, and normalizing and context-encapsulating the results of fusion sorting and re-screening to obtain search results, wherein the search results include: search fragment text, similarity score of search fragment text, knowledge base corresponding to the search fragment text and its position in the knowledge base.

[0091] It should be noted that the above steps are performed by the knowledge base retrieval routing component. For better understanding, Figure 5 The working principle diagram of the knowledge base retrieval routing component is shown below. Figure 5 The functions of the knowledge base retrieval routing component are described in detail:

[0092] The knowledge base retrieval routing component supports automatically selecting the most relevant knowledge base based on user queries or accessing multiple knowledge bases in parallel in a multi-knowledge base environment. It also performs score normalization and contextualization on retrieval results, providing high-quality generation support information for large language models. This component is particularly suitable for language model inference service systems that support Function Calling or ReAct mechanisms, offering advantages such as flexible strategies, semantic-driven nature, and resource efficiency. The component's core functions include the following:

[0093] 1) Model capability perception and retrieval initialization mechanism:

[0094] When each search task is initiated, the system first determines whether the large language model used by the current inference application supports function calling or ReAct reasoning capabilities. Based on this, it determines whether the search strategy allows the model to participate in planning. Based on this, the system extracts the search configuration of the inference application, including parameters such as top k, score threshold, reranking model, and embedding model, establishing the parameter foundation for subsequent search scheduling.

[0095] 2) Single knowledge base optimization retrieval mechanism:

[0096] When the reasoning application is configured in "Single Knowledge Base Optimization Mode," the system uses the semantic understanding of the user query by the large language model to assist in selecting the target knowledge base that best matches the current request and performs a single, efficient, targeted search on that knowledge base. This strategy effectively reduces redundant computations and multi-source conflicts and is suitable for scenarios with clear domains and concentrated knowledge. The specific process includes:

[0097] 1. The system converts candidate knowledge bases into structured tool description objects. Each object contains its unique identifier and usage description (such as knowledge type, domain label, etc.) for the large language model to identify and compare during the selection process.

[0098] For example, if the system has three candidate knowledge bases, namely "product instructions", "customer feedback records" and "sales data analysis", three corresponding tool descriptions will be generated, such as: "used to answer questions related to product functions and instructions", "contains user feedback and common questions summary", and "stores sales trends and market analysis content".

[0099] 2. The system dynamically selects the appropriate routing module based on the model's capabilities (supporting inference mechanisms such as Function Calling or ReAct). The model receives the user query and a list of knowledge base tools, combines semantic analysis with contextual judgment, and outputs the recommended target knowledge base identifier.

[0100] For example, when a user asks a query such as "What is the battery life of this product?", the large language model can prioritize tool objects related to "product description" based on semantic understanding, thereby routing the query to the most matching knowledge base for targeted retrieval.

[0101] 3. The system reads the local configuration parameters of the selected knowledge base (such as search method, embedding model, score threshold, top k, etc.) and uses these configurations to perform full-text or vector search tasks using a vector database (such as pgvector). In hybrid search scenarios, the system can initiate vector similarity queries and full-text search queries separately, merge the results of both, and select the top k relevant passages with scores above the threshold to serve as the input for subsequent knowledge generation.

[0102] 4. The returned paragraph data is encapsulated into a unified context object, along with the content, similarity score, and location information. The system also records the knowledge base selection path, query behavior, and paragraph hit count, forming a traceable link for subsequent analysis and optimization.

[0103] 3) Multi-knowledge base parallel retrieval and fusion ranking mechanism:

[0104] When the inference application is configured as "multi-knowledge base concurrent retrieval mode", the system schedules all available knowledge bases in parallel, executes independent retrieval tasks according to their respective configurations, and outputs the final results in a unified and integrated manner to form the structured context information required by the downstream generation module.

[0105] The system allocates a separate thread for each knowledge base, reads its local configuration (including search method, top k, score threshold, etc.), and executes the query. The search method is specified by the configuration, typically vector search or vector + full-text search, supported by the underlying vector database (such as pgvector).

[0106] Each thread only returns the top k text segments with scores higher than the threshold. The main thread merges and summarizes the results returned by all threads and constructs a unified document segment structure, annotating its source knowledge base, original score and metadata information.

[0107] The system uniformly fused, sorted, and re-screened all candidate paragraphs based on the global configuration parameters of the inference application. The fusion logic supports multiple strategies (such as default score sorting and preset weights), ultimately retaining the top k paragraphs that meet the global score threshold and outputting them for use by subsequent components.

[0108] The filtered paragraphs are encapsulated in a structured form as a unified context object with content, score, source and location information, which is used for prompt word construction and large model generation tasks.

[0109] Through this mechanism, multiple knowledge sources can be efficiently accessed under configuration drive, retrieval concurrency can be controlled, output format can be unified, upstream and downstream coupling can be avoided, and key support can be provided for the modular deployment and scalability of the RAG system.

[0110] 4) Retrieval result normalization and context encapsulation mechanism:

[0111] All final selected paragraph content is standardized and encapsulated into document context objects in a unified format. Each object contains metadata such as paragraph text, similarity score, source knowledge base, paragraph number, and location information, ensuring that the model input has good structural consistency and semantic interpretability. These objects can be used directly as input in generation tasks and can also be displayed as hit citations on the front end.

[0112] 5) Index hit callback and observable feedback mechanism:

[0113] During the initialization phase of a search task, the component injects a hit callback handler, which is automatically invoked upon search completion. This handler pushes the source information and metadata of the resulting hit document to a message queue or logging system, enabling real-time citation feedback and action traceability. Simultaneously, the system automatically updates the paragraph's hit count and submits related tracking tasks, ensuring a closed-loop, transparent search chain.

[0114] 6) Concurrency safety and execution performance optimization mechanism:

[0115] In multi-threaded scenarios, the system uses a thread lock mechanism to protect the security of shared structure writes and avoid concurrency conflicts. It also uses a high-precision time-consuming tracking function to record the key time nodes of each query process to form performance monitoring indicators. In scenarios where there is no need for reordering, the system automatically enables lightweight keyword matching and vector scoring paths, taking into account both performance and accuracy.

[0116] In summary, the knowledge base retrieval routing component proposed in this application combines the semantic planning capabilities of large language models with the configurable retrieval capabilities of knowledge bases. It supports both single-base optimization and multi-base concurrent modes, and integrates with vector databases for efficient querying and unified encapsulation. This component, capable of structured context generation, behavior tracking, and result normalization, significantly improves the retrieval efficiency, generation quality, and observability of RAG reasoning systems, making it a key supporting module for building multi-knowledge source intelligent question-answering systems.

[0117] In an exemplary embodiment, the above-mentioned construction of a prompt message queue based on the prompt words corresponding to the inference application configuration, the historical session information corresponding to the inference request, the search results, and the inference content can be implemented by the following steps S61-S63:

[0118] Step S61: Filtering target segments from historical conversation information whose relevance to the inference content is higher than a preset relevance threshold;

[0119] Step S62: If the total number of tokens of the prompt word, target segment, search result, and inference content exceeds a preset number, the target segment and search result are trimmed according to the preset number, and a prompt message queue is constructed based on the trimmed target segment, search result, prompt word, and inference content;

[0120] Step S63: When the total number of tokens of prompt words, target segments, retrieval results and inference contents is less than or equal to the preset number, a prompt message queue is directly constructed according to the target segments, retrieval results, prompt words and inference contents.

[0121] It should be noted that the above steps S61-S63 are executed by the prompt word construction and context enhancement component.

[0122] This component is used to integrate multiple types of input content into a structured prompt message sequence before executing an inference request, which is then called by the chat language model (LLM). Its goal is to improve response accuracy and contextual coherence in multi-turn dialogue scenarios. It is an indispensable intermediate processing module in the RAG reasoning system.

[0123] It should be noted that the construction of system prompt content focuses on the organization and integration of the following four types of core information:

[0124] System prompt words: specified by the inference application configuration, defining the model behavior boundaries and dialogue context;

[0125] Historical conversation information: extracts context from previous messages to supplement conversation continuity;

[0126] Knowledge base context: injects the paragraph content returned by the knowledge retrieval component into the prompt as auxiliary information;

[0127] The user's current question: serves as the core input for the current round of dialogue and clearly generates intent.

[0128] This component supports the following key features:

[0129] 1) Memory integration and conversation history embedding mechanism:

[0130] Based on the system's configured memory strategy and token limits, highly relevant segments are filtered from historical messages, converted into user and assistant role messages, and inserted into the prompt structure. This process preserves the critical path of the conversation context and improves the model's understanding of continuous semantics.

[0131] 2) Retrieval context and multimodal information fusion mechanism:

[0132] The paragraphs retrieved from the knowledge base are encapsulated into text content with a unified structure and injected into the prompt sequence as supplementary input for the user role. If there is multimodal information such as images and documents uploaded by the user, it can also be converted into structured prompt content and unified integrated to achieve the model's comprehensive perception of external knowledge and context.

[0133] 3) Token Budget Evaluation and Tip Structure Tailoring Mechanism:

[0134] After the prompt structure is complete, the system estimates the total number of tokens. If the context window exceeds the model's supported context, the system prioritizes pruning historical messages and knowledge base content to ensure that the system prompt and user input are fully preserved, thus ensuring the prompt's expressive integrity and contextual compactness.

[0135] Through the above mechanism, the prompt word organization and context fusion component provides structurally standardized, semantically coherent, and context-aware prompt input for large-scale model reasoning, and is a key supporting module for improving generation effects and user experience.

[0136] In an exemplary embodiment, the above-mentioned generation of the output result corresponding to the inference request by the target large language model according to the prompt message queue includes the following steps S71-S72: Step S71: Constructing a standardized request according to the model parameters of the target large language model in the inference application configuration corresponding to the inference request and the prompt message queue, wherein the standardized request adapts to the interface protocol of the target large language model; Step S72: Calling the target large language model according to the standardized request to obtain the output result of the target large language model.

[0137] In an exemplary embodiment, after the above-mentioned target large language model generates the output result corresponding to the inference request according to the prompt message queue, the method also includes the following steps S81-S82: Step S81: Encapsulate the output result into a structured output object, wherein the structured output object includes: response content, reference context, token consumption, and meta-information of the generated task; the meta-information includes: version, execution time and status information of the target large language model; Step S82: Display the structured output object, and record the structured output object in the session record corresponding to the inference request.

[0138] It should be noted that steps S71-S72 and S81-S82 are both executed by the knowledge base-driven LLM question-answering execution component (hereinafter referred to as the Q&A execution component). This component is used to submit the constructed structured prompt word input to the Large Language Model (LLM) in reasoning applications to complete the question-answer generation task. This component is located at the end of the RAG reasoning process and is responsible for outputting controllable, structured, and traceable generated responses.

[0139] The Q&A execution component first receives the prompt message sequence output by the prompt word construction and context enhancement component. This sequence incorporates the system prompt word, historical conversation context, knowledge base search content, and the user's current question. It then performs structural optimization and token budget tailoring based on the inference application configuration. The Q&A execution component no longer directly processes document segments or knowledge search results, but instead calls the model based on the encapsulated, high-quality prompts.

[0140] The Q&A execution component then constructs standardized generation requests based on the model parameters specified in the inference application (such as temperature, top_p, token cap, and whether streaming output is enabled), adapting to the interface protocols of different model service providers. The system supports both synchronous and asynchronous (SSE) call modes. Streaming mode enables real-time return of generated content and reference information, improving response speed and user interaction experience.

[0141] During the execution of the generated task, the component has a sound exception handling mechanism that supports the identification and fault tolerance of failure conditions such as timeouts, interruptions, and format exceptions. At the same time, it is deeply integrated with the system observability tracking component to automatically record metadata such as token consumption, delay duration, and model configuration used in this inference call, forming a complete traceable record chain.

[0142] Once generated, the Q&A execution component encapsulates the model response into a structured output object, which contains the final answer, the prompt snapshot used, a mapping of referenced knowledge base passages (if applicable), and relevant generation metadata. This response object is synchronously written to the session and message management system, serving as the standard source for subsequent context continuation and response display.

[0143] In summary, the question-answering execution component achieves the efficient connection between prompt word input and the question-answering capabilities of the large model. It is a key terminal execution module to ensure that the retrieval enhancement generation process has response accuracy, context consistency and tracking closed-loop integrity.

[0144] In an exemplary embodiment, the method further includes: when the inference request is the first round of dialogue request of a new session of the target reasoning application, creating a session record of the inference request; when the inference request is the dialogue request of an existing target session of the target reasoning application, updating the active time of the session record corresponding to the target session.

[0145] In an exemplary embodiment, the method further includes: during an interaction in a session, generating a message record for each interaction, wherein the message record includes: interaction input, the inference application configuration corresponding to the interaction, the relationship between the interaction and the previous interaction, and a preset response area placeholder; and recording the message record in the session record of the session.

[0146] In an exemplary embodiment, the method further includes: when an attachment file is carried in the inference request, generating an attachment index of the attachment file, and binding the attachment index to the corresponding message record.

[0147] It should be noted that the above steps are performed by the conversation and message management component, which can ensure that the reasoning application has stable context carrying capacity and interaction status recording capabilities during multiple rounds of dialogue, and dynamically create, update and persist the conversation records and message entities associated with each user request.

[0148] The core functionality of the session and message management component focuses on two aspects: session-level state management for inference applications, and message-level input and output tracking. When a user triggers an inference request, the system first determines the user identity type (e.g., end user or console account) and invocation mode (e.g., formal invocation or debug mode) based on the incoming inference application identifier and invocation source. The system then establishes or retrieves the corresponding session record based on the inference application's current configuration (including information such as the model provider, model identifier, and knowledge base configuration).

[0149] For the first request, the system creates a new session record, including the inference application identifier, model configuration information, session mode, initial input parameters, and user identity. If the session already exists, only the last active time is updated to maintain the timeliness of the conversation context.

[0150] At the session layer, the system also generates a new message record for each interaction, storing key information such as the user input, the inference configuration snapshot (specific configuration settings for this inference request, including model parameters (such as temperature and top_p), knowledge base selection, and token usage policy), parent message associations (used to build context chains), and preset response area placeholders. This data is written to the database in a structured manner and dynamically updated during the execution of the inference task, including the model-generated answer content, token consumption statistics, response latency, and related price information.

[0151] In addition, when a user request includes an uploaded file (such as a document or image), the system will automatically create an attachment metadata record and bind it to the message, ensuring that the reasoning task has a complete resource reference path when it needs to call multimodal information.

[0152] As a key supporting module in the dialogue process of reasoning applications, the conversation and message management component implements full-process coverage of organizing multi-round conversation structures, tracking message chains, and status updates, providing complete and reliable contextual data support for retrieval enhancement generation tasks.

[0153] In an exemplary embodiment, the method further includes: when the target reasoning application is started, initializing a tracking manager for the target reasoning application; and recording events in the target reasoning application in real time through the tracking manager.

[0154] In an exemplary embodiment, events in a target reasoning application are recorded in real time through a tracking manager, including: collecting and recording the following information through the tracking manager: tool usage process information, input content audit tracking, knowledge retrieval process tracking, reply generation link information, and dialogue message processing tracking; wherein, tool usage process information includes: the tool name, input and output content, and execution time of each interactive call; input content audit information includes: the audit status of the input text and whether there is sensitive information; knowledge retrieval process information includes: query text, retrieval trigger conditions, matching document content, and result information; reply generation link information includes: intermediate steps in the large language model generation process, response delays, and generation results; dialogue message processing information includes: the time points of each stage of user message reception, queuing, processing, and return.

[0155] It's important to note that the above steps are performed by the system observability and behavior tracing component, which enhances the observability and behavior traceability of conversational reasoning applications built on the Retrieval Augmentation Generation (RAG) mechanism during operation. It collects, schedules, and persists core operational and performance data throughout the entire reasoning task. This component, centered around the concept of "application-level tracing context," dynamically initializes a tracing manager at the startup of each reasoning application, enabling real-time recording and delayed processing of multiple types of events during system operation without interfering with the main process.

[0156] 1) Application-level tracing context management: The system creates a separate tracing context for each inference application, ensuring that each tracing task is bound to the corresponding application instance. Whenever a user initiates a conversation task, the system establishes a tracing task channel, allowing all behavioral data to flow and be centrally managed within this channel.

[0157] 2) Multi-scenario behavioral data collection capability: This component supports the automatic identification and collection of the following core behavioral information during the reasoning process: tool usage process information, input content audit tracking, knowledge retrieval process tracking, reply generation link information, and dialogue message processing tracking.

[0158] 3) Asynchronous Task Scheduling: The system features a unified task buffer and scheduled processing mechanism. All behavioral data first enters the task buffer, where it is automatically triggered in batches by a scheduled scheduler at set intervals. Data processing is performed asynchronously to avoid blocking the main inference process or impacting performance.

[0159] 4) Adaptive Configuration: The system supports flexible adjustment of the batch processing frequency and data volume of tracking tasks through configuration parameters to adapt to different business environments and server performance. For example, in high-concurrency scenarios, the processing frequency can be increased and the batch interval can be shortened to ensure timely processing and complete recording of tracking data.

[0160] 5) Data persistence and visualization support: All collected tracking data is processed and stored uniformly on disk, forming structured log records. This data can be connected to log analysis systems or visualization platforms to implement performance evaluation of inference services, behavior reproduction, and problem tracing. It can also be used for model tuning and troubleshooting during the R&D phase.

[0161] It should be noted that the tracing and observability management component provides the ability to record the behavior of reasoning applications from input to generation, and is a key technical foundation module to ensure high stability, high maintainability and high explainability of the system.

[0162] Obviously, the embodiments described above are only part of the embodiments of the present application, rather than all the embodiments. In order to better understand the above method, the above process is described below in conjunction with the embodiments, but it is not intended to limit the technical solutions of the embodiments of the present application. Specifically:

[0163] To implement multi-user, multi-scenario Retrieval Enhanced Generation (RAG) applications, this application's RAG reasoning system provides end-to-end support for reasoning request reception, configuration binding, knowledge retrieval, prompt construction, and model generation. The following describes the key implementation steps, combining the system's core components and deployment process:

[0164] Step 1: Create an inference application and generate a configuration version:

[0165] When users create a new inference application on the system, they must specify the initial system prompt word, associated knowledge base, model parameters, and other information. The system automatically generates a configuration entity (the inference application configuration), assigns a version number (config_version), and stores it persistently. For each subsequent configuration change (such as a knowledge base update or prompt word modification), the system generates a new version to support configuration consistency tracking and debugging analysis for historical sessions.

[0166] Step 2: Inference request reception and configuration binding judgment:

[0167] The user triggers an inference request through the front-end or API. The system parses the request parameters and extracts the app_id and session ID (session_id). The system then makes configuration binding decisions based on a three-tier priority mechanism:

[0168] 1. If the immediate configuration is explicitly passed in the request, it is used directly;

[0169] 2. If it is a historical session, the configuration version bound to the session is used;

[0170] 3. If this is a new session, the current version of the inference application is read as the binding configuration for this session. This binding logic ensures consistent model behavior and effectively avoids semantic drift or context mismatch.

[0171] Step 3: Initialize the session and message structure:

[0172] The system checks whether a bound session record exists. If not, a new session entity is created, associating information such as the app_id, config_version, and user identity. A message record is then generated, including the user's current question, request parameters, and configuration snapshots. The system response area is also initialized. If the request includes an attachment (such as a document or image), the system automatically generates an attachment index and binds it to the message for subsequent processing.

[0173] Step 4: Knowledge base retrieval task scheduling and routing selection:

[0174] Depending on the configured search mode, the system enters a single database optimization or multi-database concurrent process:

[0175] Single-database optimization mode: The system calls the Function Calling capability through the language model, performs semantic understanding and tool selection on multiple candidate knowledge bases, and initiates a targeted search after selecting the target library.

[0176] Multi-repository concurrent mode: The system creates a separate thread for each candidate knowledge base, executing search operations concurrently. All search results are aggregated, scored, and ranked. The final results are packaged as standardized context objects, along with metadata such as source identification, similarity score, and paragraph number, and are simultaneously written to the tracking system.

[0177] Step 5: Prompt word sequence construction and context trimming:

[0178] The system constructs a complete prompt word input sequence based on the current session binding configuration and search return results, including: system prompt words; recent conversation context (cut according to token budget); search paragraph content (injected as user supplementary information); current user input question.

[0179] When the total length of the token exceeds the model context window, the system executes the priority clipping strategy to ensure the semantic integrity and model compatibility of the prompt.

[0180] Step 6: Model inference execution and streaming output management:

[0181] The system constructs standard inference call requests based on the configuration, supporting both synchronous and asynchronous (SSE) modes. In streaming mode, the system transmits the response to the frontend in segments, automatically recording the following upon completion: token consumption; response latency; model version and parameter snapshots; and reference segment mapping.

[0182] This process is completed by standard execution components and integrates exception handling logic to ensure high availability and fault tolerance of the inference service.

[0183] Step 7: Request tracing and behavior data collection: When each round of inference task starts, the system initializes the tracing context channel and asynchronously records the following behaviors: tool call status (including time consumption, input and output); input review results; retrieval query path and hit information; model generation link; and dialogue message processing flow.

[0184] It should be noted that all tracking data will be batch-processed regularly by the buffer pool scheduling mechanism and stored in the log system or data platform for performance evaluation and problem reproduction.

[0185] Step 8: Response Structure Encapsulation and Persistence: After generating the output, the system encapsulates the response content, reference context, token statistics, and generated metadata into a standard response object, writes it to the message logging system, and returns it to the user. The last active time of the session is also updated to maintain the context.

[0186] Step 9: Exception Handling and State Recovery Mechanism: The system incorporates a unified exception handling mechanism, covering scenarios such as request interruptions, call failures, and content format errors. In streaming tasks, a wrapper mechanism automatically releases system state (such as flow control flags and active request counts) to prevent resource dangling. In non-streaming scenarios, manual release is supported through the API. A scheduled task is also implemented in the background to clean up expired state every minute to ensure concurrent availability and state consistency.

[0187] It should be noted that this application proposes a modular system architecture for RAG reasoning applications, integrating key mechanisms such as reasoning request concurrency control, application configuration version management, context-aware knowledge retrieval, structured prompt word construction, and model question-answering execution, with the following beneficial effects:

[0188] 1. Implementing a highly available concurrency control mechanism for inference services: This application implements a concurrent request throttling mechanism based on Redis atomic operations. This mechanism supports setting a maximum number of concurrent requests based on the inference application dimension. Combined with automatic release of streaming tasks and periodic cleanup of abnormal states, this effectively prevents system overloads caused by request accumulation. This mechanism achieves state consistency in a multi-node environment, ensuring stable system operation in high-concurrency scenarios.

[0189] 2. Ensuring configuration consistency and semantic stability across multiple conversations: This application introduces a versioning management strategy for inference application configurations and implements dynamic control and consistency maintenance of conversation configurations through a three-tiered priority binding mechanism. This mechanism ensures that users use the same version of prompt words and search parameters throughout the conversation, preventing model behavior drift and effectively improving conversation coherence and reliability.

[0190] 3. Improving the accuracy and adaptability of knowledge base retrieval: This application supports language model-assisted knowledge base selection and concurrent scheduling of multiple knowledge sources. It also achieves efficient information aggregation across knowledge bases through a unified normalization and fusion ranking strategy. Combined with the Function Calling and ReAct mechanisms, the system enables more precise semantic routing, improving the alignment of retrieval and generation tasks.

[0191] 4. Enhance the contextual awareness and structural standardization of prompt input: This application achieves a unified integration of historical dialogues, knowledge retrieval paragraphs, and multimodal information in the prompt word construction phase, and provides a token budget tailoring mechanism to ensure the standardization and semantic coherence of model input structure, effectively improving generation quality and response accuracy, and adapting to multi-round interactions and complex task scenarios.

[0192] 5. Improved system observability and fault traceability: This application includes a built-in asynchronous behavior tracing component that supports recording and analyzing key operations throughout the entire inference task process, including request processing, tool invocation, model execution, and response output. This mechanism provides comprehensive data support for model optimization, system debugging, and troubleshooting, significantly improving the maintainability and explainability of the system.

[0193] 6. System management framework that balances flexibility and governance capabilities: This application supports real-time debugging configuration injection, dynamic adjustment of inference parameters, and management of knowledge base and model versions by application dimension. It takes into account both user customization capabilities and platform governance capabilities, providing a good foundation for large-scale deployment, multi-team collaboration, and application iteration, and has broad engineering adaptability and commercial promotion prospects.

[0194] In summary, this application has significant innovation and practical value in terms of structural design, operating mechanism and function implementation, and provides key support for the large-scale deployment and high-quality service of the retrieval enhancement generation system.

[0195] For a better understanding, the following summarizes the key technical points of this application. Specifically, this application proposes a knowledge base information-driven Retrieval Enhanced Generation (RAG) reasoning system. Focusing on key aspects such as concurrency control of reasoning tasks, configuration consistency, knowledge retrieval accuracy, context construction, and response generation, a highly available and scalable system architecture is constructed. Its key technical points include:

[0196] 1. Inference Request Rate Limiting and State Management Mechanism: Through atomic rate limiting logic implemented at the Redis layer, the system precisely controls the number of concurrent requests for each inference application. Combined with an automatic release mechanism for streaming requests and a strategy for periodically clearing "zombie requests," this effectively ensures system consistency and high availability in a multi-node environment.

[0197] 2. Versioning and binding strategy for inference application configuration: Each inference application configuration supports version management. The system adopts a "three-tier priority" strategy (real-time configuration > session binding > application default) to ensure consistent model behavior across multiple rounds of conversations and avoid semantic drift.

[0198] 3. Intelligent Knowledge Base Search Routing Mechanism: This system supports both single-knowledge base optimization and multi-knowledge base concurrent search modes. The system leverages language model capabilities (such as Function Calling or ReAct) to assist in selecting the most suitable knowledge base and executing the search. Search results are uniformly packaged and normalized, improving accuracy and system efficiency.

[0199] 4. Prompt word construction and context fusion mechanism: This mechanism combines system prompt words, historical conversation content, searched paragraphs, and the user's current question to construct a structured prompt sequence. It then implements a trimming strategy based on token budgets to improve the semantic integrity of the generated context and the quality of the response.

[0200] 5. Structured conversation and message management mechanism: The system automatically manages conversation records and message entities for each round of interaction, including configuration snapshots, input and output content, attachment associations, and token statistics, providing continuous context-carrying capabilities for multi-round reasoning.

[0201] 6. Behavior Tracking and System Observability Component: This component tracks the entire inference process, covering key data such as tool calls, retrieval execution, generation latency, and token consumption. This data is written to the log system using an asynchronous scheduling mechanism for performance analysis and fault location.

[0202] In summary, this application, through the integration of the aforementioned key technical points, achieves comprehensive optimization of the RAG reasoning system in terms of concurrency control, configuration governance, retrieval scheduling, and generation management. It possesses significant advantages such as strong engineering practicality, high deployment adaptability, and good behavioral interpretability. The key modules and implementation mechanisms are the primary protection content of this invention.

[0203] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus the necessary general hardware platform, and of course it can also be implemented by hardware, but in many cases the former is a better implementation method.

[0204] The embodiment of the present application also provides a system for generating output results. Figure 6 is a structural block diagram of a system for generating output results according to an embodiment of the present application, such as Figure 6 As shown, the system includes:

[0205] The knowledge base retrieval routing component 602 is configured to, upon receiving an inference request for a target inference application, determine a target retrieval strategy based on an inference application configuration corresponding to the inference request, wherein the inference application configuration includes configuration information corresponding to the target inference application; if the target retrieval strategy is a single knowledge base retrieval strategy, determine a target knowledge base from multiple knowledge bases, and use the target knowledge base to determine a retrieval result for the inference content corresponding to the inference request; if the target retrieval strategy is a multi-knowledge base retrieval strategy, use multiple knowledge bases to jointly determine a retrieval result corresponding to the inference content;

[0206] The prompt word construction and context enhancement component 604 is used to construct a prompt message queue based on the prompt word corresponding to the reasoning application configuration, the historical session information corresponding to the reasoning request, the search results and the reasoning content;

[0207] The question-answer execution component 606 is used to generate an output result corresponding to the reasoning request according to the prompt message queue through the target large language model.

[0208] The above system addresses the fixed retrieval strategy limitations of traditional retrieval methods by determining a target retrieval strategy that matches the inference application configuration. Specifically, when the inference request corresponds to a single-knowledge base retrieval strategy, it can select the most appropriate target knowledge base for precise retrieval, avoiding the search of irrelevant knowledge bases and improving retrieval efficiency and targeting. When faced with a multi-knowledge base retrieval strategy, this method can leverage multiple knowledge bases for joint retrieval, ensuring more comprehensive information coverage and overcoming potential information blind spots associated with a single knowledge base. Furthermore, the retrieval results are combined with historical session information, system prompts, and the user's current inference content to construct a structured prompt message queue, which serves as input for the large language model. This multi-information fusion strategy enriches the model's input context, ensures input consistency, and effectively utilizes the model, thereby generating high-quality output results that better meet user needs. In other words, by intelligently selecting retrieval strategies, efficiently integrating retrieval results, and optimizing model input, this application provides a more accurate and efficient content generation method, effectively solving the challenges of retrieval strategy selection and result processing in a multi-knowledge base environment.

[0209] In an exemplary embodiment, the output result generation system also includes: an inference request concurrency management component, which is used to determine whether the current number of concurrent inference requests is greater than a preset threshold before determining the target retrieval strategy based on the inference application configuration corresponding to the inference request, when an inference request with the target inference application is obtained; if the current number of concurrent inference requests is greater than the preset threshold, the inference request is not responded to; if the current number of concurrent inference requests is less than or equal to the preset threshold, a request identifier is generated for the inference request, and the inference request is tracked and managed throughout its life cycle based on the request identifier; a knowledge base retrieval routing component 602, which is used to determine the retrieval strategy based on the inference application configuration of the target inference application corresponding to the inference request when the current number of concurrent inference requests is less than or equal to the preset threshold.

[0210] In an exemplary embodiment, the output result generation system also includes: an inference application configuration management component, which is used to create an inference application configuration of the target inference application based on the acquired configuration information during the creation process of the target inference application, wherein the inference application configuration includes: prompt words, associated knowledge base sets, retrieval configuration information, the identifier of the large language model and the model parameters of the large language model; the retrieval configuration information includes the retrieval strategy.

[0211] In an exemplary embodiment, the reasoning application configuration management component is also used to, before determining the target retrieval strategy based on the reasoning application configuration corresponding to the reasoning request, use the latest version of the reasoning application configuration associated with the target reasoning application as the reasoning application configuration corresponding to the reasoning request when the reasoning request is the first round of dialogue request for a new session of the target reasoning application and does not carry the reasoning application configuration; use the reasoning application configuration corresponding to the target session as the reasoning application configuration corresponding to the reasoning request when the reasoning request is a dialogue request for an existing target session of the target reasoning application and does not carry the reasoning application configuration; and use the carried reasoning application configuration as the reasoning application configuration corresponding to the reasoning request when the reasoning request carries the reasoning application configuration.

[0212] In an exemplary embodiment, the knowledge base retrieval routing component 602 is also used to determine whether the target large language model used by the target reasoning application supports function call reasoning capability or thought chain reasoning capability; and to obtain a retrieval strategy from the reasoning application configuration; when the target large language model supports function call reasoning capability or thought chain reasoning capability, and the retrieval strategy is a single knowledge base retrieval strategy, the target retrieval strategy is determined to be a single knowledge base retrieval strategy; when the target large language model does not support function call reasoning capability or thought chain reasoning capability, or the retrieval strategy is a multi-knowledge base retrieval strategy, the target retrieval strategy is determined to be a multi-knowledge base retrieval strategy.

[0213] In an exemplary embodiment, the knowledge base retrieval routing component 602 is also used to convert each of the multiple knowledge bases into a structured tool description object to obtain multiple tool description objects, wherein each tool description object contains a unique identifier and usage description of the corresponding knowledge base; through the target large language model, the target knowledge base is determined from the multiple knowledge bases based on the multiple tool description objects and reasoning content.

[0214] In an exemplary embodiment, the knowledge base retrieval routing component 602 is used to determine the configuration parameters of the target knowledge base, wherein the configuration parameters include: retrieval method, embedding model identifier, similarity score threshold, and number of returned fragments; based on the inference content, the configuration parameters are used to search the target knowledge base, and the retrieved content is normalized and context-encapsulated to obtain the retrieval results, wherein the retrieval results include: retrieval fragment text, the similarity score of the retrieval fragment text, and the position of the retrieval fragment text in the target knowledge base.

[0215] In an exemplary embodiment, the knowledge base retrieval routing component 602 is used to determine the configuration parameters of each knowledge base in a plurality of knowledge bases respectively, and search the plurality of knowledge bases based on the corresponding configuration parameters to obtain a plurality of retrieval sub-results, wherein the configuration parameters include: retrieval method, identifier of the embedding model, similarity score threshold, and number of returned fragments; the plurality of retrieval sub-results are fused, sorted, and re-screened, and the results of the fused sorting and re-screening are normalized and context-encapsulated to obtain a retrieval result, wherein the retrieval result includes: retrieval fragment text, similarity score of retrieval fragment text, knowledge base corresponding to retrieval fragment text, and position of retrieval fragment text in the knowledge base.

[0216] In an exemplary embodiment, the prompt word construction and context enhancement component 604 is also used to filter out target segments whose relevance to the inference content is higher than a preset relevance threshold from historical session information; when the total number of tokens of prompt words, target segments, retrieval results and inference content is greater than a preset number, the target segments and retrieval results are trimmed according to the preset number, and a prompt message queue is constructed based on the trimmed target segments and retrieval results, prompt words and inference content; when the total number of tokens of prompt words, target segments, retrieval results and inference content is less than or equal to the preset number, a prompt message queue is directly constructed based on the target segments, retrieval results, prompt words and inference content.

[0217] In an exemplary embodiment, the question-and-answer execution component 606 is also used to construct a standardized request based on the model parameters of the target large language model and the prompt message queue in the inference application configuration corresponding to the inference request, wherein the standardized request adapts to the interface protocol of the target large language model; and calls the target large language model according to the standardized request to obtain the output result of the target large language model.

[0218] In an exemplary embodiment, the question-and-answer execution component 606 is also used to generate the output result corresponding to the inference request according to the prompt message queue through the target large language model, and then encapsulate the output result into a structured output object, wherein the structured output object includes: response content, reference context, token consumption status, and metadata of the generated task; the metadata includes: version, execution time and status information of the target large language model; display the structured output object, and record the structured output object in the session record corresponding to the inference request.

[0219] In an exemplary embodiment, the output result generation system also includes: a session and message management component, which is used to create a session record of the reasoning request when the reasoning request is the first round of dialogue request of a new session of the target reasoning application; and update the active time of the session record corresponding to the target session when the reasoning request is a dialogue request of an existing target session of the target reasoning application.

[0220] In an exemplary embodiment, the conversation and message management component is also used to generate a message record for each interaction during the interaction in the conversation, wherein the message record includes: the interaction input, the reasoning application configuration corresponding to the interaction, the relationship between the interaction and the previous interaction, and a preset response area placeholder; and the message record is recorded in the conversation record of the conversation.

[0221] In an exemplary embodiment, the session and message management component is further configured to generate an attachment index of the attachment file when an attachment file is carried in the inference request, and bind the attachment index to the corresponding message record.

[0222] In an exemplary embodiment, the output result generation system also includes: a system observability and behavior tracking component, which is used to initialize a tracking manager for the target reasoning application when the target reasoning application is started; and to record events in the target reasoning application in real time through the tracking manager.

[0223] In an exemplary embodiment, the system observability and behavior tracking component is also used to collect and record the following information through the tracking manager: tool usage process information, input content audit tracking, knowledge retrieval process tracking, reply generation link information and dialogue message processing tracking; wherein, tool usage process information includes: the tool name, input and output content and execution time called for each interaction; input content audit information includes: the audit status of the input text and whether there is sensitive information; knowledge retrieval process information includes: query text, retrieval trigger conditions, matching document content and result information; reply generation link information includes: intermediate steps in the large language model generation process, response delays and generation results; dialogue message processing information includes: the time points of each stage of user message from reception, queuing, processing to return.

[0224] For the description of the features in the embodiment corresponding to the output result generation system, reference can be made to the relevant description of the embodiment corresponding to the output result generation method, which will not be repeated here.

[0225] An embodiment of the present application further provides an electronic device, comprising a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to execute the steps in any of the above-mentioned output result generation method embodiments.

[0226] An embodiment of the present application further provides a computer-readable storage medium, in which a computer program is stored, wherein the computer program is configured to execute the steps of any of the above-mentioned output result generation method embodiments when run.

[0227] In an exemplary embodiment, the computer-readable storage medium may include, but is not limited to, various media that can store computer programs, such as a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk, or an optical disk.

[0228] An embodiment of the present application further provides a computer program product, which includes a computer program. When the computer program is executed by a processor, the computer program implements the steps in any of the above-mentioned output result generation method embodiments.

[0229] An embodiment of the present application further provides another computer program product, comprising a non-volatile computer-readable storage medium, wherein the non-volatile computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps in any of the above-mentioned output result generation method embodiments are implemented.

[0230] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the above description has generally described the components and steps of each example according to their functions. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0231] The above is a detailed introduction to a method and device for generating an output result, an electronic device, a storage medium, and a computer program product provided by the present application. Specific examples are used herein to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method and core idea of ​​the present application. It should be pointed out that for ordinary technicians in this technical field, without departing from the principles of the present application, several improvements and modifications can be made to the present application, and these improvements and modifications also fall within the scope of protection of the claims of the present application.

Claims

1. A method for generating an output result, characterized in that: include: In the case of obtaining an inference request for a target inference application, determining a target retrieval strategy according to an inference application configuration corresponding to the inference request, wherein the inference application configuration has configuration information corresponding to the target inference application; In the case where the target retrieval strategy is a single knowledge base retrieval strategy, a target knowledge base is determined from multiple knowledge bases, and the target knowledge base is used to determine the retrieval result of the inference content corresponding to the inference request; in the case where the target retrieval strategy is a multi-knowledge base retrieval strategy, the multiple knowledge bases are used to jointly determine the retrieval result corresponding to the inference content; Building a prompt message queue according to the prompt word corresponding to the inference application configuration, the historical session information corresponding to the inference request, the search result and the inference content; Generate an output result corresponding to the inference request according to the prompt message queue using a target large language model; Among them, the target retrieval strategy is determined according to the reasoning application configuration corresponding to the reasoning request, including: determining whether the target large language model used by the target reasoning application supports function call reasoning capability or thought chain reasoning capability; and obtaining the retrieval strategy from the reasoning application configuration; when the target large language model supports the function call reasoning capability or the thought chain reasoning capability, and the retrieval strategy is a single knowledge base retrieval strategy, determining that the target retrieval strategy is the single knowledge base retrieval strategy; when the target large language model does not support the function call reasoning capability or the thought chain reasoning capability, or the retrieval strategy is a multi-knowledge base retrieval strategy, determining that the target retrieval strategy is the multi-knowledge base retrieval strategy.

2. The method for generating an output result according to claim 1, wherein: Before determining the target retrieval strategy according to the inference application configuration corresponding to the inference request, the method further includes: When an inference request related to a target inference application is obtained, determining whether the current number of concurrent inference requests is greater than a preset threshold; if the current number of concurrent inference requests is greater than the preset threshold, not responding to the inference request; if the current number of concurrent inference requests is less than or equal to the preset threshold, generating a request identifier for the inference request, and tracking and managing the inference request throughout its life cycle based on the request identifier; Determining a target retrieval strategy according to the inference application configuration corresponding to the inference request, including: when the current number of concurrent inference requests is less than or equal to the preset threshold, determining a retrieval strategy according to the inference application configuration of the target inference application corresponding to the inference request.

3. The method for generating an output result according to claim 1, wherein: The method further comprises: During the creation of the target reasoning application, the reasoning application configuration of the target reasoning application is created based on the acquired configuration information, wherein the reasoning application configuration includes: prompt words, associated knowledge base sets, retrieval configuration information, the identifier of the large language model and the model parameters of the large language model; the retrieval configuration information includes a retrieval strategy.

4. The method for generating an output result according to claim 1, wherein: Before determining the target retrieval strategy according to the inference application configuration corresponding to the inference request, the method further includes: If the inference request is a first-round dialogue request of a new session of the target inference application and does not carry an inference application configuration, use the latest version of the inference application configuration associated with the target inference application as the inference application configuration corresponding to the inference request; If the inference request is a dialog request for a target session in which the target inference application already exists and does not carry an inference application configuration, use the inference application configuration corresponding to the target session as the inference application configuration corresponding to the inference request; In a case where the inference request carries an inference application configuration, the carried inference application configuration is used as the inference application configuration corresponding to the inference request.

5. The method for generating an output result according to claim 1, wherein: Identify a target knowledge base from multiple knowledge bases, including: Converting each of the plurality of knowledge bases into a structured tool description object to obtain a plurality of tool description objects, wherein each tool description object includes a unique identifier of the corresponding knowledge base and a usage description; A target knowledge base is determined from a plurality of knowledge bases according to the plurality of tool description objects and the reasoning content through the target large language model.

6. The method for generating an output result according to claim 1, wherein: Determining a retrieval result of the inference content corresponding to the inference request using the target knowledge base includes: Determining configuration parameters of the target knowledge base, wherein the configuration parameters include: retrieval method, identifier of embedding model, similarity score threshold, and number of returned fragments; According to the inference content, the target knowledge base is searched using the configuration parameters, and the retrieved content is normalized and context-encapsulated to obtain a search result, wherein the search result includes: a search fragment text, a similarity score of the search fragment text, and a position of the search fragment text in the target knowledge base.

7. The method for generating an output result according to claim 1, wherein: Using the multiple knowledge bases to jointly determine a search result corresponding to the reasoning content includes: Determining configuration parameters for each of the plurality of knowledge bases, and searching the plurality of knowledge bases based on the corresponding configuration parameters to obtain a plurality of search sub-results, wherein the configuration parameters include: a search method, an identifier of an embedding model, a similarity score threshold, and a number of returned fragments; Multiple search sub-results are fused, sorted, and re-screened, and the results of the fused sorting and re-screening are normalized and context-encapsulated to obtain search results, wherein the search results include: search fragment text, the similarity score of the search fragment text, the knowledge base corresponding to the search fragment text, and the position of the search fragment text in the knowledge base.

8. The method for generating an output result according to claim 1, wherein: Constructing a prompt message queue according to the prompt word corresponding to the inference application configuration, the historical session information corresponding to the inference request, the search result, and the inference content, including: Filtering target segments from the historical conversation information, the target segments having a relevance to the inference content higher than a preset relevance threshold; When the total number of tokens of the prompt word, the target segment, the search result, and the inference content is greater than a preset number, the target segment and the search result are trimmed according to the preset number, and a prompt message queue is constructed based on the trimmed target segment, the search result, the prompt word, and the inference content; When the total number of tokens of the prompt word, the target segment, the retrieval result and the inference content is less than or equal to the preset number, a prompt message queue is directly constructed based on the target segment, the retrieval result, the prompt word and the inference content.

9. The method for generating an output result according to claim 1, wherein: Generating an output result corresponding to the inference request according to the prompt message queue using the target large language model includes: Constructing a standardized request according to the model parameters of the target large language model in the inference application configuration corresponding to the inference request and the prompt message queue, wherein the standardized request is adapted to the interface protocol of the target large language model; The target large language model is called according to the standardization request to obtain an output result of the target large language model.

10. The method for generating an output result according to claim 1, wherein: After generating an output result corresponding to the inference request according to the prompt message queue using the target large language model, the method further includes: Encapsulating the output result into a structured output object, wherein the structured output object includes: response content, reference context, token consumption status, and meta-information of the generated task; the meta-information includes: version, execution time, and status information of the target large language model; The structured output object is displayed, and the structured output object is recorded in a session record corresponding to the inference request.

11. The method for generating an output result according to claim 1, wherein: The method further comprises: In the case where the inference request is the first round of dialogue request for a new session of the target reasoning application, a session record of the inference request is created; in the case where the inference request is a dialogue request for an existing target session of the target reasoning application, the active time of the session record corresponding to the target session is updated.

12. The method for generating an output result according to claim 11, wherein: The method further comprises: During the interaction process in the session, a message record is generated for each interaction, wherein the message record includes: the interaction input, the inference application configuration corresponding to the interaction, the relationship between the interaction and the previous interaction, and the preset response area placeholder; The message record is recorded in a session record of the session.

13. The method for generating an output result according to claim 12, wherein: The method further comprises: In the case where an attachment file is carried in the inference request, an attachment index of the attachment file is generated, and the attachment index is bound to the corresponding message record.

14. The method for generating an output result according to claim 1, wherein: The method further comprises: When the target reasoning application is started, initializing a tracking manager for the target reasoning application; Events in the target reasoning application are recorded in real time by the tracking manager.

15. The method for generating an output result according to claim 14, wherein: Recording events in the target reasoning application in real time by the tracking manager includes: The tracking manager collects and records the following information: tool usage process information, input content audit tracking, knowledge retrieval process tracking, reply generation link information and dialogue message processing tracking; Among them, the tool usage process information includes: the tool name, input and output content and execution time of each interactive call; the input content review information includes: the review status of the input text and whether there is sensitive information; the knowledge retrieval process information includes: query text, retrieval trigger conditions, matching document content and result information; the reply generation link information includes: the intermediate steps in the large language model generation process, response delay and generation results; the dialogue message processing information includes: the time points of each stage from user message reception, queuing, processing to return.

16. A system for generating an output result, characterized in that: include: A knowledge base retrieval routing component is configured to, upon obtaining an inference request for a target inference application, determine a target retrieval strategy based on an inference application configuration corresponding to the inference request, wherein the inference application configuration includes configuration information corresponding to the target inference application; when the target retrieval strategy is a single knowledge base retrieval strategy, determine a target knowledge base from multiple knowledge bases, and use the target knowledge base to determine a retrieval result for the inference content corresponding to the inference request; and when the target retrieval strategy is a multi-knowledge base retrieval strategy, use the multiple knowledge bases to jointly determine a retrieval result corresponding to the inference content. A prompt word construction and context enhancement component is used to construct a prompt message queue according to the prompt word corresponding to the reasoning application configuration, the historical session information corresponding to the reasoning request, the search result and the reasoning content; A question-answer execution component, configured to generate an output result corresponding to the inference request according to the prompt message queue using a target large language model; Among them, the knowledge base retrieval routing component is also used to determine whether the target large language model used by the target reasoning application supports function call reasoning capability or thought chain reasoning capability; and to obtain the retrieval strategy from the reasoning application configuration; when the target large language model supports function call reasoning capability or thought chain reasoning capability, and the retrieval strategy is a single knowledge base retrieval strategy, the target retrieval strategy is determined to be a single knowledge base retrieval strategy; when the target large language model does not support function call reasoning capability or thought chain reasoning capability, or the retrieval strategy is a multi-knowledge base retrieval strategy, the target retrieval strategy is determined to be a multi-knowledge base retrieval strategy.

17. An electronic device, characterized in that: include: memory for storing computer programs; A processor, configured to implement the steps of the method for generating an output result as claimed in any one of claims 1 to 15 when executing the computer program.

18. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, wherein when the computer program is executed by a processor, the steps of the method for generating an output result according to any one of claims 1 to 15 are implemented.

19. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method for generating an output result according to any one of claims 1 to 15 are implemented.

Images