Low-delay intelligent question answering method and device, electronic equipment and storage medium

Abstract

The invention discloses a low-delay intelligent question answering method and device, electronic equipment and a storage medium. The method comprises the following steps: checking a current query request sent by a client, and feeding back first frame data to the client; determining whether a historical intention category and a historical tool calling track abstract matched with the current query request exist in a historical mapping set or not; if yes, generating a final query result according to the historical intention category and the historical tool calling track abstract, and pushing the final query result to the client; if not, selecting an effective historical intention category and an effective historical tool calling track abstract from the historical mapping set, converting the effective historical intention category and the effective historical tool calling track abstract into a few-sample example fragment, and taking the few-sample example fragment as a supplementary prompt of a first-stage large model; determining a tool planning result of the current query request through the first-stage large model and supplementary prompt; performing tool scheduling execution according to a tool planning result; and utilizing the second-stage large model to generate a final query result according to a scheduling execution result of the tool, and pushing the final query result to the client.

Description

Technical Field

[0001] This invention relates to the fields of artificial intelligence and natural language processing, and in particular to a low-latency intelligent question-answering method, apparatus, electronic device, and storage medium. Background Technology

[0002] With the rapid development of artificial intelligence and natural language processing technologies, intelligent question-answering systems and agents based on streaming interaction frameworks have been widely used in financial investment consulting, information consulting, and many other fields. These systems need to meet the core requirement of low-latency response. Therefore, how to achieve low-latency intelligent question answering has become an urgent technical problem to be solved. Summary of the Invention

[0003] This invention provides a low-latency intelligent question-answering method, apparatus, electronic device, storage medium, and computer program product.

[0004] According to one aspect of the present invention, a low-latency intelligent question-answering method is provided, comprising:

[0005] The system validates the current query request sent by the client and sends the first frame of streaming interaction data back to the client after the validation passes.

[0006] Determine whether there are historical intent categories and historical tool call trajectory summaries that match the current query request in the pre-built historical mapping set; wherein, the historical mapping set stores the association relationships between historical query requests and their corresponding intent categories and tool call trajectory summaries;

[0007] If they exist, the final query results are generated based on the matched historical intent categories and historical tool call trajectory summaries, and then pushed to the client in a streaming manner.

[0008] If not, select some valid historical intent categories and valid historical tool call trajectory summaries from the historical mapping set, and convert the valid historical intent categories, valid historical tool call trajectory summaries and their corresponding historical query requests into few sample example fragments, and use them as supplementary prompts for the first-stage large model;

[0009] Based on the first-stage large model and supplementary hints, the intent category of the current query request and the tool planning results are determined;

[0010] Based on the tool planning results, the tools are scheduled and executed; using the second-stage large model, based on the obtained tool scheduling and execution results, the final query results are generated and pushed to the client in a streaming manner.

[0011] According to another aspect of the present invention, a low-latency intelligent question-answering device is provided, comprising:

[0012] The verification module is used to verify the current query request sent by the client and to send the first frame of streaming interaction data back to the client after the verification is successful.

[0013] The retrieval and judgment module is used to determine whether there are historical intent categories and historical tool call trajectory summaries that match the current query request in the pre-built historical mapping set; wherein, the historical mapping set stores the association relationships between historical query requests and corresponding intent categories and tool call trajectory summaries respectively;

[0014] The direct response module is used to generate the final query results based on the matched historical intent category and historical tool call trajectory summary, if they exist, and push them to the client in a streaming manner;

[0015] The construction module is used to select a portion of valid historical intent categories and valid historical tool call trajectory summaries from the historical mapping set if they do not exist, and convert the valid historical intent categories, valid historical tool call trajectory summaries and their corresponding historical query requests into few-sample example fragments, which serve as supplementary prompts for the first-stage large model.

[0016] The first-stage processing module is used to determine the intent category of the current query request and the tool planning results through the first-stage large model and supplementary hints;

[0017] The second-stage processing module is used to schedule and execute the tool based on the tool planning results; using the second-stage large model, based on the obtained tool scheduling and execution results, it generates the final query results and pushes them to the client in a streaming manner.

[0018] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising:

[0019] At least one processor; and

[0020] A memory that is communicatively connected to at least one processor; wherein,

[0021] The memory stores a computer program that can be executed by at least one processor, such that the at least one processor is able to execute the low-latency intelligent question-answering method of the present invention.

[0022] According to another aspect of the present invention, a computer-readable storage medium is provided, which stores computer instructions for causing a processor to execute and implement the low-latency intelligent question-answering method of the present invention.

[0023] According to another aspect of the present invention, a computer program product is provided, comprising a computer program that, when executed by a processor, implements the steps in the above-described method.

[0024] The technical solution of this invention pushes the first frame of data to the client immediately after the current query request passes the validity check, avoiding misunderstandings caused by prolonged lack of response and significantly improving the user's perceived timeliness. This is especially suitable for scenarios sensitive to real-time interaction (such as financial consultation and real-time customer service). It prioritizes matching the historical mapping set; if a historical intent category and tool call trajectory summary are matched, the final result is directly generated based on the historical information, without triggering time-consuming steps such as large-scale model inference and tool scheduling. This design achieves process short-circuiting in historical hit scenarios, especially suitable for high-frequency repeated queries (such as users repeatedly querying the same type of information), significantly improving the timeliness of intelligent question answering. For scenarios where historical mappings are not matched, valid historical information is converted into a few sample fragments as supplementary prompts, providing a basis for the first stage of large-scale model inference. The model provides precise task anchors, and the few sample examples directly provide historical query association paradigms for intents / tools, allowing large models to quickly align task logic without starting from scratch. This shortens the inference time for intent recognition and tool planning, directly accelerating the timeliness of core processing steps in missed scenarios. Tool planning results are directly scheduled and executed concurrently after verification, without secondary confirmation. The original execution results of the tools are used as input for the second-stage model, without additional data transformation waiting. More importantly, the entire process uses streaming push, so users do not need to wait for the complete result to be generated before receiving it. They can obtain the process progress in real time (such as receiving intermediate data of tool execution first, and then receiving the final organized result), which avoids the perception of long waiting time for the complete result at once and makes the timeliness of the entire processing visible, further enhancing the user's perception of the system's fast processing speed.

[0025] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0026] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0027] Figure 1 This is a flowchart illustrating a low-latency intelligent question-answering method provided in an embodiment of the present invention;

[0028] Figure 2 This is a flowchart illustrating another low-latency intelligent question-answering method provided in an embodiment of the present invention;

[0029] Figure 3This is a schematic diagram of the structure of a low-latency intelligent question-answering device provided in an embodiment of the present invention;

[0030] Figure 4 This is a schematic diagram of the structure of an electronic device that implements the low-latency intelligent question-answering method of this invention. Detailed Implementation

[0031] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0032] Example 1

[0033] Figure 1 The flowchart illustrates a low-latency intelligent question-answering method provided in this embodiment of the invention. This embodiment is applicable to intelligent question-answering scenarios requiring low latency, typically financial consultation or real-time customer service scenarios. The method can be executed by a low-latency intelligent question-answering device, which can be implemented in hardware and / or software and can be configured in an electronic device.

[0034] like Figure 1 As shown, low-latency intelligent question answering methods include:

[0035] S101. Verify the current query request sent by the client, and after the verification is successful, send the first frame of streaming interaction data back to the client.

[0036] In this context, the client refers to the terminal device or application used by the user to initiate query requests, interact with the server (the system that executes the intelligent question-answering logic), and receive the results returned by the server. It serves as the interaction entry point between the user and the intelligent question-answering system. Current query request verification verifies the legitimacy of the query request sent by the client. Core verification dimensions include terminal identifier compliance, version compatibility, and user permission matching, preventing illegal or invalid requests from consuming system resources. The first frame of streaming interaction data refers to the first frame of interaction data pushed to the client after successful verification. It does not contain the final result and is only used to provide a status message indicating that the system has received the request and started processing.

[0037] Specifically, in a mobile banking app financial consultation scenario, the client (e.g., a bank app version 3.2) sends a query request (e.g., querying real-time market data for XXX fund / stock on October 20, 2024). The query request undergoes legality checks including terminal identification verification, version verification, and permission verification. Terminal identification verification primarily checks whether the client's application code (e.g., "BANK_001") is on the system whitelist (excluding unauthorized third-party apps). Version verification confirms that the client version is V3.0 or higher (versions lower than this do not support real-time market data streaming). Permission verification verifies that the user has completed real-name authentication and activated stock market data query permissions. After successful verification, the first frame of data is encapsulated according to a standardized format, and then a streaming channel is built based on the SSE (Server-SentEvents) protocol to send the encapsulated first frame of data to the client. An example of the first frame of data is: "Your query has been received. Retrieving real-time data for XXX fund / stock. Please wait."

[0038] Understandably, once a user's query request is verified, the system immediately returns the first frame of data to prevent users from experiencing system lag due to lack of response; and it filters out illegal requests in advance (such as unauthorized clients or outdated terminals) to prevent invalid requests from entering subsequent complex processes and reduce waste of system resources.

[0039] S102. Determine whether there is a historical intent category and historical tool call trajectory summary that matches the current query request in the pre-built historical mapping set.

[0040] The historical mapping set is a relational information database built upon historical session data, essentially serving as a data carrier for reusable historical processing logic for the current query. Specifically, the historical mapping set stores the associations between historical query requests and their corresponding intent categories and tool call trajectory summaries. More specifically, the historical mapping set includes a historical intent set and a historical tool set. The historical intent set includes multiple intent items, each containing at least a historical query request, its intent category, and a normalized encoding of the historical query request. The historical tool set includes multiple tool summary items, each containing at least a historical query request, a tool call trajectory summary of the historical query request, and a normalized encoding of the historical query request. The normalized encoding of the historical query request is obtained by encoding the normalized historical query request. Each intent item corresponds to a query feature identifier (e.g., the hash value of the normalized historical query request), and similarly, each tool summary item also corresponds to a query feature identifier (e.g., the hash value of the normalized historical query request). In some embodiments, the current query request is normalized and a feature identifier is generated; based on the feature identifier, a search is performed in a pre-built historical information set to determine whether there is a historical intent category and a historical tool call trajectory summary that are consistent with the current query request and have the same requirements; wherein, the historical tool call trajectory summary includes multiple tool names and key parameters corresponding to each tool name.

[0041] In specific implementation, it is determined whether there exists a historical intent category and historical tool call trajectory summary in the pre-built historical mapping set that matches the current query request, including S1021-S1023:

[0042] S1021. Normalize the current query request and calculate the current hash value of the normalized query request.

[0043] Normalization refers to preprocessing user query requests, including redundancy removal, desensitization, format standardization, and parameter extraction, to eliminate differences in the expression of the same requirement and make the current query comparable to historical queries. The current hash value is a unique feature identifier generated by a hash algorithm for the normalized current query request, which is used to quickly match related records in the historical set (same hash value or extremely high similarity indicates consistent query requirements).

[0044] Specifically, the normalization process for the current query request includes: using regular expressions to filter redundant and sensitive information in the current query, mainly removing modal particles, special symbols, repeated characters, and sensitive information while retaining the core semantics; extracting structured parameters through a named entity recognition model, classifying them according to preset dimensions, taking a financial scenario as an example, extracting the subject type as individual stock, the subject identifier as specific stock code, the time range as today, and the demand type as stock price query; reorganizing parameters based on the scenario-preset query template to generate normalized text in a unified format. Then, the SHA-256 encrypted hash algorithm can be selected to ensure that the same normalized text corresponds to a unique hash value, and the probability of hash value collisions between different texts is extremely low; in specific calculations, the normalized text is converted into a UTF-8 encoded byte stream and input into the SHA-256 algorithm to obtain the current hash value. Hash value storage: the current hash value, along with the original query and normalized text, is stored in a session cache (such as a Redis temporary key-value pair) for easy reuse in subsequent steps.

[0045] To facilitate understanding, let's take a financial consultation scenario (mobile banking app query) as an example. The user's current query request is: "Could you please check the stock code of XX Bank, which seems to be 60XXX1? What is its stock price today?". After filtering with a regular expression, we get: "XX Bank 60XXX1's stock price today". The financial NER model extracts the parameters: Request type = stock price, Subject = XX Bank (60XXX1), Time = today. It generates standardized text according to the template: Stock price query_Individual stock_60XXX1_Today. Using the SHA-256 algorithm, it generates the current hash value (0x7a3f9d8b4e2c1a0e...2d) based on the standardized text.

[0046] Understandably, text cleaning and parameter extraction eliminate differences in the expression of the same requirement (such as colloquial and formal queries), avoiding matching failures caused by different expressions but the same requirement; hash values ​​are fixed-length feature identifiers, and subsequent matching only requires hash value comparison, which is much faster than text semantic comparison, greatly improving matching efficiency; the low collision rate of the SHA-256 algorithm ensures that different requirements correspond to different hash values, avoiding mismatches; at the same time, hash values ​​cannot be decrypted in reverse, protecting user query privacy.

[0047] S1022. Match the current hash value with the hash values ​​of normalized historical query requests in the historical intent set and the hash values ​​of normalized historical query requests in the historical tool set.

[0048] S1023. Based on the matching results, determine whether there exists a historical intent category and historical tool call trajectory summary that match the current query request.

[0049] In some embodiments, the current hash value is used as the retrieval key, and key-value comparison operations are performed in parallel in the historical intent set and the historical tool set. This mainly involves matching the current hash value with the hash value of the normalized historical query request corresponding to each intent item in the historical intent set, and matching the current hash value with the hash value of the normalized historical query request corresponding to each tool summary item in the historical tool set. This process obtains the historical intent category and historical tool call trajectory summary corresponding to the current hash value, outputs it as the matching result, and records the matching status (existence / non-existence).

[0050] If a historical intent category and historical tool call trace summary that match the current query request exist, proceed to step S103; otherwise, proceed to steps S104-S106.

[0051] S103. If it exists, generate the final query results based on the matched historical intent category and historical tool call trajectory summary, and push them to the client in a streaming manner.

[0052] In some embodiments, based on the historical tool call trajectory summary, the corresponding tool call operation is executed to obtain a tool execution result that conforms to the historical intent category; for example, the tool name and key parameters in the historical tool trajectory summary are parsed, the corresponding tool API interface is called, non-core parameters are replaced, and real-time tool execution results are obtained; the tool execution results can be used as intermediate frames to push values ​​to the client for streaming interaction, so that users can intuitively understand the reasoning process. Based on the historical tool call trajectory summary, explanatory text that conforms to the historical intent category is generated; for example, based on the tool function and parameter meaning in the tool call trajectory summary, explanatory text is generated through a preset template engine; through the large model in the second stage, the final query result is generated based on the tool execution result or the explanatory text; in response to the final query result involving source information or chart information, a preset number of empty content placeholder frames carrying only source information or chart information are first pushed to the client, and then the main text content of the final query result is pushed.

[0053] The intelligent question-answering system involved in this invention comprises two stages. The first stage utilizes a selected large model or intelligent agent to identify the intent of the current query request and tool planning. The second stage utilizes the selected large model or intelligent agent to generate the final query result based on the tool execution result. By determining the historical intent category and historical tool call trajectory summary that match the current query request through a historical mapping set, and then generating the final query result based on the matched historical intent category and historical tool call trajectory summary, the step of identifying the intent of the query request and tool planning using the large model in the first stage can be omitted, which can improve the overall question-answering response speed and reduce latency. In addition, by pushing a preset number of empty content placeholder frames carrying only source information or chart information to the client and then filling in the main text, faster first-frame visibility and a smooth streaming experience can be achieved without sacrificing readability.

[0054] S104. If not, select some valid historical intent categories and valid historical tool call trajectory summaries from the historical mapping set, and convert the valid historical intent categories, valid historical tool call trajectory summaries and their corresponding historical query requests into few-sample example fragments, and use them as supplementary prompts for the first-stage large model.

[0055] Among them, the effective historical intent / tool ​​call trajectory summary refers to historical information that is semantically similar to the current query and related to the current request, which is recalled from the historical mapping set; the few sample instance fragment refers to the encapsulation of historical query requests, effective intents, historical query requests and effective tool call trajectory summaries into a very short instance text, which is used to provide task anchors for large models (avoiding reasoning from scratch).

[0056] In some embodiments, step S104 includes the following steps S1041-S1044:

[0057] S1041. Encode the current query request after normalization to obtain the current query request code.

[0058] Yes, you can perform embedding on the normalized current query request to obtain the current query request code.

[0059] S1042. Based on the current query request code, recall a preset number of candidate intent items and candidate tool summary items from the historical intent set and the historical tool set.

[0060] Each candidate intent item includes at least the intent category of the historical query request, the encoding of the normalized historical query request, and the historical query request itself; each candidate tool summary item includes at least a tool call trajectory summary of the historical query request, the encoding of the normalized historical query request, and the historical query request itself.

[0061] In practice, based on the current query request code and the preset recall quantity k (e.g., k=5), the similarity between the current query request code and the code of the normalized historical query request in each intent item in the historical intent set is calculated, and the k candidate intent items with the highest similarity are returned. Similarly, based on the current query request code and the preset recall quantity k (e.g., k=5), the similarity between the current query request code and the code of the normalized historical query request in each tool summary item in the historical tool set is calculated, and the k candidate tool summary items with the highest similarity are returned.

[0062] S1043. Calculate the comprehensive score of each recalled candidate intent item and each candidate tool summary item according to the preset scoring indicators; wherein, the preset scoring indicators include cosine similarity, BM25 score, timeliness and frequency.

[0063] The preset scoring index is a predefined multi-dimensional quantitative index system used to evaluate the matching value of candidate intent items and candidate tool summary items with the current query request, avoiding the one-sidedness of a single index. It includes four core indexes:

[0064] Cosine similarity: measures the semantic similarity between the current query request encoding and the normalized historical query request encoding (0-1, the closer to 1, the more consistent the semantics).

[0065] BM25 score: measures the degree of text matching between the current query and historical query requests (based on term frequency and inverse document frequency; the higher the score, the greater the text overlap).

[0066] Timeliness: Assess the freshness of historical query requests (e.g., 10 points for the last 7 days, 8 points for the last 30 days, and 5 points for more than 30 days).

[0067] Frequency: Evaluate the frequency of historical query requests being invoked (e.g., 10 points for 100 matches, 5 points for 10 matches, and 3 points for 1 match).

[0068] Based on the above, for each candidate intent item and candidate tool summary item, the scores of the above four indicators are calculated. Then, by weighted summation, the comprehensive score of each candidate intent item and candidate tool summary item is calculated according to the scores of the four types of indicators and the weights of the four indicators.

[0069] Understandably, multiple indicators cover semantic matching (cosine / BM25), time value (timeliness), and practical value (frequency) to avoid prioritizing old data and low-value candidates due to a single reliance on semantics.

[0070] S1044. Select effective intent items and effective tool summary items based on the combined score of each candidate intent item and each candidate tool summary item and the preset score threshold.

[0071] Threshold type selection: The preset score threshold can be a dynamic threshold or a fixed threshold, which is not limited here. When the preset score threshold is a dynamic threshold, the initial value can be 0.6, and the value range is [0.5, 0.8].

[0072] Candidate intent items with a comprehensive score greater than a preset score are considered valid intent items; candidate tool summary items with a comprehensive score greater than a preset score threshold are considered valid tool summary items.

[0073] S1045. Convert the valid intent items and valid tool summary items into few-sample example fragments respectively, and inject the obtained few-sample example fragments as boosters into the first-stage prompts to obtain supplementary prompts for the first-stage large model; wherein, the first-stage large model is determined based on the application encoding or scenario information of the user client.

[0074] Among them, few-sample example fragments refer to structured text that encapsulates valid item information according to a fixed template, with the core being to clearly demonstrate the association logic between historical query requests and intent / tools; boosting injection refers to incorporating few-sample example fragments as auxiliary information into the first-stage prompts, with the core being to reduce the reasoning difficulty of the large model; the first-stage prompts refer to a set of structured input information customized for the first-stage large model.

[0075] During implementation, valid intent items and valid tool summary items are encapsulated into few-sample example fragments according to a preset template; based on the client application encoding / scenario information, the first-stage large model is matched from the preset mapping table (e.g., FinLLM is selected for financial scenarios); the prompts are assembled according to the system instructions, few-sample examples, and current query request order to obtain supplementary prompts for the first-stage large model. The core is to transform historical references into model inference boosting information.

[0076] S105. Using the first-stage large model and supplementary hints, determine the intent category and tool planning results of the current query request.

[0077] The first-stage large model is a reasoning model specifically designed for intent recognition and tool planning. It is usually a domain-adaptive model (such as a financial LLM in a financial scenario), focusing on task decomposition and logical planning rather than text generation. The tool planning result is the tool calling scheme output by the large model, which includes the tool name, key parameters, calling order (when there are multiple tools), and data source requirements. It is the core basis for subsequent tool scheduling.

[0078] In some embodiments, the first-stage large model performs intent recognition and tool planning reasoning recognition on the current query request based on supplementary prompts to obtain streaming reasoning results; it then parses the streaming reasoning results to obtain tool planning results, and performs whitelist verification and parameter schema verification on the tool planning results to prevent unauthorized calls.

[0079] S106. Based on the tool planning results, schedule and execute the tools; using the second-stage large model, generate the final query results based on the obtained tool scheduling and execution results and push them to the client in a streaming manner.

[0080] In some embodiments, the tool planning results that have passed verification are first concurrently scheduled and executed to obtain the original execution results of the tool; post-processing operations (such as deduplication, site blocking, time normalization, etc.) are performed on the original execution results of the tool to obtain the final execution results of the tool and the source information of the final execution results of the tool; the final execution results of the tool and the source information of the final execution results of the tool are output to the client as streaming intermediate frames; based on the source information of the final execution results of the tool and the total length and scenario information of the final execution results of the tool (such as general Q&A, stock diagnosis, stock selection, financial data and information query, market interpretation), a selection is made. The second-stage large model: If the total length exceeds a length threshold, the source information and the final execution result of the tool are safely truncated. This second-stage large model is specifically designed for result processing and natural language generation. Its core function is to transform the structured data output by the tool into human-readable natural language responses. Based on the final execution result and its source information, the second-stage large model generates the final query result. First, a preset number of empty placeholder frames containing only source or chart information are pushed to the user's client, and then the main text of the final query result is pushed to the client.

[0081] In this embodiment of the invention, the first frame of data is pushed to the client immediately after the current query request passes the validity check. This avoids misunderstandings caused by prolonged lack of response from users due to system lag, significantly improving the user's perceived timeliness. It is particularly suitable for scenarios sensitive to real-time interaction (such as financial consultation and real-time customer service). Priority is given to matching the historical mapping set. If a historical intent category and tool call trajectory summary are matched, the final result is directly generated based on the historical information, without triggering time-consuming steps such as large-scale model inference and tool scheduling. This design achieves process short-circuiting in historical hit scenarios, especially suitable for high-frequency repeated queries (such as users repeatedly querying the same type of information), significantly improving the timeliness of intelligent question answering. For scenarios where historical mappings are not matched, valid historical information is converted into a few sample fragments as supplementary prompts, providing a boost to the first-stage large-scale model. It provides precise task anchors, and a small number of sample examples directly provide historical query association paradigms for intents / tools, allowing large models to quickly align task logic without starting from scratch. This shortens the inference time for intent recognition and tool planning, directly accelerating the timeliness of core processing steps in missed scenarios. Tool planning results are directly scheduled and executed concurrently after verification, without secondary confirmation. The original execution results of the tools are used as input for the second-stage model, without additional data transformation waiting. More importantly, the entire process uses streaming push, so users do not need to wait for the complete result to be generated before receiving it. They can obtain the process progress in real time (such as receiving intermediate data of tool execution first, and then receiving the final organized result), which avoids the perception of long waiting time for the complete result at once, and makes the timeliness of the entire processing visible, further enhancing the user's perception of the system's fast processing speed.

[0082] Example 2

[0083] Figure 2 A flowchart illustrating a low-latency intelligent question-answering method is provided in this embodiment of the invention. See also... Figure 2 The method includes the following steps:

[0084] S201. Obtain historical session data and annotate the historical session data to determine the intent category and tool call trajectory summary corresponding to the user's historical query requests.

[0085] Historical session data comprises complete interaction logs of past user queries and system responses recorded by the intelligent question-answering system. It includes historical user query requests, tool information invoked, and processing results, serving as the raw material for constructing historical datasets (e.g., stock query and market data return interaction records in financial apps). Annotation processing refers to the manual or automated annotation of historical session data to establish semantic and logical connections. The core is establishing the correspondence between historical query requests and intent categories, and between historical query requests and tool call trajectory summaries. Intent categories are standardized classifications of user query needs, tailored to specific business scenarios (e.g., finance, e-commerce). Tool call trajectory summaries extract core information about the tools invoked in historical queries, including tool names, key parameters, and call order (when multiple tools are used).

[0086] In practice, the system extracts all historical interaction data from the system log database, filters valid sessions (excluding invalid queries, duplicate submissions, etc.), and outputs the data in the format of user query request, tool call details, processing results, and timestamps. The extracted historical queries are cleaned (punctuation, interjections, and redundant characters are removed) to retain core semantics. Annotation is performed primarily using automatic annotation, supplemented by manual verification. During automatic annotation, a domain-adapted intent recognition model and tool trajectory extraction algorithm are invoked to batch-output preliminary annotation results of historical query requests and intent categories, as well as summaries of historical query requests and tool call trajectories. The automatic annotation results are sampled and verified to correct annotation errors and update model parameters. The final annotation results are organized into a standardized data table with fields including historical query ID, user historical query request, intent category, tool call trajectory summary, and annotation time.

[0087] S202. Normalize the user's historical query requests to obtain normalized historical query requests, and calculate the hash value of the normalized historical query requests.

[0088] The process of normalizing a user's historical query requests is similar to the process of normalizing the current query request in the above embodiment, and will not be repeated here. Calculating the hash value of the normalized historical query request includes: converting the normalized historical query request text into a UTF-8 byte stream; and calculating the hash value of the byte stream using the SHA-256 encryption algorithm.

[0089] S203. Use the hash value of the normalized historical query request as the key, and use the intent category of the historical query request, the encoding of the normalized historical query request, and the historical query request as key-value pairs and save them; construct a historical intent set based on the saved key and key-value pairs.

[0090] S204. Use the hash value of the normalized historical query request as the key, and use the tool call trajectory summary of the historical query request, the encoding of the normalized historical query request, and the historical query request as key-value pairs and save them; construct a historical tool set based on the saved key and key-value pairs.

[0091] S205. Verify the current query request sent by the client, and after the verification is successful, send the first frame of streaming interaction data back to the client.

[0092] S206. Determine whether there exists a historical intent category and historical tool call trajectory summary in the pre-built historical mapping set that matches the current query request.

[0093] The historical mapping set stores the association between historical query requests and corresponding intent categories and tool call trajectory summaries.

[0094] S207. If it exists, generate the final query result based on the matched historical intent category and the historical tool call trajectory summary, and push it to the client in a streaming manner.

[0095] S208. If not, select a portion of valid historical intent categories and valid historical tool call trajectory summaries from the historical mapping set, and convert the valid historical intent categories, valid historical tool call trajectory summaries and their respective corresponding historical query requests into few-sample example fragments, and use them as supplementary prompts for the first-stage large model.

[0096] S209. Based on the first-stage large model and the supplementary prompts, determine the intent category and tool planning result of the current query request.

[0097] S210. Based on the tool planning results, schedule and execute the tool; using the second-stage large model, generate the final query results based on the obtained tool scheduling and execution results, and push them to the client in a streaming manner.

[0098] The specific implementation process of S205-S210 can be found in the above embodiments, and will not be repeated here.

[0099] In this embodiment of the invention, by using a normalization and hash value design, different expressions of the same requirement are mapped to the same hash value, ensuring that subsequent queries can quickly match historical logic; the historical set stores the association logic between intent and tool in advance, and when a subsequent query is successfully matched, the historical tool trajectory can be directly reused to generate results without calling a large model to perform intent recognition and tool planning, thus reducing server computing power consumption.

[0100] In this embodiment, if any tool fails to execute based on the obtained tool scheduling execution status information, an exponential rollback and retry are performed, with a maximum of N retries. If the retry fails, a similar tool from another data source is used instead. If the historical intent set and historical tool set are unavailable, a rollback to BM25 / ES retrieval is performed. If BM25 / ES retrieval still fails, the system switches to the cold LLM path, which involves intent recognition and tool planning through the first-stage large model. A heartbeat mechanism is set up to maintain the SSE connection between the client and the intelligent question-answering system, supporting reconnection after disconnection. When the SSE connection is interrupted, a termination frame with an error field is returned.

[0101] Example 3

[0102] Figure 3 This is a schematic diagram of a low-latency intelligent question-answering device provided in an embodiment of the present invention. This device can execute any of the low-latency intelligent question-answering methods of the present invention. For example... Figure 3 As shown, the low-latency intelligent question-answering device includes:

[0103] The verification module 301 is used to verify the current query request sent by the client, and to send back the first frame of streaming interaction data to the client after the verification is successful.

[0104] The retrieval and judgment module 302 is used to determine whether there are historical intent categories and historical tool call trajectory summaries that match the current query request in the pre-built historical mapping set; wherein, the historical mapping set stores the association relationships between historical query requests and corresponding intent categories and tool call trajectory summaries respectively;

[0105] The direct return module 303 is used to generate the final query results based on the matched historical intent category and historical tool call trajectory summary, if they exist, and push them to the client in a streaming manner;

[0106] Module 304 is used to select a portion of valid historical intent categories and valid historical tool call trajectory summaries from the historical mapping set if they do not exist, and convert the valid historical intent categories, valid historical tool call trajectory summaries and their corresponding historical query requests into few-sample example fragments, which serve as supplementary hints for the first-stage large model.

[0107] The first-stage processing module 305 is used to determine the intent category and tool planning results of the current query request through the first-stage large model and supplementary hints;

[0108] The second-stage processing module 306 is used to schedule and execute the tool based on the tool planning results; using the second-stage large model, based on the obtained tool scheduling and execution results, it generates the final query results and pushes them to the client in a streaming manner.

[0109] In some embodiments, the historical mapping set includes a historical intent set and a historical tool set;

[0110] The device also includes a historical mapping set construction module for:

[0111] Acquire historical session data and annotate it to determine the intent category and tool call trajectory summary corresponding to the user's historical query requests;

[0112] Normalize the user's historical query requests to obtain normalized historical query requests, and calculate the hash value of the normalized historical query requests;

[0113] Use the hash value of the normalized historical query request as the key, and use the intent category of the historical query request, the encoding of the normalized historical query request, and the historical query request as key-value pairs and save them; construct a historical intent set based on the saved key and key-value pairs;

[0114] Use the hash value of the normalized historical query request as the key, and save the tool call trajectory summary of the historical query request, the encoding of the normalized historical query request, and the historical query request as key-value pairs; construct a historical tool set based on the saved key and key-value pairs.

[0115] In some embodiments, in determining whether a historical intent category and historical tool call trajectory summary matching the current query request exists in a pre-built set of historical mappings, the retrieval judgment module 302 is specifically used for:

[0116] Normalize the current query request and calculate the current hash value of the normalized query request;

[0117] The current hash value is matched with the hash values ​​of normalized historical query requests in the historical intent set and the hash values ​​of normalized historical query requests in the historical tool set, respectively.

[0118] Based on the matching results, determine whether there are historical intent categories and historical tool call trace summaries that match the current query request.

[0119] In some embodiments, the direct return module 303 is specifically used to generate the final query results based on the matched historical intent categories and historical tool call trajectory summaries, and to push them to the client in a streaming manner:

[0120] Based on the historical tool call history summary, execute the corresponding tool call operation to obtain the tool execution result that matches the historical intent category; or...

[0121] Generate explanatory text that matches the historical intent category based on the summary of historical tool call trajectories;

[0122] The final query results are generated based on the tool's execution results or explanatory text.

[0123] In response to the final query results involving source information or chart information, a preset number of empty content placeholder frames carrying only source information or chart information are first pushed to the client, and then the main text content of the final query results is pushed in.

[0124] In some embodiments, a subset of valid historical intent categories and valid historical tool call trajectory summaries are selected from the historical mapping set, and the valid historical intent categories, valid historical tool call trajectory summaries, and their respective corresponding historical query requests are converted into few-sample example fragments as supplementary hints for the first-stage large model. Specifically, the construction module 304 is used to:

[0125] The normalized current query request is encoded to obtain the current query request code;

[0126] Based on the current query request code, a preset number of candidate intent items and candidate tool summary items are retrieved from the historical intent set and the historical tool set; wherein, each candidate intent item includes at least the intent category of the historical query request, the code of the normalized historical query request, and the historical query request; each candidate tool summary item includes at least the tool call trajectory summary of the historical query request, the code of the normalized historical query request, and the historical query request;

[0127] According to the preset scoring indicators, calculate the comprehensive score of each candidate intent item and each candidate tool summary item recalled; the preset scoring indicators include cosine similarity, BM25 score, timeliness and frequency;

[0128] Based on the combined score of each candidate intent item and each candidate tool summary item and the preset score threshold, valid intent items and valid tool summary items are selected.

[0129] The valid intent items and valid tool summary items are converted into few-sample example fragments, and the resulting few-sample example fragments are injected as boosters into the first-stage prompts to obtain supplementary prompts for the first-stage large model; wherein, the first-stage large model is determined based on the application encoding or scenario information of the user client.

[0130] In some embodiments, the first-stage processing module 305 is specifically used to determine the intent category and tool planning results of the current query request through the first-stage large model and supplementary hints:

[0131] In the first stage, the large model performs intent recognition and tool planning reasoning on the current query request based on supplementary prompts, and obtains streaming reasoning results.

[0132] The streaming inference results are parsed to obtain the tool planning results, and the tool planning results are then whitelisted and parameter schema validated.

[0133] In some embodiments, the tools are scheduled and executed based on the tool planning results; using the second-stage large model, the final query results are generated based on the obtained tool scheduling and execution results and pushed to the client in a streaming manner. The second-stage processing module 306 is specifically used for:

[0134] The validated tool planning results are concurrently scheduled and executed to obtain the original execution results of the tool;

[0135] Post-processing operations are performed on the original execution results of the tool to obtain the final execution result of the tool and the source information of the final execution result of the tool;

[0136] The final execution result of the tool and the source information of the final execution result are output to the client as streaming intermediate frames;

[0137] Based on the source information of the tool's final execution result, the total length of the tool's final execution result, and the scenario information, the second-stage large model is selected;

[0138] The second stage of the large model generates the final query results based on the final execution results of the tool and the source information of the final execution results;

[0139] First, a preset number of empty placeholder frames containing only source information or chart information are pushed to the user's client. Then, the final query result text content is pushed to the client.

[0140] The low-latency intelligent question-answering device provided in the embodiments of the present invention can execute the low-latency intelligent question-answering method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the method execution.

[0141] According to embodiments of the present invention, the present invention also provides an electronic device, a readable storage medium, and a computer program product.

[0142] Example 4

[0143] Figure 4 A schematic diagram of the structure of an electronic device 10 that can be used to implement embodiments of the present invention is shown. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the invention described and / or claimed herein.

[0144] like Figure 4As shown, the electronic device 10 includes at least one processor 11 and a memory, such as a read-only memory 12 or a random access memory 13, communicatively connected to the at least one processor 11. The memory stores computer programs executable by the at least one processor. The processor 11 can perform various appropriate actions and processes based on the computer program stored in the read-only memory 12 or loaded from storage unit 18 into the random access memory 13. The random access memory 13 can also store various programs and data required for the operation of the electronic device 10. The processor 11, read-only memory 12, and random access memory 13 are interconnected via a bus 14. An input / output interface 15 is also connected to the bus 14.

[0145] Multiple components in electronic device 10 are connected to input / output interface 15, including: input unit 16; output unit 17, such as various types of displays, speakers, etc.; storage unit 18, such as disks, optical disks, etc.; and communication unit 19, such as network interface cards, modems, wireless transceivers, etc. Communication unit 19 allows electronic device 10 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0146] Processor 11 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 11 include, but are not limited to, central processing units, graphics processing units, various special-purpose artificial intelligence computing chips, various processors running machine learning model algorithms, digital signal processors, and any suitable processor, controller, microcontroller, etc. Processor 11 performs the various methods and processes described above, such as performing low-latency intelligent question answering methods.

[0147] In some embodiments, the low-latency intelligent question-answering method may be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on electronic device 10 via read-only memory 12 and / or communication unit 19. When the computer program is loaded into random access memory 13 and executed by processor 11, one or more steps of the low-latency intelligent question-answering method described above may be performed. Alternatively, in other embodiments, processor 11 may be configured to perform the low-latency intelligent question-answering method by any other suitable means (e.g., by means of firmware).

[0148] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays, application-specific integrated circuits (ASICs), application-specific standard products (ASICs), systems-on-a-chip (SoCs), complex programmable logic devices, computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0149] Computer programs used to implement the methods of the present invention can be written in any combination of one or more programming languages. These computer programs can be provided to the processor of a general-purpose computer, a special-purpose computer, or other programmable low-latency intelligent question-answering device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The computer programs can be executed entirely on the machine, partially on the machine, as a standalone software package partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0150] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory, read-only memory, erasable programmable read-only memory, optical fibers, portable compact disk read-only memory, optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0151] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device or liquid crystal display for displaying information to a user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with a user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0152] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or middleware components (e.g., application servers), or frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet. The computing system can include clients and servers. Clients and servers are generally geographically separated and typically interact via communication networks. Client-server relationships are created by computer programs running on the respective computers and having client-server relationships with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, a host product within the cloud computing service system, addressing the shortcomings of traditional physical hosts and virtual private servers, such as high management difficulty and weak business scalability.

[0153] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0154] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A low-latency intelligent question-answering method, characterized in that, include: The system verifies the current query request sent by the client and sends the first frame of streaming interaction data back to the client after the verification is successful. Determine whether a pre-built historical mapping set exists that matches the current query request in terms of historical intent category and historical tool call trajectory summary; wherein, the historical mapping set stores the association relationship between historical query requests and corresponding intent categories and tool call trajectory summaries respectively; If they exist, the final query results are generated based on the matched historical intent category and the historical tool call trajectory summary, and are pushed to the client in a streaming manner; If not, select a portion of valid historical intent categories and valid historical tool call trajectory summaries from the historical mapping set, and convert the valid historical intent categories, valid historical tool call trajectory summaries and their respective corresponding historical query requests into few-sample example fragments, which will serve as supplementary prompts for the first-stage large model; Based on the first-stage large model and the supplementary prompts, the intent category and tool planning result of the current query request are determined; Based on the tool planning results, the tools are scheduled and executed; using the second-stage large model, based on the obtained tool scheduling and execution results, the final query results are generated and pushed to the client in a streaming manner.

2. The method according to claim 1, characterized in that, The historical mapping set includes a historical intent set and a historical tool set; The process of constructing the historical mapping set includes: Historical session data is acquired and labeled to determine the intent category and tool call trajectory summary corresponding to the user's historical query requests; The user's historical query requests are normalized to obtain normalized historical query requests, and the hash value of the normalized historical query requests is calculated. Use the hash value of the normalized historical query request as the key, and use the intent category of the historical query request, the encoding of the normalized historical query request, and the historical query request as key-value pairs and save them; construct a historical intent set based on the saved key and key-value pairs; Use the hash value of the normalized historical query request as the key, and save the tool call trajectory summary of the historical query request, the encoding of the normalized historical query request, and the historical query request as key-value pairs; construct a historical tool set based on the saved key and key-value pairs.

3. The method according to claim 2, characterized in that, The determination of whether a pre-built historical mapping set contains a historical intent category and historical tool call trajectory summary that match the current query request includes: The current query request is normalized, and the current hash value of the normalized current query request is calculated. The current hash value is matched with the hash values ​​of normalized historical query requests in the historical intent set and the hash values ​​of normalized historical query requests in the historical tool set, respectively. Based on the matching results, determine whether there exists a historical intent category and historical tool call trajectory summary that match the current query request.

4. The method according to claim 2, characterized in that, The final query results are generated based on the matched historical intent category and the historical tool call trajectory summary, and are streamed to the client, including: Based on the historical tool call trajectory summary, the corresponding tool call operation is executed to obtain the tool execution result that matches the historical intent category; or... Based on the summary of the historical tool call trajectory, generate explanatory text that matches the historical intent category; The final query result is generated based on the execution result of the tool or the explanatory text; In response to the fact that the final query result involves source information or chart information, a preset number of empty content placeholder frames carrying only source information or chart information are first pushed to the client, and then the main text content of the final query result is pushed in addition.

5. The method according to claim 3, characterized in that, Select a subset of valid historical intent categories and valid historical tool call trajectory summaries from the historical mapping set, and convert the valid historical intent categories, valid historical tool call trajectory summaries, and their respective corresponding historical query requests into few-sample example fragments, which serve as supplementary hints for the first-stage large model, including: The normalized current query request is encoded to obtain the current query request code; Based on the current query request code, a preset number of candidate intent items and candidate tool summary items are recalled from the historical intent set and the historical tool set; wherein, each candidate intent item includes at least the intent category of the historical query request, the code of the normalized historical query request, and the historical query request; each candidate tool summary item includes at least a summary of the tool call trajectory of the historical query request, the code of the normalized historical query request, and the historical query request; According to the preset scoring indicators, calculate the comprehensive score of each recalled candidate intent item and each candidate tool summary item; wherein, the preset scoring indicators include cosine similarity, BM25 score, timeliness and frequency; Based on the combined score of each candidate intent item and each candidate tool summary item and the preset score threshold, valid intent items and valid tool summary items are selected. The effective intent items and the effective tool summary items are converted into few-sample example fragments, and the resulting few-sample example fragments are injected as boosters into the first-stage prompts to obtain supplementary prompts for the first-stage large model; wherein, the first-stage large model is determined based on the application encoding or scenario information of the user client.

6. The method according to claim 1, characterized in that, Based on the first-stage large model and the supplementary hints, the intent category and tool planning results of the current query request are determined, including: In the first stage, the large model performs intent recognition and tool planning reasoning on the current query request based on supplementary prompts, and obtains streaming reasoning results. The streaming inference results are parsed to obtain tool planning results, and the tool planning results are then validated using a whitelist and parameter schema.

7. The method according to claim 6, characterized in that, Based on the tool planning results, schedule and execute the tools. Using the second-stage large model, based on the obtained tool scheduling and execution results, the final query results are generated and streamed to the client, including: The validated tool planning results are concurrently scheduled and executed to obtain the original execution results of the tool; Post-processing operations are performed on the original execution results of the tool to obtain the final execution result of the tool and the source information of the final execution result of the tool; The final execution result of the tool and the source information of the final execution result are output to the client as streaming intermediate frames; Based on the source information of the final execution result of the tool, the total length of the final execution result of the tool, and the scenario information, the second-stage large model is selected; The second-stage large model generates the final query results based on the final execution result of the tool and the source information of the final execution result of the tool; First, a preset number of empty content placeholder frames carrying only source information or chart information are pushed to the user's client, and then the main text content of the final query result is pushed to the client.

8. A low-latency intelligent question-answering device, characterized in that, include: The verification module is used to verify the current query request sent by the client, and to send back the first frame of streaming interaction data to the client after the verification is successful. The retrieval and judgment module is used to determine whether there exists a historical intent category and a historical tool call trajectory summary that match the current query request in the pre-built historical mapping set; wherein, the historical mapping set stores the association relationship between historical query requests and corresponding intent categories and tool call trajectory summaries respectively; The direct return module is used to generate the final query result based on the matched historical intent category and the historical tool call trajectory summary, if it exists, and push it to the client in a streaming manner; The construction module is used to select a portion of valid historical intent categories and valid historical tool call trajectory summaries from the historical mapping set if they do not exist, and convert the valid historical intent categories, valid historical tool call trajectory summaries and their respective corresponding historical query requests into few-sample example fragments, which serve as supplementary prompts for the first-stage large model. The first-stage processing module is used to determine the intent category and tool planning result of the current query request through the first-stage large model and the supplementary prompts; The second-stage processing module is used to schedule and execute the tool based on the tool planning results; using the second-stage large model, based on the obtained tool scheduling and execution results, it generates the final query results and pushes them to the client in a streaming manner.

9. An electronic device, characterized in that, include: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that cause a processor to execute the method of any one of claims 1-7.

Images