A question and answer method, related devices and computer program product

Abstract

The application discloses a question and answer method, related equipment and a computer program product, and relates to the technical field of natural language processing. Information retrieval is performed based on a user input original question to obtain a search result corresponding to the original question. In the case of meeting a first condition, the original question is input into a large language model configured for semantic analysis to obtain at least one structured information of the original question. According to each structured information, a reconstructed question is generated. Information retrieval is performed according to the reconstructed question to obtain a search result corresponding to the reconstructed question. According to the search results corresponding to the original question and the reconstructed question respectively, a target reply result to the original question is generated. The application accurately captures key constraints in the original question, significantly improves the pertinence of the search result, and thus improves the accuracy of the reply result.

Description

Technical Field

[0001] This application relates to the field of natural language processing technology, and more specifically, to a question-answering method, related equipment, and computer program products. Background Technology

[0002] RAG (Retrieval-Augmented Generation) technology combines language models and information retrieval techniques. Specifically, when a RAG system needs to answer a question, it first retrieves information related to the question from a knowledge base or document collection, and then uses the retrieved information to guide the generation of the response text, thereby improving the quality and accuracy of the response.

[0003] However, during the search process in the RAG system, the user's input questions are often poorly worded or contain long-tailed expressions, making it difficult to accurately understand the user's true intent. This results in low accuracy of search results and affects the quality and accuracy of the final generated response. Summary of the Invention

[0004] In view of the above problems, this application is made to provide a question-and-answer method, related equipment, and computer program product to improve the quality and accuracy of the final generated response. The specific solution is as follows:

[0005] Firstly, this application provides a question-and-answer method, including:

[0006] Obtain the original question input by the user;

[0007] Information retrieval is performed based on the original question to obtain the retrieval results corresponding to the original question, as well as the matching degree between the retrieval results and the original question;

[0008] Under the condition that the first condition is met, the original question is input into the configured large language model for semantic parsing to obtain at least one structured information of the original question. The structured information is a structured representation of the logical relationship between any two entities in the original question. The first condition indicates that the search results do not match the solution requirements of the original question.

[0009] Based on each piece of structured information, a reconstruction problem is generated;

[0010] Information retrieval is performed based on the reconstruction question to obtain the retrieval results corresponding to the reconstruction question, as well as the matching degree between the retrieval results and the reconstruction question;

[0011] Based on the query source weight corresponding to the question type of each search result, the query source weight represents the degree of closeness between the question of the corresponding question type and the user's true intent. The question types include: the original question and the reconstructed question.

[0012] The matching degree corresponding to each search result and the query source weight are fused to obtain the fused confidence score of each search result;

[0013] The search results corresponding to the fusion confidence scores that meet the preset conditions are combined with the large language model configured for the original question input to generate the target response result for the original question.

[0014] In one possible design, in another implementation of the first aspect of the embodiments of this application, the process of semantically parsing the original problem into a configured large language model to obtain at least one structured piece of information about the original problem includes:

[0015] Obtain a first prompt instruction format template, which includes a first task instruction and an original question slot. The first task instruction is used to instruct the large language model to perform semantic parsing on the original question in the original question slot and generate at least one triplet using the parsed entities and the logical relationships between the entities.

[0016] The original question is filled into the original question slot to obtain the target first prompt instruction. The target first prompt instruction is then input into the configured large language model to obtain at least one triple generated by the large language model as structured information.

[0017] In one possible design, in another implementation of the first aspect of the embodiments of this application, the process of generating a reconstruction problem based on each piece of structured information includes:

[0018] The configured large model is invoked to instruct the large model to reference each of the structured information and generate a reconstruction problem.

[0019] In one possible design, in another implementation of the first aspect of the embodiments of this application, the process of invoking a configured large model to instruct the large model to reference each of the structured information and generate a reconstruction problem includes:

[0020] Obtain a second prompt instruction template, which includes a second task instruction and a structured information slot. The second task instruction is used to instruct the large model to generate a reconstruction question for retrieval based on the structured information in the structured information slot.

[0021] The structured information is filled into the structured information slot to obtain the target second prompt instruction. The target second prompt instruction is input into the large model to obtain the reconstruction problem generated by the large model.

[0022] In one possible design, in another implementation of the first aspect of the embodiments of this application, the second task instruction is further used to instruct the large model to replace the names of entities in the structured information with standardized entity names during the process of generating a reconstruction question for retrieval based on the structured information in the structured information slot;

[0023] And / or, the second task instruction is further used to instruct the large model to perform synonym replacement on relation words representing logical relationships between entities in the structured information during the process of generating a reconstruction question for retrieval based on the structured information in the structured information slot.

[0024] In one possible design, in another implementation of the first aspect of the embodiments of this application, the process of determining whether the first condition is met includes:

[0025] The configured large model is invoked to instruct the large model to refer to the search results corresponding to the original question, determine whether the search results match the answer requirements of the original question, and generate a judgment result.

[0026] In a second aspect, an electronic device is provided, comprising: a memory and a processor;

[0027] The memory is used to store programs;

[0028] The processor is configured to execute the program to implement the question-and-answer method described in any of the first aspects of this application.

[0029] Thirdly, a readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the question-and-answer method described in any of the first aspects of this application.

[0030] Fourthly, a computer program product is provided, including a computer program that, when executed by a processor, implements the question-and-answer method described in any of the first aspects of this application.

[0031] By employing the aforementioned technical solution, when the search results for the original question do not match the required answers, this application leverages the ability of a large language model to identify core intent boundaries and deep semantic structures. This allows for semantic parsing of the original question, accurately locating entities and complex relationships within them using semantic information, and extracting structured information. This accurately captures the key constraints of the original question, avoiding false recalls caused by ambiguous surface semantics or long-tailed representations. Subsequently, a reconstructed question is generated based on each structured piece of information, precisely expressing its semantic boundaries. This serves as the basis for subsequent searches, significantly improving the relevance of the search results. Finally, the search results for both the original and reconstructed questions are combined to generate the response, avoiding the limitations of a single search path and thus improving the accuracy of the response. Attached Figure Description

[0032] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the scope of this application. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings:

[0033] Figure 1 A schematic diagram of an implementation system architecture for the question-answering method provided in this application embodiment;

[0034] Figure 2 A flowchart illustrating an implementation of a question-and-answer method provided in this application embodiment;

[0035] Figure 3 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0036] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0037] The current RAG system suffers from particularly high recall and precision, stemming from inherent limitations in the semantic matching mechanism upon which the retrieval process relies. This deficiency is exacerbated when faced with non-standard user queries or queries involving long-tail content, making it difficult for the system to accurately capture the user's true intent and directly impacting the quality and reliability of the final generated answer.

[0038] Specifically, the primary challenge stems from the "semantic similarity trap." Since vector retrieval generally relies on surface-level semantic matching methods such as cosine similarity, it is essentially a fuzzy matching mechanism, making it highly susceptible to interference from negative words, synonym confusion, and contextual bias. For example, when a user asks "XX does not produce what," the model may over-focus on the entity "XX" and ignore the negative connotation expressed by "does not produce," incorrectly recalling a large amount of positive content related to the XX product. Or, when asking "how to avoid catching a cold," the potential association between "avoid" and words like "prevention" and "symptoms" may lead to the inclusion of non-target documents describing cold symptoms or transmission routes in the search results.

[0039] Meanwhile, the "long-tail query failure" further exposes the model's insufficient adaptability to vertical domains. In professional scenarios such as technology, healthcare, and law, users often need to perform highly precise queries, such as "the function of the buffer_size parameter in the tf.data.Dataset.map function of TensorFlow 2.0" or "medication contraindications for patients with diabetes and hypertension." However, existing vector models have limited semantic understanding of subtle differences in technical terms (such as "buffer_size" vs. "batch_size"), spelling variations of code identifiers (such as "user_id" vs. "userid"), and rare entity names. They often fail to accurately identify key features and instead introduce similar but inaccurate or even completely irrelevant content due to the "semantic generalization" phenomenon, thus significantly reducing the accuracy of the retrieval process.

[0040] In summary, the RAG system struggles to accurately understand the user's true intent when faced with complex expressions, negative semantics, and specialized long-tail queries, resulting in low accuracy of search results and consequently affecting the quality and accuracy of the final generated responses.

[0041] To address the aforementioned issues, this application provides a question-and-answer method. The question-and-answer method of this application embodiment will be described in detail below with reference to the accompanying drawings.

[0042] See Figure 1 , Figure 1 A schematic diagram of a system architecture is shown. The system may include a terminal 100 and a server 200, wherein the server 200 may include one or more servers (…). Figure 1 (This example uses a server as an illustration).

[0043] Either terminal 100 or server 200 can be used independently to execute the question-and-answer method provided in the embodiments of this application. Alternatively, terminal 100 and server 200 can also be used collaboratively to execute the question-and-answer method provided in the embodiments of this application.

[0044] In this application embodiment, the terminal 100 can be a mobile phone, tablet computer, teaching large screen, wearable device, vehicle-mounted device, conference terminal, augmented reality (AR) / virtual reality (VR) device, laptop computer, ultra-mobile personal computer (UMPC), netbook, personal digital assistant (PDA), etc., and this application embodiment does not impose any restrictions on it.

[0045] The product form of server 200 in this application embodiment may include, but is not limited to, cloud servers (running as virtual server instances on physical servers), micro servers (suitable for small office or home environments), storage servers (focused on data storage and management), GPU servers (equipped with graphics processing units, suitable for high-performance computing and graphics processing tasks), and modular servers (allowing flexible configuration and expansion of hardware components as needed).

[0046] This application provides a question-and-answer method, which is illustrated by applying the method to a computer device. Specifically, the computer device may be... Figure 1 The system consists of terminal 100 or a combination of terminal 100 and server 200. (Refer to...) Figure 2 The present application provides a flowchart of a question-and-answer method, which may include steps S100 to S160, and these steps are described in detail below.

[0047] Step S100: Obtain the original question input by the user.

[0048] This step is the starting point and driving force of the entire question-and-answer process. Users can submit their natural language questions (Query) that need answers through the computer device interface, i.e., the original question. The original question may contain ambiguous, vague, or ellipsis statements, or have complex semantic structures (such as negation, conditions, and technical terms).

[0049] Once the computer device receives the original question, it uses it as the basic input for all subsequent processing flows, accurately conveying the user's original intent and semantic information without any modification or filtering, thus providing a true source of information for subsequent processes such as structured parsing.

[0050] Step S110: Perform information retrieval based on the original question to obtain the retrieval results corresponding to the original question.

[0051] This step can be understood as an initial query / initial retrieval without any processing or rewriting. Through retrieval methods such as vector retrieval, relevant information most likely to contain the answer to the original question is selected from the knowledge base and used as the retrieval results corresponding to the original question.

[0052] Step S120: Determine whether the first condition is met. The first condition indicates that the search results do not match the answer requirements of the original question. If yes, proceed to step S130; otherwise, generate the target answer to the original question based on the search results.

[0053] This step assesses the sufficiency of the association between the retrieved context and the original question, determining whether it is sufficient to generate an answer that meets the requirements of the original question; that is, determining whether the search results corresponding to the original question match the requirements of the original question.

[0054] It is understandable that the search results obtained in step S110 may not match the requirements of the original question, that is, the search results may be inaccurate, irrelevant (low precision) or missing key information (low recall). Step S120 is used to investigate the possibility of low precision and low recall of the search results, thereby determining the method of generating a response to the original question.

[0055] In one possible implementation, the process of determining whether the first condition is met may include: judging the search results based on preset rules and simple indicators to determine whether they meet the first condition.

[0056] For example, a minimum threshold is set for the cosine similarity, which represents the relevance between the vector retrieval results and the original question in step S110. If the highest cosine similarity among all retrieval results is lower than this minimum threshold, it is determined that the retrieval results corresponding to the original question meet the first condition, i.e., the retrieval results do not match the solution requirements of the original question. Conversely, if the lowest cosine similarity among all retrieval results is higher than this minimum threshold, it is determined that the retrieval results corresponding to the original question do not meet the first condition, i.e., the retrieval results match the solution requirements of the original question.

[0057] In another possible implementation, the process of determining whether the first condition is met includes: calling the configured large model to instruct the large model to refer to the search results corresponding to the original question, determining whether the search results match the answer requirements of the original question, and generating a judgment result.

[0058] Specifically, after calling the configured large model, the original question and its corresponding search results need to be passed to the large model as key inputs. You can also add prompts, such as "Evaluate the provided context information to determine whether it contains enough information to answer the given question", to guide the large model to clarify and perform the judgment task and generate the final judgment result.

[0059] When the judgment result is "No" (the search result matches the answer requirement of the original question), the search result can be directly used to generate the answer to the original question, simplifying the question-and-answer process. Conversely, when the judgment result is "Yes" (the search result does not match the answer requirement of the original question), an answer will not be forcibly generated based on the error information. Instead, steps S130 and subsequent steps are triggered to reconstruct / rewrite the original question, thereby ensuring the honesty of the RAG system when it cannot provide a high-quality answer and avoiding misleading users.

[0060] Step S130: Input the original question into the configured large language model for semantic parsing to obtain at least one structured piece of information about the original question.

[0061] This step uses semantic analysis to deconstruct the entities within the original, unstructured, and ambiguous question input by the user, as well as the logical relationships connecting these entities. Based on the logical relationships between any two entities, it generates one or more structured and explicit pieces of structured information, providing a basis for subsequent precise retrieval. The structured information is a structured representation of the logical relationships between any two entities in the original question.

[0062] Step S140: Generate a reconstruction problem based on each structured piece of information.

[0063] This step is crucial for self-optimization and refinement after the initial search results in step S110 fail to accurately answer the original question. Specifically, the poorly worded original question is rewritten into one or more new, clearer, more complete, and boundary-controlled query questions. Within the semantic boundaries defined by the entities and logical relationships between entities in each structured piece of information, new query questions (defined as reconstruction questions) are generated.

[0064] Understandably, the process of generating a new query question is not simply a restatement of the original question, but rather uses structured information to eliminate ambiguity, vagueness, and logical gaps in the original question, generating a new query question with clearer semantics and a more focused intent, thereby guiding users to find information relevant to their true intent during the retrieval process.

[0065] For example, if the structured information is "Company XX does not produce game consoles", the goal of this step is to generate a query that clearly expresses this negative intent, such as "list the types of game consoles that Company XX has never produced" or "which game console products does Company XX not produce".

[0066] Step S150: Perform information retrieval based on the reconstruction question to obtain the retrieval results corresponding to the reconstruction question.

[0067] Using the "reconstruction question" generated in step S140 as new retrieval input, the retrieval operation is performed again. It is understandable that because the retrieval basis (i.e., the reconstruction question) upon which step S140 relies has eliminated ambiguity to the greatest extent and explicitly included key logical relationships, it bypasses problems such as the "semantic similarity trap" encountered during the initial retrieval, enabling the retrieval of a set of document fragments with significantly higher relevance and significantly less noise, thus laying a solid contextual foundation for finally generating an accurate and reliable answer.

[0068] Step S160: Based on the search results corresponding to the original question and the reconstructed question, generate the target response result for the original question.

[0069] This step combines the search results for the original question and the search results for the reconstructed question to generate a final answer that responds to the original question.

[0070] Understandably, the accuracy of the target response generated in this step depends on the contextual understanding and generation capabilities of the large language model. It needs to identify and prioritize the highly accurate information from the search results of the reconstructed question, while also carefully considering the relevant background or complementary details that may exist in the search results of the original question. Finally, it integrates all valuable information into a coherent, natural, and direct text that answers the user's original intent, thus obtaining the target response.

[0071] In one possible implementation, the process of generating a target response to the original question based on the retrieval results corresponding to the original question and the reconstructed question includes: 1) 2) ...

[0072] Understandably, by using the fusion confidence score corresponding to each search result, relatively reliable search results are selected from the search results of the dual channels (original question search and reconstructed question search), and the search results are dynamically integrated using a large language model (LLM) to generate target answer results for the original question, thereby improving the accuracy and reliability of the final answer.

[0073] The fusion confidence score of each search result reflects its contribution to solving the original problem. It can be evaluated from multiple aspects, such as the semantic matching degree between the search result and the original problem, the source credibility of the search result (e.g., Wikipedia > ordinary web pages), and the timeliness of the search result.

[0074] Specifically, in this embodiment of the application, when performing a search based on the original question and the reconstructed question, the output search results also include a matching score with the corresponding query / question, such as a similarity score. Based on this, different weights are assigned to search results from different question sources. If the question source is the original question, it means that the search result is the most direct response to the user's original input, assuming that the original question has not deviated from the user's true intent, but also acknowledging that the original question may have insufficient recall or semantic ambiguity leading to inaccurate matching scores. Therefore, it is given a relatively low query source weight.

[0075] If the source of the query is a refactored query, it represents a search result that has undergone LLM intent parsing and optimization. Because the refactored query is more clearly worded, the matching score of this search result is believed to be more accurate, and it covers content that the original query result might have missed. Therefore, it is given a higher weight for its query source.

[0076] In this embodiment, the fusion confidence score of a search result is derived from two aspects: its matching score with the corresponding question (the original question or a reconstructed question), and the query source weight of its corresponding question type / question source (the original question or a reconstructed question). The matching score and query source weight of each search result are weighted and fused to obtain the fusion confidence score, which measures the overall contribution of the search result to solving the original question.

[0077] Furthermore, by utilizing the fusion confidence score of each search result and preset conditions, search results that meet the conditions are filtered out. In one possible implementation, a threshold (e.g., fusion confidence score > 0.7) or a ranking rule (e.g., taking the TOP-N results) is set to filter search results with high fusion confidence scores. These are then used as high-confidence context fragments, forming a prompt along with the original question, which is then input into the large language model. The prompt can include role settings (e.g., "You are an expert assistant"), task instructions ("Please accurately answer the user's original question based on the following high-confidence context fragments"), the original question, high-confidence context fragments (arranged by priority), and output requirements (e.g., structured, concise, citing sources).

[0078] The large language model follows the prompt to generate a natural language answer (defined as the target response result) and provides feedback to the user. Understandably, because the high-confidence context fragments input are filtered through fusion confidence and are highly relevant and have clear intent, the probability of LLM "fabricating facts" (illusion) is greatly reduced, and the fluency, professionalism, and accuracy of the answer are also guaranteed.

[0079] In summary, the question-answering method provided in this application, when the retrieval results for the original question do not match the required answers, leverages the ability of a large language model to identify core intent boundaries and deep semantic structures to perform semantic parsing on the original question. By using semantic information, it accurately locates entities and complex relationships between entities within the original question, extracting structured information to accurately capture key constraints in the original question and avoid false recalls caused by ambiguous surface semantics or long-tailed representations. Subsequently, based on each piece of structured information, a reconstructed question that accurately expresses its semantic boundaries is generated as the basis for subsequent retrieval, significantly improving the relevance of the retrieval results. Finally, the retrieval results from both the original question and the reconstructed question are combined to generate the answer, avoiding the limitations of a single retrieval path and thus improving the accuracy of the answer.

[0080] Next, other possible implementations of the question-answering method provided in this application will be described in detail through the following embodiments.

[0081] In one possible implementation, step S130, the process of inputting the original question into a configured large language model for semantic parsing to obtain at least one structured piece of information about the original question, includes: obtaining a first prompt format template, the first prompt format template including a first task instruction and an original question slot, the first task instruction being used to instruct the large language model to perform semantic parsing on the original question in the original question slot, and to generate at least one triplet using the parsed entities and the logical relationships between the entities; filling the original question into the original question slot to obtain a target first prompt, and inputting the target first prompt into the configured large language model to obtain at least one triplet generated by the large language model as structured information.

[0082] This process leverages a well-configured large language model to understand the underlying intent and precise boundaries of the original problem, overcomes the ambiguity and complexity of natural language, and transforms it into at least one structured triplet of information, providing clear guidance and constraints for the next step of precise reconstruction.

[0083] Specifically, a prompt template is obtained to guide the large language model in semantic parsing. This prompt template contains the first task instruction and the original question slot. Optionally, the prompt template can be retrieved from a pre-set template library containing templates for different semantic parsing needs, or it can be manually written by developers according to specific business scenarios and requirements to ensure the accuracy and relevance of the template.

[0084] The original question slot is a blank area used to place the original question to be processed. It is a variable position that will be filled with specific original questions in practical applications, thereby realizing the semantic parsing of various specific original questions. The first task instruction describes the specific tasks and task objectives that the large language model needs to perform, providing a clear direction for the large language model. In the embodiments of this application, the first task instruction is used to instruct the large language model to perform semantic parsing of the original questions in the original question slot, and to generate at least one triplet using the parsed entities and the logical relationships between entities.

[0085] Optionally, the first task instruction can be optimized according to task requirements. For example, it can be instructed to perform optimization operations in the process of generating structured information, or to generate structured information in a preset format.

[0086] By optimizing the first task instruction, such as explicitly defining entity types, relational constraints, and structured output formats, the ambiguity and randomness of large language models in generating structured information can be effectively reduced, thereby improving the accuracy, consistency, and task adaptability of structured information.

[0087] In one example, the first prompt template could be: "You are an expert in extracting structured knowledge from natural language. For issues in the IT operations domain, please complete the following tasks:"

[0088] 1. Extract knowledge graph triples from each original question, in the format: entity1, relation, entity2;

[0089] 2. If an original question contains multiple entities or relations, extract multiple triples;

[0090] 3. Allow reasonable inferences based on common sense (e.g., supplementing information omitted in a subject-verb-object structure);

[0091] 4. Standardize synonyms and abbreviations into standard forms;

[0092] 5. The output should be structured content, suitable for knowledge bases or vector retrieval systems.

[0093] Original question: {}.

[0094] Furthermore, the user-inputted original question, such as "XX does not produce what", is filled into the original question slot "{}" in the first prompt instruction format template, the task object of the first task instruction is updated, and the target first prompt instruction is obtained.

[0095] The target first prompt instruction is input into the large language model. Based on its own algorithm and training data, the large language model performs semantic parsing of the input original question following the instructions of the first task instruction.

[0096] Through semantic parsing, the large language model can identify key elements (entities and entity relationships) in the original question, including but not limited to core entities (such as "XX Company" and "cold"), attributes, actions, and the logical relationships between them (such as "produce / do not produce", "avoid / prevent", "parameter effect", and "taboo"). It may also include key modifiers such as time range, negation words (such as "not"), and conditional restrictions (such as "compounded with hypertension"). Furthermore, based on the identified entities and their logical relationships, at least one triple is generated, such as {Entities:XX Company,Relations:Do not produce,Entities:X}. These triples, as structured information, can more clearly express the semantic content of the original question.

[0097] Furthermore, a reconstruction problem is generated based on each piece of structured information, and multiple methods can be used to generate reconstruction problems based on structured information.

[0098] In one possible implementation, rule template generation is used, pre-defining a series of rule templates for different structured information. For example, if the structured information is a triple in the form of "subject-action-object", for a triple such as "XX-release-new mobile phone", the rule template can be set as "what company released what product", and the elements in the triple are filled into the corresponding positions in the template to generate the reconstruction question "what product did XX company release".

[0099] In another possible implementation, the process of generating a refactoring problem based on each piece of structured information may include: invoking a configured large model to instruct the large model to reference each piece of structured information and generate a refactoring problem.

[0100] This embodiment of the application takes into account the powerful language understanding and generation capabilities of the large model and selects the large model to reconstruct the problem. Specifically, the configured large model is invoked, and structured information and related prompts are input into the large model, instructing the large model to generate a reconstruction problem based on this structured information. For example, structured information such as "Subject: XX; Relationship: Does not produce; Object: Electronic products" and the prompt "Please generate a suitable problem based on the following structure information" are input into the large model. The large model, with its powerful language understanding and generation capabilities, generates a reconstruction problem such as "XX does not produce what electronic products".

[0101] Understandably, large models, with their powerful language understanding and generation capabilities, can accurately grasp the semantic logic in structured information and generate questions that conform to the norms. They can process information of various types and emerging fields without having to pre-set a large number of rules or rely on specific knowledge graphs. They are highly adaptable and flexible, and greatly improve work efficiency.

[0102] Specifically, the process of calling the configured large model to instruct it to refer to each structured information and generate a reconstruction question includes: obtaining a second prompt format template, which includes a second task instruction and structured information slots; the second task instruction instructs the large model to generate a reconstruction question for retrieval based on the structured information in the structured information slots; filling the structured information into the structured information slots to obtain the target second prompt instruction; and inputting the target second prompt instruction into the large model to obtain the reconstruction question generated by the large model.

[0103] This process utilizes a pre-configured large model. Following a specific format of prompts and instructions, structured information is accurately populated into the large model, which then generates a retrieval-ready reconstruction question. The key lies in obtaining a suitable prompt and instructions template to guide the large model in understanding the task and accurately populating the structured information to obtain a reconstruction question that meets the requirements.

[0104] First, obtain the second prompt instruction template, which contains the second task instruction and structured information slots. The second task instruction clarifies the task the large model needs to complete: generating a reconstruction question for retrieval based on structured information. The structured information slots are reserved positions for subsequent filling with specific structured information. Considering that step S130 may generate multiple pieces of structured information, the second prompt instruction template can contain multiple structured information slots to meet the filling requirements.

[0105] Optionally, the second prompt format template can be a variety of different prompt templates designed in advance based on common structured information types and retrieval needs, or it can be a prompt template generated using a generative model in natural language processing based on questions and structured information pairs in historical data.

[0106] Optionally, the second task instruction can be optimized according to task requirements, thereby instructing the large model to generate a reconstruction problem that meets the task requirements. In one possible implementation, the second task instruction is also used to instruct the large model to replace the names of entities in the structured information with standardized entity names during the process of generating a reconstruction problem for retrieval based on the structured information in the structured information slots; and / or, the second task instruction is also used to instruct the large model to perform synonym replacement on the relation words representing the logical relationships between entities in the structured information during the process of generating a reconstruction problem for retrieval based on the structured information in the structured information slots.

[0107] The aforementioned second task instruction is used to instruct the large model to generate multiple variations or more standardized representations of the elements contained in the structured information, thereby improving the retrieval performance of the reconstruction problem.

[0108] Optionally, the second task instruction is used to instruct the large model to perform synonym / near-synonym replacement, terminology standardization / complete expression, domain-specific variants / supplements, and mining of potential implicit concepts for various elements (entities, relation words, etc.) in the structured information during the generation of the reconstruction problem. It can also instruct the large model on details such as the output format of the final output reconstruction problem.

[0109] For example, the second prompt template could be: "You are an expert in generating query questions based on structured information (triples), specializing in IT operations. Please follow these steps:"

[0110] 1. Generating query questions from triples:

[0111] 1) Create a natural language problem for each triple;

[0112] 2) Ensure the question fully covers relational semantics;

[0113] 3) Use standardized entity names;

[0114] 2. Final output requirements:

[0115] 1) Combine all questions into one line, without line breaks;

[0116] 2) Use double hash symbols "##" to separate the issues;

[0117] 3) The output must be a single-line string with no leading or trailing spaces;

[0118] 4) Never include newline or format characters in the output.

[0119] Extracted knowledge graph triples: {knowledge_graph_triples};

[0120] Output the generated problem only in the required format.

[0121] Furthermore, the structured information corresponding to the original question in this Q&A session is filled into the structure information slot of the second prompt instruction format template, namely {knowledge_graph_triples} in the example above, to form the target second prompt instruction.

[0122] The target second prompt instruction is input into the large model. Based on its internal algorithm and training data, the large model understands the meaning of the target second prompt instruction and follows the instructions of the second task instruction to expand the question within the semantic boundaries defined by the structured information in the structured information slot. For example, for the original question "XX does not produce what", it expands the question within the boundaries of the structured information {topic: XX formula, negation relation: production, object: X}.

[0123] Understandably, during the expansion process, the large model, according to the task instructions of the second task, requires replacing the names of entities in the structured information with standardized entity names, and changing the entities in the structured information to standardized entity names in relevant professional fields, thus processing various elements in the structured information. For example, "XX" is explicitly defined as "XX formula," and the negative relation "does not produce" is replaced with "not involved," "not involved," "not entered... field," etc. Optionally, relevant concepts are added to professional terms, such as expanding "avoiding colds" to "cold prevention measures" in the medical field. Alternatively, the reconstruction problem generated by the large model is derived from relevant concepts derived from the structured information, such as the object "X" being derived as possibly "consumer electronics," "automobiles," "food," etc., generating a statement like "XX formula does not produce what consumer electronics products."

[0124] Based on this, the large model ultimately generates a set or a group of candidate reconstructed questions. For example, "What does XX not produce?" might be rewritten as "List the products or business areas that XX company is not involved in." This accurately covers the core intent of the original question and, compared to the original question, expresses it more clearly, completely, and searchably. At the same time, due to the constraints of structured information, it avoids intent shift (noise) caused by over-expansion.

[0125] Furthermore, information retrieval is performed based on the reconstructed questions generated above to obtain the retrieval results corresponding to the reconstructed questions. Based on the retrieval results corresponding to the original questions and the reconstructed questions respectively, the target answer results for the original questions are generated.

[0126] In summary, this application significantly improves the performance of the RAG system in handling complex queries by introducing LLM-driven structured information extraction and question rewriting techniques and integrating a confidence weighting mechanism. This improvement is mainly reflected in the following aspects:

[0127] 1. Effectively mitigates semantic similarity traps and improves retrieval precision and intent understanding accuracy: LLM-based entity relation extraction can deeply understand the deep semantic structure of the original question (such as negation, limitation, and conditional relations) and identify the core intent boundaries. Dynamically generated boundary-controllable reconstruction questions enable the reconstruction questions to accurately express key constraints, effectively avoiding false recalls caused by surface semantic ambiguity.

[0128] Meanwhile, by assigning higher query source weights to the retrieval results of the rewritten question recall, noise is further filtered out, ensuring that the core intent of the information input into the LLM is clear and highly relevant, greatly reducing the probability of "answering the wrong question".

[0129] 2. Significantly improves recall and content adaptability for long-tail professional queries: In vertical fields such as law and technology, LLM's structured parsing can accurately locate professional terms, fine-grained parameters, and complex relationships between entities. Combined with dynamic query expansion designed for specific domain knowledge bases, it can generate high-quality rewriting questions that include professional variations, synonyms, hyponyms, and core concept breakdowns. This effectively covers long-tail entity expressions and differences in expression within the knowledge base, overcoming the "zero recall" or "low relevance recall" problems caused by subtle differences in terminology or sparse expression in traditional vector retrieval. This significantly improves the effective recall rate, ensuring the professionalism and accuracy of the generated answers.

[0130] 3. Enhancing the overall robustness and reliability of the RAG system: A boundary-controlled problem rewriting approach ensures that the generated reconstructed problems do not deviate from the user's original intent, avoiding excessive expansion and the introduction of noise. Parallel database retrieval combined with a confidence-weighted fusion mechanism integrates the advantages of the original query (fidelity) and the rewritten query (high coverage / high accuracy), much like multi-expert voting, selecting the most reliable contextual information as the basis for generation. The final generated answer maintains the fluency of the large language model while systematically ensuring its factual accuracy, domain expertise, and intent relevance, thus improving user trust and the system's practical value.

[0131] This application also provides an electronic device in its embodiments. (See reference...) Figure 3 The diagram illustrates a structural schematic suitable for implementing the electronic device in the embodiments of this application. The electronic device in the embodiments of this application may include, but is not limited to, fixed terminals such as mobile phones, tablets, large-screen teaching displays, wearable devices, etc. Figure 3 The electronic device shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of this application.

[0132] like Figure 3 As shown, the electronic device may include a processing unit (e.g., a central processing unit, a graphics processing unit, etc.) 1, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 2 or a program loaded from a storage device 8 into a random access memory (RAM) 3, to implement the question-and-answer method of the foregoing embodiments of this application. When the electronic device is powered on, the RAM 3 also stores various programs and data required for the operation of the electronic device. The processing unit 1, ROM 2, and RAM 3 are interconnected via a bus 4. An input / output (I / O) interface 5 is also connected to the bus 4.

[0133] Typically, the following devices can be connected to I / O interface 5: input devices 6 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 7 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 8 including, for example, memory cards, hard drives, etc.; and communication devices 9. Communication device 9 allows electronic devices to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 3 Electronic devices with various devices are shown, but it should be understood that it is not required to implement or have all of the devices shown. More or fewer devices may be implemented or have alternatively.

[0134] This application also provides a computer program product including computer-readable instructions, which, when executed on an electronic device, cause the electronic device to implement any of the question-and-answer methods provided in this application.

[0135] This application also provides a computer-readable storage medium that carries one or more computer programs. When the one or more computer programs are executed by an electronic device, the electronic device can implement any of the question-and-answer methods provided in this application.

[0136] It should also be noted that the device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. In addition, in the device embodiment drawings provided in this application, the connection relationship between modules indicates that they have a communication connection, which can be implemented as one or more communication buses or signal lines.

[0137] Through the above description of the embodiments, those skilled in the art can clearly understand that this application can be implemented by means of software plus necessary general-purpose hardware, or it can be implemented by special-purpose hardware including application-specific integrated circuits, special-purpose CPUs, special-purpose memory, special-purpose components, etc. Generally, any function performed by a computer program can be easily implemented by corresponding hardware, and the specific hardware structure used to implement the same function can also be diverse, such as analog circuits, digital circuits, or special-purpose circuits. However, for this application, software program implementation is more often the preferred implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a readable storage medium, such as a computer floppy disk, USB flash drive, mobile hard disk, ROM, RAM, magnetic disk, or optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, training equipment, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0138] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product.

[0139] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions may be transmitted from one website, computer, training device, or data center to another website, computer, training device, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a training device or data center that integrates one or more available media. The available media may be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., DVDs), or semiconductor media (e.g., solid-state drives (SSDs)).

[0140] The various embodiments in this specification are described in a progressive manner. Each embodiment focuses on the differences from other embodiments. The various embodiments can be combined as needed, and the same or similar parts can be referred to each other.

Claims

1. A question and answer method, characterized by, The method comprises: obtaining a user input original question; performing information retrieval based on the original question to obtain a retrieval result corresponding to the original question and a matching degree between the retrieval result and the original question; in a case where a first condition is met, performing semantic analysis on the original question input configured large language model to obtain at least one structured information of the original question, the structured information being a structured representation of a logical relationship between any two entities in the original question, the first condition representing that the retrieval result does not match a solution requirement of the original question; generating a reconstruction question according to each structured information; performing information retrieval according to the reconstruction question to obtain a retrieval result corresponding to the reconstruction question and a matching degree between the retrieval result and the reconstruction question; obtaining a query source weight corresponding to a problem type of each retrieval result, the query source weight representing an approaching degree between a problem of the corresponding problem type and a real intention of a user, the problem type including the original question and the reconstruction question; fusing the matching degree and the query source weight corresponding to each retrieval result to obtain a fusion confidence score of each retrieval result; generating a target answer result for the original question in the large language model configured for the original question, the target answer result corresponding to the fusion confidence score meeting a preset condition; the process of generating a reconstruction question according to each structured information comprises: obtaining a second prompt instruction prompt format template, the second prompt instruction prompt format template including a second task instruction and a structured information slot, the second task instruction being used to instruct a large model to generate a reconstruction question for retrieval according to structured information in the structured information slot; filling the structured information into the structured information slot to obtain a target second prompt instruction prompt, and inputting the target second prompt instruction prompt into the large model to obtain the reconstruction question generated by the large model.

2. The question and answer method of claim 1, wherein, the process of performing semantic analysis on the original question input configured large language model to obtain at least one structured information of the original question comprises: obtaining a first prompt instruction prompt format template, the first prompt instruction prompt format template including a first task instruction and an original question slot, the first task instruction being used to instruct a large language model to perform semantic analysis on an original question in the original question slot and generate at least one triple by using analyzed entities and logical relationships between the entities; filling the original question into the original question slot to obtain a target first prompt instruction prompt, and inputting the target first prompt instruction prompt into the configured large language model to obtain at least one triple generated by the large model as structured information.

3. The question answering method of claim 1, wherein, the second task instruction is also used to instruct the large model to replace names of entities in the structured information with standardized entity names in the process of generating a reconstruction question for retrieval according to structured information in the structured information slot. And / or, the second task instruction is further used to instruct the large model to perform synonym replacement on a relationship word in the structured information that represents a logical relationship between entities in a process of generating a reconstructed question for retrieval according to structured information in the structured information slot.

4. The question answering method of claim 1, wherein, The process of determining whether the first condition is met includes: The configured large model is called to instruct the large model to refer to a retrieval result corresponding to the original question, determine whether the retrieval result matches the answer requirement of the original question, and generate a judgment result.

5. An electronic device, comprising: Including: A memory and a processor; The memory is configured to store a program. The processor is configured to execute the program to implement the steps of the question and answer method according to any one of claims 1-4.

6. A readable storage medium, having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the steps of the question and answer method according to any one of claims 1-4.

7. A computer program product comprising a computer program, characterized in that, The computer program is executed by the processor to implement the steps of the question and answer method according to any one of claims 1-4.

Images