Automated Question Answering Methods for Intelligent Engine Systems Based on Large Language Models

Abstract

This application relates to an automated question-answering method for an intelligent engine system based on a large language model. The method includes: extracting logical primitives from a business knowledge base and constructing a dependency graph based on the dependencies between these primitives; generating a semantic requirement value for each logical primitive based on the semantic association between the question text and the logical primitives, and calculating the compensation priority of the logical primitives based on the similarity between the question text and the logical primitives, and the node degree features of the logical primitives in the dependency graph; determining the logical singularity missing degree of each logical primitive based on the semantic requirement value and the compensation priority, and identifying logical primitives with a logical singularity missing degree greater than a preset threshold as logical singularities; determining the parsing order of the logical singularities based on the dependency graph, and parsing the logical singularities according to the parsing order to obtain the parsing results; and generating an answer for the question text based on the parsing results. This application can improve the accuracy of answers in complex business scenarios.

Description

Technical Field

[0001] This application relates to the field of natural language processing technology, and in particular to an automated question answering method for an intelligent engine system based on a large language model. Background Technology

[0002] With the rapid development of large language model technology, intelligent question-answering systems have gradually replaced traditional customer service robots based on rule matching or retrieval ranking, becoming the core carrier for automated interaction between enterprises and customers / consumers. Current mainstream technical solutions typically adopt a retrieval-enhanced generation (RAG) architecture, which matches user questions with text fragments in a knowledge base using vector similarity, and then uses the retrieval results as context input to a large model to generate an answer. This type of method alleviates the illusion problem of large models to some extent and improves the flexibility of knowledge updates, and is widely used in e-commerce consultation, after-sales support, and other scenarios.

[0003] However, the existing RAG architecture still faces a key technical bottleneck when dealing with complex enterprise business scenarios: the semantic disconnect between the retrieval and generation stages. The text fragments retrieved by the system based on vector similarity are often literally related to the question, forcing large models to guesswork based on incomplete context when generating answers, making it difficult to guarantee the accuracy and interpretability of the responses. Summary of the Invention

[0004] To address the aforementioned technical problems, the purpose of this application is to provide an automated question-answering method for an intelligent engine system based on a large language model. The specific technical solution adopted is as follows: Firstly, an automated question-answering method for an intelligent engine system based on a large language model is provided, the method comprising: Extract logical primitives from the business knowledge base and construct a dependency graph based on the dependencies between the logical primitives; Based on the semantic association between the user-input question text and the logical primitives, a semantic requirement value for each logical primitive is generated. Then, based on the similarity between the question text and each logical primitive, and the node degree feature of each logical primitive in the dependency graph, the compensation priority of each logical primitive is calculated. The compensation priority represents the degree of compensation priority of the logical primitive as a preceding node in the dependency graph. Based on semantic requirements and compensation priority, the logical singularity missing degree of each logical primitive is determined, and logical primitives with a logical singularity missing degree greater than a preset threshold are identified as logical singularities; the logical singularity missing degree characterizes the importance of the logical primitive in compensating the problem text. Based on the dependency graph, determine the parsing order of logical singularities, and parse the logical singularities according to the parsing order to obtain the parsing results; Based on the analysis results, generate an answer to the question text.

[0005] Optionally, logical primitives are extracted from the business knowledge base, and a dependency graph is constructed based on the dependencies between the logical primitives, including: Extract logical primitives from the business knowledge base; these logical primitives include entity primitives, attribute primitives, relation primitives, rule primitives, and process primitives. Determine a unique identifier for each logical primitive and associate it with a set of synonyms and a set of necessary role slots; the set of necessary role slots represents the set of elements necessary for a semantically complete description of the logical primitive. Parse the step description text of each business process to identify the sequence and prerequisites between steps, and transform the sequence and prerequisites into directed dependencies between logical primitives; The logical primitives contained in the condition part of the business rule are used as premise primitives, and the logical primitives contained in the conclusion part are used as successor primitives. Based on the directed dependencies between the premise primitives and the successor primitives, a dependency graph is constructed; where the nodes of the dependency graph are the logical primitives.

[0006] Optionally, based on the semantic association between the user-input question text and the logical primitives, a semantic requirement value for each logical primitive is generated, including: The sum of semantic similarities between all interrogative words in the question text and each logical primitive is calculated to obtain the interrogative focus intensity of each logical primitive. Based on the number of unfilled roles in the downstream nodes of each logical primitive in the dependency graph and the number of roles in the necessary role slot set of the downstream nodes of each logical primitive, the unfilled role strength of each logical primitive is generated; where unfilled roles indicate: roles in the necessary role slot set of the logical primitive that do not appear in the user question text. Determine the contextual keyword strength of each logical unit based on the relevance between the keywords in the question text and each logical unit; The semantic requirement value of each logical primitive is obtained by weighted fusion of the question focus strength, unfilled role strength and context keyword strength.

[0007] Optionally, based on the similarity between the question text and each logical primitive, and the node degree feature of each logical primitive in the dependency graph, the compensation priority of each logical primitive is calculated, including: Based on the similarity between the keywords in the question text and each logical primitive, the question text is matched with the logical primitives, and the set of logical primitives that are successfully matched is determined as the initial set of activated nodes. For each logical primitive, calculate the shortest directed path distance from the initial set of activated nodes to each logical primitive in the dependency graph; Calculate the criticality factor of each logical primitive based on its out-degree and in-degree in the dependency graph; the node degree characteristics of a logical primitive include its out-degree and in-degree. The compensation priority of each logical primitive is calculated based on the shortest directed path distance and criticality factor of each logical primitive.

[0008] Optionally, based on the out-degree and in-degree of each logical primitive in the dependency graph, a criticality factor for each logical primitive is calculated, including: Find the maximum out-degree of all nodes in the dependency graph; The first coefficient of each logic primitive is obtained by the ratio of its out-degree to its maximum out-degree. The second coefficient of each logical primitive is obtained by taking the logarithm of the sum of the in-degree of each logical primitive and the preset value. Based on the first and second coefficients of each logical primitive, the criticality factor of each logical primitive is obtained.

[0009] Optionally, based on the first and second coefficients of each logical primitive, the criticality factor of each logical primitive is obtained, including: The criticality factor of each logical primitive is obtained by multiplying the first coefficient and the second coefficient of each logical primitive.

[0010] Optionally, the logical singularity missing degree of each logical primitive is determined based on the semantic requirement value and compensation priority, including: For each logical primitive, the root mean square of the semantic requirement value and the compensation priority is calculated to obtain the logical singularity missing degree of each logical primitive.

[0011] Optionally, the resolution order of logical singularities is determined based on the dependency graph, including: Extract a subgraph containing all logical singularities from the dependency graph; Calculate the in-degree of each logical singularity in the subgraph, and add logical singularities with an in-degree of zero as the current nodes to be parsed to the parsing sequence; Remove the currently unparsed node from the subgraph, update the in-degree of the remaining logical singularities in the subgraph, and repeat the process of adding newly generated logical singularities with an in-degree of zero to the parsing sequence until all logical singularities have been added to the parsing sequence, thus obtaining the parsing order of the logical singularities.

[0012] Optionally, logical singularities are parsed according to the parsing order to obtain the parsing results, including: Based on the data source of the logical singularity, the logical singularity is parsed in the parsing order. If the data source of the logical singularity is a structured database, the parsing result is obtained through structured query. If the data source of the logical singularity is a rule base, the parsing result of the premise logical singularity is substituted into the preset logical expression of the business rule to obtain the parsing result. If the data source of the logical singularity is a process base, the corresponding step description text is retrieved to obtain the parsing result.

[0013] Optionally, based on the parsing results, an answer to the question text is generated, including: If the question text matches the preset question template, the parsing result will be filled into the corresponding placeholder in the question template to generate an answer for the question text; If the question text does not match the preset question template, the parsing result is concatenated with the question text to form a structured prompt, which is then input into the large language model to generate an answer to the question text.

[0014] Based on common knowledge in the field, the above-mentioned preferred conditions can be combined arbitrarily to obtain various preferred embodiments of this application.

[0015] This application offers the following advantages: It extracts logical primitives from a business knowledge base and constructs a dependency graph; generates semantic requirement values ​​based on the semantic association between the user's question text and the logical primitives; calculates compensation priority based on the similarity between the question text and the logical primitives and the node degree characteristics of the logical primitives in the dependency graph; determines the logical singularity missingness based on the semantic requirement value and the compensation priority; identifies logical primitives with a missingness greater than a preset threshold as logical singularities; determines the parsing order according to the dependency graph; and generates the answer after parsing. By accurately identifying the logical gaps necessary for the answer but currently missing information through logical singularity missingness, it avoids semantic fragmentation problems. Through the dual determination of compensation priority and logical singularity missingness, and the ordered parsing of the dependency graph, it provides causal and complete contextual support for large language models, significantly improving the accuracy, interpretability, reliability, and processing efficiency of the question-answering system in complex business scenarios. Attached Figure Description

[0016] To more clearly illustrate the technical solutions and advantages in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 This is a flowchart of an automated question-answering method for an intelligent engine system based on a large language model, as shown in one embodiment. Figure 2This is a schematic diagram of the structure of an automated question-answering system based on a large language model intelligent engine system in one embodiment; Figure 3 This is a schematic diagram of the structure of an electronic device in one embodiment. Detailed Implementation

[0018] To further illustrate the technical means and effects adopted by this application to achieve the intended inventive purpose, the following, in conjunction with the accompanying drawings and preferred embodiments, details the specific implementation, structure, features, and effects of an automated question-answering method for an intelligent engine system based on a large language model proposed in this application. In the following description, different "one embodiment" or "another embodiment" do not necessarily refer to the same embodiment. Furthermore, specific features, structures, or characteristics in one or more embodiments can be combined in any suitable form.

[0019] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains.

[0020] The following, with reference to the accompanying drawings, details a specific scheme for an automated question-answering method based on a large language model intelligent engine system provided in this application. For example... Figure 1 As shown, the method includes: S11. Extract logical primitives from the business knowledge base and construct a dependency graph based on the dependencies between logical primitives.

[0021] Integrate heterogeneous data from multiple sources to form a unified business knowledge base, which mainly includes the following components: 1. Structured database: Stores order information (order number, user ID, product SKU, order time, order status), product information (product ID, name, inventory, price, warranty period), user information (user ID, membership level, points), etc. This data is accessed through SQL interface or API call.

[0022] 2. Unstructured document library: Contains text materials such as product manuals, frequently asked questions (FAQ), after-sales service policies, and operation guides, stored in vectorized form, supporting semantic search.

[0023] 3. Rule base: Stores business logic rules, such as warranty conditions (within the warranty period and not caused by human error), return and exchange rules (7-day no-reason return), VIP benefits, etc. Each rule is represented by a logical expression.

[0024] 4. Process Library: Stores standardized business processes, such as return process, repair steps, refund application process, etc.; each process is described in a structured form, including a list of steps, required materials, precautions, etc.

[0025] In one embodiment, logical primitives are extracted from a business knowledge base, and a dependency graph is constructed based on the dependencies between the logical primitives, including: Extract logical primitives from the business knowledge base; these logical primitives include entity primitives, attribute primitives, relation primitives, rule primitives, and process primitives. Determine a unique identifier for each logical primitive and associate it with a set of synonyms and a set of necessary role slots; the set of necessary role slots represents the set of elements necessary for a semantically complete description of the logical primitive. Parse the step description text of each business process to identify the sequence and prerequisites between steps, and transform the sequence and prerequisites into directed dependencies between logical primitives; The logical primitives contained in the condition part of the business rule are used as premise primitives, and the logical primitives contained in the conclusion part are used as successor primitives. Based on the directed dependencies between the premise primitives and the successor primitives, a dependency graph is constructed; where the nodes of the dependency graph are the logical primitives.

[0026] Extract logical primitives from the business knowledge base and establish prerequisite dependencies between primitives, specifically: 1. By analyzing business knowledge, define the types of logical primitives, which include: entity primitives, attribute primitives, relation primitives, procedural primitives, and rule primitives. Specifically: Entity-based primitives: orders, products, users, coupons, etc.; Attribute-based primitives: order status, product inventory, product price, user membership level, etc.; Relational primitives: an order includes products, a user owns an order, and the products belong to a category, etc. Process-oriented primitives: refund process, return process, repair process, exchange process, etc.; Rule-based basic elements: whether it meets the warranty conditions, whether it supports 7-day no-reason return, whether it is a VIP user, etc.

[0027] 2. Assign a unique identifier to each of the above logical primitives and associate it with its data source (e.g., database fields come from structured databases, document paragraphs come from unstructured document libraries, rule IDs come from rule libraries, and process IDs come from process libraries) and a set of synonyms. The set of synonyms can be automatically expanded from the domain dictionary, WordNet, or by using word vector clustering from the business corpus (e.g., synonyms for the primitive order status include order status, order progress, and shipping status).

[0028] 3. Determine the necessary role slot set for each logical primitive. For example, the order logic primitive contains {order number, user ID, order time}, and the refund logic primitive contains {order number, refund reason, refund amount}. These role slots can be other primitives or basic attributes and are recorded in the primitive attributes and associated with a unique identifier.

[0029] For each business process (such as a return process) in the process library, its step description text is parsed. Dependency parsing is used to identify the sequence and preconditions between steps, and these sequences and preconditions are transformed into directed dependencies between logical primitives. For each business rule in the rule library, the logical primitives contained in the condition part of the business rule are used as prerequisite primitives, and the logical primitives contained in the conclusion part are used as successor primitives. Prerequisite and successor relationships between logical primitives include, for example: to obtain the order status, the order number must first be obtained; to determine whether the warranty conditions are met, the purchase date, current date, and reason for damage are needed; to execute the refund process, the order number, refund reason, and refund amount are needed.

[0030] Construct a dependency graph based on the directed dependencies between the premise primitives and the successor primitives. , Where V is the set of all logical primitive nodes and E is the set of all directed edges, if logical primitive It is a logical primitive The premise is that there exists a directed edge. , which means yes The necessary prerequisites are defined in the dependency graph, where nodes represent logical primitives and directed edges point from prerequisite primitives to successor primitives. The dependency graph can be obtained through expert definition, rule parsing, or automatic extraction from business processes, and is maintained and updated regularly.

[0031] S12. Based on the semantic association between the user-input question text and the logical primitives, generate the semantic requirement value for each logical primitive, and calculate the compensation priority of each logical primitive based on the similarity between the question text and each logical primitive, and the node degree feature of each logical primitive in the dependency graph.

[0032] Among them, the compensation priority characterizes the degree of compensation priority of the logical primitive as a preceding node in the dependency graph, that is, the degree of urgency of the logical primitive being compensated first at the graph structure level.

[0033] When a user submits issue text H through any channel, the following preprocessing operations are performed: 1. Text cleaning: Remove irrelevant characters (such as emojis and extra spaces), standardize Chinese and English punctuation, and convert traditional Chinese to simplified Chinese.

[0034] 2. Word segmentation and part-of-speech tagging: Use a domain-adaptive word segmentation tool to segment words and tag them with parts of speech (nouns, verbs, adjectives, etc.).

[0035] 3. Dependency Syntax Analysis: This involves analyzing syntactic structures, identifying interrogative words and their governing core components, and extracting dependency relationships such as subject-verb-object, attributive, adverbial, and complement, thus providing a foundation for subsequent calculation of the interrogative focus intensity.

[0036] 4. Keyword extraction: Nouns, verbs, and adjectives are selected from the word segmentation results, and then filtered through a domain dictionary to retain business-related words, forming a keyword set T.

[0037] In one embodiment, based on the semantic association between the user-input question text and the logical primitives, a semantic requirement value for each logical primitive is generated, including: The sum of semantic similarities between all interrogative words in the question text and each logical primitive is calculated to obtain the interrogative focus intensity of each logical primitive. Based on the number of unfilled roles in the downstream nodes of each logical primitive in the dependency graph and the number of roles in the necessary role slot set of the downstream nodes of each logical primitive, the unfilled role strength of each logical primitive is generated; where unfilled roles indicate: roles in the necessary role slot set of the logical primitive that do not appear in the user question text. Determine the contextual keyword strength of each logical unit based on the relevance between the keywords in the question text and each logical unit; The semantic requirement value of each logical primitive is obtained by weighted fusion of the question focus strength, unfilled role strength and context keyword strength.

[0038] User questions inherently contain a strong demand for specific information. This demand is not only reflected in the question words, but also in the tension between the mentioned and unmentioned information. Therefore, it is necessary to extract the demand signals from the question text.

[0039] For the user's question text H, dependency parsing is used to identify the interrogative words (such as what, why, how, how much) and their governing core components. Then, for each interrogative word, a set of logical primitive types that it may point to is predefined: ,in, Represents the q-th interrogative word. As an inclusion symbol, P represents the set of primitive types that the interrogative word is most likely to point to; for example, "how much" maps to all numerical primitives (such as inventory, price), and "how" maps to process primitives (such as maintenance steps, application process).

[0040] For any logical primitive p, if p appears in the interrogative word... In the set P of logical primitive types, the interrogative words are calculated. The semantic similarity with logical primitive p is calculated, which can be obtained by pre-training word vectors; otherwise, the similarity is considered to be 0. The sum of the semantic similarities between all interrogative words and logical primitive p is calculated to obtain the interrogative focus strength of logical primitive p. If the question text does not contain an explicit interrogative word, then all Set it to 0. The larger the value, the stronger the user's question is directed at p.

[0041] Each logical primitive p defines a set of necessary role slots (such as subject, object, time, condition, etc.), which semantically constitute a complete description of the logical primitive p; for example, the logical primitive "refund" requires the roles of order number, refund reason, and refund amount. By identifying the filled role slots in the question text H (i.e., the dependencies related to p, such as specific order number, amount, etc.) through a semantic dependency graph, and checking which necessary role slots have not yet been filled, the number of filled roles for each logical primitive p can be counted.

[0042] When a necessary role for a logical primitive p is not present in the problem, a missing role signal is generated. This signal not only affects the current primitive p but may also propagate through the dependency graph to other downstream primitives that depend on that role. Therefore, we need to obtain the set of all downstream nodes reachable from p in the dependency graph. (Including p itself), for each downstream node Calculate the proportion of missing characters. ,in ,in, For example, the minimum value. To avoid division by zero, .

[0043] Each downstream node of the computational logic primitive p The confidence level mentioned in the question If the question directly contains 'q' or its synonyms, then the confidence level is... If the value is 1, then calculate the maximum cosine similarity between the word vectors of all words in the question and q. The cosine similarity ranges from -1 to 1. If the similarity is less than 0, then it is forcibly truncated to 0; if it is greater than or equal to 0, then the original value is retained to obtain the confidence score. .

[0044] Increase the unfilled character strength of primitive p. Defined as: the proportion of missing roles among all downstream nodes reachable from p (including p itself). confidence level with each downstream node The weighted sum of the unfilled character strengths The calculation formula is: Unfilled character strength The larger the value, the wider the impact of the absence of p on the role, and the higher the compensation requirement.

[0045] Extract keywords (nouns, verbs, adjectives) from the question text, but only retain words relevant to business logic (filtered using a domain dictionary). For each keyword t in the question, calculate its relevance to each logical primitive p. Specifically, obtain the cosine similarity and PMI (point mutual information) between the keyword and primitive p. The PMI value can be obtained by offline statistical analysis of the co-occurrence frequency of keywords and primitive names in documents (such as product manuals and FAQs) in the business knowledge base, and the PMI value will be mapped to the [0,1] interval according to minimum-maximum normalization. If the corpus is insufficient, only the cosine similarity can be used. The average of the cosine similarity and PMI is taken as the relevance between keyword t and logical primitive p. The relevance of all keywords in the context to logical primitive p is summed to obtain the context keyword strength of logical primitive p. , indicating the degree to which the words in the question suggest p.

[0046] The semantic demand value of each logical primitive p is obtained by weighting and fusing the intensity of the question focus, the intensity of the unfilled role, and the intensity of the context keywords. Semantic requirement value The calculation formula is: ; in, For preset weights, such as 0.5 0.3 The value is 0.2, which can be adjusted according to the business scenario (e.g., appropriately increased in customer service scenarios). ), The range of values ​​for is controlled between [0,1] by min-max normalization, which represents the degree to which the logical primitive p is needed in the current problem from a semantic point of view.

[0047] In one embodiment, the compensation priority of each logical primitive is calculated based on the similarity between the question text and each logical primitive, and the node degree feature of each logical primitive in the dependency graph, including: Based on the similarity between the keywords in the question text and each logical primitive, the question text is matched with the logical primitives, and the set of logical primitives that are successfully matched is determined as the initial set of activated nodes. For each logical primitive, calculate the shortest directed path distance from the initial set of activated nodes to each logical primitive in the dependency graph; Calculate the criticality factor of each logical primitive based on its out-degree and in-degree in the dependency graph; the node degree characteristics of a logical primitive include its out-degree and in-degree. The compensation priority of each logical primitive is calculated based on the shortest directed path distance and criticality factor of each logical primitive.

[0048] Specifically, based on the out-degree and in-degree of each logical primitive in the dependency graph, the criticality factor of each logical primitive is calculated, including: Find the maximum out-degree of all nodes in the dependency graph; The first coefficient of each logic primitive is obtained by the ratio of its out-degree to its maximum out-degree. The second coefficient of each logical primitive is obtained by taking the logarithm of the sum of the in-degree of each logical primitive and the preset value. Based on the first and second coefficients of each logical primitive, the criticality factor of each logical primitive is obtained.

[0049] Specifically, based on the first and second coefficients of each logical primitive, the criticality factor of each logical primitive is obtained, including: The criticality factor of each logical primitive is obtained by multiplying the first coefficient and the second coefficient of each logical primitive.

[0050] The preset value can be set according to the actual situation, for example, 1.

[0051] Relying solely on semantic features can lead to biases, such as overlooking crucial nodes not mentioned in the problem but necessary for reasoning. Therefore, it is necessary to introduce structural information from a logical dependency graph, as described above. This describes the preconditions between logical primitives. Based on the question text H, the question text is matched with the logical primitives through synonym matching or semantic similarity calculation. For example, for each logical primitive p, if the cosine similarity between any word in the question text and primitive p is > 0.8, then logical primitive p is considered to have successfully matched the question text, and this logical primitive is determined as an active node. This results in an initial set of active nodes consisting of multiple successfully matched logical primitives, i.e., multiple active nodes. .

[0052] For any logical primitive p, in the dependency graph In the process, calculate from the currently active node set Shortest directed path distance to p Because the dependencies are directional, the path must follow the direction of the edge from the previous primitive to the successor primitive. If there exists an active node... Make existence from The directed path to p, then The minimum number of edges for this path; if it is unreachable, then... .from The shorter the distance to p, the easier it is to deduce p from known information, so the direct need for compensation to p may be lower; conversely, the longer the distance, the more independent p is, and it may be necessary to actively obtain it.

[0053] In a dependency graph, the criticality of a node is determined by its topological role, especially when it serves as a hub for multiple subsequent inference branches. In a dependency graph, the logical primitive p is a node, and the out-degree of logical primitive p... The in-degree is the number of nodes that depend on p (i.e., how many nodes need p as a prerequisite). Let p be the number of prerequisites that p depends on. Nodes with high out-degree are often the hubs of reasoning; their absence can prevent many subsequent inferences from proceeding. Therefore, the criticality factor of the logical primitive p is... for: ,in, The maximum out-degree of all nodes in the dependency graph. This represents the out-degree of the logical primitive p. Perform maximum value normalization. For extremely small values, for example, Logarithmic terms This represents the smooth introduction of the in-degree influence of the logical primitive p using a logarithmic function. The product of the two is the criticality factor of the p-th primitive. After Min-Max normalization, the value is mapped to the 0-1 interval. The higher the out-degree or in-degree, the larger the criticality factor.

[0054] Combining the above shortest directed path distance and key factors Calculate the compensation priority of the logical primitive p. , specifically: If p can reach ( ≠ Then the compensation priority The calculation formula is: , This is a positive scaling factor, for example, 0.5, representing the shortest directed path distance. The closer and more critical factors The lower the value, the lower the priority of compensation; the longer the shortest directed path distance or the higher the criticality factor, the higher the priority of compensation.

[0055] If p is unreachable ( Then the compensation priority The calculation formula is: ,in, , This is the amplification factor for unreachable nodes (e.g., 2.0), because unreachable nodes cannot be derived and must be obtained from external sources, thus requiring higher compensation.

[0056] Compensation priority The calculation formula makes the logical primitives with shortest directed path distances but low criticality factors... Smaller logical primitives with longer shortest directed path distances or higher criticality factors. Larger.

[0057] S13. Based on the semantic requirement value and compensation priority, determine the logical singularity missing degree of each logical primitive, and identify logical primitives with a logical singularity missing degree greater than a preset threshold as logical singularities.

[0058] Among them, the logical singularity missingness represents the importance of logical primitives in compensating for the question text, that is, it represents the comprehensive urgency of logical primitives as logical breakpoints that are necessary for the answer but are currently missing, in the context of combining the semantics of the user's question and the graph structure.

[0059] The preset threshold can be set according to the actual situation, for example, 0.6.

[0060] In one embodiment, the logical singularity missing degree of each logical primitive is determined based on the semantic requirement value and compensation priority, including: For each logical primitive, the root mean square of the semantic requirement value and the compensation priority is calculated to obtain the logical singularity missing degree of each logical primitive.

[0061] Based on the semantic demand values ​​obtained above and compensation priority Calculate the logical singularity missing degree for each logical primitive p. Logical singularity missing degree The calculation formula is: ; The formula uses the root mean square instead of a simple average because: when the semantic demand value and compensation priority At the same time, when it is higher It will increase significantly; if any item is low, then This will be lowered. This characteristic can more sensitively capture nodes that exhibit high urgency of absence in both semantic and structural dimensions; that is, the absence of such a node will cause the entire reasoning chain to break or the answer to become unavailable. As an empirical scoring model for measuring the priority of missing information, the following calculations were made: Then, for all candidate nodes Perform global normalization again (e.g., min-max normalization) to ensure the final... The value range is [0,1]. The larger the value, the more urgently the logic primitive needs compensation.

[0062] Given an initial set of activated nodes V and a set of candidate answer nodes Q that the user's question intent points to, the candidate answer node set Q can be determined in the following way: First, the user's question text H is classified according to intent. A mapping relationship between common question types and answer primitives is pre-defined; for example, "refund process" corresponds to the process primitive "refund process". Then, a set of candidate answer nodes Q is generated based on the intent classification of the question text H. If the intent classification cannot determine the answer, the semantic similarity between the question text H and each logical primitive p is calculated. The top k primitives with the highest semantic similarity (e.g., k=3 or 5) are selected as candidate answer nodes, forming Q. Semantic similarity can be calculated by averaging word vectors and then calculating cosine similarity, or by using a pre-trained sentence vector model.

[0063] The steps for fault identification are as follows: Determine the set of candidate answer nodes. This refers to the type of answer a user might expect for their question. For each candidate answer node... In the global dependency graph G, all necessary prerequisite primitives, including direct and indirect prerequisites, are obtained by reverse traversal (along the reverse direction of the directed edges), forming a dependency closure, which is the set of all necessary prerequisites. For each logical primitive p, the following is calculated: Determine a preset threshold (e.g., 0.6), and... Logical primitives exceeding a preset threshold are identified as logical singularities. The nodes corresponding to these singularities are considered faults requiring immediate compensation, and are then processed according to... The values ​​are sorted in descending order to obtain a priority compensation list. If some nodes... Values ​​below the threshold but still within the dependency closure of candidate answers can be marked as minor faults and processed as needed in subsequent steps.

[0064] Through the above-mentioned fault identification steps, the priority of information compensation can be dynamically determined, avoiding blind retrieval and providing a complete and causally related context for large models to generate accurate answers. For example, in e-commerce after-sales scenarios, if a user asks "How to apply for a refund?", the system may identify that the logical primitive "refund process" has high semantic demand, but if the nodes it depends on, such as order number and order status, are not mentioned in the question, these dependent nodes may be listed as faults due to their high structural urgency, thus prioritizing the retrieval of relevant order information.

[0065] S14. Based on the dependency graph, determine the parsing order of logical singularities, and parse the logical singularities according to the parsing order to obtain the parsing results.

[0066] During the fault identification stage, the logical primitive nodes (i.e. logical singularities) that most need compensation in the current state are identified. These nodes represent the precise information necessary to answer the user's question. Since there may be prerequisite dependencies between logical singularities (i.e., parsing a singularity may require obtaining the value of its prerequisite node first), the parsing order must be determined according to the directionality of the dependency relationship.

[0067] In one embodiment, determining the parsing order of logical singularities based on the dependency graph includes: Extract a subgraph containing all logical singularities from the dependency graph; Calculate the in-degree of each logical singularity in the subgraph, and add logical singularities with an in-degree of zero as the current nodes to be parsed to the parsing sequence; Remove the currently unparsed node from the subgraph, update the in-degree of the remaining logical singularities in the subgraph, and repeat the process of adding newly generated logical singularities with an in-degree of zero to the parsing sequence until all logical singularities have been added to the parsing sequence, thus obtaining the parsing order of the logical singularities.

[0068] The following topological sorting steps are performed on the current set of logical singularities using the dependency graph G, specifically: 1. Extract a subgraph from the dependency graph that contains only nodes with logical singularities and directed edges (prerequisite relations); 2. Calculate the in-degree of each node in the subgraph (i.e., the number of other singularities it depends on); 3. Find the node with an in-degree of zero (i.e., the node that does not depend on any other singularity) and add it to the parsing sequence as the current node to be parsed. This node is the first batch of nodes that can be parsed immediately. Then, parse these nodes in turn. After parsing each node p, mark its result as known and remove p and all its outgoing edges from the subgraph. Update the in-degree of the remaining nodes. Repeat this process to add newly generated logical singularities with an in-degree of zero to the parsing sequence until all logical singularities have been added to the parsing sequence, thus obtaining the parsing order of the logical singularities. 4. If a cycle exists in the subgraph (theoretically, the dependency graph should be a directed acyclic graph, but this may occur due to modeling errors), a complete topological sort cannot be performed. In this case, a heuristic strategy is used: calculate each logical singularity. Prioritize parsing The node with the highest value is selected, and the prerequisite dependency edge with the lowest confidence or call frequency in the cycle is disconnected. The original graph is then downgraded to a directed acyclic graph and reordered; or a multi-round dialogue is triggered to clarify the issue to the user.

[0069] In one embodiment, logical singularities are parsed according to the parsing order to obtain the parsing results, including: Based on the data source of the logical singularity, the logical singularity is parsed in the parsing order. If the data source of the logical singularity is a structured database, the parsing result is obtained through structured query. If the data source of the logical singularity is a rule base, the parsing result of the premise logical singularity is substituted into the preset logical expression of the business rule to obtain the parsing result. If the data source of the logical singularity is a process base, the corresponding step description text is retrieved to obtain the parsing result.

[0070] Based on the data source of the logical singularity and its preset data source mapping relationship (see the associated data source in the primitive definition in step one), each logical primitive p is parsed according to the parsing order. Let the value of each logical primitive p after parsing be... This value can be a number, a boolean value, a string, or structured text, etc. For example: if p is an order number, then... It could be 20250313001; if p is "whether it meets the warranty conditions", then... Possibly "Yes / No"; if p is the refund process, then Specific steps may be provided in the text.

[0071] The specific analysis process is as follows: 1. If the logical singularity originates from a structured database, corresponding to the structured data (such as database fields or API interfaces), directly initiate a query request to obtain the parsed results. (Structured query results); Confidence level upon successful query The confidence level is 1 when the test fails. It is 0.

[0072] 2. If the logical singularity originates from an unstructured document library, the parsing result is obtained through semantic retrieval. Then the confidence level Similarity score for semantic retrieval.

[0073] 3. If the logical singularity originates from the rule base and corresponds to a business rule (such as "whether it meets the warranty conditions"), then the parsing result of the already parsed premise singularity (such as the purchase date) is substituted into the logical expression corresponding to the rule for logical judgment to obtain the parsing result. (Yes / No), Confidence Level Take the minimum confidence score of all premise nodes (if all premises are structured data, then...). The confidence level is set to 1. If the premise contains uncertain information, the rule confidence level can be set.

[0074] 4. If the logical singularity originates from the process library and corresponds to a process description (such as "return process"), then retrieve the corresponding standardized step text from the process library to obtain the parsed result. (Step text), confidence level The value is 1.

[0075] After parsing, each logical singularity p is assigned a structured parsing result. and confidence level These results are stored as triples: This information is used as the basis for generating subsequent answers; if parsing fails, the missing logical singularity information is marked, and the reason is explained in subsequent answers. Simultaneously, the parsing path (e.g., which data source was used, which rule was applied) is recorded to provide interpretability in the answer. Subsequently, a confidence check mechanism is executed: the confidence level of each logical singularity is... The results are compared with a preset reliability threshold to select valid parsing results for logical singularities with a confidence level greater than the reliability threshold, so as to generate subsequent answers. The reliability threshold can be set according to the actual situation, such as 0.85 or 0.9.

[0076] S15. Based on the analysis results, generate the answer to the question text.

[0077] In one embodiment, based on the parsing results, an answer to the question text is generated, including: If the question text matches the preset question template, the parsing result will be filled into the corresponding placeholder in the question template to generate an answer for the question text; If the question text does not match the preset question template, the parsing result is concatenated with the question text to form a structured prompt, which is then input into the large language model to generate an answer to the question text.

[0078] After obtaining the analytical results of all logical singularities with high confidence, select the answer generation method based on the user's question type: Templated answers: If the user's question text can be mapped to a predefined common question template (such as querying order status or inquiring about warranty policies), the system will match the corresponding template from the template library, fill the template placeholder with the parsing results of logical singularities, and generate a structured answer; the template can be preset by business experts to ensure that the key information is accurately positioned and expressed in a standardized manner.

[0079] Dynamic generation: For complex problems without corresponding templates, the parsing results of logical singularities are concatenated with the user's question text to form a structured prompt, which is then input into the large language model. The instruction requires the model to organize language based on the provided facts and not to fabricate or tamper with logical singularity information.

[0080] After the answer is generated, the key information in the answer is automatically extracted and compared with the analysis result of the logical singularity to ensure consistency. If a contradiction is found (such as the model output not matching the analysis result of the logical singularity), the correction mechanism is triggered to force the replacement with the correct value or regenerate.

[0081] It should be understood that, although Figure 1 The steps in the flowchart are shown sequentially as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order in which these steps are executed, and they can be performed in other orders. Figure 1 At least some of the steps in the process may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least some of the sub-steps or stages of other steps.

[0082] This application also provides an automated question-answering system based on a large language model and an intelligent engine system, such as... Figure 2 As shown, the system includes: Extraction module 21 is used to extract logical primitives from the business knowledge base and construct a dependency graph based on the dependency relationships between logical primitives; The calculation module 22 is used to generate the semantic requirement value of each logical primitive based on the semantic association between the question text input by the user and the logical primitive, and to calculate the compensation priority of each logical primitive based on the similarity between the question text and each logical primitive and the node degree feature of each logical primitive in the dependency graph; the compensation priority represents the compensation priority of the logical primitive as a preceding node in the dependency graph. The determination module 23 is used to determine the logical singularity missing degree of each logical primitive based on the semantic requirement value and the compensation priority, and to determine the logical primitive with a logical singularity missing degree greater than a preset threshold as a logical singularity; the logical singularity missing degree characterizes the importance of the logical primitive for the compensation of the problem text; The parsing module 24 is used to determine the parsing order of logical singularities based on the dependency graph, and to parse the logical singularities according to the parsing order to obtain the parsing results; The generation module 25 is used to generate answers to the question text based on the parsing results.

[0083] For the system embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to in the description of the method embodiments. The system embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this application according to actual needs.

[0084] Figure 3This is a schematic diagram of the structure of an electronic device according to an example embodiment of this application. The electronic device includes a memory, a processor, and a computer program stored in the memory and used to run on the processor. When the processor executes the computer program, it implements the method described in any of the above embodiments. Figure 3 The electronic device 30 shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of this application.

[0085] like Figure 3 As shown, the electronic device 30 can be manifested as a general-purpose computing device, such as a server device. The components of the electronic device 30 may include, but are not limited to: at least one processor 31, at least one memory 32, and a bus 33 connecting different system components (including memory 32 and processor 31).

[0086] Bus 33 includes a data bus, an address bus, and a control bus.

[0087] The memory 32 may include volatile memory, such as random access memory 321 and / or cache memory 322, and may further include read-only memory 323.

[0088] The memory 32 may also include a program tool 325 (or utility) having a set (at least one) program module 324, such program module 324 including but not limited to: an operating system, one or more application programs, other program modules and program data, each or some combination of these examples may include an implementation of a network environment.

[0089] The processor 31 executes various functional applications and data processing, such as the methods provided in any of the above embodiments, by running computer programs stored in the memory 32.

[0090] Electronic device 30 can also communicate with one or more external devices 34 (e.g., keyboard, pointing device, etc.). This communication can be made through input / output interface 35. Furthermore, electronic device 30 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public network, such as the Internet) via network adapter 36. As shown, network adapter 36 communicates with other modules of electronic device 30 via bus 33. It should be understood that, although not shown in the figure, other hardware and / or software modules can be used in conjunction with electronic device 30, including but not limited to: microcode, device drivers, redundant processors, external disk drive arrays, RAID (disk array) systems, tape drives, and data backup storage systems.

[0091] It should be noted that although several units / modules or sub-units / modules of the electronic device have been mentioned in the detailed description above, this division is merely exemplary and not mandatory. In fact, according to the embodiments of this application, the features and functions of two or more units / modules described above can be embodied in one unit / module. Conversely, the features and functions of one unit / module described above can be further divided and embodied by multiple units / modules.

[0092] This application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method provided in any of the above embodiments.

[0093] The readable storage medium may be more specifically adopted, including but not limited to: portable disk, hard disk, random access memory, read-only memory, erasable programmable read-only memory, optical storage device, magnetic storage device, or any suitable combination thereof.

[0094] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0095] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the method described in any of the above embodiments.

[0096] The program code for executing the computer program product of this application can be written in any combination of one or more programming languages. The program code can be executed entirely on the user device, partially on the user device, as a standalone software package, partially on the user device and partially on a remote device, or entirely on a remote device.

[0097] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0098] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of the application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these modifications and improvements all fall within the protection scope of this application.

[0099] The various embodiments in this specification are described in a progressive manner. The same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on describing the differences from other embodiments.

Claims

1. An automated question-answering method for an intelligent engine system based on a large language model, characterized in that, The method includes: Logical primitives are extracted from the business knowledge base, and a dependency graph is constructed based on the dependencies between logical primitives; among them, logical primitives include entity primitives, attribute primitives, relation primitives, rule primitives, and process primitives; Based on the semantic association between the user-input question text and the logical primitives, a semantic requirement value for each logical primitive is generated. Then, based on the similarity between the question text and each logical primitive, and the node degree feature of each logical primitive in the dependency graph, the compensation priority of each logical primitive is calculated. The compensation priority represents the degree of compensation priority of the logical primitive as a preceding node in the dependency graph. Based on semantic requirements and compensation priority, the logical singularity missing degree of each logical primitive is determined, and logical primitives with a logical singularity missing degree greater than a preset threshold are identified as logical singularities; the logical singularity missing degree characterizes the importance of the logical primitive in compensating the problem text. Based on the dependency graph, determine the parsing order of logical singularities, and parse the logical singularities according to the parsing order to obtain the parsing results; Based on the analysis results, generate an answer to the question text; The calculation of the compensation priority for each logical primitive based on the similarity between the question text and each logical primitive, and the node degree feature of each logical primitive in the dependency graph, includes: Based on the similarity between the keywords in the question text and each logical primitive, the question text is matched with the logical primitives, and the set of logical primitives that are successfully matched is determined as the initial set of activated nodes. For each logical primitive, calculate the shortest directed path distance from the initial set of activated nodes to each logical primitive in the dependency graph; Calculate the criticality factor of each logical primitive based on its out-degree and in-degree in the dependency graph; the node degree characteristics of a logical primitive include its out-degree and in-degree. Calculate the compensation priority of each logical primitive based on the shortest directed path distance and criticality factor of each logical primitive. The determination of the logical singularity missing degree of each logical primitive based on semantic requirement value and compensation priority includes: For each logical primitive, the root mean square of the semantic requirement value and the compensation priority is calculated to obtain the logical singularity missing degree of each logical primitive.

2. The automated question-answering method for an intelligent engine system based on a large language model as described in claim 1, characterized in that, The step of extracting logical primitives from the business knowledge base and constructing a dependency graph based on the dependencies between logical primitives includes: Extract logical primitives from the business knowledge base; Determine a unique identifier for each logical primitive and associate it with a set of synonyms and a set of necessary role slots. The set of necessary role slots represents the set of elements necessary for a semantically complete description of a logical primitive. The role slots in the set of necessary role slots are basic attributes and are associated with the unique identifier. Parse the step description text of each business process to identify the sequence and prerequisites between steps, and transform the sequence and prerequisites into directed dependencies between logical primitives; The logical primitives contained in the condition part of the business rule are used as premise primitives, and the logical primitives contained in the conclusion part are used as successor primitives. Based on the directed dependencies between the premise primitives and the successor primitives, a dependency graph is constructed; where the nodes of the dependency graph are the logical primitives.

3. The automated question-answering method for an intelligent engine system based on a large language model as described in claim 1, characterized in that, The step of generating a semantic requirement value for each logical primitive based on the semantic association between the user-input question text and the logical primitives includes: The sum of semantic similarities between all interrogative words in the question text and each logical primitive is calculated to obtain the interrogative focus intensity of each logical primitive. Based on the number of unfilled roles in the downstream nodes of each logical primitive in the dependency graph and the number of roles in the necessary role slot set of the downstream nodes of each logical primitive, the unfilled role strength of each logical primitive is generated; where unfilled roles indicate: roles in the necessary role slot set of the logical primitive that do not appear in the user question text. Determine the contextual keyword strength of each logical unit based on the relevance between the keywords in the question text and each logical unit; The semantic requirement value of each logical primitive is obtained by weighted fusion of the question focus strength, unfilled role strength and context keyword strength.

4. The automated question-answering method for an intelligent engine system based on a large language model as described in claim 1, characterized in that, The calculation of the criticality factor for each logical primitive based on its out-degree and in-degree in the dependency graph includes: Find the maximum out-degree of all nodes in the dependency graph; The first coefficient of each logic primitive is obtained by the ratio of its out-degree to its maximum out-degree. The second coefficient of each logical primitive is obtained by taking the logarithm of the sum of the in-degree of each logical primitive and the preset value. Based on the first and second coefficients of each logical primitive, the criticality factor of each logical primitive is obtained.

5. The automated question-answering method for an intelligent engine system based on a large language model as described in claim 4, characterized in that, The criticality factor of each logical primitive is obtained based on the first and second coefficients of each logical primitive, including: The criticality factor of each logical primitive is obtained by multiplying the first coefficient and the second coefficient of each logical primitive.

6. The automated question-answering method for an intelligent engine system based on a large language model as described in claim 1, characterized in that, The step of determining the parsing order of logical singularities based on the dependency graph includes: Extract a subgraph containing all logical singularities from the dependency graph; Calculate the in-degree of each logical singularity in the subgraph, and add logical singularities with an in-degree of zero as the current nodes to be parsed to the parsing sequence; Remove the currently unparsed node from the subgraph, update the in-degree of the remaining logical singularities in the subgraph, and repeat the process of adding newly generated logical singularities with an in-degree of zero to the parsing sequence until all logical singularities have been added to the parsing sequence, thus obtaining the parsing order of the logical singularities.

7. The automated question-answering method for an intelligent engine system based on a large language model as described in claim 6, characterized in that, The process of parsing logical singularities according to the parsing order to obtain the parsing results includes: Based on the data source of the logical singularity, the logical singularity is parsed in the parsing order. If the data source of the logical singularity is a structured database, the parsing result is obtained through structured query. If the data source of the logical singularity is a rule base, the parsing result of the premise logical singularity is substituted into the preset logical expression of the business rule to obtain the parsing result. If the data source of the logical singularity is a process base, the corresponding step description text is retrieved to obtain the parsing result.

8. The automated question-answering method for an intelligent engine system based on a large language model as described in claim 1, characterized in that, The step of generating an answer to the question text based on the parsing results includes: If the question text matches the preset question template, the parsing result will be filled into the corresponding placeholder in the question template to generate an answer for the question text; If the question text does not match the preset question template, the parsing result is concatenated with the question text to form a structured prompt, which is then input into the large language model to generate an answer to the question text.

Images