Method, device and medium for dynamically constructing a rule base based on a large language model

Abstract

The present application discloses a method, device, and medium for dynamically constructing a rule base based on a large language model, which relates to the field of rule base construction. The method includes: based on a pre-set hierarchical template structure, injecting meta-knowledge through a domain knowledge graph corresponding to standard data to generate dynamic prompt words; outputting corresponding semantic triples through a large language model and performing semantic enhancement on the semantic triples; performing symbolic processing and vectorization processing to obtain rule vectors and generate corresponding specified rules; locally dynamically updating a rule subset in the rule base through data update of an external specified knowledge base, and dynamically updating the rule weights of the specified rules determined by confidence. The deep cooperation between the large language model and the symbol system introduces a logical verification layer through local dynamic update to formally constrain the generation results of the large language model, ensuring the completeness of rule logic while retaining the generation ability.

Description

Technical Field

[0001] This application relates to the field of rule base construction, and specifically to a method, device, and medium for dynamically constructing a rule base based on a large language model. Background Art

[0002] Currently, the rule base construction technology faces multiple technical bottlenecks in the general field.

[0003] The mainstream solutions in traditional schemes can be divided into two categories: manually driven and data driven. Manually driven technologies rely on domain experts to manually write rule logic. Although they can ensure the accuracy of rules, the average time-consuming for a single manually written rule even exceeds 30 minutes. They have the defects of high labor costs, long iteration cycles, and it is difficult to cover the implicit logic in complex scenarios. Data driven technologies automatically extract rules from labeled data through machine learning methods such as decision trees and association rule mining. However, the rules generated by them often lack semantic coherence, and there may be a phenomenon of "rule semantic discontinuity". Especially when dealing with unstructured texts (such as contract terms and technical documents), the practicality of rules decreases due to the difficulty in deeply analyzing context associations. Summary of the Invention

[0004] To solve the above problems, this application proposes a method for dynamically constructing a rule base based on a large language model, including:

[0005] Receiving heterogeneous data input from multiple sources and performing feature normalization to obtain standard data;

[0006] Based on a pre-set hierarchical template structure, injecting meta-knowledge through the domain knowledge graph corresponding to the standard data to generate dynamic prompt words;

[0007] Using the large language model, taking the dynamic prompt words as input, outputting corresponding semantic triples, performing semantic enhancement on the semantic triples, and calculating the confidence level for the semantic triples;

[0008] Performing symbolic processing and vectorization processing on the semantic triples to obtain rule vectors, and generating corresponding specified rules according to the rule vectors and the corresponding confidence levels, and adding them to the rule base;

[0009] Based on the triggering of a preset event, through the data update of an external specified knowledge base, locally dynamically updating the rule subset in the rule base, and dynamically updating the rule weights of the specified rules determined by the confidence level according to the time decay factor and the user feedback factor.

[0010] In one example, based on a pre-set hierarchical template structure, injecting meta-knowledge through the domain knowledge graph corresponding to the standard data to generate dynamic prompt words, specifically including:

[0011] Determine a pre-set hierarchical template structure; wherein, the hierarchical template structure includes an atomic rule layer, a composite rule layer, and a constraint condition layer;

[0012] Through the atomic rule layer, set the field units included in the prompt word; and through the composite rule layer, set the logical relationship between the field units; and through the constraint condition layer, add constraint conditions on the basis of the field units to generate a prompt word template.

[0013] Through the domain knowledge graph corresponding to the standard data, obtain the metadata corresponding to the field unit, and supplement the content of the prompt word template through the metadata to obtain the corresponding dynamic prompt word.

[0014] In one example, semantic enhancement of the semantic triple and confidence calculation for the semantic triple specifically include:

[0015] For the semantic triple, uniformly map the entity information included therein;

[0016] Perform syntactic structure analysis on the standard data, extract the corresponding modification information, and perform semantic enhancement on the semantic triple based on the modification information.

[0017] Determine the corresponding conditional trigger probability, and calculate the confidence of the semantic triple based on the semantic similarity between the semantic triple and the corresponding domain knowledge graph, the structural integrity of the semantic triple, and the context coherence of the semantic triple.

[0018] In one example, perform symbolic processing and vectorization processing on the semantic triple to obtain a rule vector, specifically including:

[0019] Convert the semantic triple into an abstract syntax tree for symbolic processing;

[0020] Encode the abstract syntax tree through a graph neural network to obtain a rule vector;

[0021] Wherein, in the graph neural network, during the update process of the rule vector, perform mean pooling operation based on the embedding vectors of the child nodes in the previous layer iteration, and splice with the rule vector obtained in the previous layer to obtain the rule vector of the next layer.

[0022] In one example, perform local dynamic update on the rule subset in the rule base through data update of an external specified knowledge base, specifically including:

[0023] Determine the specified knowledge base corresponding to the multi-source input, monitor the specified knowledge base, and extract the updated event feature vectors;

[0024] Based on the event feature vectors, perform similarity matching with each rule vector in the rule base to obtain the corresponding first rule subset;

[0025] Obtain user feedback data, and determine the corresponding second rule subset based on the feedback satisfaction corresponding to the user feedback data, and / or the rule trigger times corresponding to the user feedback data;

[0026] Perform local dynamic updates on the specified rules included in the first rule subset and / or the second rule subset;

[0027] During the local dynamic update process, if the rule base is in an online state, compare and test the updated specified rule with the specified rule before the update to obtain the corresponding execution effect difference; if the rule base is in an offline state and the execution effect difference is higher than the preset minimum gain threshold, sort the specified rules according to the updated rule weights.

[0028] In one example, dynamically update the rule weights of the specified rules determined by the confidence level according to the time decay factor and the user feedback factor, specifically including:

[0029] Determine the initial rule weight corresponding to the specified rule according to the confidence level, and determine the specified domain corresponding to the specified rule;

[0030] Determine the first update coefficient according to the update frequency of the associated policy text in the specified domain, and determine the second update coefficient according to the user feedback data;

[0031] Determine the corresponding time decay coefficient according to the first update coefficient, the second update coefficient, and the domain adaptation weight coefficient corresponding to the specified domain, and determine the corresponding time decay factor according to the time decay coefficient and the current rule weight;

[0032] Determine the user feedback factor according to the user feedback gain coefficient and the user feedback data;

[0033] Determine the vector update factor according to the update result of the rule vector corresponding to the specified rule and the semantic similarity gain coefficient;

[0034] Dynamically update the rule weights according to the time decay factor, the user feedback factor, and the vector update factor.

[0035] In one example, the method further includes:

[0036] Extract the attention matrices of each layer of the large language model, and determine the attention heads corresponding to the specified rules;

[0037] Based on the attention heads, determine the influence scores corresponding to each tokenization unit in the standard data;

[0038] According to the influence scores, associate each node of the abstract syntax tree with the tokenization unit to obtain the importance weights corresponding to the nodes of the abstract syntax tree;

[0039] Obtain and display the rule traceability graph through rendering; wherein, the node size in the rule traceability graph is positively correlated with the importance weight, and the color depth of the edge reflects the logical dependence strength.

[0040] In one example, the method further includes:

[0041] Parse according to the abstract syntax tree to obtain a sequence of logical units;

[0042] Based on the sequence of logical units, generate corresponding explanatory texts according to the structured template;

[0043] Generate an adjustment description based on the local dynamic update.

[0044] On the other hand, the present application also proposes a device for dynamically constructing a rule base based on a large language model, including:

[0045] At least one processor; and,

[0046] A memory communicatively connected to the at least one processor; wherein,

[0047] The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the method for dynamically constructing a rule base based on a large language model as described in any of the above examples.

[0048] On the other hand, the present application also proposes a non-volatile computer storage medium storing computer-executable instructions, and the computer-executable instructions are set to: the method for dynamically constructing a rule base based on a large language model as described in any of the above examples.

[0049] The method for dynamically constructing a rule base based on a large language model proposed by the present application can bring the following beneficial effects:

[0050] Deep collaboration between the large language model and the symbolic system, introducing a logical verification layer through local dynamic update to formally constrain the generation results of the large language model, while retaining the generation ability and ensuring the completeness of the rule logic.

[0051] Through a dynamic closed-loop management mechanism, a real-time closed-loop of "generation - verification - feedback - optimization" is constructed, and the rule weights are dynamically adjusted according to the time decay factor and the user feedback factor.

[0052] Information interaction is carried out between each layer through a unified data bus. Among them, the control bus monitors external knowledge change events in real time (for example, policy and regulation updates). When a preset trigger condition is detected, the incremental update process of the rule base is automatically started. The service interface layer provides a rule version comparison function, supporting backtracking the historical rule status through timestamps to ensure that the system has the ability to quickly roll back in case of faults.

[0053] An interpretability enhancement design is set up, which combines attention weight visualization and rule dependency graphs to realize a traceable inference chain from the original text to the generated rules. Brief Description of the Drawings

[0054] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation to the present application. In the drawings:

[0055] Figure 1 is a schematic flowchart of the method for dynamically constructing a rule base based on a large language model in an embodiment of the present application;

[0056] Figure 2 is a schematic architecture diagram of a rule base dynamic construction system in a certain situation in an embodiment of the present application;

[0057] Figure 3 is a schematic diagram of the method for dynamically constructing a rule base based on a large language model in a certain situation in an embodiment of the present application;

[0058] Figure 4 is a schematic diagram of the device for dynamically constructing a rule base based on a large language model in an embodiment of the present application. Detailed Description of the Embodiments

[0059] To make the purpose, technical solutions, and advantages of the present application clearer, the technical solutions of the present application will be clearly and completely described below in conjunction with the specific embodiments of the present application and the corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present application.

[0060] The following will detail the technical solutions provided by each embodiment of the present application in conjunction with the drawings.

[0061] To address the above technical problems, rule generation can be based on reinforcement learning. However, its rule generation framework requires a pre-set reward mechanism and is difficult to adapt to the dynamic requirements of multi-objective complex scenarios. Rule verification can also be based on knowledge graphs. However, although this verification method can establish rule associations, its graph construction relies on manual maintenance and lacks real-time performance, making it difficult to meet the business environment with high-frequency updates.

[0062] Generally speaking, the core defects of traditional solutions mainly stem from the limitations of the underlying architecture.

[0063] Traditional rule generation methods highly rely on structured data input and manual feature engineering, making it difficult to capture the implicit logical relationships in natural language and resulting in insufficient rule generalization ability. For example, when policies and regulations are updated, a large amount of manpower is required to re-verify the logical consistency of the static rule library. Although manual writing and static templates can ensure logical controllability, they sacrifice response speed and generalization ability.

[0064] Solutions based on knowledge graphs and others attempt to establish rule associations through graph neural networks, but their graph construction still relies on manual annotation and has weak real-time update capabilities.

[0065] Although data-driven and deep learning solutions improve the degree of automation, they are limited by insufficient semantic understanding depth. This contradiction is more prominent in the digital environment with high-frequency policy iterations and fragmented business scenarios.

[0066] Traditional solutions are either limited by the rigidity of pre-set conditions or troubled by overloaded resource consumption, and it is difficult to achieve full-process automation of rule generation, verification, and optimization. It is also difficult to balance the interpretability and dynamic adaptability of rules, which has become the key obstacle restricting the implementation of intelligent rule library technology.

[0067] As Figure 1 shown, the embodiments of this application provide a method for dynamically constructing a rule library based on a large language model, which is applied in a rule library dynamic construction system (hereinafter referred to as the system for short).

[0068] As Figure 2 shown, the system includes a data perception layer, a semantic parsing layer, a dynamic update layer, and a service interface layer, forming a full-process closed loop from multi-source data input to rule service output. The system adopts a hierarchical decoupling design and realizes the full-life cycle dynamic management of rule generation, verification, optimization, and deployment through the deep cooperation of the semantic understanding ability of the large language model (LLM) and the symbolic logic verification mechanism.

[0069] At the system architecture level, the data perception layer includes a heterogeneous data access module and a feature normalization module, supporting the unified structured processing of multi-modal inputs such as policy texts, user behavior logs, and business work orders. The semantic parsing layer deploys a context-aware rule extraction engine, adopting a multi-level attention mechanism to enhance the large language model's ability to capture implicit logic, and converting unstructured text into weighted semantic triples (Subject-Predicate-Object, SPO). The dynamic update layer uses an incremental learning framework and designs a dynamic adjustment algorithm for rule weights based on a time decay factor and a feedback gain coefficient. The service interface layer realizes microservices deployment through a rule API gateway and provides a visual interpretation platform for the rule traceability graph, supporting the reverse traceability of the decision-making path.

[0070] As Figure 1 shown, the method includes:

[0071] S101: Receive heterogeneous data with multi-source inputs and perform feature normalization to obtain standard data.

[0072] The heterogeneous data access module is used to receive heterogeneous data with multi-source inputs. The sources of multi-source inputs can include different websites, databases, articles, manuals, etc. The data obtained from different sources may have structural differences, pattern incompatibilities, encoding representation differences, file format differences, etc., which are called heterogeneous data.

[0073] For heterogeneous data, it is difficult to process directly. At this time, feature normalization is performed through the feature normalization module. Among them, feature normalization can include basic data format normalization, numerical feature normalization, categorical feature normalization, etc., which are respectively used to convert the same feature from different sources into a consistent data type (for example, converting a numerical value in string format to a floating point type, converting "yes / no" to a boolean type, etc.), scale the data to a fixed range or a fixed value, and select a suitable feature encoding to encode the data (for example, unique encoding, label encoding, target encoding, etc.).

[0074] S102: Based on a pre-set hierarchical template structure, perform meta-knowledge injection through the domain knowledge graph corresponding to the standard data to generate dynamic prompt words.

[0075] As Figure 3 shown, the rule generation process mainly includes the following several stages:

[0076] In the rule extraction stage and the rule generation stage, it is mainly used to drive the extraction of atomic rules and composite rules from the original text of the large language model based on a dynamically generated domain adaptation Prompt template, and generate a candidate rule set with confidence scores.

[0077] In the rule verification phase and the dynamic optimization phase, the recalculation of rule weights is triggered according to the real-time feedback data stream (such as user behavior and external knowledge updates), and a multi-objective optimization algorithm with constraint conditions is used to update the rule base.

[0078] In the service deployment phase, the verified rule base is converted into executable code, and a low-latency inference service is provided externally through the API gateway.

[0079] In the rule extraction phase and the rule generation phase, based on the semantic parsing ability of the large language model, the semantic parsing layer adopts multi-level dynamic prompt engineering and structured representation technology to realize the automatic conversion from unstructured text to executable rules. Through the dynamic generation mechanism of domain-adapted multi-level prompt templates, the large language model is driven to extract semantic triples (Subject-Predicate-Object, SPO) from the input text.

[0080] Determine the preset hierarchical template structure; among them, the hierarchical template structure includes an atomic rule layer, a composite rule layer, and a constraint condition layer. Among them, the three layers included in the hierarchical template structure are fixed, however, the content included in each layer may be different in different scenarios and domains, thus obtaining a dynamic prompt template. The dynamic prompt template is improved according to the meta-knowledge of the knowledge graph to obtain the final dynamic prompt.

[0081] In the hierarchical template structure, there are at least three levels of prompt hierarchies. The first level is the atomic rule layer, which is used to extract atomic rules. Among them, the atomic rule is the smallest description unit in the prompt, which usually refers to the field unit included in the prompt. The second level is the composite rule layer, which expresses the logical relationship between each atomic rule layer. The third level is the constraint condition layer, which is used to add corresponding constraint conditions additionally. For example, domain constraint conditions are added to describe the domain to which the currently obtained standard data belongs, and corresponding constraint conditions can also be added to the existing atomic rule layer and composite rule layer.

[0082] Based on this, through the atomic rule layer, the field units included in the prompt are set; through the composite rule layer, the logical relationship between each field unit is set; and through the constraint condition layer, constraint conditions are added on the basis of the field units to generate a prompt template.

[0083] For example, in the legal field, the field units set in the atomic rule layer may include "subject type", "trigger condition", "legal consequence", etc. The composite rule layer represents the logical relationships among these field units. In the legal field, this logical relationship is usually a same-level logical relationship, while in other fields, the logical relationships may include dependency relationships, inclusion and being included relationships, etc. In the constraint condition layer, domain constraint conditions can be added through role definitions, and other constraint conditions can be added based on requirements (such as word count limits, context information recognition limits, etc.).

[0084] Through the domain knowledge graph corresponding to the standard data, obtain the metadata corresponding to the field unit, and through the metadata, supplement the content of the prompt template to obtain the corresponding dynamic prompt.

[0085] The dynamic generation of the prompt template depends on the injection of meta-knowledge of the domain knowledge graph. Extract the core concepts of the domain through a graph neural network (such as "constitutive elements", "exemption circumstances", etc. in legal provisions), and embed them into the context of the prompt (such as embedding as a constraint condition in the constraint condition layer, or improving the field unit) to enhance the ability of the large language model to capture implicit logical relationships.

[0086] Taking the legal field as an example, the hierarchical template structure can be:

[0087] [Role Definition] As a legal expert, extract the rights and obligations relationships from the following clauses:

[0088] [Input Text] {Legal provisions to be analyzed}

[0089] [Output Format] Subject - Predicate - Object:

[0090] - Subject Type: <Legal Person / Natural Person>

[0091] - Trigger Condition: <Time / Event Constraint>

[0092] - Legal Consequence: <Description of Behavioral Consequence>

[0093] S103: Through the large language model, take the dynamic prompt as the input, output the corresponding semantic triple, perform semantic enhancement on the semantic triple, and calculate the confidence level for the semantic triple.

[0094] During the rule verification phase, the semantic parsing layer receives dynamic prompt words generated based on the current domain and calls the large language model to output corresponding semantic triples. For example, when the input text is "If the lessee fails to pay the rent without justifiable reasons for a cumulative period of 30 days, the lessor may terminate the lease contract and claim damages.", the output semantic triples can be: Subject: lessee, Predicate: fails to pay the rent, Object: lessor.

[0095] This semantic triple is an intermediate representation form of the rule. However, directly using this triple to generate the rule may result in inaccurate rule descriptions due to the relatively weak semantic content contained in the triple.

[0096] Based on this, semantic enhancement processing is performed on the original semantic triple SPO output by the large language model.

[0097] For the semantic triple, the entity information contained in it is uniformly mapped. For example, based on the joint model of BERT - BiLSTM - CRF for entity disambiguation and linking, entities with different expressions (such as "Party A" and "lessee") are uniformly mapped to the canonical nodes in the knowledge graph.

[0098] For the standard data, syntactic structure analysis is performed to extract the corresponding modification information, and based on the modification information, semantic enhancement of the semantic triple is carried out. For example, through dependency syntactic analysis, modification information such as conditional adverbials and time constraints is identified, thereby performing semantic enhancement on the semantic triple and adding corresponding time constraints, conditional constraints, etc.

[0099] According to the time constraint in the modification information, the corresponding rule effective time interval is determined. Within this time constraint, according to the conditional constraint of the conditional adverbial, the trigger times within the rule effective time interval are determined, and the conditional trigger probability is obtained by combining domain knowledge.

[0100] Meanwhile, based on the semantic similarity between the semantic triple and the corresponding domain knowledge graph, the structural integrity of the semantic triple, and the context coherence of the semantic triple, confidence calculation is performed on the semantic triple.

[0101] At this time, according to the conditional trigger probability and the obtained confidence, they can be used as supplements after semantic enhancement of the semantic triple and added to the semantic enhancement process together for subsequent rule generation. Its expression can be as shown in Formula 1:

[0102] Formula 1;

[0103] Among them, is the original semantic triple, is the semantic triple after semantic enhancement, They are respectively the start point and the end point of the rule effective time, used to represent the rule effective time interval. is the condition trigger probability. is the confidence level.

[0104] The calculation expression of the confidence level can be as shown in Formula 2:

[0105] Formula 2;

[0106] Among them, is the semantic similarity between the semantic triple and the domain knowledge base (which can be calculated based on the cosine similarity of Sentence-BERT). is the structural integrity of the semantic triple (verified according to the preset grammar rules for the field completeness). is the context coherence of the semantic triple (determined by quantifying the logical consistency through the entropy value of the self-attention weights of the large language model). is a configurable weight coefficient, and the sum of them is 1.

[0107] In the semantic triple after such semantic enhancement, it not only contains the effective content in the conditional adverbial, time constraint and other modification information, but also contains the corresponding condition trigger probability and confidence level, so as to facilitate the subsequent implementation of functions such as priority sorting and iteration of the rule according to the condition trigger probability and confidence level after generating the rule.

[0108] S104: Perform symbolic processing and vectorization processing on the semantic triple to obtain a rule vector, and generate a corresponding specified rule according to the rule vector and the corresponding confidence level, and add it to the rule library.

[0109] The semantic triple obtained at this time can actually be used as a new rule. However, for the convenience of storing and referencing this rule, it can be subjected to symbolic processing and vectorization processing, and the confidence level obtained above is added to the obtained rule vector to improve this rule. For the convenience of description here, the obtained new rule is called a specified rule. After generating the specified rule, it can be added to the rule library to realize the construction of the rule library.

[0110] Specifically, at the symbolic level, the semantic triple is converted into an abstract syntax tree for symbolic processing.

[0111] Define ANTLR grammar rules to convert into an abstract syntax tree (AST). For example, the original standard data is "If the lessee fails to pay the rent without justifiable reasons for a cumulative period of 30 days, the lessor may terminate the lease contract and claim damages.", and the obtained For "If the lessee fails to pay the rent overdue for more than 30 days, the lessor has the right to terminate the contract", through the corresponding symbolic processing, the parsed AST can be: IF (the lessee overdue pays the rent) WITH Duration>30d THEN (the lessor executes to terminate the contract).

[0112] At the vectorization level, through the graph neural network, the abstract syntax tree is encoded to obtain the rule vector. Among them, the GraphSAGE graph neural network can be used to encode the AST to generate the rule vector .

[0113] Among them, in the graph neural network, in order to gradually capture and fuse the multi-level dependency relationships and complex semantic associations between rules, the rule vector usually needs to be generated iteratively. In the update process of the rule vector, based on the embedding vectors of the child nodes in the previous layer iteration, a mean pooling operation is performed, and concatenated with the rule vector obtained in the previous layer to obtain the rule vector of the next layer. Its update formula can be as shown in Formula 3:

[0114] Formula 3;

[0115] Among them, is the embedding vector of the child nodes in the k-th layer iteration, the function adopts the mean pooling operation, is the vector concatenation, is the trainable parameter matrix, the rule vector of the k-th (corresponding to the previous layer in the above text) layer, is the rule vector of the k+1-th layer (corresponding to the next layer in the above text).

[0116] The specified rule obtained at this time can be stored through the dual engines of the Neo4j graph database and the relational database. The graph database stores the logical dependency relationships between rules (for example, the causal relationship between the "terminate the contract" rule and the "rent payment" rule), and the relational database records the metadata of the specified rule (including confidence , effective time , version number, etc.). The two are associated and queried and jointly inferred through the unique rule ID.

[0117] Through this technical collaboration, the system solves the semantic fault problem in traditional rule generation. For example, when parsing the clause "If the deposit is not paid within 10 working days after the contract is signed, it is regarded as automatically waived", the system not only extracts the triple (contract signer, not paid, deposit), but also attaches the time constraint ( ) and the action modifier (automatically waived), and finally generates an executable rule with a complete logical chain. And, the confidence As the initial value of the rule weight is recorded in the database and passed to the dynamic update layer to form an internal consistency closed loop of the technical solution.

[0118] S105: Based on the triggering of a preset event, through the data update of an external specified knowledge base, locally dynamically update the rule subset in the rule base, and dynamically update the rule weight of the specified rule determined by the confidence according to the time decay factor and the user feedback factor.

[0119] In the dynamic optimization stage, during the use of the rule base after it is constructed, each specified rule in the rule base can also be dynamically updated based on the actual situation. The dynamic update module realizes the continuous optimization and adaptive adjustment of the rule base based on the incremental learning framework and the real-time feedback mechanism.

[0120] The preset event can be set based on requirements. For example, the preset event includes that the content of the monitored external knowledge base is updated, the satisfaction of the user feedback data does not meet the expectation, the trigger rate of the rules in the rule base is abnormal, etc. At this time, the dynamic update of the rule base can be triggered.

[0121] During the dynamic update process, it can include two levels of updates, namely, updating the rule weight of the added specified rule and updating the content of the specified rule.

[0122] During the update process of the content of the specified rule, the incremental learning process is triggered through dual-channel event listening.

[0123] Determine the specified knowledge base corresponding to the multi-source input, monitor the specified knowledge base, and extract the updated event feature vector. The specified knowledge base is determined in advance. According to the different specified domains to which the specified rules belong, the corresponding authoritative specified knowledge bases are screened as the monitoring objects. For example, it can be the specified knowledge base corresponding to policies and regulations, industry standards. When the specified knowledge base is updated, the corresponding updated event feature vector is extracted .

[0124] Based on the event feature vector, perform similarity matching with each rule vector in the rule base to obtain the corresponding first rule subset. Expand the similarity matching between the event feature vector and the rule base, and screen out the affected rule subset as the first rule subset. For example, select the specified rules with a similarity higher than a certain value but not 100% similar to form the first rule subset.

[0125] Meanwhile, obtain user feedback data, and determine the corresponding second rule subset based on the feedback satisfaction corresponding to the user feedback data and / or the number of rule trigger times corresponding to the user feedback data. The user feedback data can be obtained by analyzing the user interaction log, determining the trigger times of each rule by obtaining the retrieval records of the rules in the rule library by the user, and corresponding user satisfaction questionnaires can also be set up to obtain the user's satisfaction with the use of the rule library.

[0126] When the number of times a certain rule is triggered within a time window exceeds the adaptive threshold, or the user satisfaction score is continuously lower than the threshold for a certain number of times, add this rule to the second rule subset. Among them, the adaptive threshold can be as shown in Formula Four:

[0127] Formula Four;

[0128] Among them, is the adaptive threshold, is the historical average trigger times, is a preset constant value.

[0129] At this time, the specified rules in the first rule subset and the second rule subset respectively represent the rules for updating the original content, or the rules that are frequently triggered recently and the users are not satisfied with. Therefore, local dynamic updates are performed on the specified rules included in the first rule subset and / or the second rule subset.

[0130] During the local dynamic update process, incremental learning adopts an online-offline hybrid mode to ensure the continuity of the service. Among them, the online state and offline state of the rule library indicate whether the rule library is being used.

[0131] When the rule library is in the online state, compare and test the updated specified rule with the specified rule before the update to obtain the corresponding difference in execution effect. For example, in the online state, put the rule set to be updated (corresponding to the union of the first rule subset and the second rule subset) into the shadow rule library, and compare the execution effect differences between the new and old rule sets through A / B testing. And the difference in execution effect can be as shown in Formula Five:

[0132] Formula Five;

[0133] Among them, is the difference in execution effect, is the rule effect index, which can be selected and set manually. For example, it can include the customer service problem-solving rate, rule trigger frequency, etc. 、 respectively represent the values of the rule effect indicators corresponding to the k-th specified rule after and before the update.

[0134] The rule base is in an offline state, and the execution effect difference is higher than the preset minimum gain threshold, that is ( is the preset minimum gain threshold), it is considered that this update is effective, and the specified rules are sorted according to the updated rule weights. The higher the updated rule weight, the higher its priority, and the more forward the sorting. The calculation process of the rule weights can use a multi-objective optimization algorithm with constraints to solve the optimal weight combination. Among them, the constraint conditions can include rule effect indicators, and the objectives of the multi-objective optimization algorithm can include the optimal solution achieved by each indicator in the rule effect indicators.

[0135] After this local update, the rule base is hot-deployed to the production environment through the version control gateway, and the old version is retained in the rollback buffer area.

[0136] While updating the content included in the specified rule, the update of the rule weight of the specified rule can also be executed.

[0137] During the update process of the rule weights, through the dynamic coupling calculation of the time decay factor and the user feedback factor, combined with the rule weight adjustment algorithm, it is ensured that the rule base evolves accurately with the change of the external environment.

[0138] In the above text, when generating the specified rule, its corresponding confidence level has been obtained. At this time, according to the confidence level, the initial rule weight corresponding to the specified rule is determined, and the specified domain corresponding to the specified rule is determined. Here, the obtained confidence level can be directly used as the initial rule weight. Among them, the rule weight represents the reliability and importance of the specified rule. The higher the rule weight, the more reliable and important the rule.

[0139] According to the update frequency of the associated policy text in the specified domain, the first update coefficient is determined, and the second update coefficient is determined according to the user feedback data. The associated policy text in the specified domain can be determined through a specific knowledge base in this domain, and this knowledge base can be pre-marked manually. And the user feedback data can be obtained by analyzing the user interaction logs.

[0140] Generally speaking, for the convenience of description, it can be set that the update frequency of the associated policy text is positively correlated with the first update coefficient, and the user satisfaction in the user feedback data is positively correlated with the second update coefficient.

[0141] Determine the corresponding timeliness attenuation coefficient according to the first update coefficient, the second update coefficient, and the domain adaptation weight coefficient corresponding to the specified domain. The domain adaptation weight coefficient is set based on different domains and is used to describe the weights of the first update coefficient and the second update coefficient. The sum of the domain adaptation weight coefficients corresponding to the first update coefficient and the second update coefficient is 1.

[0142] The formula for the timeliness attenuation coefficient can be as shown in Formula 6:

[0143] Formula 6;

[0144] Where, is the timeliness attenuation coefficient of the i-th specified rule, is the domain adaptation weight coefficient, is the first update coefficient, is the second update coefficient, represents the update frequency of the associated policy text, is the length of the monitoring time window, is the variance of the user feedback data (covering the number of rule triggers etc.) of the i-th specified rule, is the maximum value of the variances of the user feedback data of all specified rules.

[0145] The timeliness attenuation coefficient is used to describe the attenuation speed of the rule weight of each rule as time increases. At this time, according to the timeliness attenuation coefficient and the current rule weight, determine the corresponding timeliness attenuation factor.

[0146] At the same time, the user feedback factor can also be determined according to the user feedback gain coefficient and the user feedback data. The user feedback gain coefficient is preset and is positively correlated with the effective trigger rate of the rules in the user behavior data stream, and is used to represent the correlation degree between the update speed of the rule weight and the user feedback data.

[0147] Determine the vector update factor according to the update result of the rule vector corresponding to the specified rule and the semantic similarity gain coefficient. The update result of the rule vector represents the semantic similarity degree between the current specified rule after update and before update. The semantic similarity gain coefficient is preset and is used to represent the correlation degree between the update speed of the rule weight and the semantic similarity degree.

[0148] Based on this, the timeliness attenuation factor, the user feedback factor, and the vector update factor have been obtained. According to these three factors, the dynamic update of the rule weight can be realized, and it can be as shown in Formula 7:

[0149] Formula 7;

[0150] Where, , are the rule weights of the i-th specified rule after and before the update, is the time decay factor, is the user feedback factor, is the vector update factor, is the time decay coefficient, is the time difference between before and after the weight rule update, is the user feedback gain coefficient, is the number of times the i-th specified rule has been triggered before the update, is the maximum number of times all specified rules have been triggered, is the semantic similarity gain coefficient, The function is based on the rule vector and the real-time updated text vector (corresponding to the updated rule vector) for cosine similarity calculation.

[0151] The dynamic update layer and the semantic parsing layer form a closed-loop feedback. For example, when the user behavior log shows that the trigger rate of a certain risk control rule has increased abnormally, the system will automatically increase and increase its weight , and at the same time trigger the semantic street west village to reparse the associated policy text to verify the effectiveness of the rule. The rule vector is used for similarity matching and semantic gain calculation during this process, achieving the joint optimization of data-driven and semantic-driven.

[0152] The deep collaboration between the large language model and the symbol system introduces a logical verification layer through local dynamic updates to formally constrain the generation results of the large language model, ensuring the completeness of the rule logic while retaining the generation ability.

[0153] Through the dynamic closed-loop management mechanism, a real-time closed-loop of "generation - verification - feedback - optimization" is constructed, and the rule weights are dynamically adjusted according to the time decay factor and the user feedback factor.

[0154] Information is exchanged between each layer through a unified data bus. Among them, the control bus monitors external knowledge change events in real time (such as policy and regulation updates). When a preset trigger condition is detected, the incremental update process of the rule library is automatically started. The service interface layer provides a rule version comparison function, supporting the backtracking of historical rule states through timestamps to ensure that the system has the ability to quickly roll back in case of faults.

[0155] An interpretability enhancement design is set up, combined with attention weight visualization and rule dependency graphs, to realize a traceable inference chain from the original text to the generated rules.

[0156] In one embodiment, such as Figure 2As shown, in the service deployment stage, in the service interface layer, a rules API gateway and a rules traceability graph (also known as an interactive traceability graph) can also be set up. Among them, the rules API gateway is set up for the API gateway of the specified rules, including functions such as import, export, routing, invocation, and coordination of the specified rules, and corresponding API gateways are set up respectively. The rules traceability graph is a visual traceability graph for the specified rules, realizing the visual traceability of the rules decision-making process and constructing a multi-granularity interpretation path from the original text to the generated rules.

[0157] In the construction of the rules generation traceability graph, the attention matrices of each layer of the large language model are extracted, and the attention heads corresponding to the specified rules are determined. Among them, the attention heads corresponding to the specified rules can include the attention heads corresponding to logical connectives and conditional adverbs.

[0158] Based on the attention heads, the influence scores corresponding to each token unit in the standard data are determined, that is, the influence scores of each token unit (token) in the input text on the rules generation are calculated, which can be shown as Formula VIII:

[0159] Formula VIII;

[0160] Among them, is the influence score corresponding to the j-th token unit in the standard data, is the weight of the j-th token unit in the k-th attention head of the l-th layer, is the pre-defined set of rule-related attention heads, is the k-th attention head, n is the total number of attention heads, and L is the total number of layers of the large language model.

[0161] Based on this, according to the influence scores, the nodes of the abstract syntax tree are associated with the token units, and the importance weights corresponding to the nodes of the abstract syntax tree are obtained, which can be shown as Formula IX:

[0162] Formula IX;

[0163] Among them, is the importance weight of the m-th node of the abstract syntax tree, is all the token units associated with the m-th node, is the current j-th token unit, is the context weight of the j-th token unit, is the term frequency-inverse document frequency value of the j-th token unit.

[0164] The rule traceability graph is obtained through rendering and displayed. For example, an interactive rule traceability graph is rendered through a force-directed graph layout algorithm. Among them, the size of the nodes in the rule traceability graph is positively correlated with the importance weight, and the color depth of the edges reflects the strength of logical dependence, intuitively showing the key logical path of rule generation.

[0165] Meanwhile, the generation of natural language explanations corresponding to specified rules can also be generated. The system parses according to the abstract syntax tree (the rule AST corresponding to the specified rule) to obtain a sequence of logical units. Based on the sequence of logical units, the corresponding explanatory text is generated according to the structured template. For example, for the rule "terminate the contract after 30 days overdue", the explanation is generated: IF the lessee fails to pay the rent overdue for more than 30 days THEN the lessor has the right to terminate the contract BECAUSE Article 5.2 of the contract terms and the historical case

[2023] Min Zhong Zi No. X. Among them, No. X here is an exemplary description, which is determined based on actual documents in actual work. The semantic faithfulness of the explanation is verified by the cosine similarity between the rule vector and the explanation text vector to ensure that the explanation content is consistent with the original rule logic.

[0166] Moreover, based on local dynamic updates, adjustment instructions are generated. In the dynamic update scenario, the explanatory content is updated synchronously: when the rule weight changes exceed the threshold, adjustment instructions are generated (for example, "the weight decays due to policy updates"). The service interface layer and the semantic parsing layer cooperate deeply, and the attention weights in the traceability graph are fed back to optimize the Prompt template, strengthening the guidance of keyword fields, forming a closed-loop interpretability enhancement system.

[0167] As Figure 4 shown, the embodiment of the present application also provides a device for dynamically constructing a rule base based on a large language model, including:

[0168] At least one processor; and,

[0169] A memory communicatively connected to the at least one processor; wherein,

[0170] The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the method for dynamically constructing a rule base based on a large language model as described in any one of the above embodiments.

[0171] The embodiment of the present application also provides a non-volatile computer storage medium storing computer-executable instructions, and the computer-executable instructions are set to: the method for dynamically constructing a rule base based on a large language model as described in any one of the above embodiments.

[0172] The various embodiments in this application are described in a progressive manner. For the same or similar parts among the embodiments, reference can be made to each other. Each embodiment focuses on the differences from other embodiments. In particular, for the device and medium embodiments, since they are basically similar to the method embodiments, the description is relatively simple. For the relevant parts, reference can be made to the corresponding descriptions in the method embodiments.

[0173] The devices and media provided in the embodiments of this application correspond one-to-one with the methods. Therefore, the devices and media also have beneficial technical effects similar to those of their corresponding methods. Since the beneficial technical effects of the methods have been described in detail above, the beneficial technical effects of the devices and media will not be elaborated here.

[0174] The above description is only for the embodiments of this application and is not intended to limit this application. For those skilled in the art, various modifications and changes can be made to this application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of this application shall be included within the scope of the claims of this application.

Claims

1. A method for dynamically constructing a rule base based on a large language model, characterized in that The method includes: Receiving heterogeneous data from multiple sources and performing feature normalization to obtain standard data; Based on a pre-set hierarchical template structure, injecting meta-knowledge through the domain knowledge graph corresponding to the standard data to generate dynamic prompt words; Using a large language model, taking the dynamic prompt words as input, outputting corresponding semantic triples, performing semantic enhancement on the semantic triples, and calculating the confidence of the semantic triples; Performing symbolic processing and vectorization processing on the semantic triples to obtain rule vectors, and generating corresponding specified rules according to the rule vectors and the corresponding confidence, and adding them to the rule base; Based on the triggering of a pre-set event, locally dynamically updating the rule subset in the rule base by updating the data of an external specified knowledge base; And dynamically updating the rule weights of the specified rules determined by the confidence according to the time decay factor and the user feedback factor, specifically including: determining the initial rule weight corresponding to the specified rule according to the confidence, and determining the specified domain corresponding to the specified rule; determining the first update coefficient according to the update frequency of the associated policy text in the specified domain, and determining the second update coefficient according to the user feedback data; determining the corresponding time decay coefficient according to the first update coefficient, the second update coefficient, and the domain adaptation weight coefficient corresponding to the specified domain, and determining the corresponding time decay factor according to the time decay coefficient and the current rule weight; determining the user feedback factor according to the user feedback gain coefficient and the user feedback data; determining the vector update factor according to the update result of the rule vector corresponding to the specified rule and the semantic similarity gain coefficient; dynamically updating the rule weights according to the time decay factor, the user feedback factor, and the vector update factor.

2. The method for dynamically constructing a rule base based on a large language model according to claim 1, wherein Based on a pre-set hierarchical template structure, injecting meta-knowledge through the domain knowledge graph corresponding to the standard data to generate dynamic prompt words, specifically including: Determining the pre-set hierarchical template structure; wherein, the hierarchical template structure includes an atomic rule layer, a composite rule layer, and a constraint condition layer; Setting the field units included in the prompt words through the atomic rule layer; setting the logical relationship between the field units through the composite rule layer; and adding constraint conditions on the basis of the field units through the constraint condition layer to generate a prompt word template; Obtaining the metadata corresponding to the field units through the domain knowledge graph corresponding to the standard data, and supplementing the content of the prompt word template through the metadata to obtain the corresponding dynamic prompt words.

3. The method for dynamically constructing a rule base based on a large language model according to claim 1, wherein Performing semantic enhancement on the semantic triples and calculating the confidence of the semantic triples, specifically including: For the semantic triples, uniformly mapping the entity information contained therein; Performing syntactic structure analysis on the standard data, extracting the corresponding modification information, and performing semantic enhancement on the semantic triples based on the modification information; Determine the corresponding conditional trigger probability, and calculate the confidence of the semantic triple based on the semantic similarity between the semantic triple and the corresponding domain knowledge graph, the structural integrity of the semantic triple, and the context coherence of the semantic triple.

4. The method for dynamically constructing a rule base based on a large language model according to claim 1, wherein Perform symbolic processing and vectorization processing on the semantic triple to obtain a rule vector, specifically including: Convert the semantic triple into an abstract syntax tree and perform symbolic processing; Encode the abstract syntax tree through a graph neural network to obtain a rule vector; Among them, in the graph neural network, during the update process of the rule vector, a mean pooling operation is performed based on the embedding vectors of the child nodes in the previous layer iteration, and concatenated with the rule vector obtained in the previous layer to obtain the rule vector of the next layer.

5. The method for dynamically constructing a rule base based on a large language model according to claim 1, characterized in that Perform local dynamic update on the rule subset in the rule base through data update of an external specified knowledge base, specifically including: Determine the specified knowledge base corresponding to the multi-source input, monitor the specified knowledge base, and extract the updated event feature vector; Based on the event feature vector, perform similarity matching with each rule vector in the rule base to obtain the corresponding first rule subset; Obtain user feedback data, and determine the corresponding second rule subset based on the feedback satisfaction corresponding to the user feedback data and / or the rule trigger times corresponding to the user feedback data; Perform local dynamic update on the specified rules included in the first rule subset and / or the second rule subset; During the local dynamic update process, if the rule base is in an online state, compare and test the updated specified rule with the specified rule before update to obtain the corresponding execution effect difference; if the rule base is in an offline state and the execution effect difference is higher than the preset minimum gain threshold, sort the specified rules according to the updated rule weights.

6. The method for dynamically constructing a rule base based on a large language model according to claim 4, wherein The method further includes: Extract the attention matrices of each layer of the large language model and determine the attention heads corresponding to the specified rules; Based on the attention heads, determine the influence scores corresponding to each token unit in the standard data; According to the influence scores, associate each node of the abstract syntax tree with the token unit to obtain the importance weights corresponding to the nodes of the abstract syntax tree; Obtain and display a rule traceability graph through rendering; wherein, the node size in the rule traceability graph is positively correlated with the importance weight, and the color depth of the edge reflects the logical dependence strength.

7. The method for dynamically constructing a rule base based on a large language model according to claim 4, wherein The method further includes: Parse according to the abstract syntax tree to obtain a logical unit sequence; Based on the logical unit sequence, generate corresponding explanatory text according to a structured template; Generate an adjustment description based on the local dynamic update.

8. A rule base dynamic construction device based on a large language model, characterized in that, Including: At least one processor; And, A memory communicatively connected to the at least one processor; wherein, The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the method for dynamically constructing a rule base based on a large language model according to any one of claims 1 to 7.

9. A non-volatile computer storage medium stores computer-executable instructions, characterized in that, The computer-executable instructions are configured to implement the method for dynamically constructing a rule base based on a large language model according to any one of claims 1 to 7.

Images