Large model and dynamic knowledge base collaborative classification decision-making method, system and equipment

Abstract

The invention provides a large model and dynamic knowledge base collaborative classification decision-making method, system and device, and relates to the field of artificial intelligence and information processing, and the method comprises the steps: carrying out the feature extraction of to-be-classified data, and constructing a dynamic knowledge graph; dividing knowledge of different domains into structural knowledge and empirical knowledge to form a cross-domain rule; classifying the to-be-classified data by using a large language model to obtain the confidence of a classification result; matching the dynamic knowledge graph with a cross-domain rule, and calculating a rule matching degree; respectively dividing the confidence coefficient and the rule matching degree into a high level, a middle level and a low level; performing classification decision according to the levels of the confidence coefficient and the rule matching degree; when the confidence coefficient is a high level, adopting a big language model classification result; judging the rule matching degree when the confidence degree is a middle level or a low level; and triggering manual intervention when the confidence degree is a low level and no matching rule exists. According to the invention, the defect that the prior art cannot meet the requirements of high precision, low investment, strong controllability and cross-scene at the same time is overcome.

Description

Technical Field

[0001] This invention belongs to the field of artificial intelligence and information processing technology, and in particular relates to a collaborative classification decision-making method, system and device for large models and dynamic knowledge bases. Background Technology

[0002] As enterprises accelerate their digital transformation, the demand for real-time and accurate classification of unstructured text (such as transaction records in the financial sector, user reviews in e-commerce operations, and patient complaints in the medical field) is surging. Current technologies primarily rely on static rules or pure AI models for classification, but both approaches have drawbacks: Static rule solution: limited coverage, unable to recognize variant expressions (such as "yyds" referring to positive reviews in e-commerce operations), new categories require manual rule configuration, and maintenance costs are high.

[0003] Pure AI model solution: Poor domain transferability: Models trained for financial scenarios cannot be directly used for medical classification; High risk of black-box testing: Model illusions can lead to misjudgments, requiring manual review; High computing power costs: Small and medium-sized enterprises find it difficult to afford training / inference expenses.

[0004] In summary, the core contradiction of existing technologies is: The contradiction between efficiency and scale: linear human resource input cannot lead to scalable growth; The contradiction between precision and generalization: domain knowledge is difficult to transfer across scenarios; The contradiction between controllability and intelligence: intelligence and reliability cannot be achieved simultaneously. Summary of the Invention

[0005] Therefore, the technical problem to be solved by the present invention is to provide a collaborative classification decision-making method, system and device for large models and dynamic knowledge bases, which overcomes the inability of existing technologies to simultaneously meet the requirements of high accuracy, low investment, strong controllability and cross-scenario compatibility.

[0006] In a first aspect, the present invention provides a collaborative classification decision-making method using a large model and a dynamic knowledge base, comprising: Feature extraction steps: Extract features from the data to be classified; Steps for constructing a dynamic knowledge graph: Construct a dynamic knowledge graph based on the extracted features; Steps to construct cross-domain rules: Differentiate knowledge from different domains into structured knowledge and empirical knowledge to form cross-domain rules; Classification steps: Classify the data to be classified using a large language model and obtain the confidence score of the classification results; match the dynamic knowledge graph with cross-domain rules and calculate the rule matching degree; divide the confidence score and rule matching degree into high, medium and low levels by setting thresholds respectively. Fusion decision-making steps: Classify decisions based on the level of confidence and rule matching: When the confidence level is high, the classification result of the large language model is adopted; When the confidence level is medium or low, the rule matching degree is judged as follows: When the rule matching degree is high, the rule matching result is adopted; When the rule matching degree is medium, the classification results of the large language model and the rule matching results are fused. When the rule matching degree is low, the classification result of the large language model is adopted; When the confidence level is low and there is no matching rule, manual intervention is triggered.

[0007] Furthermore, in the feature extraction step, the data to be classified is converted into a standardized format through a domain adaptation interface, which connects to data sources from different domains.

[0008] Furthermore, the feature extraction includes identifying entity information and its attributes in the data to be classified, extracting the time sequence patterns, correlations and contextual information of events, and parsing the semantics of unstructured information.

[0009] Furthermore, in the classification step, the large language model uses a structured prompting engineering template to classify the data to be classified, constraining the output format and range.

[0010] Furthermore, the large language model is a general-purpose large model that is not trained and requires only lightweight fine-tuning.

[0011] Furthermore, in the step of constructing cross-domain rules, structural knowledge refers to deterministic logical rules; empirical knowledge refers to rules involving probability, empirical judgment, or fuzzy boundaries, which are modeled through probabilistic logical fitting.

[0012] Furthermore, in the fusion decision-making step, in scenarios with structured verification or strong rule constraints, deterministic rules are executed first.

[0013] Furthermore, the method also includes an output step, the output of which includes classification, confidence level, rule matching degree, and the decision process of the fusion decision step.

[0014] Secondly, this invention provides a collaborative classification decision-making system based on a large model and a dynamic knowledge base, comprising: Feature extraction module: Extracts features from the data to be classified; Dynamic knowledge graph construction module: Constructs dynamic knowledge graphs based on extracted features; Construct cross-domain rule modules: Distinguish between structural knowledge and empirical knowledge from different domains to form cross-domain rules; Classification module: Uses a large language model to classify the data to be classified and obtains the confidence of the classification results; matches the dynamic knowledge graph with cross-domain rules and calculates the rule matching degree; and divides the confidence and rule matching degree into high, medium and low levels by setting thresholds. Fusion Decision Module: Makes classification decisions based on the level of confidence and rule matching. When the confidence level is high, the classification result of the large language model is adopted; When the confidence level is medium or low, the rule matching degree is judged as follows: When the rule matching degree is high, the rule matching result is adopted; When the rule matching degree is medium, the classification results of the large language model and the rule matching results are fused. When the rule matching degree is low, the classification result of the large language model is adopted; When the confidence level is low and there is no matching rule, manual intervention is triggered.

[0015] Thirdly, the present invention provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the above-described method.

[0016] Beneficial effects: This invention classifies data to be classified using a large language model, obtains the confidence level of the classification results, matches dynamic knowledge graphs with cross-domain rules, calculates the rule matching degree, and then makes a fusion decision based on the level of confidence and rule matching degree. This breaks through the accuracy ceiling of rule-based solutions and AI solutions, and achieves high accuracy classification with consistent cross-domain results. It adopts a general-purpose large model that is not trained and is lightweight and fine-tuned, with zero training cost, eliminating the computational power and data dependence generated by model fine-tuning, and solving the resource bottleneck of small and medium-sized enterprises; In the fusion decision-making process, in scenarios with structured verification or strong rule constraints, deterministic rules are executed first, which overcomes the risk of random misjudgment by AI black box models and ensures the reliability of classification results in key scenarios. In addition, a domain-independent classification framework was established, enabling cross-scenario migration and reducing the secondary investment required for system reconstruction. Attached Figure Description

[0017] To make the content of this invention easier to understand, the invention will be further described in detail below with reference to specific embodiments and accompanying drawings.

[0018] Figure 1 This is a flowchart of the method in Embodiment 1 of the present invention; Figure 2 This is a schematic diagram of the architecture of Embodiment 2 of the present invention; Figure 3 This is a schematic diagram of the banking architecture of Embodiment 2 of the present invention; Figure 4 This is a schematic diagram of a computer device according to Embodiment 3 of the present invention. Detailed Implementation

[0019] The present invention will now be described in detail with reference to the accompanying drawings and embodiments. The principles and features of the present invention are described below with reference to the accompanying drawings. It should be noted that, unless otherwise specified, the embodiments and features described in these embodiments can be combined with each other. The embodiments given are only for explaining the present invention and are not intended to limit the scope of the present invention.

[0020] In existing technologies, classification is mainly performed using static rule-based schemes or AI model-based schemes, but both of these schemes have drawbacks: Static rule scheme: Coverage bottleneck: Static rules cannot cover all scenarios; Variant handling failure: Variant representations of the same concept cannot be accurately identified; Maintenance costs: Each new category tag requires manual configuration of rules or expressions, which is costly and inefficient.

[0021] AI model solutions: Domain transfer failure: A classifier trained on financial transaction data cannot be directly used for classifying medical complaints; Black-box risk: Occasional hallucinations may lead to fatal errors, requiring extensive manual review and are difficult to automate; Computing costs: Model training and inference are costly, making them unaffordable for small and medium-sized enterprises.

[0022] Example 1 Therefore, this embodiment provides a collaborative classification decision-making method using a large model and a dynamic knowledge base, such as... Figure 1 As shown, it includes: Feature extraction steps: Extract features from the data to be classified; Steps for constructing a dynamic knowledge graph: Construct a dynamic knowledge graph based on the extracted features; Steps to construct cross-domain rules: Differentiate knowledge from different domains into structured knowledge and empirical knowledge to form cross-domain rules; Classification steps: Classify the data to be classified using a large language model and obtain the confidence score of the classification results; match the dynamic knowledge graph with cross-domain rules and calculate the rule matching degree; divide the confidence score and rule matching degree into high, medium and low levels by setting thresholds respectively. Fusion decision-making steps: Classify decisions based on the level of confidence and rule matching: When the confidence level is high, the classification result of the large language model is adopted; When the confidence level is medium or low, the rule matching degree is judged as follows: When the rule matching degree is high, the rule matching result is adopted; When the rule matching degree is medium, the classification results of the large language model and the rule matching results are fused. When the rule matching degree is low, the classification result of the large language model is adopted; When the confidence level is low and there is no matching rule, manual intervention is triggered.

[0023] In some optional embodiments, during the feature extraction step, the data to be classified is converted into a standardized format through a domain adaptation interface, which connects to data sources from different domains.

[0024] In some optional embodiments, the feature extraction includes identifying entity information and its attributes in the data to be classified, extracting the time sequence patterns, correlations and contextual information of the events, and parsing the semantics of unstructured information.

[0025] In some optional embodiments, during the classification step, the large language model uses a structured prompting engineering template to classify the data to be classified, constraining the output format and range.

[0026] In some alternative embodiments, the large language model is a non-trained, lightweight, fine-tuned general-purpose large model.

[0027] In some optional embodiments, in the step of constructing cross-domain rules, the structural knowledge is deterministic logical rules; the empirical knowledge is rules involving probability, empirical judgment, or fuzzy boundaries, which are modeled by probabilistic logical fitting.

[0028] In some optional embodiments, in the fusion decision step, deterministic rules are executed first in scenarios with structured verification or strong rule constraints.

[0029] In some optional embodiments, the method further includes an output step, the output of which includes classification, confidence level, rule matching degree, and the decision process of the fusion decision step.

[0030] Example 2 This embodiment provides a collaborative classification decision-making system using a large model and a dynamic knowledge base. Through a collaborative decision-making mechanism combining untrained general-purpose large model control and a dynamic rule arbitration engine, it achieves a dual breakthrough in classification accuracy and controllability. The overall system adopts a layered design, with clear responsibilities for each layer, standardized interfaces, and support for cross-domain reuse, such as... Figure 2 As shown, the architecture of this embodiment is as follows: 1. Input layer: Receives heterogeneous input data from multiple sources.

[0031] Provide domain-specific adaptation interfaces to convert raw data into a standardized format required by downstream processing modules and adapt to data sources in specific domains (such as ERP systems, e-commerce platform APIs, medical HIS systems, etc.).

[0032] 2. Feature Processing Layer: Multi-source feature extraction is performed, and the core tasks include: Entity attribute extraction: Identify the core entities in the data (such as users, products, accounts, patients) and their key attributes (ID, name, type, status, etc.).

[0033] Temporal context extraction: Analyzing the time sequence patterns, correlations, and contextual information of events or transactions.

[0034] Original semantic parsing: understanding the key semantics of unstructured information such as text descriptions and summaries.

[0035] Based on the above characteristics, a dynamic knowledge graph is constructed to capture the dynamic patterns of relationships between entities and the development of events.

[0036] 3. Knowledge Construction Layer: Human experts are involved at the core, differentiating knowledge types: Structured knowledge: Clear and explicit business rules (e.g., expense reimbursements must be linked to a valid invoice number). These are then distilled into deterministic logical rules (if-then).

[0037] Empirical knowledge: Rules involving probability, empirical judgment, or fuzzy boundaries (e.g., users who frequently purchase high-end goods are more likely to accept recommendations for new products). Modeling is achieved through probabilistic logic fitting (e.g., Bayesian networks, lightweight machine learning models).

[0038] The structured knowledge is uniformly stored in a cross-domain rule base. This rule base supports management and retrieval by domain identifier.

[0039] 4. Decision Integration Layer: The decision fusion engine is the core driving module: Large Model Semantic Parsing: Invokes a non-trained / lightly tuned general large model (LLM). Guides the large model for preliminary classification or intent recognition through structured micro-cues engineering templates, strictly constraining its output format and range, and suppressing illusions.

[0040] Rule distillation execution: Based on the dynamic knowledge graph features of the current input, accurately call and execute the matching rules (structural and empirical) in the cross-domain rule base, and calculate the rule matching score.

[0041] Confidence Arbitrator: Based on the initial confidence level (high / medium / low) of the large model and the rule matching results, executes a dynamic decision-making strategy. High confidence output: Directly adopt the results of the large model.

[0042] Low to medium confidence output: Triggers rule matching arbitration. When the rule matching degree is high, the result of rule refinement / correction is adopted; when the matching degree is average, both may be merged or a specified rule may be given priority; when the matching degree is low or there is no matching rule, the result of the large model is retained but marked as low confidence.

[0043] Low confidence and rules cannot support: Decision rejection, triggering manual intervention.

[0044] Output the final decision (classification / label) and judgment path (including initial judgment, triggered rules and their weights, and arbitration logic).

[0045] 5. Output layer: Output the final classification label or behavior recognition result, including the necessary confidence score, detailed information, and an interpretable decision path.

[0046] The architecture of this embodiment has the following advantages: 1. Pluggable type and rule libraries: Building cross-domain rules decouples type definitions and rule logic from the underlying engine. When targeting different domains (such as e-commerce, healthcare, and finance), only the corresponding type library needs to be configured and domain-specific rules need to be stored in the cross-domain rule library, without modifying the core engine logic.

[0047] 2. Standardized micro-hint templates: unify the input and output formats of the large model (e.g., require output {type:str,reason:str,confidence:enum[high,medium,low]}), significantly reducing the risk of formatting errors and illusions.

[0048] 3. Prioritize the execution of deterministic rules: In well-defined scenarios (such as structured validation and strong rule constraints), prioritize the execution of 100% accurate deterministic rules (implemented through regular expressions) to avoid unnecessary model calls and improve system efficiency and stability.

[0049] like Figure 3As shown, this embodiment uses bank transaction text classification as an example to demonstrate how the present invention can be instantiated in a specific field.

[0050] 1. Input layer: Domain-specific interface: Connects to bank customers' ERP / OA systems, receiving standardized bank transaction text (including transaction time, counterparty name, amount, summary, etc.) and enterprise-specific data (employee list, supplier whitelist, industry classification attributes, historical transaction records).

[0051] 2. Feature Processing Layer: Transaction feature extraction: focusing on bank transaction dimensions.

[0052] Employee attribute matching: Identify whether the other party's account name matches the employee list (key entity attributes).

[0053] Transaction timing analysis: Identify periodic payments and transactions at unusual times (timing context).

[0054] Payment Semantic Analysis: In-depth analysis of the "Summary" and "Description" text fields (raw semantic analysis).

[0055] Build a dynamic transaction graph: link the transaction parties (our / the other party's account), transaction details (amount, time, description), and background information (employees, suppliers).

[0056] 3. Knowledge Construction Layer: Banking expert: Extracting knowledge from the financial field.

[0057] Structural knowledge (deterministic logical rules): such as "the payee's name exists in the employee list and the summary does not contain the keywords 'salary' or 'bonus' → type = reimbursement".

[0058] Empirical knowledge (probabilistic model fitting): such as the probability weights of training models based on historical data to identify subtypes of "payment" transactions (such as prepayment / final payment / periodic payment).

[0059] Store in the banking rules database: Store bank-specific rules (e.g., FIN-012, FIN-087, FIN-103, FIN-201), labeled as the "Banking" domain.

[0060] 4. Instantiation of the decision fusion layer: Flow-based decision engine: Executes general decision fusion logic, calls the bank's proprietary semantic classification model (fine-tuned LLM), matches and executes the banking rule base.

[0061] Confidence Arbitrator: Makes the final decision based on a threshold strategy optimized for banking scenarios.

[0062] Based on bank transaction records, the transaction type can be identified. See Table 1 for the input: Table 1 Amount ¥12,850.00 summary Payment settlement describe 2025 Q3 Appliance Payment Contract ZD2025-087 Respondent's name XX Electric Appliances Co., Ltd. This party's name XX Technology Co., Ltd. time 2025-08-18 14:22:31 Model classification output result: { "type": "loan payment", "confidence": "medium" } Arbitration triggered with medium confidence, rule engine matching result: rule_scores = { "FIN-012": ("XX Electric Co., Ltd." in "electric appliances") and (12850 > 10000) → 0.92, "FIN-087": ("contract" in text) and (12850 % 1 != 0) → 0.88, "FIN-103": ("settlement" in text) and ("2025Q3" in text) → 0.95, "FIN-201": (the name of our party belongs to the "technology industry") and ("electric appliances" in the name of the other party) → 0.90 } Arbitration decision output result: { "type": "loan payment", "subtype": "periodic payment", "confidence": 0.95, "decision_path": "LLM initial judgment: loan payment (0.83)", "Rule triggered: FIN-103 (0.95)", "Decision result: loan payment - periodic payment" } This embodiment is used in the scenario of automatically generating vouchers for bank statements during the tax period. Through the verification of monthly voucher data generated by a certain enterprise, the statistical results are as follows: The amount of data processed is approximately 100,000 records; The accuracy rate has increased from 77% (traditional solution) to 95%; The average daily volume of processed bank statements by a single accountant has increased from 200 to 1000; The manual intervention rate has decreased from 35% to 7.5%.

[0063] Example 3 ​Based on the same inventive concept, this embodiment also provides an electronic device. The method corresponding to the electronic device can be the method in the foregoing embodiments, and its problem-solving principle is similar to that method. The electronic device provided in this application embodiment includes: at least one processor; and a memory communicatively connected to the at least one processor; the memory stores instructions executable by the at least one processor, which are executed by the at least one processor to enable the at least one processor to execute the methods and / or technical solutions of the foregoing embodiments of this application.

[0064] The electronic device can be a user device, or a device formed by integrating user devices and network devices through a network, or it can be an application running on the aforementioned devices. The user device includes, but is not limited to, various terminal devices such as computers, mobile phones, tablets, smartwatches, and smart bands. The network device includes, but is not limited to, network hosts, single network servers, multiple network server sets, or cloud computing-based computer sets, and can be used to implement some processing functions when setting an alarm clock. Here, the cloud consists of a large number of hosts or network servers based on cloud computing. Cloud computing is a type of distributed computing, consisting of a virtual computer composed of a group of loosely coupled computer sets.

[0065] Figure 4 The structure of an apparatus suitable for implementing the methods and / or technical solutions of this embodiment is shown. The apparatus 300 includes a central processing unit (CPU) 301, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 302 or a program loaded from a storage portion 308 into a random access memory (RAM) 303. Various programs and data required for system operation are also stored in the RAM 303. The CPU 301, ROM 302, and RAM 303 are interconnected via a bus 304. An input / output interface (I / O interface) 305 is also connected to the bus 304.

[0066] The following components are connected to I / O interface 305: an input section 306 including a keyboard, mouse, touchscreen, microphone, infrared sensor, etc.; an output section 307 including cathode ray tubes, liquid crystal displays, LED displays, OLED displays, etc., and speakers, etc.; a storage section 308 including one or more computer-readable media such as hard disks, optical disks, magnetic disks, semiconductor memory, etc.; and a communication section 309 including network interface cards such as LAN cards, modems, etc. The communication section 309 performs communication processing via a network such as the Internet.

[0067] Obviously, the above embodiments are merely illustrative examples for clear explanation and are not intended to limit the implementation. Those skilled in the art will recognize that other variations or modifications can be made based on the above description. It is neither necessary nor possible to exhaustively list all possible implementations here. However, obvious variations or modifications derived therefrom are still within the scope of protection of this invention.

Claims

1. A collaborative classification decision-making method using a large model and a dynamic knowledge base, characterized in that, include: Feature extraction steps: Extract features from the data to be classified; Steps for constructing a dynamic knowledge graph: Construct a dynamic knowledge graph based on the extracted features; Steps to construct cross-domain rules: Differentiate knowledge from different domains into structured knowledge and empirical knowledge to form cross-domain rules; Classification steps: Use a large language model to classify the data to be classified and obtain the confidence level of the classification results; The dynamic knowledge graph is matched with cross-domain rules to calculate the rule matching degree; by setting thresholds, the confidence degree and rule matching degree are divided into high level, medium level and low level respectively. Fusion decision-making steps: Classify decisions based on the level of confidence and rule matching: When the confidence level is high, the classification result of the large language model is adopted; When the confidence level is medium or low, the rule matching degree is judged as follows: When the rule matching degree is high, the rule matching result is adopted; When the rule matching degree is medium, the classification results of the large language model and the rule matching results are fused. When the rule matching degree is low, the classification result of the large language model is adopted; When the confidence level is low and there is no matching rule, manual intervention is triggered.

2. The method according to claim 1, characterized in that, In the feature extraction step, the data to be classified is converted into a standardized format through a domain adaptation interface, which connects to data sources from different domains.

3. The method according to claim 2, characterized in that, The feature extraction includes identifying entity information and its attributes in the data to be classified, extracting the time sequence patterns, correlations and contextual information of events, and parsing the semantics of unstructured information.

4. The method according to claim 1, characterized in that, In the classification step, the large language model uses a structured prompting engineering template to classify the data to be classified, constraining the output format and range.

5. The method according to claim 4, characterized in that, The large language model is a general-purpose large model that is not trained and requires only lightweight fine-tuning.

6. The method according to claim 1, characterized in that, In the step of constructing cross-domain rules, structural knowledge refers to deterministic logical rules; empirical knowledge refers to rules involving probability, empirical judgment, or fuzzy boundaries, which are modeled through probabilistic logical fitting.

7. The method according to claim 1, characterized in that, In the fusion decision-making step, in scenarios with structured verification or strong rule constraints, deterministic rules are executed first.

8. The method according to claim 1, characterized in that, The method further includes an output step, the output of which includes classification, confidence level, rule matching degree, and the decision process of the fusion decision step.

9. A collaborative classification decision-making system using a large model and a dynamic knowledge base, characterized in that, include: Feature extraction module: Extracts features from the data to be classified; Dynamic knowledge graph construction module: Constructs dynamic knowledge graphs based on extracted features; Construct cross-domain rule modules: Distinguish between structural knowledge and empirical knowledge from different domains to form cross-domain rules; Classification module: Uses a large language model to classify the data to be classified and obtains the confidence level of the classification results; The dynamic knowledge graph is matched with cross-domain rules to calculate the rule matching degree; by setting thresholds, the confidence degree and rule matching degree are divided into high level, medium level and low level respectively. Fusion Decision Module: Makes classification decisions based on the level of confidence and rule matching. When the confidence level is high, the classification result of the large language model is adopted; When the confidence level is medium or low, the rule matching degree is judged as follows: When the rule matching degree is high, the rule matching result is adopted; When the rule matching degree is medium, the classification results of the large language model and the rule matching results are fused. When the rule matching degree is low, the classification result of the large language model is adopted; When the confidence level is low and there is no matching rule, manual intervention is triggered.

10. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the method of any one of claims 1 to 8.

Images