A multimodal rule fusion dataset construction and dynamic adaptation method and system

Abstract

A kind of multimodal rule fusion dataset construction and dynamic adaptation method, comprising the following steps: (1) domain feature self-sensing step: input the rule document of target domain, text data or image / video type multimodal data, automatically extract domain features by natural language processing or visual analysis, and configure adaptation parameters for core engine;(2) multi-source heterogeneous data fusion step: the structured, semi-structured or unstructured data of target domain is uniformly converted into "entity-attribute-value-timestamp" four-tuple format;(3) rule analysis and representation step;(4) QIR-KG dataset construction step;(5) dynamic association and evolution step.The application adopts the design idea of "QIR-KG domain-independent kernel+plug-in architecture", and is enhanced in parameter calibration, visual processing, cloud edge end cooperation, conflict resolution, exception handling and the like, to ensure the implementability and robustness of the scheme.

Description

Technical Field

[0001] This invention relates to the fields of data processing and artificial intelligence technology, and in particular to a method, system and medium for constructing and dynamically adapting a multimodal rule fusion dataset. Background Technology

[0002] In heavily regulated areas such as customs supervision, financial risk control, medical quality control, industrial quality inspection, and government approval, business decisions often require combining domain rules with data queries. However, existing technologies have the following shortcomings when handling such needs: 1. General technical shortcomings in the construction of traditional rule-based datasets The lack of a unified core data structure as a carrier, the single method of dataset construction and domain binding, the rule representation is mostly in the form of simple IF-THEN, which cannot support complex rules such as threshold calculation, process steps, and time sequence relationships. Furthermore, the static updates of datasets, the high cost of implementation and application, and the lack of interpretability make it difficult for even pure text rule datasets to meet the core requirements of strong supervision for in-depth rule analysis, dynamic evolution, and full life cycle traceability.

[0003] 2. New industry defects caused by multimodal fusion As visual signals from the physical world (such as customs waterlines, industrial defect images, and medical images) become important inference bases for heavily regulated operations, existing technologies, lacking a unified data structure, cannot effectively align and fuse multimodal inputs such as images / videos with text rules. Furthermore, they cannot achieve rule representation and inference based on visual states, or construct multimodal rule fusion datasets to meet the multimodal business needs of the industry.

[0004] Therefore, existing technologies lack a unified core data structure to support the construction of rule-based datasets (plain text / multimodal) and cannot meet the upgrade requirements of multimodal fusion. Summary of the Invention

[0005] For ease of understanding, the core terms are defined as follows: The QIR-KG data structure is a data organization method that includes natural language request units (Q units), intent and instruction units (I units), rule logic expression units (R units), and knowledge graph embedding units (KG units), serving as a unified carrier for constructing rule fusion datasets. Physically, it can be stored in layers or merged.

[0006] QIR-KG dataset: A dataset instance consisting of multiple data records conforming to the QIR-KG data structure, with each record corresponding to complete information for one rule inference.

[0007] Multimodal rule fusion dataset: A dataset built with the QIR-KG data structure as the core carrier, supporting the collaborative representation of multimodal inputs such as text, images, and videos with rules.

[0008] Rule structure hash: A unique identifier generated based on the rule logic content and its dependent external variables, used to detect version changes of the rule logic.

[0009] Data record structure hash: A unique identifier generated by the rule structure hash referenced by the data record and the field structure of the record itself, used to detect changes in the logical structure of the data record.

[0010] Dynamic quality fingerprint: A set of multi-dimensional quality indicators attached to each QIR-KG data record, including at least content hash, data record structure hash, quality score, source credibility, timeliness, domain weight, diversity contribution, visual quality dimension, and fusion quality score.

[0011] Fusion Quality Score: The score representing the overall quality of data records in the dynamic quality fingerprint, which is calculated by weighting multiple quality dimensions.

[0012] Visual quality dimension: A metric used in dynamic quality fingerprinting to evaluate the quality of image or video input.

[0013] Visual entities: Entities detected and identified from images or videos, which can be embedded into KG units of the QIR-KG data structure.

[0014] Visual rules: The triggering conditions are rules based on visual detection results, encapsulated in the R unit of the QIR-KG data structure, and adopt a unified representation framework with text rules.

[0015] Relationships between rules: Logical relationships such as input dependencies and output mappings exist between rules, which are used to determine the order in which rules are executed.

[0016] Association type: The time-series association category between rules and queries, including pre-rule type, post-rule type, parallel rule type, and alternative rule type.

[0017] UDTP protocol: A unified data transmission protocol format for encapsulating and transmitting QIR-KG datasets, including a protocol header, data body, and verification information, and supports version compatibility.

[0018] To address the shortcomings of existing technologies, this invention provides a method for constructing and dynamically adapting a multimodal rule fusion dataset based on the QIR-KG data structure. It adopts the design concept of "QIR-KG domain-independent kernel + plug-in architecture" and enhances aspects such as data structure design, parameter calibration, visual processing, cloud-edge-device collaboration, conflict resolution, and anomaly handling to ensure the feasibility and robustness of the solution.

[0019] In a first aspect, the present invention provides a QIR-KG data structure for the construction and dynamic adaptation of multimodal rule fusion datasets, including: The Natural Language Request Unit is used to store the original text request and multimodal input references; The intent and instruction unit is used to store request intents, data acquisition instructions, and multimodal task instructions. The rule logic expression unit is used to store the three-layer abstract structure of text rules or visual rules and the rule structure hash; The knowledge graph embedding unit is used to store vector representations of text entities and visual entities, as well as entity information. Each unit can be linked across units, and can be extended to other units as needed. Each unit can also be extended internally as needed.

[0020] This data structure provides a unified core carrier for subsequent steps such as dataset construction, dynamic evolution, and lightweight adaptation.

[0021] Secondly, this invention provides a method for constructing and dynamically adapting a multimodal rule fusion dataset based on the aforementioned QIR-KG data structure, comprising the following steps: Step 1: Domain Feature Self-Awareness Input the target domain's rule documents, multi-source data descriptions, and typical requests, and automatically extract domain features through natural language processing and visual analysis; based on the perception results, automatically configure the optimal parameters for the core engine and write them into the global metadata management module.

[0022] Step 2: Multi-source heterogeneous data fusion Multi-source data is uniformly converted into a "entity-attribute-value-timestamp" quadruple format; cross-data source entity associations are established, and multi-source data is weighted and fused to generate knowledge input for constructing QIR-KG data records.

[0023] Step 3: Rule Parsing and Representation Domain rules are parsed into a three-layer abstract structure including triggering conditions, execution logic, and output conclusions. A unique rule structure hash and version information are generated for each rule and stored in the rule base.

[0024] Step 4: QIR-KG Dataset Construction The fused data record is generated according to the QIR-KG data structure. The fused data record logically includes the natural language request unit, intent and instruction unit, rule logic expression unit and knowledge graph embedding unit.

[0025] Step 5: Dynamic Association and Evolution Construct a dynamic association graph containing entity nodes, rule nodes, and request nodes, and update the weights of the association edges based on the co-occurrence log.

[0026] Step 6: Dynamic Quality Fingerprint Generation Calculate a multidimensional dynamic quality fingerprint for each data record; when the hash structure of a data record changes, a version management mechanism is triggered to generate a new version record, and the old record is marked as obsolete and archived according to its historical value.

[0027] Step 7: Conflict Resolution and Optimization A weighted scoring mechanism is used to resolve conflicts between multiple rules in the rule nodes; the weight adjustment results and user feedback during the conflict resolution process are regularly optimized offline to optimize the macro weight configuration and synchronized to the dynamic quality fingerprint.

[0028] Step 8: Lightweight Reasoning and Clipping Different algorithms are used to prune the dataset according to the deployment scenarios of cloud, edge, and mobile devices, and symbolic execution verification is used to ensure the equivalence of the rules before and after pruning.

[0029] Step 9: UDTP Protocol Encapsulation and Format Conversion The dataset is encapsulated into a unified protocol format and converted into the format required by the target training framework; the UDTP protocol supports version compatibility.

[0030] Step 10: Quality assessment, exception handling, and global metadata management The quality assessment step evaluates the dataset from multiple dimensions, and data items below the threshold are automatically triggered to update the process; the anomaly handling step adopts different processing strategies according to the anomaly level; the global metadata management step stores dataset-level metadata through the global metadata management module to achieve automatic evolution of the dataset.

[0031] Compared with the prior art, the beneficial effects of the present invention are as follows: 1. Supported by a unified data structure By defining the QIR-KG data structure as a unified core carrier, natural language requests, intent commands, rule logic, and knowledge graphs are embedded and integrated into standardized data records. This solves the problems of traditional datasets being structurally chaotic, domain-bound, and incompatible with multimodal inputs, providing a consistent data foundation for the construction, evolution, and lightweight deployment of rule fusion datasets.

[0032] 2. Multimodal unified representation To address the problem that existing technologies cannot effectively align visual signals with text rules, the QIR-KG data structure is used to encapsulate multimodal inputs and rule logic in a unified manner, enabling visual spatial relationships and video temporal states to be directly used as rule triggering conditions, thus achieving a seamless connection between the visual state of the physical world and rule reasoning.

[0033] 3. Dynamic evolution of rules To address the issues of static updates and version chaos in traditional rule datasets, this paper achieves automatic detection and minute-level synchronization of rule changes through rule structure hashing and data record structure hashing. The dataset can dynamically evolve as business rules are adjusted, solving the pain point of high timeliness requirements for rules in areas with strong regulatory oversight.

[0034] 4. Full lifecycle traceability To address the lack of quality assessment and version traceability mechanisms in existing datasets, this paper implements fully auditable data records from generation and evolution to application by incorporating dynamic quality fingerprints and source metadata, thus meeting the rigid requirements of interpretability and compliance evidence chains in highly regulated scenarios.

[0035] 5. Scenario-adaptive lightweight deployment To address the issue that a single dataset is difficult to adapt to the differentiated computing power environments of cloud, edge, and device, an intelligent pruning algorithm and symbolic execution verification are used to achieve adaptive compression of the dataset size while ensuring the equivalence of rule logic, thereby reducing the deployment cost for multi-scenario implementation. Attached Figure Description

[0036] Figure 1 This is a diagram of the overall system architecture of the present invention; Figure 2 This is a schematic diagram of the core QIR-KG data structure of the present invention; Figure 3 This is a flowchart of the dynamic quality fingerprint generation process of the present invention; Figure 4 This is a schematic diagram of the dynamic correlation spectrum of the present invention; Figure 5 This is a flowchart of the intelligent cropping engine of the present invention. Detailed Implementation

[0037] The specific embodiments of the present invention are described below with reference to the accompanying drawings. It should be understood that the described embodiments are for illustrative purposes only and are not intended to limit the scope of the invention.

[0038] 1. Overall Architecture and Plug-in Specifications like Figure 1 As shown, the system comprises dataset construction groups, dataset evolution and optimization groups, and application groups, all coordinated through a global metadata management module. The system uses the QIR-KG data structure as its core and employs a plug-in architecture to extend domain-specific functionalities.

[0039] The functional groups are positioned as follows: Dataset Construction Group: Complete the mapping and rule parsing of multi-source data to the QIR-KG data structure to generate the basic dataset; Dataset Evolution and Optimization Groups: Enables dynamic quality assessment, association evolution, and conflict resolution, continuously optimizing data quality; Application Deployment Group: Responsible for intelligent trimming, protocol encapsulation and format conversion, supporting multi-scenario deployment in cloud, edge and terminal.

[0040] Global metadata management module: Independent of the three major groups, it communicates bidirectionally with all modules through the message bus, stores dataset-level metadata, and ensures consistency and traceability.

[0041] Core architecture definition: Domain-independent kernel: composed of QIR-KG data structure module, dynamic quality fingerprint module, and UDTP protocol module.

[0042] Plug-in architecture: Includes domain feature plugins, rule parsing plugins, conflict resolution plugins, pruning algorithm plugins, data augmentation plugins, visual feature extraction plugins, and visual rule parsing plugins. Users can select and configure as needed; in pure text scenarios, multimodal plugins are not loaded or are implemented empty.

[0043] 1.1 Unified Plugin Interaction Specification All plugins interact with the kernel using a standardized JSON format, containing the following fixed fields: request_id (string): a unique identifier for the request, used for tracing; input (object): plugin-specific input parameters; output (object): the plugin's execution result; status (object): contains status codes and error messages (such as success, parameter error, execution timeout, etc.). Communication protocols support HTTP / REST or gRPC.

[0044] 1.2 Core Plug-in Interface Example Visual feature extraction plugin: The input parameters include: media_type (image or video), data (Base64 encoded data), task_type (such as entity detection, semantic segmentation, key point detection, etc.), and parameters (including configurable parameters such as confidence threshold and maximum number of detections).

[0045] The output includes: a list of detected entities (including location, confidence, feature embedding, etc.) and visual quality assessment results (sharpness, illumination, etc.).

[0046] Rule parsing plugin: Input parameters include: rule_text (raw rule text), source (source information), and domain (domain identifier).

[0047] The output includes: rule ID, rule type, triggering condition, execution logic, output conclusion, rule source, and confidence level.

[0048] Visual rule parsing plugin: The plugin has built-in visual spatial relation operators (such as above, below, contain, overlap) that support the conversion of visual rules into a unified three-layer abstract structure.

[0049] 2. Dataset Construction Group 2.1 Domain Feature Self-Awareness Input the target domain's rule documents, multi-source data descriptions, and typical requests (which may include images / videos). The system will automatically extract the following features using natural language processing and visual analysis: (1) Rule type distribution recognition: The pre-trained language model is used to classify the rule text with multiple labels to identify the proportion of five types of rules: condition judgment type, threshold calculation type, process step type, model reasoning type, and knowledge statement type. At the same time, the visual task type involved in the rule is also identified.

[0050] (2) Request Intent Clustering: After vectorization and dimensionality reduction of the request text, clustering is performed to identify the main intent types, including data retrieval, rule verification, compliance judgment, process verification, knowledge query, visual status query, etc.

[0051] (3) Terminology system construction: construct a domain terminology network by word embedding method, identify core entities and their synonyms, and provide semantic support for field mapping; for visual entities, establish a mapping relationship between visual concepts and text labels.

[0052] (4) Data source feature extraction: Extract data format, data size, update frequency, and visual data related parameters such as image resolution and video frame rate.

[0053] Cold start enhancement: When the number of labeled samples in the domain is less than a preset threshold, the system enters cold start mode. If a small number of labeled samples exist, they are used after consistency verification; if there are no labeled samples, a zero-shot strategy is adopted—semantic analysis of the rule document is performed based on a large language model, domain features are initially extracted by combining the data source description, and a general domain rule template library is loaded. Initial rules are generated through semantic similarity matching, and consistency checks are performed on the generated rules. If the checks fail, the rules are marked for manual review. For visual entities, a general visual language model is called to perform zero-shot recognition and match it with text labels. If no match is found, a new entity is created and marked with low confidence.

[0054] Once the number of labeled samples exceeds the threshold, the cold start ends, and the system enters the parameter optimization phase. The core parameters are optimized with the validation set inference accuracy as the objective function. After the optimization results are confirmed to be effective, the configuration is updated through the global metadata management module.

[0055] Based on the perception results, the optimal parameters are automatically configured for the core engine and written to the global metadata management module; each core parameter has a preset baseline value based on the domain and scenario, and is dynamically optimized based on feedback during operation. For pure text scenarios that do not involve visual input, visual-related parameters are not included in the configuration and calculation.

[0056] 2.2 Multi-source heterogeneous data fusion Convert multi-source data into a unified "entity-attribute-value-timestamp" four-tuple format: (1) Structured data: Data record fields are directly mapped to quadruples, and the timestamp is taken as the record update time.

[0057] (2) Semi-structured data: Parse according to the data format, extract fields based on the preset schema to generate four tuples, and obtain the timestamp from the response information.

[0058] (3) Unstructured data: Text data is generated into quadruplets through entity recognition and attribute extraction; Image / video data calls the visual feature extraction plugin to detect visual entities, extract position coordinates, confidence and recognize text, and generate quadruplets frame by frame, with the timestamp based on the device acquisition time.

[0059] For multimodal scenarios, the visual detection confidence threshold is determined based on the domain-preset benchmark value. Detection results below the lower threshold can be discarded or trigger manual review.

[0060] Main timeline alignment: Select the core data type of the domain as the main timeline, and map other data source timestamps to the main timeline through event matching or nearest neighbor interpolation, and record the alignment method and time offset.

[0061] Cross-data source entity association: Within the same modality, text entities are associated based on a combination of string similarity and semantic similarity, while visual entities are associated based on a combination of bounding box overlap and visual feature similarity; cross-modal entities are associated based on semantic similarity. When the similarity is below a preset threshold, manual review is triggered. If the review timeout is exceeded, temporary associations are established in descending order of data source credibility and marked as pending confirmation.

[0062] Multi-source data weighted fusion: At the entity attribute granularity, numerical attributes are weighted and averaged according to source credibility, while categorical attributes prioritize high-credibility data sources. The fused quadruples are used to construct KG units for QIR-KG data records, providing unified knowledge input for rule-based reasoning.

[0063] 2.3 Rule Parsing and Representation Domain rules are parsed into a three-layer abstract structure, supporting rule types such as conditional judgment, threshold calculation, process step, model reasoning, knowledge statement, visual judgment, and video temporal sequence. (1) Triggering condition: Boolean logic expression, supports text conditions and visual spatial relation operators, and threshold can be set.

[0064] (2) Execution logic: a description of steps, whether single-step or multi-step, serial or parallel, cyclic or branching.

[0065] (3) Output conclusions: deterministic conclusions, probabilistic results, or operation instructions.

[0066] After being processed by the classification model, rules with a confidence level below a preset threshold are marked for manual review. The parsed rules include metadata such as source information, priority, and tags, which are used for rule structure hash calculations and stored in the rule database.

[0067] The rule structure hash is generated as follows: the rule abstract syntax tree is standardized and serialized, concatenated with sorted external variables, and then calculated using a hash algorithm. Each rule receives a unique identifier and its version information is recorded. The parsed three-layer abstract structure and rule structure hash are directly mapped to the corresponding fields of the R unit in the QIR-KG data record for subsequent reference.

[0068] 2.4 Definition and Construction of QIR-KG Dataset like Figure 2 As shown, the QIR-KG data structure consists of the following six components: (1) Natural Language Request Unit (Q Unit): Stores the original user input and query type, and can be extended to multimodal input references.

[0069] (2) Intent and Instruction Unit (I Unit): Stores intent type, parameters and data acquisition instructions, and can be extended to multimodal task instructions.

[0070] (3) Rule logic expression unit (R unit): Stores the three-layer abstract structure of rules, rule structure hash, relationship between rules and association type, and can expand the unified encapsulation of visual rules.

[0071] (4) Knowledge Graph Embedding Unit (KG Unit): Stores the identifier, type and vector representation of text entities, and can extend the fusion vector representation of visual entities.

[0072] (5) Output results: Store structured output results, confidence level, reasoning and rule references, and expand the visual evidence field.

[0073] (6) Metadata: storage domain, difficulty, quality fingerprint, data source and update log, which can be expanded to include visual quality dimensions.

[0074] For detailed definitions of the core fields and multimodal extended fields of each unit, please refer to the glossary and... Figure 2 Indication.

[0075] Construction Instructions: Each QIR-KG data record is generated as follows: entities and attributes are extracted from the fused quadruplets in Section 2.2 and filled into KG cells; the three-layer rule structure and rule structure hash parsed in Section 2.3 are filled into R cells; the user's original request and multimodal input references are filled into Q cells; intent recognition results and data acquisition instructions are filled into I cells; the output results can initially be empty or filled by annotations; and the quality fingerprint is calculated based on the record content and filled into the metadata fields.

[0076] The QIR-KG dataset achieves a unified representation of plain text and multimodal scenarios through the aforementioned units: core fields of each unit are enabled in plain text scenarios, and extended fields are enabled in multimodal scenarios, without modifying the core structure. Each record includes a version control field, and physical storage can be implemented in a tiered or merged manner.

[0077] Example of a data record structure (JSON serialization): {"record_id": "[string: unique identifier for the record]", "version": "[integer: version number]", "dataset_id": "[string: identifier for the dataset]", "q_layer": { "query": "[string: original user request]", "query_type": "[string: query type]", "multimodal_input": { "type": "[string: image / video]", "url": "[string: resource address]", "key_frames": "[integer array: keyframe index]"}}, "i_layer": { "intent": { "type": "[string: intent type]", "params": { "[parameter name]": "[parameter value]"}}, "data_fetch_instruction": "[string: data fetch instruction]","instruction": { "task_type": "[string: task type]", "command": "[string: specific instruction]"}}, "r_layer": { "rule_id": "[string: rule ID]", "rule_type": "[string: rule type]", "rule_structure_hash": "[string: rule structure hash]", "priority": "[integer: priority]", "effective_time": "[string: effective time, ISO8601 format]", "rule_relations": [ { "target_rule_id": "[string: association rule ID]", "relation_type": "[string: relationship type]"} ], "association_type": "[string: association type]", "trigger_condition": { "operator": "[string: logical operator]", "conditions": [ { "field": "[string: field name]", "exists": "[boolean value]"} ]}, "execution_logic": { "type": "[string: execution type]", "condition": "[string: condition expression]", "then": { "steps": "[String array: Execution steps]"}, "else": { "action": "[String: Exception handling action]"}}, "output_conclusion": "[String: Output conclusion]", "source": { "filename": "[String: Source filename]", "clause": "[String: Clause number]", "confidence": "[Floating-point number: Confidence level]"}}, "kg_layer": { "entities": [ { "id": "[string: entity ID]", "type": "[string: entity type]", "attributes": { "[attribute name]": "[attribute value]"}, "embedding": "[floating-point array: vector representation]"} ], "cross_modal_links": [ { "visual_entity_id": "[string: visual entity ID]", "text_entity_id": "[string: text entity ID]", "similarity": "[floating-point number: similarity]"} ]}, "output_result": { "output": { "result": "[Inference result, type depends on the specific rule]", "confidence": "[Floating-point number: confidence level]"}, "inference_reasoning": "[String: reason for inference]", "rule_reference": "[String array: referenced rule ID]", "visual_evidence": { "frame": "[Integer: keyframe number]", "bbox": "[Integer array: bounding box coordinates]", "url": "[String: resource address]"}}, "metadata": { "domain": "[string: domain]", "difficulty": "[floating-point number: difficulty score]", "quality_fingerprint": { "content_hash": "[string: content hash]", "record_structure_hash": "[string: record structure hash]", "quality_score": "[floating-point number: quality score]", "source_credibility": "[floating-point number: source credibility]", "timeliness": "[floating-point number: timeliness]", "domain_weight": "[floating-point number: domain weight]", "diversity_contribution": "[floating-point number: diversity contribution]", "visual_quality": { "clarity": "[floating-point number: clarity]", "illumination": "[floating-point number: lighting conditions]", "occlusion": "[floating-point number: occlusion degree]", "frame_stability": "[floating-point number: frame rate stability]", "aggregate_score": "[floating-point number: visual quality aggregate score]"}, "fusion_score": "[floating-point number: fusion quality score]"}, "update_log": [ { "time": "[string: timestamp, ISO8601 format]", "version": "[integer: updated version]", "content": "[string: description of the change]"} ]}} The above JSON serialization format example is only used to illustrate the field composition and hierarchical relationship of the QIR-KG data structure. Those skilled in the art can choose the storage method, adjust the field name, order or nesting method according to actual needs, which does not constitute a limitation on the scope of protection of this invention.

[0078] 3. Dataset Evolution and Optimization Groups 3.1 Dynamic Quality Fingerprint Calculate a multidimensional dynamic quality fingerprint for each QIR-KG data record (e.g.) Figure 3 As shown), this includes content hash, data record structure hash, quality score, source credibility, timeliness, domain weight, diversity contribution, and visual quality dimension score. The quantification methods for each dimension are as follows: Quality score: Calculated based on a comprehensive assessment of data integrity, consistency, and accuracy.

[0079] Source credibility: Preset baseline value based on data source type.

[0080] Timeliness: Calculated based on decay over natural days.

[0081] Domain weights: determined based on topic model similarity.

[0082] Diversity contribution: calculated as 1 minus the maximum similarity with existing samples.

[0083] Visual quality dimensions are weighted by sharpness, lighting conditions, degree of occlusion, and frame rate stability.

[0084] The fusion quality score is calculated by weighting the above six dimensions, and the weights can be dynamically configured according to the scenario. If the data record does not contain visual input, the visual quality dimension score is not included in the calculation, and the other dimensions are normalized and then weighted.

[0085] The rule structure hash is generated by concatenating the standardized serialization of the rule abstract syntax tree with external variables and then calculating it using a hash algorithm. The data record structure hash is recalculated when the rule changes or the record structure changes, using a combination of timed and triggered methods to detect changes. When a rule changes, the version number of all data records referencing that rule is incremented and the structure hash is recalculated; old records are marked as discarded and archived hierarchically according to their historical value.

[0086] After the dynamic quality fingerprint is calculated, the scores of each dimension are written into the metadata field of the corresponding QIR-KG data record. The quality scores are then used for quality-aware scheduling, pruning selection, and evolutionary decision-making.

[0087] 3.2 Dynamic Relationships and Evolution: such as Figure 4The system shown constructs a dynamic association graph containing text entity nodes, visual entity nodes, rule nodes, and request nodes. The feature vector of a new node is composed of its corresponding embedding. Initial associations are established based on rule references, semantic similarity, and visual co-occurrence. Association edges include rule dependency types and rule-data general association types.

[0088] Edge weight update: Updated using an exponential moving average based on the co-occurrence log. Co-occurrence refers to the simultaneous fulfillment of the visual entity detection result and the rule triggering condition in the same data record, or the co-occurrence frequency of visual entity appearance and rule triggering within the same time window being ≥1.

[0089] The dynamic association graph is constructed based on entities, rules, and request nodes in the QIR-KG dataset. Node embeddings and edge weights are updated by a graph neural network and then written back to the KG unit and metadata of the corresponding data record, realizing the dynamic evolution of the knowledge layer.

[0090] 3.3 Conflict Resolution and Optimization A macro-weighting configuration and reinforcement learning collaborative mechanism are adopted to resolve multi-rule conflicts. The resolution dimensions include rule priority, business level, effective time, execution cost, result impact, and visual detection confidence.

[0091] For multiple rules that are hit simultaneously, the rule with the higher priority takes precedence; if the priorities are the same, the overall score is calculated using the following formula: Score(r)=α×priority_norm+β×domain_weight+γ×history_accuracy+δ×quality Here, priority_norm is the normalization priority, domain_weight is the domain matching degree, history_accuracy is the historical accuracy, and quality is the data fusion quality score. When the number of historical requests is insufficient, the history_accuracy item is not included, and the weight is redistributed proportionally.

[0092] Reinforcement learning is used to optimize the aforementioned weight coefficients offline, while online inference directly uses the current weights to calculate the score. The reward function is defined as: R = α×ΔAcc + β×ΔCompliance – γ×ΔTime Wherein, ΔAcc is the percentage increase in accuracy, ΔCompliance is the percentage increase in compliance, and ΔTime is the percentage change in inference time; the weight coefficients α, β, and γ can be configured according to the optimization goals of the business scenario, satisfying α+β+γ=1.

[0093] When visual rules conflict with textual rules, the final score of the rule is dynamically adjusted based on the confidence level of the visual detection. Score_final = Score_original × (1 + β × (confidence - 0.5)) Where confidence is the visual detection confidence level (value 0~1), and β is the adjustment coefficient. If the scores are still the same after adjustment, the visual rule takes precedence.

[0094] The weight adjustment results and user feedback during the conflict resolution process are periodically used for offline optimization of macro-weight configuration and synchronized to the dynamic quality fingerprint. The conflict resolution module operates in a closed loop with the QIR-KG dataset: it reads rule dependencies, structure hashes, quality fingerprints, priorities, and version information from data records, performs a comprehensive scoring to select the best result, and writes the optimization results back to the corresponding records after resolution. 3.4 Data Augmentation and Course Learning Diverse samples are generated through a plug-in enhancement framework: text enhancement includes synonym replacement and sentence transformation; visual enhancement includes geometric transformation, color adjustment, noise injection, blur simulation, and video temporal cropping.

[0095] Low-quality visual data is preprocessed using a lightweight restoration model, and low frame rate videos are padded with frame interpolation to complete the frame sequence. The effectiveness of the augmented samples is verified by either the original intent classification model or the visual inspection model.

[0096] The difficulty of the samples is calculated based on rule length, data size, inference depth, and visual complexity to construct a course learning dataset. The data augmentation module uses the QIR-KG dataset as the sole data source, extracts the content and difficulty level of each unit from the records, generates augmented samples through rule combination, entity replacement, and intent expansion, and grades them according to difficulty; the optimization results and newly added samples are stored back into the dataset, forming an iterative closed loop.

[0097] 4. Application Groups 4.1 Lightweight Reasoning and Clipping The workflow of the intelligent cropping engine is as follows: Figure 5 As shown, a differentiated pruning strategy is adopted according to the deployment scenario: the cloud retains all samples; the edge selects the core set based on the weighted distance of the fusion quality score and semantic embedding; and the mobile terminal adopts a greedy coverage algorithm to prioritize the retention of samples with high fusion quality scores and many coverage rule IDs.

[0098] Video data is sampled using keyframes, and a preset number of frames before and after the rule trigger time are forcibly retained. The symbolic execution verification process involves selecting the rule's trigger condition and output conclusion as the core verification nodes, performing stratified sampling verification by rule type, and fully verifying the core rules. If verification fails, repairable rules automatically adjust parameter thresholds and retry; otherwise, the process rolls back to the previous valid version and logs the exception.

[0099] Edge model compression is based on layer importance assessment and pruning to ensure that accuracy loss does not exceed a preset tolerance. The incremental synchronization mechanism pushes rules in a tiered manner according to priority, and the priority scheduling queue uses a weighted strategy to ensure that urgent rules are transmitted first. The basic rule base is dynamically maintained during offline periods, and rules are downgraded and the user is notified when they expire. Offline inference data is stored in categories and incrementally synchronized after network recovery; in case of conflicts, inference is re-based on the cloud version.

[0100] Format conversions for different downstream tasks include: (1) Execution strategy model training: Extract input-output pairs from Q-units and output_result, and concatenate multimodal features with text as the model input sequence.

[0101] (2) Rule knowledge base construction: Extract rule information and quality fingerprints from R units and metadata, construct rule tables, dependency tables and quality tables, and support version verification and quality-aware scheduling.

[0102] (3) Construction of interpretable rule base: Extract rule source information and applicable scenarios from R unit to provide a basis for the interpretation generation system to trace.

[0103] New rule-triggered records and quality feedback generated at the edge are incrementally transmitted back to the cloud, and after review, they are stored in the QIR-KG dataset to drive the continuous evolution of the dataset.

[0104] 4.2 UDTP Protocol and Format Conversion Define a UDTP protocol in JSON format to encapsulate the QIR-KG dataset, supporting version compatibility. The protocol includes the following parts: (1) Protocol header: contains protocol version, dataset identifier, timestamp and data type fields.

[0105] (2) Data body: contains QIR-KG record array.

[0106] (3) Check tail: includes data body checksum and optional digital signature.

[0107] Version compatibility mechanism: Newly added fields are backward compatible; older client versions ignore unknown fields; newer client versions use default values ​​for missing fields.

[0108] The format conversion module converts the QIR-KG dataset into the format required by the target training framework, and supports the extraction of input-output pairs from the records to generate training samples.

[0109] 4.3 Quality Assessment and Global Metadata Management The generated QIR-KG dataset undergoes multi-dimensional quality evaluation, including accuracy, coverage, consistency, usability, and visual annotation accuracy. Data items falling below a preset threshold automatically trigger an update process. Anomaly handling employs a tiered strategy (minor, moderate, severe), with fallback measures (such as Kalman filter prediction, insufficient evidence mode, or manual review) triggered when visual entities overlap or detection fails. Quality evaluation results are written back to the global metadata management module and associated with the quality fingerprint of the corresponding data record, forming a quality closed loop.

[0110] The global metadata management module operates independently of each functional group, communicating bidirectionally with all modules via a message bus. It stores metadata such as dataset identifiers, version numbers, quality summaries, coverage scenarios, rule base indexes, conflict resolution weights, evaluation thresholds, and update logs. Each module reads its configuration from this module and writes back the execution results, enabling the automatic evolution of the dataset.

[0111] 5. Core parameter configuration example For ease of implementation, the table below summarizes the main configurable parameters and their typical values ​​involved in this specification. It should be understood that the following values ​​are for illustrative purposes only and do not constitute a limitation on the scope of protection of this invention. Those skilled in the art can make conventional adjustments according to actual application scenarios, data scale, and accuracy requirements.

[0112] Parameter name chapter default value Recommended range illustrate Cold start labeled sample number threshold 2.1 50 30～100 When the number of labeled samples is lower than this value, enter cold start mode. Label consistency matching threshold 2.1 0.9 0.8～0.95 Validation pass criteria for a small number of labeled samples Zero-sample rule generates similarity threshold 2.1 0.7 0.6～0.9 Lower bound of semantic similarity for matching general rule templates Rule generation confidence threshold 2.1 0.5 0.3～0.7 If the rule confidence level is below this value, it will be marked as requiring manual review. Threshold under visual detection confidence 2.2 0.5 0.3～0.6 Visual inspection results with a confidence level below this value can be discarded or manually reviewed. Cross-modal entity association similarity threshold 2.2 0.8 0.7～0.9 Manual review is triggered when the cross-modal entity semantic similarity is below this value. Rule classification confidence threshold 2.3 0.7 0.6～0.8 If the confidence level of the rule classification result is lower than this value, it will be marked as pending review. Lower limit of the number of conflict resolution history requests 3.3 10 5～20 If the number of historical requests is lower than this value, the historical accuracy rate will not be included in the conflict resolution score. Conflict resolution linear weighting coefficients (α, β, γ, δ) 3.3 (0.3, 0.2, 0.2,0.3) Each coefficient ranges from 0.1 to 0.5, and the sum can be 1. These correspond to the weights of priority, domain matching degree, historical accuracy, and data quality, respectively. Reinforcement learning reward function weights (α_r, β_r, γ_r) 3.3 (0.6, 0.3, 0.1) The coefficients range from 0.1 to 0.8, satisfying α_r + β_r + γ_r = 1. The reward weights correspond to improvements in accuracy, compliance, and inference time, respectively. Visual rule adjustment coefficient β_v 3.3 0.3 0.1～0.5 Confidence Adjustment Strength When Visual Rules Conflict with Text Rules Symbolic execution confidence reduction step size 4.1 0.05 0.01～0.1 The step size for decreasing the confidence threshold of the trigger condition each time verification fails. Symbol execution retry limit 4.1 3 1～5 Automatically adjust the maximum number of retries when verification fails. Tolerance for compression accuracy loss in edge-end models 4.1 5% 1%～10% Maximum allowable accuracy loss ratio for edge pruning compression Offline degradation days threshold at the edge 4.1 7 3～30 If the edge device remains offline for more than this number of days, a downgrade will be triggered and the user will be notified. Note: The symbols α, β, and γ in the conflict resolution linear weighting coefficient and the reinforcement learning reward function weights are defined independently in their respective formulas and do not affect each other.

[0113] The algorithm combination used in this invention forms a functional dependency relationship around the construction and dynamic adaptation of the QIR-KG dataset, and the synergistic effect is as follows: (1) Pre-trained language models, visual detection models and clustering algorithms work together to achieve complete extraction of domain features and parameter calibration.

[0114] (2) The rule parsing plugin and the visual rule parsing plugin map text rules and visual rules to the R unit in a unified manner to realize the fusion representation of multiple types of rules.

[0115] (3) Multi-source data fusion, entity association and main time axis alignment and collaboration unify heterogeneous data into a quadruple format to resolve time and logic conflicts.

[0116] (4) The QIR-KG data structure, dynamic quality fingerprint and hierarchical archiving work together to form a high-quality dataset that is traceable, assessable and evolvable.

[0117] (5) Dynamic association graph, graph neural network and exponential moving average work together to realize the self-evolution of node association and rapid synchronization of rule changes.

[0118] (6) Macro weight allocation, reinforcement learning and dependency perception work together to achieve dynamic resolution and continuous optimization of multi-rule conflicts.

[0119] (7) Text enhancement, visual enhancement, repair model and course learning work together to improve the generalization ability of dataset and sample coverage.

[0120] (8) Core set selection / greedy coverage, symbolic execution, layer importance pruning and incremental synchronization are coordinated to realize the lightweight deployment of the dataset for cloud-edge-device scenarios, and ensure the equivalence and real-time performance of rule logic.

[0121] The above algorithm combination is seamlessly integrated in terms of data flow and synergistically enhances the overall effect, achieving a comprehensive technical effect that a single algorithm cannot achieve.

[0122] To reduce repetitive descriptions, the following embodiments are all based on the same general process. Each step has been described in detail in the invention content and specific implementation methods. The embodiments only illustrate the differences. The general process includes: domain feature self-awareness, multi-source data fusion, rule parsing and representation, QIR-KG dataset construction, dynamic quality fingerprint calculation, dynamic association and evolution, conflict resolution, data augmentation and curriculum learning, lightweight pruning, quality assessment, and practical application.

[0123] Example 1: Customs Vessel Risk Inquiry (Plain Text Scenario) This embodiment demonstrates the independent construction and application of the QIR-KG data structure in a plain text scenario, corresponding to the case where the multimodal input is empty in claim 1.

[0124] The administrator uploads the "Ship Risk Management Regulations" document, a description of the ship database table structure, and typical requests. The system enters a cold start (the number of labeled samples is below the threshold), generating initial rules using zero-sample feature extraction and general template matching; following the general process steps: 1. Domain Feature Self-Awareness: Extract rule type distribution, terminology system, and data source features, and configure core parameters.

[0125] 2. Multi-source data fusion: Connect to the ship database and convert the vessel table data into a "entity-attribute-value-timestamp" quadruple, such as (Oceanship, origin_region, high-risk area A, t).

[0126] 3. Rule parsing and representation: Parse Article 3 of the "Regulations on Ship Risk Management" to generate a three-layer abstract structure of rule R001 (condition judgment type).

[0127] 4. QIR-KG Dataset Construction: Generate QIR-KG records. Q cells store the query "Check the risk level of Yuanyang", I cells store the SQL command, R cells store rule R001 and structure hash, and KG cells store the vector representation of the ship entity "Yuanyang".

[0128] 5. Dynamic quality fingerprint calculation: The fusion quality score of 0.92 is calculated based on integrity, source credibility, timeliness, etc.

[0129] 6. Dynamic association and evolution: Establish associations between rule node R001, request node Q001, and data node D001, and update the graph.

[0130] 7. Conflict Resolution: There are no rule-based conflicts in this scenario, so skip this step.

[0131] 8. Data Augmentation and Course Learning: Perform synonym replacement and sentence transformation on the request text, and add it to the dataset according to difficulty level.

[0132] 9. Lightweight cropping: Full data is exported from the cloud, and the k-center algorithm is used for cropping at the edge.

[0133] 10. Quality assessment: The accuracy and coverage of the assessment dataset all meet the threshold requirements.

[0134] 11. Practical Application: The generated dataset is converted to a new format, and the resulting risk assessment model can be deployed on a customs edge server. This model receives user-input vessel query requests (such as "Check the risk level of the ocean liner"), parses them in real time, and outputs the risk level. The inference results can be linked to the rule source and quality fingerprint in the QIR-KG data records to ensure traceability.

[0135] This embodiment fully demonstrates the process of constructing a rule fusion dataset according to the method of this application, starting from the original rule document, data source description, and business samples. The final generated dataset not only retains the original business semantics but also lays a data foundation for subsequent advanced applications such as version tracking, quality awareness, and knowledge reasoning through the injection of deep information such as structural hashing, quality fingerprinting, and relationships between rules. The dataset integrates scattered and heterogeneous rules and data sources into a unified and computable structured dataset, solving the core problems of information silos, version chaos, and lack of quality in traditional rule management.

[0136] Example 2: Customs draft survey (multimodal scenario) This embodiment demonstrates the QIR-KG data structure's native support for multimodal rule fusion datasets, supporting multimodal scenarios with image / video input.

[0137] Customs officers uploaded the "Regulations for Ship Draft Weight Measurement" document, a ship database description, and typical video samples, guiding users to annotate a small number of key frames and pass the consistency check.

[0138] Following the general process steps: 1. Domain Feature Self-Awareness: Identifies visual rule types, visual entity terms, and data source features (resolution 1920×1080, frame rate 25fps).

[0139] 2. Multi-source data fusion: Connect to the database to obtain ship parameters; call the visual feature extraction plugin to perform video frame extraction and detection, extract the position and scale numbers of the waterline and load line, and generate a "visual entity-attribute-value-timestamp" quadruple.

[0140] 3. Rule parsing and representation: Parse the procedure measurement rules, generate R001 (visual detection type) and R004 (threshold calculation type), and establish the dependency relationship R001→R004.

[0141] 4. QIR-KG Dataset Construction: Generate QIR-KG records, add video URLs and keyframe indexes to the Q unit, add visual detection instructions to the I unit, encapsulate visual rules in the R unit, and weightedly fuse visual entity features and text entity features in the KG unit.

[0142] 5. Dynamic quality fingerprint calculation: Visual quality dimensions (such as sharpness 0.92, illumination 0.85, etc.) are used to calculate a fusion quality score of 0.91.

[0143] 6. Dynamic association and evolution: Establish association edges between rule nodes R001 and R004 and visual entity nodes, and update the graph.

[0144] 7. Conflict resolution: No conflict, skip.

[0145] 8. Data Augmentation and Course Learning: Adjust brightness, add noise, and perform temporal cropping on videos. Use frame interpolation to repair low frame rate segments and classify them according to visual complexity.

[0146] 9. Lightweight Pruning: Keyframes are pruned at the edge using a dynamic k-value algorithm, and symbolic execution is used to verify the equivalence of core rules; on the mobile end, a greedy coverage algorithm is used to prioritize the retention of samples that cover the core rules and have high fusion quality scores, and these samples are converted into a lightweight inference format.

[0147] 10. Quality assessment: Assess the accuracy of visual annotations (IoU ≥ 0.8) and ensure they meet the threshold requirements.

[0148] 11. Practical Application: The generated dataset is converted to a new format to train a lightweight model. The lightweight model is loaded onto an edge server, and frames are extracted and detected after users upload videos. The rule engine then calculates the ship's draft.

[0149] This embodiment verifies the construction and application of a QIR-KG data structure multimodal rule fusion dataset.

[0150] Example 3: Application Reasoning Scenarios of the Water Draft Weighing Model A lightweight visual feature extraction model was trained based on the QIR-KG multimodal rule fusion dataset constructed in Example 2. The model was deployed on a customs edge server to receive real-time ship draft video streams.

[0151] Inference process: Video frame extraction → Call the visual feature extraction model to detect the waterline and load line positions → Read R unit rules from QIR-KG data records, perform condition matching and logical calculation → Output results along with visual evidence (keyframe URL, detection box coordinates).

[0152] Actual results: The output includes depth values, confidence scores, and traceable visual evidence fields. When rules change, the cloud-based QIR-KG dataset update triggers a change in the rule structure hash. The edge device obtains the updated rules and associated data records every minute through an incremental synchronization mechanism, automatically loads the new rules, and executes the judgment without manual intervention.

[0153] This embodiment fully demonstrates the entire closed-loop process from the construction of the QIR-KG multimodal rule fusion dataset to model training, edge deployment, and online inference, verifying the dataset's practicality and dynamic adaptability in real business scenarios.

[0154] Example 4: Special Scenario - Multi-rule Conflict Resolution This embodiment demonstrates a conflict resolution mechanism based on the QIR-KG data structure.

[0155] The user inputs "Assess the credit risk of client Wang Wu," and the system matches three rules: R101 (recommend approval), R102 (recommend rejection), and R103 (income threshold rule). All three rules have the same priority (7), and a comprehensive score needs to be calculated.

[0156] Score = α×priority_norm + β×domain_weight + γ×history_accuracy +δ×quality, with example weights α=0.3, β=0.2, γ=0.2, δ=0.3. The scores for each rule are as follows: R101: 0.80, R102: 0.76, R103: 0.825 R103 has the highest score (0.825), so the R103 conclusion is output first (approval is recommended, but additional verification of income proof is required). If the number of historical requests is less than 10, a modified formula is used, employing only priority_norm, domain_weight, and quality, with weights distributed proportionally.

[0157] If there is a conflict between visual rules and text rules, the visual weight is dynamically adjusted based on the confidence level of visual detection. When the confidence level is ≥0.8, the weight is increased to 0.25.

[0158] Example 5: Special Scenario - Cloud-Edge-Device Incremental Synchronization This embodiment demonstrates a cloud-edge-device collaboration mechanism based on the QIR-KG dataset.

[0159] The industrial quality inspection edge network is unstable. The full QIR-KG dataset is stored in the cloud, while the cropped dataset is stored at the edge. Synchronization mechanism: Emergency rules are pushed based on business priority: Level 1 is real-time, Level 2 is every 5 minutes, and Level 3 is every 1 hour.

[0160] The priority scheduling queue uses FIFO plus weight to ensure that urgent rules are transmitted first.

[0161] Offline downgrade: The basic rule base dynamically maintains core rules with a business coverage rate of ≥90%. Rules that have been offline for more than 7 days will be downgraded and the user will be notified.

[0162] Data feedback: Inference data generated during offline periods is stored in categories according to timestamps. After the network is restored, it is incrementally synchronized. In case of conflicts, inference is re-inferred based on cloud rules.

[0163] Example 6: Special Scenario - Dataset Cold Start This example demonstrates the construction of a QIR-KG dataset when there are insufficient labeled samples. The number of labeled samples, confidence level, and number of recommended samples are provided as examples and can be adjusted according to domain requirements. In the newly established "High-tech Enterprise Certification" field, if the number of labeled samples is less than 50, it will enter a cold start phase. We use a large language model to perform zero-shot analysis of documents and extract core terms (R&D investment ratio, number of patents) and rule types.

[0164] Load the general rule template library for government approvals, generate initial rules through semantic similarity matching, and mark rules with a confidence score of <0.5 for manual review.

[0165] Five high-value samples are recommended for users to annotate. After annotation, they will be added to the QIR-KG dataset after consistency verification.

[0166] Once the number of labeled samples exceeds 50, the cold start ends and the system automatically optimizes the parameters.

[0167] Example 7: Special Scenario - Exception Handling This example demonstrates an anomaly handling process based on the QIR-KG dataset.

[0168] If a frame in the visual inspection has a confidence score of 0.55 (slight anomaly), inter-frame interpolation is used to complete the visual entity position in that frame, and inference continues. If the subsequent rule parsing confidence score is 0.45 (general anomaly), the rule is marked as having low confidence, and the inference result is accompanied by a "low confidence score in rule parsing" message. If three consecutive visual inspections yield no results (serious anomaly), the process is immediately stopped, manual review is triggered, and the anomaly information is written to the global metadata management module.

Claims

1. A QIR-KG data structure for constructing and dynamically adapting multimodal rule fusion datasets, characterized in that: include: A natural language request unit for storing at least one of a raw text request and a multimodal input reference; The intent and instruction unit is used to store at least one of the following: request intent and data acquisition instruction, and multimodal task instruction; The rule logic expression unit is used to store the three-layer abstract structure of text rules or visual rules, as well as the rule structure hash; The knowledge graph embedding unit is used to store vector representations of text entities and visual entities, entity information, and cross-modal entity associations; Among them, the rule structure hash is used to uniquely identify the logical version of the rule; Cross-modal entity association establishes a correspondence between visual entities and text entities based on semantic similarity; cross-unit associations are established between units, and each unit can be independently extended and new units can be added.

2. The QIR-KG data structure according to claim 1, characterized in that: The QIR-KG data structure is physically stored in independent hierarchical layers or merged storage.

3. The QIR-KG data structure according to claim 1, characterized in that: The QIR-KG data structure is applied to at least one stage of the construction, use, evolution, or adaptation of a multimodal rule fusion dataset.

4. A method for constructing and dynamically adapting a multimodal rule fusion dataset based on the QIR-KG data structure, characterized in that: include: Multi-source heterogeneous data is uniformly converted into structured data units, and the main timeline is aligned to map timestamps from different data sources to the main timeline. The domain rules are parsed into a three-layer abstract structure including triggering conditions, execution logic, and output conclusion, and a rule structure hash is generated. A fused data record is generated using the QIR-KG data structure described in claim 1, wherein the fused data record logically includes the natural language request unit, intent and instruction unit, rule logic expression unit, and knowledge graph embedding unit; Cross-modal entity associations are established based on semantic similarity, and visual entities are associated with text entities and written into knowledge graph embedding units; The dataset is pruned differently based on the deployment scenarios of cloud, edge, and mobile devices, and symbolic execution verification is used to ensure that the rule logic is equivalent before and after pruning. Construct a dynamic association graph containing entity nodes, rule nodes, and request nodes, and update the weights of the association edges based on the co-occurrence log.

5. The method for constructing and dynamically adapting a multimodal rule fusion dataset based on the QIR-KG data structure according to claim 4, characterized in that: It also includes a dynamic quality fingerprint generation step, wherein the dynamic quality fingerprint is a set of multi-dimensional quality indicators for each data record, including at least content hash, data record structure hash, quality score, source credibility, timeliness, domain weight, diversity contribution, visual quality dimension and fusion quality score; The fused quality score is calculated by weighting multiple quality dimensions in the dynamic quality fingerprint; When the hash of the data record structure changes, the version management mechanism is triggered to generate a new version record and mark the old record as obsolete and archive it according to its historical value. The visual quality dimension is obtained by quantizing and weighting at least one of the following sub-dimensions: image sharpness, lighting conditions, degree of occlusion, and frame rate stability.

6. The method for constructing and dynamically adapting a multimodal rule fusion dataset based on the QIR-KG data structure according to claim 4, characterized in that: It also includes conflict resolution and optimization steps, using a weighted scoring mechanism to resolve rule conflicts; The weighted scoring mechanism uses at least one of the following as scoring dimensions: rule priority, business level, effective time, execution cost, and result impact. For multiple rules that are applied simultaneously, calculate the overall score for each rule and apply the rule with the highest score first. When visual rules conflict with text rules, the overall score weight of the visual rules is dynamically adjusted based on the confidence level of visual detection. The weight adjustment results and user feedback during the conflict resolution process are used to optimize the macro weight configuration offline and are simultaneously updated to the dynamic quality fingerprint.

7. The method for constructing and dynamically adapting a multimodal rule fusion dataset based on the QIR-KG data structure according to claim 4, characterized in that: The cloud platform adopts a full retention strategy. At the edge, a weighted distance based on fused quality score and semantic embedding is used to select key samples; For mobile devices, samples with more coverage rule IDs and higher fusion quality scores are preferred. The symbol execution verification includes: selecting the triggering conditions and output conclusions of the rules as core verification nodes, performing hierarchical sampling verification according to rule type, and automatically adjusting the threshold or rolling back to the previous valid version when verification fails.

8. The method for constructing and dynamically adapting a multimodal rule fusion dataset based on the QIR-KG data structure according to claim 4, characterized in that: It also includes UDTP protocol encapsulation and format conversion steps, which encapsulate the dataset into a unified protocol format and convert it into the format required by the target training framework; The UDTP protocol includes a protocol header, data body, and verification information, and supports version compatibility. When a lower version client parses a higher version protocol, it ignores unknown fields, and when a higher version client reads lower version data, it uses default values ​​for missing fields. During the transformation, it supports extracting input-output pairs from data records to generate training samples.

9. The method for constructing and dynamically adapting a multimodal rule fusion dataset based on the QIR-KG data structure according to claim 4, characterized in that: It also includes quality assessment steps, exception handling steps, and global metadata management steps; The quality assessment step evaluates the dataset in at least one of the following dimensions: accuracy, coverage, consistency, and usability. Data items falling below a threshold automatically trigger an update process. The anomaly handling steps categorize anomalies into three levels: minor, moderate, and severe. Minor anomalies are filled using inter-frame interpolation, moderate anomalies are marked with low confidence and inference continues, and severe anomalies stop inference and trigger manual review. The global metadata management process stores dataset-level metadata. Each module reads its configuration from the global metadata and writes it into the execution results, enabling the dataset to evolve automatically.

10. The method for constructing and dynamically adapting a multimodal rule fusion dataset based on the QIR-KG data structure according to claim 4, characterized in that: The unified conversion of multi-source heterogeneous data includes: for image or video data, detecting visual entities and extracting location coordinates, confidence, and feature embedding information through a visual feature extraction plugin, and converting them into a "visual entity-attribute-value-timestamp" quadruple format, consistent with the quadruple format of text data; The method of generating fused data records using the QIR-KG data structure described in claim 1 includes: setting up multimodal input references in the natural language request unit, setting up multimodal task instructions in the intent and instruction unit, setting up a three-layer abstract structure encapsulation of visual rules in the rule logic expression unit, and weighted concatenation of visual entity features and text entity features in the knowledge graph embedding unit to achieve a fused vector representation of text entities and visual entities. The cross-modal entity association includes: triggering manual review when the semantic similarity is lower than a preset threshold; if the manual review response times out, the system selects entities in descending order of data source credibility to establish temporary associations, and adds a "pending manual confirmation" label to the temporary association results.

11. The method for constructing and dynamically adapting a multimodal rule fusion dataset based on the QIR-KG data structure according to claim 4, characterized in that: The main timeline alignment step includes: selecting the core data type of the target domain as the main timeline, mapping the timestamps of other data sources to the main timeline through event matching or nearest neighbor interpolation, and recording the alignment method and time offset.

12. The method for constructing and dynamically adapting a multimodal rule fusion dataset based on the QIR-KG data structure according to claim 4, characterized in that: It also includes a cold start step: when the number of labeled samples in the domain is less than a preset threshold, the system enters the cold start mode; if there are no labeled samples, a zero-shot domain feature extraction strategy is adopted, semantic analysis of the rule document is performed based on the large language model, domain features are initially extracted in combination with the data source description, and the initial rules are generated by loading the domain general rule template library through semantic similarity matching; for visual entities, the general visual language model is called to perform zero-shot recognition and match with text labels, and if there is no match, a new entity is created and marked with low confidence.

13. A multimodal rule fusion dataset construction and dynamic adaptation system based on QIR-KG data structure, characterized in that: The system is used to execute the multimodal rule fusion dataset construction and dynamic adaptation method based on the QIR-KG data structure as described in any one of claims 4 to 12, including: The module includes a domain feature self-awareness module, a multimodal data fusion module, a rule parsing and representation module, a QIR-KG dataset construction module, and a dynamic association and evolution module. And at least one of the following modules: dynamic quality fingerprint module, conflict resolution and optimization module, lightweight inference and pruning module, UDTP protocol and format conversion module, quality assessment module, exception handling module, and global metadata management module; The system enables bidirectional communication between modules through a message bus.

14. A computer-readable storage medium, characterized in that: It stores a computer program, which, when executed by a processor, implements the method for constructing and dynamically adapting a multimodal rule fusion dataset based on the QIR-KG data structure as described in any one of claims 4 to 12.

15. The computer-readable storage medium according to claim 14, characterized in that: It also stores a dataset with the QIR-KG data structure as described in claim 1, or a QIR-KG dataset constructed using the multimodal rule fusion dataset construction and dynamic adaptation method based on the QIR-KG data structure as described in claim 4.

Images