Collaborative processing and intelligent conversion method of multi-modal business data

By aligning spatiotemporal features and optimizing resources, combined with a rule-based decision-making mechanism, the problems of cross-modal data fragmentation and rule engine latency in multimodal business data processing have been solved, achieving efficient multimodal data fusion and intelligent decision-making, and improving enterprise data governance efficiency and compliance capabilities.

CN121029411BActive Publication Date: 2026-07-07SHAANXI KEXINTONG SOFT INFORMATION TECHNOLOGY CO LTD

Patent Information

Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
SHAANXI KEXINTONG SOFT INFORMATION TECHNOLOGY CO LTD
Filing Date
2025-08-21
Publication Date
2026-07-07

AI Technical Summary

Technical Problem

Traditional methods suffer from problems such as cross-modal data fragmentation, delayed rule engine updates, high error correlation rates, and poor cross-modal desensitization in multimodal business data processing, leading to rigid resource scheduling, difficulties in compliance auditing, and difficulty in meeting the needs of highly sensitive scenarios.

Method used

By coupling cross-modal temporal correlation through a spatiotemporal feature alignment engine, it detects semantic conflicts between images and text, generates RAG semantic units, constructs a four-dimensional priority label to divide the resource pool, dynamically allocates resources, and generates business decision content by combining conflict arbitration identifiers and rule nodes, thus constructing a full-link semantic thinking chain and realizing hot updates and compliance report generation.

Benefits of technology

It enables precise fusion of multimodal data and intelligent decision-making, improves data governance efficiency and compliance capabilities, reduces the risk of resource contention, supports rapid rule iteration and hot deployment, and meets the auditing needs of high compliance scenarios.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN121029411B_ABST
    Figure CN121029411B_ABST
Patent Text Reader

Abstract

The present application belongs to the technical field of multi-modal intelligent data governance, and discloses a method for collaborative processing and intelligent conversion of multi-modal business data, comprising: fusing multi-source data, coupling cross-modal time sequence association, detecting graphic-text semantic conflict, generating RAG semantic unit and space-time alignment semantic package; constructing four-dimensional priority label to divide resource pool categories, dynamically allocating resource pool computing resources, generating low-redundancy feature set, triggering forced preemption strategy when resource competition occurs, and synchronously generating resource arbitration log; injecting dynamic rule nodes according to data types, performing confidence arbitration combined with conflict arbitration identifier, generating business decision content and generating rule correction decision package; constructing a full-link semantic thinking chain, synchronously capturing system operation track, and then generating a double-channel traceability report; identifying residual sensitive fields, triggering hot update to load corresponding rules, quantifying governance effectiveness to generate a compliance report, updating the industry rule library in reverse, and optimizing the resource allocation strategy.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention relates to the field of multimodal intelligent data governance technology, and more specifically, to a method for collaborative processing and intelligent transformation of multimodal business data. Background Technology

[0002] During the intelligent transformation of enterprises, the demand for collaborative processing of multimodal business data (text, images, voice, tables) has surged. However, traditional methods face three major bottlenecks: fragmentation of cross-modal data (such as contradictions between medical images and reports), high update latency of rule engines, and lack of interpretable paths in generative models. These problems lead to rigid resource scheduling, difficulties in compliance auditing, and difficulty in meeting the needs of highly sensitive scenarios (such as finance and healthcare).

[0003] Mainstream solutions to the above problems still have some significant drawbacks: CLIP and other models only statically concatenate features and cannot capture dynamic business relationships (such as the causal relationship between stock prices and policy texts); the RAG slicing mechanism can lead to the fragmentation of the text and image context and an excessively high rate of incorrect associations; hard-coded risk control strategies require service interruption for updates and cannot achieve millisecond-level hot deployment; cross-modal desensitization is ineffective, the voice feature reconstruction rate is too high, and a data governance closed loop is missing. Summary of the Invention

[0004] To overcome the aforementioned deficiencies of the prior art and to achieve the above objectives, the present invention provides the following technical solution: a method for collaborative processing and intelligent conversion of multimodal business data, comprising:

[0005] S1: Integrate multi-source input data, couple cross-modal temporal correlation through a spatiotemporal feature alignment engine, detect semantic conflicts between images and text, generate business-adaptive RAG semantic units based on conflict detection results, embed conflict arbitration identifiers, and integrate them into a spatiotemporal aligned semantic package;

[0006] S2: Based on the spatiotemporally aligned semantic package, construct a four-dimensional priority label to classify resource pools and dynamically allocate computing resources to the resource pools; convert natural language instructions into database queries to generate a low-redundancy feature set; trigger a forced preemption strategy when resource contention occurs and synchronously generate resource arbitration logs.

[0007] S3: Inject dynamic rule nodes based on the data type of the low-redundancy feature set, perform confidence arbitration in conjunction with the conflict arbitration identifier, generate business decision content, connect rule correction records in parallel, and generate rule correction decision package;

[0008] S4: Based on the rule-correction decision package, construct a full-link semantic thinking chain, synchronously capture the system operation trajectory, and then align and generate a dual-channel traceability report;

[0009] S5: Scan the dual-channel traceability report, identify residual sensitive fields, trigger hot update to load the corresponding rules; then quantify the governance effectiveness to generate a compliance report, reverse update the industry rule base, and optimize resource allocation strategies.

[0010] Furthermore, the method of coupling cross-modal temporal correlation through the spatiotemporal feature alignment engine includes:

[0011] The original multi-source data is processed to unify its format, and the text and image features of the multi-source data are extracted.

[0012] Standardize the resolution of image features and the vector dimension of text features, and extract the timestamps of various data types for unified timestamp processing;

[0013] Furthermore, a spatiotemporal feature alignment engine is constructed to perform spatiotemporal feature alignment on text and image features of all types of data, generating spatiotemporal feature vectors.

[0014] Furthermore, the generation method of the spatiotemporal alignment semantic package includes:

[0015] Based on spatiotemporal feature vectors, the numerical deviation between images and text is evaluated, semantic conflicts are detected, and an industry standard terminology library is matched to identify the conflict type and generate a conflict arbitration identifier.

[0016] Retrieve compliance rules that match semantically conflicting content from a pre-built industry knowledge vector library;

[0017] An enhanced context is constructed by combining semantically conflicting content, compliance rules, and corresponding original multi-source data;

[0018] Based on the enhanced context, a business-adaptive RAG semantic unit is generated, and then the conflict arbitration identifier is embedded into the RAG semantic unit to generate a spatiotemporally aligned semantic package.

[0019] Furthermore, the method for dynamically allocating computing resources to the resource pool includes:

[0020] Based on spatiotemporally aligned semantic packages, computing resource priorities are allocated according to data types;

[0021] By combining the numerical deviation between images and text, the urgency of tasks based on timestamps, and the intensity of tasks' demand for computing resources, task priorities are evaluated, and four-dimensional priority labels are generated.

[0022] The resource pool is divided into a hardware resource pool and an elastic resource pool. Hardware resources are allocated based on four-dimensional priority tags, and preemptive scheduling is implemented according to preset preemption rules. The resource allocation strategy is dynamically adjusted in conjunction with the elastic resource pool to generate a resource allocation scheme.

[0023] Furthermore, the methods for synchronously generating resource arbitration logs include:

[0024] The fuzzy natural language instructions are transformed into precise database queries, and the database query results are merged with image features to generate a low-redundancy feature set.

[0025] When the resource pool load is detected to be below expectations, a forced preemption strategy is triggered:

[0026] Based on task priority, all tasks that do not meet the task priority requirements are interrupted, computing resources are allocated to other tasks, and resource arbitration logs are synchronously recorded and generated.

[0027] Furthermore, the methods for generating the business decision content include:

[0028] Match industry rule bases based on task type and data type;

[0029] Then, based on the task type, the corresponding rule class node in the industry rule library is dynamically loaded;

[0030] Apply rule nodes to the low-redundancy feature set to obtain rule matching results;

[0031] And combine the conflict arbitration identifier and resource arbitration log to conduct confidence-based arbitration;

[0032] If the arbitration result meets expectations, business decision content is generated; if the arbitration result does not meet expectations, an early warning mechanism is triggered.

[0033] Furthermore, the generation method of the rule correction decision package includes:

[0034] The system matches the business decision content with an industry standard terminology database, and then performs compliance checks on the matched business decision content.

[0035] If compliance verification fails, an automatic correction operation will be performed to generate a compliance label;

[0036] Record automatic correction operations and combine them with correction timestamps to generate rule correction records;

[0037] Integrate rule correction records and corrected business decision content marked with compliance labels to generate a rule correction decision package.

[0038] Furthermore, the generation method of the dual-channel traceability report includes:

[0039] Based on the rule correction decision package, structured reasoning steps are generated through rule correction records, and the reasoning steps are bound to the original rule correction decision records to construct a full-link semantic thinking chain;

[0040] Synchronously record the complete trajectory and timestamp of system operations as a system operation chain;

[0041] Align the semantic thinking chain with the system operation chain in time and integrate them to generate dual-channel data; perform multi-dimensional analysis based on the dual-channel data to generate a dual-channel traceability report.

[0042] Furthermore, the methods for triggering the hot update loading rule include:

[0043] Detect sensitive fields remaining in the dual-channel traceability report, identify the type of sensitive field, trigger the rule hot update mechanism, and match the compliance rules in the corresponding industry rule base;

[0044] Then, the rules engine API is invoked to load new compliant rules in real time to update business decision content and verify the update effect.

[0045] Furthermore, the methods for generating the compliance report include:

[0046] Based on the updated results, define and quantify governance effectiveness assessment indicators, integrate all effectiveness assessment indicators, and generate a compliance report.

[0047] Based on compliance reports, the industry rule base is dynamically updated, new compliance rules are added, and the matching logic of existing rules is adjusted; the allocation strategies of hardware resource pool and elastic resource pool are optimized simultaneously.

[0048] The technical effects and advantages of the collaborative processing and intelligent conversion method for multimodal business data in this invention are as follows:

[0049] This invention focuses on highly compliant scenarios such as finance and healthcare. Through five core links—dynamic alignment, resource optimization, rule-based decision-making, interpretability, and closed-loop evolution—it achieves accurate fusion of multimodal data and intelligent decision-making, significantly improving enterprise data governance efficiency and compliance capabilities.

[0050] First, by dynamically aligning the spatiotemporal data, a cross-modal semantic association chain is constructed. Based on the cross-modal attention mechanism, the temporal causal relationship between multimodal data is dynamically captured, eliminating the semantic gaps caused by single-modal processing and improving the accuracy of data fusion in complex scenarios.

[0051] Secondly, through intelligent resource scheduling strategies, task priorities are dynamically adapted, and resources are dynamically allocated based on the urgency of tasks and resource needs, prioritizing critical tasks, reducing the risk of resource contention and optimizing utilization.

[0052] Then, through a rule-driven decision-making mechanism, illegal outputs are intercepted and automatically corrected, generated content is verified in real time and automatic correction is triggered, reducing the risk of business violations and supporting rapid iteration and hot deployment of rules;

[0053] Next, through a fully auditable design, transparent operation traceability is achieved, and an explainable decision path is generated to meet the audit requirements of high compliance scenarios and improve the transparency and credibility of the model.

[0054] Finally, through a closed-loop self-evolution mechanism, the model parameters and rule base are continuously optimized, the rule base is dynamically updated based on compliance feedback, and the model is calibrated in reverse to adapt to business changes and regulatory updates, thereby reducing governance risks.

[0055] This invention, through a five-step closed loop of "dynamic alignment → resource optimization → rule-based decision-making → interpretability → closed-loop evolution," deeply integrates multi-model scheduling, RAG enhancement, rule engine, and full-link thinking chain capabilities, and is expected to become the core engine for enterprise intelligent transformation, improving the security, efficiency, and robustness of data processing. Attached Figure Description

[0056] Figure 1 This is a schematic diagram of the collaborative processing and intelligent conversion method for multimodal business data according to the present invention;

[0057] Figure 2 This is a schematic diagram of the process for detecting text-image semantic conflicts and generating RAG semantics in the collaborative processing and intelligent conversion method for multimodal business data of the present invention.

[0058] Figure 3 This is a schematic diagram of the collaborative processing and intelligent conversion system for multimodal business data according to the present invention. Detailed Implementation

[0059] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0060] Example 1

[0061] Please see Figure 1 and Figure 2 As shown in this embodiment, the collaborative processing and intelligent conversion method for multimodal business data includes:

[0062] S1: Integrate multi-source input data, couple cross-modal temporal correlation through a spatiotemporal feature alignment engine, detect semantic conflicts between images and text, generate business-adaptive RAG semantic units based on conflict detection results, embed conflict arbitration identifiers, and integrate them into a spatiotemporal aligned semantic package;

[0063] S2: Based on the spatiotemporally aligned semantic package, construct a four-dimensional priority label to classify resource pools and dynamically allocate computing resources to the resource pools; then convert natural language instructions into database queries to generate a low-redundancy feature set; when resource contention occurs, trigger a forced preemption strategy and synchronously generate resource arbitration logs.

[0064] S3: Inject dynamic rule nodes based on the data type of the low-redundancy feature set, and then combine them with the conflict arbitration identifier to perform confidence arbitration, generate business decision content, connect rule correction records in parallel, and generate rule correction decision package;

[0065] S4: Based on the rule-correction decision package, construct a full-link semantic thinking chain, synchronously capture the system operation trajectory, and then align and generate a dual-channel traceability report;

[0066] S5: Scan the dual-channel traceability report, identify residual sensitive fields, trigger hot update to load the corresponding rules; then quantify the governance effectiveness to generate a compliance report, reverse update the industry rule base, and optimize resource allocation strategies;

[0067] The raw multi-source data includes images (image scans such as PDF scans, images of paper documents)), text (raw text reports), voice (voice commands such as telephone recordings, voice memos) and structured tables (such as structured tables such as Excel and CSV).

[0068] The method of fusing raw multi-source data involves preprocessing the multi-source data, that is, unifying the format of the raw multi-source data and extracting the text and image features of the multi-source data. Specifically:

[0069] For image data preprocessing, OCR tools (such as Tesseract OCR, Google CloudVision API) can be used to recognize and extract text content from the image, and then pre-trained image models (such as ResNet-50, ViT) can be used to extract image features (such as color histogram, edge detection, object location).

[0070] For preprocessing of voice commands, speech recognition APIs (such as Google Speech-to-Text, DeepSpeech) can be used to convert speech into text features. Alternatively, spectral features (such as MFCC, Mel spectrum) or semantic features (such as through speech embedding models) can be extracted from the speech and then converted into text features.

[0071] Text preprocessing can be divided into two parts: for structured tables, key fields are extracted using regular expressions as text features (such as "factory vacancy rate" and "contract amount"); for unstructured text, word vector encoding is performed using BERT tools as text features.

[0072] The resolution of image features and the vector dimension of text features are standardized, and the timestamps of various types of data are extracted and processed uniformly to provide a basis for subsequent spatiotemporal alignment.

[0073] Specifically, regular expressions or NLP tools (such as spaCy and Dateparser) can be used to identify dates and times in text to obtain timestamps in the text; for speech and image data, timestamps in speech can be extracted after semantic-to-text conversion, while timestamps in images need to be identified through OCR.

[0074] Furthermore, a spatiotemporal feature alignment engine is constructed to perform spatiotemporal feature alignment on text and image features of all types of data, generating spatiotemporal feature vectors;

[0075] Specifically, the construction of the spatiotemporal feature alignment engine needs to be based on the pre-trained CLIP (Contrastive Language-Image Pretraining) model. The CLIP model is used to align image and text features. The CLIP model structure consists of two parts: an image encoder (such as ViT-B / 32 encoding (outputting the embedding vector of 2048-dimensional features)) and a text encoder (such as Transformer encoding (outputting the embedding vector of 512-dimensional features)). Then, a cross-modal attention mechanism is introduced on the CLIP model. The optimization objective of the attention mechanism is set to minimize the cross-modal feature mapping bias, thereby dynamically optimizing the feature weights of the image and text.

[0076] Then, by horizontally concatenating the embedding vectors of image features and text features, a spatiotemporal feature vector is generated. For example, if the image embedding vector is 512-dimensional and the text embedding vector is 512-dimensional, then the spatiotemporal feature vector is 1024-dimensional.

[0077] During the processing, anomaly handling mechanisms can be added, including removing low-quality samples (such as blurry images and meaningless text) through noise filtering methods, rotating or cropping images, and replacing text with synonyms, which can enhance the robustness of the CLIP model.

[0078] It should be noted that the purpose of this part is to transform multi-source heterogeneous data (text, images, semantics, structured tables) into a unified spatiotemporal feature vector through a spatiotemporal feature alignment engine, so as to provide a foundation for subsequent conflict detection and semantic unit generation.

[0079] Based on spatiotemporal feature vectors, the numerical deviation between images and text is evaluated, semantic conflicts are detected, and an industry standard terminology library is matched to identify the conflict type and generate a conflict arbitration identifier.

[0080] Specifically, the method for evaluating the numerical deviation between images and text is to calculate the norm of the difference between the image feature embedding vector and the text feature embedding vector in the spatiotemporal feature vector, and then divide it by the text feature embedding vector. If the numerical deviation is greater than the preset deviation threshold (which can be adjusted according to industry sensitivity), it is marked as a semantic conflict.

[0081] It should be noted that the original multi-source input data are different types of data for the same event. That is, the image feature embedding vector and the text feature embedding vector are descriptions of the same event from different perspectives. For example, the image feature embedding vector represents the area of ​​an object as 50px, while the text feature embedding vector represents the area of ​​an object as 2cm. If there is a semantic conflict, it means that the conflicting content is 50px and 2cm.

[0082] Regular expressions and NLP models (such as BERT) are used to extract conflicting content (such as 50px vs 2cm) and compare it with keywords and categories from industry standard terminology databases (such as GB standards, GB / T standards, GB / Z standards, ISO standards or IEC standards) to identify and determine the conflict type (such as inconsistent units).

[0083] Integrate conflict type (such as inconsistent units), numerical deviation value (such as 0.3), and timestamp to generate a conflict arbitration identifier;

[0084] Use FAISS or Milvus vector databases to build industry knowledge vector libraries (such as "factory area unit conversion rules"), then convert semantically conflicting content (i.e. conflict descriptions) into vectors, and retrieve compliance rules that match semantically conflicting content from pre-built industry knowledge vector libraries (such as the industry rule engine Drools) through similarity search.

[0085] By combining semantic conflict content, compliance rules, and corresponding original multi-source data, an enhanced context (such as a Prompt template, which includes conflict descriptions, rules, and business scenario constraints) is constructed.

[0086] Example of an enhanced context Prompt template:

[0087] [Conflict Description]: The image is labeled as 50px, and the text description is 2cm;

[0088] [Rule]: 1cm = 37.795px;

[0089] [Business Scenario Constraints (Tasks)]: Please generate semantic units that are adapted to the business scenario;

[0090] Based on the enhanced context, LLM (such as GPT-4) tools are used to combine rules and business scenario constraints to generate business-adaptive RAG semantic units, and then conflict arbitration identifiers are embedded into the RAG semantic units (for example, the conflict arbitration identifiers are appended to the RAG semantic units in JSON format) to generate spatiotemporally aligned semantic packages.

[0091] An example RAG semantic unit: "Object area needs verification. The current image annotation is 50px (approximately 1.32cm), which deviates from the text description of 2cm. It is recommended to correct it to 2cm."

[0092] Exemplary industry application validation:

[0093] In the medical field: Detecting unit conflicts between medical images and reports (such as "dosage unit error") and generating correction suggestions;

[0094] In the financial sector: verify the consistency between the contract amount description and the numerical value in the scanned image (e.g., "currency conversion error").

[0095] It should be noted that the purpose of this part is to generate business-adaptive RAG semantic units by detecting semantic conflicts between images and text, and to embed conflict arbitration identifiers to provide structured input for subsequent decision-making. Since this part combines vector retrieval, context enhancement and generative models, it can ensure the accuracy of conflict detection and the business adaptability of correction suggestions.

[0096] Based on spatiotemporally aligned semantic packets, computing resources are prioritized according to data type (image, text, speech). (That is, data type weights are allocated based on experience and the degree of hardware support for different types of data. For example, if the GPU can process image data well, then image data will be processed by the GPU first, while text data is more suitable for the CPU. Therefore, the data type weight of image data to the GPU can be higher than that of text data. For example, the data type weight of image data to the GPU is 0.8, and the data type weight of text data to the GPU is 0.6, which is used to guide the resource allocation strategy.)

[0097] By combining the numerical deviation between images and text (to reflect the business importance of the task), the task urgency based on timestamps (to judge the timeliness of the task based on timestamps, such as the closer to the deadline, the higher the urgency of the task; the absolute difference between the current timestamp and the task deadline can be used as the specific task urgency value to reflect the time sensitivity of the task), and the intensity of the task's demand for computing resources (i.e., the degree to which the task consumes hardware computing resources (such as the number of GPU cores and memory usage) to reflect the resource consumption characteristics of the task), the task priority is evaluated and a four-dimensional priority label is generated.

[0098] Specifically, the task priority evaluation method is based on the task type (such as text task, image task), combined with the corresponding data type weight, numerical deviation, task urgency and demand intensity, and then assigning a weight to each of these four factors according to industry needs, and then weighting and integrating them to obtain a task priority score, which is used as the four-dimensional priority label of the task.

[0099] For example, suppose in a medical scenario, there is a need to analyze emergency room images and medical records;

[0100] The task background includes Task 1: CT image analysis of emergency patients (image task);

[0101] Task 2: Text analysis of patient electronic medical records (text task);

[0102] Objective: To dynamically allocate GPU / CPU resources based on four-dimensional priority tags, prioritizing critical tasks;

[0103] Assume the weights of the four dimensions are 0.5, 0.3, 0.2, and 0.2, respectively; CT images have a data type weight of 0.8 for GPU and 0.2 for CPU; medical record text has a data type weight of 0.6 for GPU and 0.3 for CPU.

[0104] Task 1 (CT images): Data type weight 0.8 (GPU preferred), numerical deviation 0.2 (the larger the numerical deviation, the more important the task), timestamp urgency 0.9 (patient's vital signs are unstable), resource demand intensity 0.8 (high resource consumption, such as 3D reconstruction);

[0105] Therefore, the corresponding task priority is:

[0106] 0.5⋅0.8+0.3⋅0.2+0.2⋅0.9+0.2⋅0.8=0.4+0.06+0.18+0.16=0.80;

[0107] Task 2 (Medical Record Text): Data type weight 0.3 (CPU priority), numerical bias 0.7, timestamp urgency 0.5 (routine outpatients), resource demand intensity 0.4 (low resource consumption, such as text summaries);

[0108] Therefore, the corresponding task priority is:

[0109] 0.5⋅0.3+0.3⋅0.7+0.2⋅0.5+0.2⋅0.4=0.15+0.21+0.1+0.08=0.54;

[0110] Among these, CT images require GPU acceleration for processing, while medical record text is suitable for CPU processing; there are inconsistencies between the diagnostic results in CT images and medical record text (e.g., the image shows a tumor, but the medical record does not record it); emergency patients need to be prioritized, while general outpatient patients can be delayed; CT image processing requires a large amount of GPU resources, while text analysis has lower resource requirements;

[0111] The resource pool is divided into a hardware resource pool (divided by data type, such as GPU A6000 for images and vCPU 16 cores for text) and an elastic resource pool (containerization technologies such as Kubernetes are introduced to build an elastic resource pool for dynamically expanding resources, such as temporarily adding GPU cores). Hardware resources are allocated based on four-dimensional priority labels, and preemptive scheduling is implemented according to preset preemption rules (such as interrupting low-priority tasks when a high-priority task (such as a task with a priority > 0.7) enters the task (such as interrupting low-priority tasks with a priority < 0.4)). Preemptive scheduling allocates exclusive resources to critical tasks. For example, the task priority (0.98) of emergency patient CT image analysis (image task) is much higher than the priority of other tasks, so medical CT image analysis is a critical task, and the task processing exclusively uses the GPU, temporarily interrupting the resources of other low-priority tasks in the GPU. The resource allocation strategy is dynamically adjusted in conjunction with the elastic resource pool to generate a resource allocation scheme (including hardware resource allocation strategy and preemption rules).

[0112] Specifically, the resource allocation scheme is generated by dynamically adjusting the resource allocation strategy in conjunction with the elastic resource pool. The method is to monitor the utilization rate of the hardware resource pool (such as CPU / memory / GPU) in real time through tools such as Prometheus. When the resource pool load of a certain hardware exceeds the preset load threshold (such as 85%), elastic expansion is triggered to build an elastic expansion pool (such as adding vCPU (virtual central processing unit)). Then, high-priority tasks are directly allocated to the hardware resource pool, and low-priority tasks are allocated to the elastic resource pool.

[0113] For example, suppose there is task 1 (CT image analysis of emergency patients (image task)) and task 2 (text analysis of patient electronic medical records (text task)).

[0114] Task 1 has a priority of 0.80, and Task 2 has a priority of 0.54.

[0115] The resource allocation strategy is as follows:

[0116] For the hardware resource pool, Task 1 preempts the GPU resource pool to prioritize the execution of 3D reconstruction of CT-impacted data and AI-assisted diagnosis; Task 2 is allocated to the CPU resource pool to perform structured analysis of medical record texts.

[0117] For the elastic resource pool, if GPU resources are insufficient to process task 1, Kubernetes will be triggered to dynamically scale up and temporarily allocate GPU nodes to task 1.

[0118] If the GPU is idle after Task 1 is completed, upgrade Task 2 to GPU acceleration (such as NLP model fine-tuning).

[0119] The following are exemplary results of this processing: CT images of emergency patients are processed first, and doctors can obtain diagnostic results in a very short time (e.g., 5 minutes) and provide emergency treatment to patients more quickly; while medical record text analysis is delayed for a longer time (e.g., 30 minutes) and does not affect the emergency treatment of patients.

[0120] It should be noted that the purpose of this part is to construct a four-dimensional priority label to classify resource pools through spatiotemporally aligned semantic packages, and to allocate computing resources in combination with dynamic scheduling strategies, so as to provide a foundation for subsequent low-redundancy feature set generation and resource contention processing. This part integrates industry-adapted priority classification methods and elastic resource management technology, which can ensure that critical tasks are executed first, while reducing resource waste.

[0121] The fuzzy natural language instructions are transformed into precise database queries, and the database query results are merged with image features to generate a low-redundancy feature set.

[0122] Specifically, Chat2SQL tools (such as LLM-based instruction parsers) can be used to identify key entities in natural language instructions;

[0123] Example: If the input command is "What is the factory vacancy rate?", the corresponding key entities are "factory" and "vacancy rate".

[0124] After identifying key entities, the Chat2SQL tool is used to perform intent recognition on ambiguous natural language commands and generate accurate SQL based on the context.

[0125] The precise SQL query results are combined with image features (such as heat maps of vacant areas in factory layout diagrams) to generate a low-redundancy feature set.

[0126] One approach to merging the data involves compressing key fields (e.g., "vacancy rate = 35%)) using TF-IDF or BERT on the precise SQL query results of the text, then extracting core region features (e.g., "factory buildings") using principal component analysis (PCA) or edge detection on the image features, and finally integrating them together using a multimodal fusion model (e.g., ViLBERT) as a low-redundancy feature set (e.g., "factory building vacancy rate = 35%)".

[0127] When the resource pool load is detected to be below expectations, a forced preemption strategy is triggered:

[0128] Based on task priority, all tasks that do not meet the task priority requirements are interrupted, and computing resources are allocated to other tasks. It is necessary to ensure that GPU tasks are only migrated to the GPU resource pool, and to synchronously record and generate resource arbitration logs (including task ID, preemption time, and reason). The resource arbitration logs are used for subsequent resource scheduling optimization and the generation of audit reports.

[0129] Specifically, when the resource pool load exceeds a preset load threshold (e.g., 85%), it is determined that the resource pool does not meet expectations and a forced preemption strategy is triggered.

[0130] Tasks with priorities lower than the preset low priority threshold (e.g., task priority < 0.6) are judged as tasks that do not meet the task priority requirements.

[0131] After interrupting these unmet tasks in the hardware resource pool, the freed-up computing resources are allocated to tasks that meet the task priority requirements, and then the preempted tasks are migrated to the elastic resource pool.

[0132] Example: Task 1 (task priority = 0.55, non-urgent text analysis), Task 2 (task priority = 0.7, urgent image processing);

[0133] Therefore, Task 1 was terminated, computing resources were allocated to Task 2, Task 2 continued to execute, and Task 1 was migrated to the elastic resource pool for execution.

[0134] The system automatically matches industry rule bases based on task type (i.e., the domain / scenario type of the task analyzed from the original multi-source input data, such as medical / financial / industrial, etc.) and data type (e.g., "factory vacancy rate" is a financial attribute).

[0135] For example: in medical scenarios, the SNOMED-CT rule base can be loaded (e.g., "tumor location must be marked on emergency CT images"); in financial scenarios, SEC compliance rules can be loaded (e.g., "a vacancy rate > 50% requires triggering a financial statement risk warning").

[0136] Then, through the Spring Boot dynamic injection mechanism, the corresponding rule class node in the industry rule base is dynamically loaded according to the task type (for example, if the task type is medical, the medical rule class node, i.e., the medical rule engine, is loaded; if it is financial, the financial rule engine is loaded).

[0137] Apply rule nodes to low-redundancy feature sets (such as "factory vacancy rate = 35%)" to obtain rule matching results;

[0138] An exemplary way to obtain rule matching results is to perform Boolean logic judgments through the rule engine to obtain a rule matching score, which is then used as the rule matching result.

[0139] Example application scenario: In a medical setting, if the CT image rule matching score is 0.9, which is greater than the preset matching score threshold of 0.8, it is judged as a high match, and the tumor location annotation is considered to conform to SNOMED-CT.

[0140] In a financial context, a candlestick chart volatility rule matching score of 0.8 indicates a match, meaning the volatility is consistent with the financial statement risk disclosure.

[0141] And combine the conflict arbitration identifier and resource arbitration log to conduct confidence-based arbitration;

[0142] One confidence level arbitration method is as follows: assign a weight to the rule matching score (e.g., 0.9 indicates a high match), the conflict arbitration flag (blinded, e.g., 0 indicates a conflict, 1 indicates no conflict), and the resource overload record in the resource arbitration log (also binarized, e.g., 0 indicates that the resource pool load is greater than the load threshold, 1 indicates normal), and then calculate the confidence level by weighted fusion;

[0143] Example: Assume the weights of the three are 0.6, 0.3, and 0.1 respectively;

[0144] In a medical scenario: if the rule matching score for emergency CT images is 0.9, no conflict is 1, and the resource pool is normal (1), then the confidence level is 0.6 × 0.9 + 0.3 × 1 + 0.1 × 1 = 0.94.

[0145] Financial scenarios:

[0146] If the candlestick chart volatility rule matching score is 0.8, the conflict is 0, and the resource pool overload is 0, then: confidence level = 0.6 × 0.8 + 0.3 × 0 + 0.1 × 0 = 0.48;

[0147] If the arbitration result is greater than or equal to the preset confidence threshold (e.g., 0.8), it is determined that the arbitration result meets expectations, and business decision content is generated, including a low-redundancy feature set, conflict arbitration identifier, resource arbitration log, and compliance scenario content.

[0148] Among them, compliance scenarios include generating diagnostic reports (such as "CT images of an emergency patient show a tumor, further examination is recommended") in the medical field, and generating risk warnings (such as "The factory vacancy rate is 35%, which is lower than the SEC compliance threshold, and no warning is required") in the financial field.

[0149] If the arbitration result is less than the confidence threshold, it is determined that the arbitration result does not meet expectations and triggers an early warning mechanism (such as sending the corresponding scenario content to the manual review queue). For example, in the medical scenario, conflicting CT images and medical record texts are submitted to doctors for review, and in the financial scenario, conflicting K-line volatility and financial statement data are submitted to the compliance team for review.

[0150] The system matches the business decision content with an industry standard terminology database, and then performs compliance checks on the matched business decision content. If the compliance check fails, an automatic correction operation is performed to generate compliance tags (including industry compliance tags (such as medical scenario compliance) and correction status tags (such as automatic correction)).

[0151] Specifically, compliance verification compares business decision content with an industry standard terminology library. If non-standard expressions or semantic gaps are found, compliance verification will fail.

[0152] For example, "the area of ​​the object is 50px" is a non-standard unit and a non-standard expression, which will trigger a compliance check failure.

[0153] In a medical setting, if the business decision is "CT image shows tumor" but the tumor location is not labeled, it is considered semantically missing and will trigger a compliance check failure.

[0154] In financial scenarios, if a business decision states "factory vacancy rate = 35%" but the risk is not clearly disclosed, it also constitutes semantic omission and triggers a compliance verification failure.

[0155] The automatic correction operation is as follows: use a predefined unit mapping table (e.g., 50px → 2cm) to map to industry standard units (e.g., if a non-standard unit of 50px is identified, map it to the industry standard unit of 2cm). At the same time, semantic supplementation is performed on semantically missing content, that is, supplementing missing key information (e.g., adding tumor location annotations in medical scenarios).

[0156] For example, the original business decision content contained the information: "CT images show a tumor" → after correction: "CT images show a tumor in the upper lobe of the right lung";

[0157] Record the automatic correction operation and combine it with the correction timestamp to generate a rule correction record (including the original content before correction, the content after correction, the correction timestamp, and the responsible unit). The responsible unit refers to the operation object. Examples of responsible units include System_Auto_Correction (system process automatic correction) or User_Admin (administrator account).

[0158] Integrate rule correction records and corrected business decision content marked with compliance labels to generate a rule correction decision package;

[0159] If the corrected content meets industry standards, a rule correction decision package is generated; if the correction fails or there are uncovered rules, a manual intervention warning is triggered.

[0160] Exemplary application scenarios:

[0161] In a medical setting, the input for business decision-making is: "CT images of an emergency patient show a tumor."

[0162] Compliance verification: There is a semantic conflict in the conflict arbitration mark (the location of the tumor is not recorded in the medical record);

[0163] Automatic correction operations: Supplement tumor location label (e.g., "right upper lobe"); Unit standardization (e.g., 50px → 2cm);

[0164] Generate compliance label: Compliant in medical settings, automatically corrected;

[0165] Record supplementary tumor location and unit correction operations, generate rule correction records, and finally obtain the rule correction decision package;

[0166] Based on the rule correction decision package, structured reasoning steps are generated through rule correction records, and the reasoning steps are bound to the original rule correction decision records to construct a full-link semantic thinking chain;

[0167] Specifically, the rule correction records (such as "50px" → "2cm") are formatted into JSON or XML format to adapt to the input requirements of intelligent agents (such as DeepSeek-Reasoner);

[0168] Next, the agent API is called, and the formatted rule correction record is used as input to generate structured reasoning steps (such as keyword extraction → database association → risk calculation).

[0169] Exemplary real-world application scenarios:

[0170] Rule correction record: Original content: "Factory vacancy rate = 35%", revised content: "Factory vacancy rate = 35% (below the SEC compliance threshold, no warning required)";

[0171] The constructed full-link semantic thinking chain is as follows: "Keyword extraction": "Identify the missing information 'risk disclosure' in the original content" → "Database association": "Query SEC compliance rules and supplement risk warnings" → "Risk calculation": "Assess the impact of missing risk disclosure on financial statements";

[0172] The system synchronously records the complete trajectory of system operations (including operation type (e.g., speech-to-text, SQL query), operation parameters (e.g., SQL query statement, voice file path)) and timestamps, serving as the system operation chain. The semantic thinking chain is time-aligned with the system operation chain via NTP or PTP protocols to ensure minimal timestamp error (e.g., <10ms), thus achieving precise alignment. Time-aligned dual-channel data (including the semantic thinking chain and the system operation chain) is then integrated and generated. Multi-dimensional analysis is performed based on the dual-channel data to generate a dual-channel traceability report.

[0173] Multi-dimensional analysis includes compliance verification and performance analysis;

[0174] Compliance verification involves comparing the semantic thinking chain reasoning steps with industry standards (such as SEC and SNOMED-CT) to verify the rationality of the corrections.

[0175] Performance analysis involves analyzing the high-frequency, time-consuming operations in the statistical system's operation chain (such as SQL query efficiency).

[0176] Integrate semantic thinking chain, system operation chain and multi-dimensional analysis results to generate a structured audit report as a dual-channel traceability report;

[0177] The NLP model (such as BERT-Classifier) ​​is used to detect sensitive fields (such as "patient ID=1234567") remaining in the dual-channel traceability report and identify the type of sensitive field (such as the identity ID of medical examination records). If a sensitive field is detected, a rule hot update mechanism is triggered to match the compliance rules in the corresponding industry rule base (such as the privacy compliance rules in the HIPAA Act).

[0178] Then, the rule engine API (such as Drools rule engine) is called to load the new compliant rules in real time to update the business decision content, and the compliance verification is re-executed on the updated business decision content to verify the update effect, ensure that sensitive fields are de-identified (e.g., "patient ID=****"), and record the rule update event synchronously.

[0179] Based on the updated results, define and quantify governance effectiveness assessment indicators, integrate all effectiveness assessment indicators, and generate a compliance report (including the defined and quantified effectiveness assessment indicators and specific problem descriptions).

[0180] Exemplary indicators for evaluating governance effectiveness include, for example, the coverage rate of sensitive field de-identification and the compliance verification pass rate;

[0181] Then, based on the defined governance effectiveness evaluation indicators, these indicators are quantified into numerical values. For example, the ratio of the number of de-identified fields to the total number of sensitive fields is taken as the sensitive field de-identification coverage rate; the ratio of the number of compliance verifications passed to the total number of verifications is taken as the compliance verification pass rate.

[0182] Based on compliance reports, the industry rule base is dynamically updated, new compliance rules are added, and the matching logic of existing rules is adjusted; the allocation strategies of hardware resource pool and elastic resource pool are optimized simultaneously to form a closed loop of governance-feedback-optimization.

[0183] An exemplary method is to extract effectiveness evaluation indicators and specific problem descriptions from the generated compliance report, and define the key analysis directions as the types of sensitive fields that are not covered (e.g., a certain type of medical record number is not recognized due to format changes), rule matching failure cases (e.g., a file path fails to pass compliance verification due to naming rule adjustments), and rule conflicts or redundancy (e.g., multiple rules produce contradictory desensitization strategies for the same field).

[0184] The new compliance rule is as follows: Based on the validity assessment indicators, identify the types of sensitive fields that are not covered, and then add new rule definitions (such as the extended format of medical record numbers; for example, if a medical record number in the format "123456-ABC" is found to be unrecognized, add a new rule to describe its characteristics (a combination of numbers, hyphens, and uppercase letters)).

[0185] The newly defined rules will be added to the industry rule base as new rule entries, specifying the field matching pattern (e.g., regular expressions), de-identification strategy (e.g., replacing [identity ID=1234] with [identity ID=****]), and applicable scenarios (e.g., medical text, financial statements).

[0186] Adjust the existing rule matching logic as follows: make logical corrections to rules that fail to match, for example, expand the matching range: expand the "6-digit number" rule to "6-10 digits + letter combination";

[0187] Enhance semantic recognition: Supplement contextual judgment capabilities through natural language processing models (such as BERT) to avoid misjudgments;

[0188] If multiple rules conflict, increase the priority of the key rule (e.g., in a medical scenario, prioritize the medical record number rule over the general numbering rule); if rules are redundant, merge or delete inefficient rules (e.g., delete duplicate file path matching rules).

[0189] The allocation strategy for simultaneously optimizing the hardware resource pool and the elastic resource pool is as follows:

[0190] Resource allocation strategies are analyzed from compliance reports and business decisions to extract key information, such as: resource gaps for high-priority tasks (e.g., compliance verification tasks are delayed due to insufficient resources), abnormal utilization of elastic resources (e.g., the elastic resource pool fails to expand in time during peak periods), and hardware resource preemption conflicts (e.g., low-priority tasks occupy resources required by high-priority tasks).

[0191] Based on the extracted key information, optimize the hardware resource pool and the elastic resource pool;

[0192] Hardware resource pool optimization includes preemption priority adjustment and resource allocation threshold optimization;

[0193] Priority adjustment: Redefine the hardware resource preemption rules based on task urgency (e.g., compliance verification tasks have the highest priority); Example: Increase the preemption priority of "compliance verification tasks" from "medium" to "high" to ensure that it has priority in obtaining computing resources;

[0194] Resource allocation threshold optimization: Adjust the allocation threshold of hardware resources (such as triggering preemption when CPU utilization exceeds 80%) to avoid resource waste or overload;

[0195] Elastic resource pool optimization includes adjusting dynamic expansion strategies and optimizing resource reclamation mechanisms;

[0196] Dynamic scaling strategy adjustment: Based on historical load data, optimize the scaling trigger conditions for elastic resources (e.g., when the task queue length exceeds 500, scale up by 20%).

[0197] Introduce a predictive expansion mechanism to allocate resources in advance based on timestamp trends (such as peak hours from 14:00 to 16:00 daily);

[0198] Resource recycling mechanism optimization: Set recycling thresholds for elastic resources (such as automatically releasing idle resources within 30 minutes after a task is completed) to reduce operating costs;

[0199] Each time the rule base is updated and resource policies are adjusted, the compliance verification process is rerun to generate a new compliance report; the optimization effect is verified by comparing the metrics of the new and old reports (such as a 5% increase in de-identification coverage).

[0200] If the new rule causes a misjudgment (such as incorrectly de-identifying normal fields), an early warning mechanism will be triggered to suspend the rule's effect and roll back to a stable version; if the resource allocation strategy causes a performance bottleneck (such as an increase in task timeout rate), resource arbitration log analysis will be triggered immediately to adjust the preemption priority or expand elastic resources.

[0201] Incorporate successfully optimized rules and strategies into the industry knowledge base to form standardized processes (such as "Medical Data De-identification Rule Update SOP").

[0202] Link resource allocation strategies to business scenarios (such as setting up stricter compliance verification and resource protection in the financial industry).

[0203] By continuously using data-driven iteration and anomaly feedback mechanisms, the stability and adaptability of the governance process are ensured, thereby forming a complete closed loop of evaluation-optimization-verification-iteration.

[0204] Example 2

[0205] Please see Figure 3 As shown, parts not described in detail in this embodiment are described in Embodiment 1. A collaborative processing and intelligent conversion system for multimodal business data is provided, including:

[0206] Cross-modal alignment engine: It integrates multi-source input data, couples cross-modal temporal correlation through a spatiotemporal feature alignment engine, detects semantic conflicts between images and text, generates business-adaptive RAG semantic units based on the conflict detection results, embeds conflict arbitration identifiers, and integrates them into a spatiotemporal alignment semantic package;

[0207] Resource optimization and scheduling module: Based on spatiotemporal aligned semantic packets, construct four-dimensional priority labels to classify resource pools and dynamically allocate computing resources to resource pools; convert natural language commands into database queries to generate low-redundancy feature sets; trigger a forced preemption strategy when resource contention occurs and synchronously generate resource arbitration logs;

[0208] Rule Decision Verification Unit: Injects dynamic rule nodes based on the data type of the low-redundancy feature set, performs confidence arbitration in conjunction with the conflict arbitration identifier, generates business decision content, connects rule correction records in parallel, and generates a rule correction decision package;

[0209] Explainable audit trail system: Based on rule-correction decision packages, it constructs a semantic thought chain across the entire chain, synchronously captures the system operation trajectory, and then aligns and generates dual-channel traceability reports;

[0210] Closed-loop self-evolution optimization engine: Scans dual-channel traceability reports, identifies residual sensitive fields, triggers hot updates to load corresponding rules; then quantifies governance effectiveness to generate compliance reports, reverse-updates the industry rule base, and optimizes resource allocation strategies.

[0211] Example 3

[0212] This embodiment discloses an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the operation mode of the above-described collaborative processing and intelligent conversion method for multimodal service data.

[0213] Since the electronic device described in this embodiment is the electronic device used to implement the collaborative processing and intelligent conversion method for multimodal service data in the embodiments of this application, those skilled in the art can understand the specific implementation method and various variations of the electronic device in this embodiment based on the collaborative processing and intelligent conversion method for multimodal service data described in the embodiments of this application. Therefore, how the electronic device implements the method in the embodiments of this application will not be described in detail here. Any electronic device used by those skilled in the art to implement the collaborative processing and intelligent conversion method for multimodal service data in the embodiments of this application falls within the scope of protection of this application.

[0214] The above formulas are all dimensionless calculations. The formulas are derived from software simulations based on a large amount of collected data to obtain the most recent real-world results. The preset parameters and thresholds in the formulas are set by those skilled in the art according to the actual situation.

[0215] The above description is merely a preferred embodiment of the present invention, and the scope of protection of the present invention is not limited to the above embodiments. All technical solutions falling within the scope of the present invention's concept are within the scope of protection of the present invention. It should be noted that for users of ordinary technical skills, any improvements and modifications made without departing from the principles of the present invention should also be considered within the scope of protection of the present invention.

Claims

1. A method for collaborative processing and intelligent conversion of multi-modal business data, characterized in that, include: S1: Integrate multi-source input data, couple cross-modal temporal correlation through a spatiotemporal feature alignment engine, detect semantic conflicts between images and text, generate business-adaptive RAG semantic units based on conflict detection results, embed conflict arbitration identifiers, and integrate them into a spatiotemporal aligned semantic package; S2: Based on the spatiotemporally aligned semantic package, construct a four-dimensional priority label to classify resource pools and dynamically allocate computing resources to the resource pools; convert natural language instructions into database queries to generate a low-redundancy feature set; trigger a forced preemption strategy when resource contention occurs and synchronously generate resource arbitration logs. S3: Inject dynamic rule nodes based on the data type of the low-redundancy feature set, perform confidence arbitration in conjunction with the conflict arbitration identifier, generate business decision content, connect rule correction records in parallel, and generate rule correction decision package; S4: Based on the rule-correction decision package, construct a full-link semantic thinking chain, synchronously capture the system operation trajectory, and then align and generate a dual-channel traceability report; S5: Scan the dual-channel traceability report, identify residual sensitive fields, trigger hot update to load the corresponding rules; then quantify the governance effectiveness to generate a compliance report, reverse update the industry rule base, and optimize resource allocation strategies.

2. The collaborative processing and intelligent conversion method for multimodal business data according to claim 1, characterized in that, The method of coupling cross-modal temporal correlation through a spatiotemporal feature alignment engine includes: The original multi-source data is processed to unify its format, and the text and image features of the multi-source data are extracted. Standardize the resolution of image features and the vector dimension of text features, and extract the timestamps of various data types for unified timestamp processing; Furthermore, a spatiotemporal feature alignment engine is constructed to perform spatiotemporal feature alignment on text and image features of all types of data, generating spatiotemporal feature vectors.

3. The collaborative processing and intelligent conversion method for multimodal business data according to claim 2, characterized in that, The methods for generating the spatiotemporal aligned semantic package include: Based on spatiotemporal feature vectors, the numerical deviation between images and text is evaluated, semantic conflicts are detected, and an industry standard terminology library is matched to identify the conflict type and generate a conflict arbitration identifier. Retrieve compliance rules that match semantically conflicting content from a pre-built industry knowledge vector library; An enhanced context is constructed by combining semantically conflicting content, compliance rules, and corresponding original multi-source data; Based on the enhanced context, a business-adaptive RAG semantic unit is generated, and then the conflict arbitration identifier is embedded into the RAG semantic unit to generate a spatiotemporally aligned semantic package.

4. The collaborative processing and intelligent conversion method for multimodal business data according to claim 3, characterized in that, The methods for dynamically allocating computing resources to the resource pool include: Based on spatiotemporally aligned semantic packages, computing resource priorities are allocated according to data types; By combining the numerical deviation between images and text, the urgency of tasks based on timestamps, and the intensity of tasks' demand for computing resources, task priorities are evaluated, and four-dimensional priority labels are generated. The resource pool is divided into a hardware resource pool and an elastic resource pool. Hardware resources are allocated based on four-dimensional priority tags, and preemptive scheduling is implemented according to preset preemption rules. The resource allocation strategy is dynamically adjusted in conjunction with the elastic resource pool to generate a resource allocation scheme.

5. The collaborative processing and intelligent conversion method for multimodal business data according to claim 4, characterized in that, The methods for synchronously generating resource arbitration logs include: The fuzzy natural language instructions are transformed into precise database queries, and the database query results are merged with image features to generate a low-redundancy feature set. When the resource pool load is detected to be below expectations, a forced preemption strategy is triggered: Based on task priority, all tasks that do not meet the task priority requirements are interrupted, computing resources are allocated to other tasks, and resource arbitration logs are synchronously recorded and generated.

6. The collaborative processing and intelligent conversion method for multimodal business data according to claim 5, characterized in that, The methods for generating the business decision content include: Match industry rule bases based on task type and data type; Then, based on the task type, the corresponding rule class node in the industry rule library is dynamically loaded; Apply rule nodes to the low-redundancy feature set to obtain rule matching results; And combine the conflict arbitration identifier and resource arbitration log to conduct confidence-based arbitration; If the arbitration result meets expectations, business decision content is generated; if the arbitration result does not meet expectations, an early warning mechanism is triggered.

7. The collaborative processing and intelligent conversion method for multimodal business data according to claim 6, characterized in that, The generation methods of the rule correction decision package include: The system matches the business decision content with an industry standard terminology database, and then performs compliance checks on the matched business decision content. If compliance verification fails, an automatic correction operation will be performed to generate a compliance label; Record automatic correction operations and combine them with correction timestamps to generate rule correction records; Integrate rule correction records and corrected business decision content marked with compliance labels to generate a rule correction decision package.

8. The collaborative processing and intelligent conversion method for multimodal business data according to claim 7, characterized in that, The dual-channel traceability report is generated in the following ways: Based on the rule correction decision package, structured reasoning steps are generated through rule correction records, and the reasoning steps are bound to the original rule correction decision records to construct a full-link semantic thinking chain; Synchronously record the complete trajectory and timestamp of system operations as a system operation chain; Align the semantic thinking chain with the system operation chain in time and integrate them to generate dual-channel data; perform multi-dimensional analysis based on the dual-channel data to generate a dual-channel traceability report.

9. The collaborative processing and intelligent conversion method for multimodal business data according to claim 8, characterized in that, The methods for triggering the hot update loading rules include: Detect sensitive fields remaining in the dual-channel traceability report, identify the type of sensitive field, trigger the rule hot update mechanism, and match the compliance rules in the corresponding industry rule base; Then, the rules engine API is invoked to load new compliant rules in real time to update business decision content and verify the update effect.

10. The collaborative processing and intelligent conversion method for multimodal business data according to claim 9, characterized in that, The compliance report is generated in the following ways: Based on the updated results, define and quantify governance effectiveness assessment indicators, integrate all effectiveness assessment indicators, and generate a compliance report. Based on compliance reports, the industry rule base is dynamically updated, new compliance rules are added, and the matching logic of existing rules is adjusted; the allocation strategies of hardware resource pool and elastic resource pool are optimized simultaneously.