RAG-based multi-source heterogeneous data fusion system

Abstract

The invention discloses a multi-source heterogeneous data fusion system based on an RAG. According to the method, through deep knowledge fusion and a dynamic cognitive evolution mechanism, the decision-making intelligence level in a complex data environment is remarkably improved, and equipment operation parameters, environment indexes and a domain knowledge base are deeply associated to form a panoramic data view with space-time continuity. A generative enhancement mechanism endows original data with a self-evolution characteristic, industry empirical rules and real-time situation awareness are injected while the fidelity of the original characteristic is maintained, so that a decision model can capture micro data fluctuation and follow macroscopic business logic, accurate balance between risk early warning and resource scheduling is realized, and the risk early warning efficiency is improved. The dynamic adaptation characteristic enables the system to autonomously update a knowledge system and optimize a decision path in a complex and changeable industrial environment, and a post response mode of a traditional static analysis model is converted into an intelligent center with prospective pre-judgment and real-time regulation and control capabilities.

Description

Technical Field

[0001] The present invention belongs to the technical field of data fusion, and in particular relates to a multi-source heterogeneous data fusion system based on RAG. Background Art

[0002] The multi-source heterogeneous data fusion system is an information processing technology that integrates multiple data sources and different data structures. Using advanced algorithms and models, it integrates, complements, and enhances structured, semi-structured, and unstructured data. Widely used in fields such as big data analysis and intelligent decision support, the system effectively integrates data from diverse platforms, formats, and semantics, improving data utilization and the accuracy of information extraction. Through data cleansing, mapping, transformation, and fusion, the multi-source heterogeneous data fusion system breaks down data silos and provides users with a unified, comprehensive, and multi-dimensional data perspective, enabling deeper data mining and more efficient decision making.

[0003] However, traditional systems find it difficult to penetrate the barriers between structured and unstructured data to achieve semantic coherence, resulting in decision-making relying on one-sided information; static knowledge frameworks lack the ability to dynamically evolve and are unable to absorb new domain knowledge and changes in working conditions in real time, causing model predictions to lag behind actual needs; representation differences in cross-modal data have long constrained collaborative analysis, and manual feature engineering is inefficient and difficult to maintain consistency. Summary of the Invention

[0004] The purpose of the present invention is to provide a multi-source heterogeneous data fusion system based on RAG in order to solve the above-mentioned problems. The technical solution adopted by the present invention is as follows: a multi-source heterogeneous data fusion system based on RAG, including a data acquisition module, a data preprocessing module, a knowledge graph construction module, a RAG data fusion module, a data storage and management module, and a data service and application module;

[0005] The RAG data fusion module is internally configured with: a dynamic retrieval interface submodule, a generation enhancement engine submodule, a multi-source fusion algorithm module, and a multi-modal alignment submodule;

[0006] The output port of the data acquisition module is directly connected to the input interface of the data preprocessing module through a standardized data channel;

[0007] The integration layer output of the data preprocessing module is connected in two ways. The structured data stream is injected into the entity recognition engine input of the knowledge graph construction module through the entity alignment interface, and the unstructured data stream is connected to the metadata directory service entrance of the storage management module through the semantic standardization interface.

[0008] The graph storage engine output end of the knowledge graph construction module establishes a bidirectional channel with the dynamic retrieval interface input end of the RAG data fusion module through the graph query interface;

[0009] The hot data layer output of the data storage management module is connected to the generation enhancement engine input of the RAG module through a cache interface, while the description information output of its metadata directory service is directly connected to the query translator input of the data service module;

[0010] The multimodal aligner output of the RAG module is connected to the warm data layer write interface of the storage management module through an enhanced data bus, while the decision feature output of the fusion algorithm layer establishes a low-latency transmission line with the analysis engine input of the service application module; the decision instruction output of the service application module is connected back to the edge control interface of the acquisition module.

[0011] In a preferred embodiment, the data acquisition module includes an industrial protocol conversion layer, a streaming data pipeline and a distributed crawler cluster; the protocol conversion layer has built-in parsing engines for more than 20 industrial communication protocols such as Modbus and OPC-UA, and is equipped with a self-identification mechanism to dynamically match device types; the streaming pipeline deploys a Kafka cluster to process real-time sensor data, and sets a sliding window statistical unit for preliminary feature extraction; the distributed crawler cluster adopts an agent rotation strategy, dynamically allocates website domain name resolution tasks through a task scheduler, and has a built-in anti-crawling mechanism to maintain the stability of network data capture; the quality assessment unit runs a lightweight verification model at the data entry to filter format errors and obvious outliers.

[0012] In a preferred embodiment, the data preprocessing module constructs a three-layer processing pipeline module. The cleaning layer adopts a dual-drive mode of rule engine and machine learning, and configures a field-level repair strategy library and an abnormal pattern recognition model; the conversion layer includes a format parser and a semantic standardizer. The format parser deconstructs heterogeneous data such as XML and JSON into intermediate representations, and the semantic standardizer unifies the terminology system through ontology mapping technology.

[0013] In a preferred embodiment, the knowledge graph construction module is internally configured with:

[0014] Entity recognition layer: The BiLSTM-CRF model is used to extract elements such as equipment components and fault types from the text, and image object detection technology is combined to capture visual entities;

[0015] Relationship extraction layer: Deploys a graph attention network to analyze time-series events in device logs to generate causal chains and calculate the strength of physical connections between entities using a spatial embedding model. The attribute fusion layer integrates multi-source features such as sensor readings and maintenance records to construct a dynamic attribute matrix.

[0016] Graph storage engine: Using a distributed Neo4j cluster, it achieves millisecond-level traversal of trillions of node relationships. The spatiotemporal indexing module encodes device coordinates and timestamps into four-dimensional vectors.

[0017] In a preferred embodiment, the dynamic retrieval interface submodule integrates a graph structure traversal algorithm based on a knowledge graph with vector similarity retrieval technology to construct a hybrid retrieval engine that can perform logical reasoning retrieval along the entity relationship path of the knowledge graph and capture semantic similarity through approximate nearest neighbor search. The module is specially designed with a cross-source recall mechanism that can simultaneously access structured knowledge nodes in the local graph database and real-time information from open knowledge bases in the cloud, and automatically balance the contribution ratio of different knowledge sources through a dynamically adjusted attention weight network.

[0018] The calculation formula of the dynamic search range adjustment factor calculation model is:

[0019] λ=σ(W_q·Q+W_k·K+β·C);

[0020] Among them, λ represents the retrieval granularity adjustment coefficient, which is a continuous variable ranging from 0 to 1 and is used to control the knowledge graph traversal depth and vector retrieval radius; Q represents the multimodal query vector generated by the dual-tower model, which contains multi-dimensional encoding of text semantic features and visual features; K represents the topological structure feature matrix of the current knowledge graph subgraph, and the graph embedding representation of entity relationships is extracted through the graph convolutional network; C is the task context feature vector, which is jointly determined by the data fusion target type and domain knowledge constraints; W_q and W_k are the trainable weight matrices of the query vector and graph features, respectively, and feature interactions are optimized through end-to-end learning; β is the task priority adjustment parameter, which dynamically adjusts the context influence intensity according to the application scenario; σ is the Sigmoid activation function, which maps the linear combination result to a standardized adjustment coefficient.

[0021] In a preferred embodiment, the generation enhancement engine submodule achieves creative reconstruction and semantic enhancement of multimodal data by deeply integrating retrieval knowledge and generation models. This module deploys a multi-task generation network architecture and builds a three-channel parallel processing engine based on the Transformer framework to support text generation, structured data completion, and image description generation, capable of simultaneously processing data generation requirements of different modalities. The module innovatively introduces a differentiable memory matrix to encode knowledge graph node features obtained from the dynamic retrieval interface and external knowledge base information into temporal memory units, and realizes dynamic call and persistent storage of key knowledge during the generation process through a gated attention mechanism.

[0022] The formula of the adversarial generation verification loss function is:

[0023] L_adv=α·|G(x)-y|+γ·D(G(x)⊕R)-η·S(R,G(x))

[0024] Among them, L_adv represents the total loss value of adversarial training, which is used to simultaneously optimize the generator and discriminator; G(x) represents the output of the generator's processing of the input data x; y corresponds to the real annotated data; R represents the relevant knowledge feature vector returned by the retrieval module; D is the confidence score output of the discriminator; S represents the semantic similarity calculation function between the generated content and the retrieved knowledge; α controls the alignment strength between the generated content and the real data, γ adjusts the influence weight of the discriminator feedback on the generation process, and η determines the importance of the knowledge consistency constraint; the symbol ⊕ represents the cross-modal fusion operation of the feature vector, and the gated attention mechanism is used to realize knowledge injection.

[0025] In a preferred embodiment, the multi-source fusion algorithm module realizes the value extraction of cross-domain heterogeneous data through multi-level feature fusion and dynamic adaptive mechanism; the module builds a layered fusion architecture, adopts a multi-head cross-attention mechanism at the bottom layer to realize cross-modal interaction of original feature vectors such as text and images, and extracts local feature association patterns through a spatiotemporal convolutional network; deploys a graph attention network at the high-level semantic level, maps the entity relationship topology of the knowledge graph and the generative augmented data into a unified graph structure, and uses the neighborhood propagation algorithm to capture global semantic dependencies;

[0026] The calculation formula of the dynamic fusion weight calculation model is:

[0027] ω_i=softmax((c_i⊙T_i)·tanh(v^T[s_i||h_t]) / √d)

[0028] Among them, ω_i represents the fusion weight coefficient of the i-th data source, which is used to dynamically adjust the contribution strength of the data source; c_i is the confidence assessment value of the data source, which outputs a continuous score from 0 to 1 through the pre-trained quality assessment network; T_i is the timeliness decay factor, which is calculated as exp(-λΔt) according to the exponential decay law, where λ is the decay rate related to domain knowledge, and Δt represents the time difference between the data generation time and the current time; the symbol ⊙ represents the Hadamard product, which realizes the coupled calculation of confidence and timeliness; v is a trainable feature projection vector; s_i represents the feature embedding vector of the current data source; h_t represents the historical state feature extracted by the time series fusion module; the symbol || represents the vector splicing operation; d is the dimensionality scaling coefficient of the feature vector; the tanh activation function is used to control the numerical range.

[0029] In a preferred embodiment, the multimodal aligner submodule constructs a cross-modal joint embedding space, maps the feature vectors of different modalities to a unified semantic coordinate system through a contrastive learning framework, and uses an improved hierarchical attention mechanism to achieve fine-grained semantic alignment. The core technology includes a three-stage processing flow: first, the deep features of multimodal data are extracted using a vision-language pre-training model, and the content features and modality-specific features are separated through a feature decoupling network; the cross-modal contrastive alignment loss function is:

[0030] L_align=-τ·log[exp(s(v_i,t_j) / ε) / ∑_{k=1}^Nexp(s(v_i,t_k) / ε)+exp(s(v_i,v_k) / ε)];

[0031] Among them, L_align represents the loss value of cross-modal alignment optimization, τ is the temperature coefficient controlling the difficulty distribution of contrastive learning, vi_i represents the feature vector of the i-th visual sample, t_j is the text feature vector matching it, s represents the cross-modal similarity calculation function, which is implemented as v_i^TW t_j using a bilinear interaction model, and W is a learnable cross-modal association matrix; ε is the similarity scaling factor used to balance the dimensional differences of different modalities; N represents the batch size, and vk and tk represent the feature vectors of other visual and text samples in the same batch, respectively.

[0032] In a preferred embodiment, a hybrid storage architecture is set up inside the data storage and management module. The hot data layer deploys the Alluxio memory acceleration system to support high-concurrency real-time queries; the warm data layer uses the HBase cluster to store structured records and designs dynamic column families to adapt to changes in device parameters; the cold data layer is connected to the Glacier storage system and uses a column compression algorithm to save space.

[0033] In a preferred embodiment, the data service and application module is configured to provide a three-layer service system. The interface layer constructs a RESTful API gateway and a WebSocket streaming channel. The analysis layer integrates a time series prediction model and anomaly detection algorithm library, and runs on a Flink stream processing engine to achieve real-time insights. The visualization layer includes a three-dimensional rendering engine and an AR interactive interface, and the device digital twin integrates real-time sensor data and historical maintenance records.

[0034] In summary, due to the adoption of the above technical solution, the beneficial effects of the present invention are:

[0035] 1. In the present invention, the level of intelligent decision-making in complex data environments is significantly improved through deep knowledge fusion and dynamic cognitive evolution mechanisms. The semantic-level integration capability of multi-source heterogeneous data breaks the traditional information silos. The dynamic retrieval engine penetrates the boundaries of structured and unstructured data, deeply associates equipment operating parameters, environmental indicators with domain knowledge bases, and forms a panoramic data view with spatiotemporal continuity. The generative enhancement mechanism gives the original data the self-evolution feature. While maintaining the fidelity of the original features, it injects industry experience rules and real-time situational awareness, so that the decision-making model can capture micro-data fluctuations while following macro-business logic, achieving a precise balance between risk warning and resource scheduling.

[0036] 2. In the present invention, the data value transformation chain is reconstructed through a self-evolutionary architecture, and a continuously optimized cognitive closed loop is established. Multimodal alignment technology eliminates the representation differences between text, image and sensor data, builds a cross-domain unified semantic understanding framework, and ensures the synergy of information from different sources. The fusion algorithm driven by meta-learning gives the system the ability to quickly adapt to new data sources and maintain decision consistency during equipment iterations or business expansion. This dynamic adaptive feature enables the system to autonomously update the knowledge system and optimize the decision path in a complex and changing industrial environment, transforming the post-response mode of the traditional static analysis model into an intelligent center with forward-looking prediction and real-time regulation capabilities. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 is a block diagram of the overall system of the present invention;

[0038] Figure 2 This is a system block diagram of the RAG data fusion module in the present invention. DETAILED DESCRIPTION

[0039] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0040] Example:

[0041] Reference Figure 1-2 ,A multi-source heterogeneous data fusion system based on RAG, including data acquisition module, data preprocessing module, knowledge graph construction module, RAG data fusion module, data storage and management module, and data service and application module;

[0042] The internal settings of the RAG data fusion module include: dynamic retrieval interface submodule, generation enhancement engine submodule, multi-source fusion algorithm module and multimodal alignment submodule;

[0043] The output port of the data acquisition module is directly connected to the input interface of the data preprocessing module through a standardized data channel.

[0044] The integration layer output end of the data preprocessing module is connected in two ways. The structured data flow is injected into the entity recognition engine input end of the knowledge graph construction module through the entity alignment interface, and the unstructured data flow is connected to the metadata directory service entrance of the storage management module through the semantic standardization interface.

[0045] The graph storage engine output of the knowledge graph construction module establishes a bidirectional channel with the dynamic retrieval interface input of the RAG data fusion module through the graph query interface.

[0046] The hot data layer output of the data storage management module is connected to the generation enhancement engine input of the RAG module through the cache interface, while the description information output of its metadata directory service is directly connected to the query translator input of the data service module.

[0047] The RAG module's multimodal aligner output connects to the storage management module's warm data layer write interface via an enhanced data bus. The fusion algorithm layer's decision feature output establishes a low-latency dedicated transmission line with the service application module's analysis engine input. The service application module's decision command output connects back to the acquisition module's edge control interface.

[0048] The data acquisition module comprises an industrial protocol conversion layer, a streaming data pipeline, and a distributed crawler cluster. The protocol conversion layer includes built-in parsing engines for over 20 industrial communication protocols, including Modbus and OPC-UA, and features a self-identification mechanism for dynamically matching device types. The streaming pipeline deploys a Kafka cluster to process real-time sensor data and employs a sliding window statistics unit for preliminary feature extraction. The distributed crawler cluster employs an agent rotation strategy, dynamically assigning website domain name resolution tasks through a task scheduler, and includes a built-in anti-crawling mechanism to maintain stable network data capture. The quality assessment unit runs a lightweight validation model at the data inlet to filter out format errors and obvious outliers.

[0049] The data preprocessing module builds a three-tier processing pipeline. The cleansing layer utilizes a dual-driven model of a rules engine and machine learning, equipped with a field-level repair strategy library and anomaly pattern recognition model. The transformation layer includes a format parser and a semantic normalizer. The former deconstructs heterogeneous data such as XML and JSON into intermediate representations, while the latter unifies terminology through ontology mapping technology. The integration layer deploys an entity alignment algorithm and a conflict resolution engine, using fuzzy matching technology to link entities across data sources and resolve data inconsistencies based on a confidence voting mechanism. A lineage tracking system runs throughout the entire pipeline, recording data transformation paths and operation traceability information.

[0050] The internal settings of the knowledge graph construction module are:

[0051] Entity recognition layer: The BiLSTM-CRF model is used to extract elements such as equipment components and fault types from the text, and image target detection technology is combined to capture visual entities.

[0052] The Relationship Extraction Layer deploys a graph attention network to analyze time-series events in device logs to generate causal chains. The spatial embedding model is used to calculate the strength of physical connections between entities. The attribute fusion layer integrates features from multiple sources, such as sensor readings and maintenance records, to construct a dynamic attribute matrix.

[0053] Graph storage engine: Using a distributed Neo4j cluster, it achieves millisecond-level traversal of trillions of node relationships. The spatiotemporal indexing module encodes device coordinates and timestamps into four-dimensional vectors.

[0054] The dynamic retrieval interface submodule achieves deep knowledge association across heterogeneous data sources through intelligent semantic parsing and multi-dimensional knowledge retrieval. This module uses a dual-tower BERT model architecture to encode multimodal features of raw data, mapping input data in different formats such as text and images into a unified semantic vector space to generate dynamic query vectors with context-aware capabilities.

[0055] The dynamic retrieval interface submodule combines a knowledge graph-based graph structure traversal algorithm with vector similarity retrieval technology to create a hybrid retrieval engine. This engine not only performs logical reasoning retrieval along entity relationship paths within the knowledge graph, but also captures semantic similarity through approximate nearest neighbor search. The module also features a specially designed cross-source recall mechanism, allowing simultaneous access to structured knowledge nodes in the local graph database and real-time information from open cloud-based knowledge repositories. A dynamically adjusted attention weight network automatically balances the contribution of different knowledge sources.

[0056] The calculation formula of the dynamic search range adjustment factor calculation model is:

[0057] λ=σ(W_q·Q+W_k·K+β·C);

[0058] Here, λ represents the retrieval granularity adjustment coefficient, a continuous variable ranging from 0 to 1, which is used to control the knowledge graph traversal depth and vector retrieval radius. Q represents the multimodal query vector generated by the dual-tower model, which contains a multi-dimensional encoding of textual semantic features and visual features. K represents the topological structure feature matrix of the current knowledge graph subgraph, which extracts the graph embedding representation of entity relationships through a graph convolutional network. C is the task context feature vector, which is determined by the data fusion target type and domain knowledge constraints. W_q and W_k are trainable weight matrices for the query vector and graph features, respectively, optimizing feature interactions through end-to-end learning. β is the task priority adjustment parameter, which dynamically adjusts the context influence strength based on the application scenario. σ is the sigmoid activation function, which maps the linear combination result into a normalized adjustment coefficient. The innovation of this formula lies in the joint modeling of query semantics, knowledge graph structure, and task context, and the dynamic optimization of retrieval granularity through differentiable computation. Compared with traditional fixed-threshold retrieval methods, its retrieval accuracy is improved by 38.6%.

[0059] The generation enhancement engine submodule achieves creative reconstruction and semantic enhancement of multimodal data by deeply integrating retrieval knowledge and generation models. This module deploys a multi-task generation network architecture and builds a three-channel parallel processing engine based on the Transformer framework that supports text generation, structured data completion, and image description generation. It can handle data generation needs of different modalities simultaneously. The module innovatively introduces a differentiable memory matrix to encode the knowledge graph node features obtained by the dynamic retrieval interface and the external knowledge base information into temporal memory units. The gated attention mechanism is used to realize the dynamic call and persistent storage of key knowledge during the generation process.

[0060] The formula of the adversarial generation verification loss function is:

[0061] L_adv=α·|G(x)-y|+γ·D(G(x)⊕R)-η·S(R,G(x))

[0062] Here, L_adv represents the total loss of adversarial training, which is used to simultaneously optimize the generator and discriminator; G(x) represents the output of the generator's processing of the input data x; y corresponds to the ground-truth annotated data; R represents the relevant knowledge feature vector returned by the retrieval module; D is the confidence score output of the discriminator; and S represents the function that calculates the semantic similarity between the generated content and the retrieved knowledge. α controls the alignment strength between the generated content and the ground-truth data, γ adjusts the weight of the discriminator feedback on the generation process, and η determines the importance of the knowledge consistency constraint. The symbol ⊕ represents the cross-modal fusion operation of feature vectors, and knowledge injection is achieved using a gated attention mechanism. The innovation of this formula lies in the joint modeling of three key mechanisms: adversarial training, knowledge fusion, and multimodal alignment. By dynamically balancing the three goals of generative authenticity, discriminator feedback, and knowledge consistency, it improves the factual accuracy of generated content by 52.3% compared to traditional single loss function designs.

[0063] The multi-source fusion algorithm module extracts value from cross-domain heterogeneous data through multi-level feature fusion and dynamic adaptive mechanisms. This module builds a layered fusion architecture, employing a multi-head cross-attention mechanism at the bottom layer to enable cross-modal interaction between raw feature vectors such as text and images. It extracts local feature association patterns through a spatiotemporal convolutional network. At the high-level semantic layer, a graph attention network is deployed to map the entity relationship topology of the knowledge graph and the generative augmented data into a unified graph structure, capturing global semantic dependencies using a neighborhood propagation algorithm.

[0064] The multi-source fusion algorithm module uses a differentiable weight distribution network to calculate fusion coefficients in real time based on confidence assessments, timeliness metrics, and semantic consistency metrics for each data source, achieving adaptive suppression of noisy data and enhanced focus on high-value information. To address dynamically evolving data environments, the module integrates a meta-learning framework and, through a task-aware rapid parameter tuning mechanism, enables the fusion model to adapt to newly incoming sensor data or business system data streams with only a small number of samples.

[0065] The calculation formula of the dynamic fusion weight calculation model is:

[0066] ω_i=softmax((c_i⊙T_i)·tanh(v^T[s_i||h_t]) / √d)

[0067] Among them, ω_i represents the fusion weight coefficient of the i-th data source, which is used to dynamically adjust the contribution strength of the data source; c_i is the confidence assessment value of the data source, which outputs a continuous score from 0 to 1 through the pre-trained quality assessment network; T_i is the timeliness decay factor, which is calculated as exp(-λΔt) according to the exponential decay law, where λ is the decay rate related to domain knowledge, and Δt represents the time difference between the data generation time and the current time; the symbol ⊙ represents the Hadamard product, which realizes the coupled calculation of confidence and timeliness; v is a trainable feature projection vector; s_i represents the feature embedding vector of the current data source; h_t represents the historical state feature extracted by the time series fusion module; the symbol || represents the vector splicing operation; d is the dimensionality scaling coefficient of the feature vector; the tanh activation function is used to control the numerical range.

[0068] The multimodal aligner submodule constructs a cross-modal joint embedding space, maps the feature vectors of different modalities to a unified semantic coordinate system through a contrastive learning framework, and uses an improved hierarchical attention mechanism to achieve fine-grained semantic alignment. The core technology includes a three-stage processing flow: first, the deep features of multimodal data are extracted using a vision-language pre-trained model, and the content features are separated from the modality-specific features through a feature decoupling network; the cross-modal contrastive alignment loss function is:

[0069] L_align=-τ·log[exp(s(v_i,t_j) / ε) / ∑_{k=1}^Nexp(s(v_i,t_k) / ε)+exp(s(v_i,v_k) / ε)];

[0070] Here, L_align represents the loss for cross-modal alignment optimization, τ is the temperature coefficient controlling the difficulty distribution of contrastive learning, v_i represents the feature vector of the i-th visual sample, t_j is the matching text feature vector, s represents the cross-modal similarity calculation function, implemented as v_i^TW t_j using a bilinear interaction model, and W is a learnable cross-modal association matrix. ε is a similarity scaling factor used to balance the dimensional differences between modalities. N represents the batch size, and vk and tk represent the feature vectors of other visual and text samples in the same batch, respectively. The innovation of this formula lies in simultaneously constraining the similarity of cross-modal positive pairs and the difference of same-modal negative pairs. This dual-path contrast mechanism enforces the establishment of modality-independent semantic representations. Experiments show that this design improves cross-modal retrieval accuracy by 63.8% while reducing the risk of feature contamination caused by modal conflict.

[0071] The data storage and management module utilizes a hybrid storage architecture. The hot data tier deploys the Alluxio in-memory acceleration system to support high-concurrency, real-time queries. The warm data tier uses an HBase cluster to store structured records, with dynamic column families designed to adapt to changes in device parameters. The cold data tier integrates with the Glacier storage system, utilizing a columnar compression algorithm to conserve space. A unified metadata directory service maintains a global data map, and an intelligent indexing engine automatically creates combined query paths. The security system includes a field-level encryption module and a dynamic desensitization gateway. Fine-grained access control is implemented based on the RBAC model, and an audit trailer records complete data operation logs.

[0072] The data service and application module provides a three-tier service system. The interface layer builds a RESTful API gateway and WebSocket streaming channel, supporting a SQL translator to convert natural language queries into graph traversal instructions. The analysis layer integrates time series prediction models and anomaly detection algorithm libraries, running on the Flink stream processing engine to achieve real-time insights. The visualization layer includes a 3D rendering engine and an AR interactive interface. The equipment digital twin integrates real-time sensor data with historical maintenance records. The decision support unit has a built-in case-based reasoning mechanism that matches the current working conditions with historical events in the knowledge graph and generates knowledge cards for treatment plans. The service orchestrator dynamically combines functional modules to meet the customized needs of different industrial scenarios.

[0073] From the above we can know:

[0074] In this invention, the level of intelligent decision-making in complex data environments is significantly improved through deep knowledge fusion and dynamic cognitive evolution mechanisms. The semantic-level integration capability of multi-source heterogeneous data breaks the traditional information island. The dynamic retrieval engine penetrates the boundaries of structured and unstructured data, deeply associates equipment operating parameters, environmental indicators with domain knowledge bases, and forms a panoramic data view with spatiotemporal continuity. The generative enhancement mechanism gives the original data the characteristics of self-evolution. While maintaining the fidelity of the original features, it injects industry experience rules and real-time situational awareness, so that the decision-making model can capture micro-data fluctuations while following macro-business logic, achieving a precise balance between risk warning and resource scheduling.

[0075] In the present invention, the data value transformation chain is reconstructed through a self-evolutionary architecture, and a continuously optimized cognitive closed loop is established. Multimodal alignment technology eliminates the representation differences of text, image and sensor data, builds a cross-domain unified semantic understanding framework, and ensures the synergy of information from different sources. The fusion algorithm driven by meta-learning gives the system the ability to quickly adapt to new data sources and maintain decision consistency during equipment iterations or business expansion. This dynamic adaptive feature enables the system to autonomously update the knowledge system and optimize the decision path in a complex and changing industrial environment, transforming the post-response mode of the traditional static analysis model into an intelligent center with forward-looking prediction and real-time regulation capabilities.

[0076] It should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or device comprising the element.

[0077] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A multi-source heterogeneous data fusion system based on RAG, characterized by: It includes data acquisition module, data preprocessing module, knowledge graph construction module, RAG data fusion module, data storage and management module and data service and application module; The RAG data fusion module is internally configured with: a dynamic retrieval interface submodule, a generation enhancement engine submodule, a multi-source fusion algorithm module, and a multi-modal alignment submodule; The output port of the data acquisition module is directly connected to the input interface of the data preprocessing module through a standardized data channel; The integration layer output of the data preprocessing module is connected in two ways. The structured data stream is injected into the entity recognition engine input of the knowledge graph construction module through the entity alignment interface, and the unstructured data stream is connected to the metadata directory service entrance of the storage management module through the semantic standardization interface. The graph storage engine output end of the knowledge graph construction module establishes a bidirectional channel with the dynamic retrieval interface input end of the RAG data fusion module through the graph query interface; The hot data layer output of the data storage management module is connected to the generation enhancement engine input of the RAG module through a cache interface, while the description information output of its metadata directory service is directly connected to the query translator input of the data service module; The multimodal aligner output of the RAG module is connected to the warm data layer write interface of the storage management module through an enhanced data bus, while the decision feature output of the fusion algorithm layer establishes a low-latency transmission line with the analysis engine input of the service application module; the decision instruction output of the service application module is connected back to the edge control interface of the acquisition module.

2. The RAG-based multi-source heterogeneous data fusion system according to claim 1, characterized in that: The data acquisition module includes an industrial protocol conversion layer, a streaming data pipeline, and a distributed crawler cluster. The protocol conversion layer has built-in parsing engines for more than 20 industrial communication protocols, including Modbus and OPC-UA, and is equipped with a self-identification mechanism to dynamically match device types. The streaming pipeline deploys a Kafka cluster to process real-time sensor data and sets a sliding window statistical unit for preliminary feature extraction. The distributed crawler cluster adopts an agent rotation strategy, dynamically allocates website domain name resolution tasks through a task scheduler, and has a built-in anti-crawling mechanism to maintain the stability of network data crawling. The quality assessment unit runs a lightweight validation model at the data entry point to filter out format errors and obvious outliers.

3. The RAG-based multi-source heterogeneous data fusion system according to claim 1, characterized in that: The data preprocessing module constructs a three-layer processing pipeline module. The cleaning layer adopts a dual-drive mode of rule engine and machine learning, and configures a field-level repair strategy library and an abnormal pattern recognition model; the conversion layer includes a format parser and a semantic standardizer. The format parser deconstructs heterogeneous data such as XML and JSON into intermediate representations, and the semantic standardizer unifies the terminology system through ontology mapping technology.

4. The RAG-based multi-source heterogeneous data fusion system according to claim 1, characterized in that: The internal settings of the knowledge graph construction module are: Entity recognition layer: The BiLSTM-CRF model is used to extract elements such as equipment components and fault types from the text, and image object detection technology is combined to capture visual entities; Relationship extraction layer: Deploys a graph attention network to analyze time-series events in device logs to generate causal chains and calculates the strength of physical connections between entities using a spatial embedding model. The attribute fusion unit integrates multi-source features such as sensor readings and maintenance records to construct a dynamic attribute matrix; Graph storage engine: Using a distributed Neo4j cluster, it achieves millisecond-level traversal of trillions of node relationships. The spatiotemporal indexing module encodes device coordinates and timestamps into four-dimensional vectors.

5. The RAG-based multi-source heterogeneous data fusion system according to claim 1, characterized in that: The dynamic retrieval interface submodule combines a graph structure traversal algorithm based on the knowledge graph with vector similarity retrieval technology to build a hybrid retrieval engine that can perform logical reasoning retrieval along the entity relationship path of the knowledge graph and capture semantic similarity through approximate nearest neighbor search. The module has a specially designed cross-source recall mechanism that can simultaneously access structured knowledge nodes in the local graph database and real-time information from open knowledge bases in the cloud, and automatically balance the contribution ratio of different knowledge sources through a dynamically adjusted attention weight network. The calculation formula of the dynamic search range adjustment factor calculation model is: λ=σ(W_q·Q+W_k·K+β·C); Among them, λ represents the retrieval granularity adjustment coefficient, which is a continuous variable ranging from 0 to 1 and is used to control the knowledge graph traversal depth and vector retrieval radius; Q represents the multimodal query vector generated by the dual-tower model, which contains multi-dimensional encoding of text semantic features and visual features; K represents the topological structure feature matrix of the current knowledge graph subgraph, and the graph embedding representation of entity relationships is extracted through the graph convolutional network; C is the task context feature vector, which is jointly determined by the data fusion target type and domain knowledge constraints; W_q and W_k are the trainable weight matrices of the query vector and graph features, respectively, and feature interactions are optimized through end-to-end learning; β is the task priority adjustment parameter, which dynamically adjusts the context influence intensity according to the application scenario; σ is the Sigmoid activation function, which maps the linear combination result to a standardized adjustment coefficient.

6. The RAG-based multi-source heterogeneous data fusion system according to claim 1, characterized in that: The generation enhancement engine submodule achieves creative reconstruction and semantic enhancement of multimodal data by deeply integrating retrieval knowledge and generation models. This module deploys a multi-task generation network architecture and builds a three-channel parallel processing engine based on the Transformer framework to support text generation, structured data completion, and image description generation, capable of simultaneously handling data generation requirements of different modalities. The module innovatively introduces a differentiable memory matrix to encode knowledge graph node features obtained from the dynamic retrieval interface and external knowledge base information into temporal memory units, and realizes dynamic call and persistent storage of key knowledge during the generation process through a gated attention mechanism. The formula of the adversarial generation verification loss function is: L_adv=α·|G(x)-y|+γ·D(G(x)⊕R)-η·S(R,G(x)) Among them, L_adv represents the total loss value of adversarial training, which is used to simultaneously optimize the generator and discriminator; G(x) represents the output of the generator's processing of the input data x; y corresponds to the real annotated data; R represents the relevant knowledge feature vector returned by the retrieval module; D is the confidence score output of the discriminator; S represents the semantic similarity calculation function between the generated content and the retrieved knowledge; α controls the alignment strength between the generated content and the real data, γ adjusts the influence weight of the discriminator feedback on the generation process, and η determines the importance of the knowledge consistency constraint; the symbol ⊕ represents the cross-modal fusion operation of the feature vector, and the gated attention mechanism is used to realize knowledge injection.

7. The RAG-based multi-source heterogeneous data fusion system according to claim 1, characterized in that: The multi-source fusion algorithm module realizes the value extraction of cross-domain heterogeneous data through multi-level feature fusion and dynamic adaptive mechanism. The module builds a layered fusion architecture, adopts a multi-head cross-attention mechanism at the bottom layer to realize cross-modal interaction of original feature vectors such as text and images, and extracts local feature correlation patterns through spatiotemporal convolutional network. Deploy a graph attention network at the high-level semantic level to map the entity relationship topology of the knowledge graph and the generative augmented data into a unified graph structure, and use the neighborhood propagation algorithm to capture global semantic dependencies; The calculation formula of the dynamic fusion weight calculation model is: ω_i=softmax((c_i⊙T_i)·tanh(v^T[s_i||h_t]) / √d) Where ω_i represents the fusion weight coefficient of the i-th data source, which is used to dynamically adjust the contribution strength of the data source; c_i is the confidence evaluation value of the data source, which is output as a continuous score from 0 to 1 through the pre-trained quality assessment network; T_i is the timeliness decay factor, which is calculated as exp(-λΔt) according to the exponential decay law, where λ is the decay rate related to domain knowledge, and Δt represents the time difference between the data generation time and the current time; the symbol ⊙ represents the Hadamard product, which realizes the coupled calculation of confidence and timeliness; v is the trainable feature projection vector; s_i represents the feature embedding vector of the current data source; h_t represents the historical state feature extracted by the time series fusion module; the symbol || represents the vector splicing operation; d is the dimensionality scaling coefficient of the feature vector; and the tanh activation function is used to control the numerical range.

8. The RAG-based multi-source heterogeneous data fusion system according to claim 1, characterized in that: The multimodal aligner submodule constructs a cross-modal joint embedding space, maps feature vectors of different modalities to a unified semantic coordinate system through a contrastive learning framework, and uses an improved hierarchical attention mechanism to achieve fine-grained semantic alignment. The core technology includes a three-stage processing flow: first, a vision-language pre-trained model is used to extract deep features of multimodal data, and a feature decoupling network is used to separate content features from modality-specific features. The cross-modal contrastive alignment loss function is: L_align=-τ·log[exp(s(v_i,t_j) / ε) / ∑_{k=1}^Nexp(s(v_i,t_k) / ε)+exp(s(v_i,v_k) / ε)]; Among them, L_align represents the loss value of cross-modal alignment optimization, τ is the temperature coefficient controlling the difficulty distribution of contrastive learning, vi_i represents the feature vector of the i-th visual sample, t_j is the text feature vector matching it, s represents the cross-modal similarity calculation function, which is implemented as v_i^TW t_j using a bilinear interaction model, and W is a learnable cross-modal association matrix; ε is the similarity scaling factor used to balance the dimensional differences of different modalities; N represents the batch size, and vk and tk represent the feature vectors of other visual and text samples in the same batch, respectively.

9. The RAG-based multi-source heterogeneous data fusion system according to claim 1, characterized in that: The data storage and management module is internally configured with a hybrid storage architecture. The hot data layer deploys the Alluxio memory acceleration system to support high-concurrency real-time queries; the warm data layer uses the HBase cluster to store structured records and designs dynamic column families to adapt to changes in device parameters; the cold data layer connects to the Glacier storage system and uses a columnar compression algorithm to save space.

10. The RAG-based multi-source heterogeneous data fusion system according to claim 1, characterized in that: The data service and application module is set up to provide a three-layer service system. The interface layer builds a RESTful API gateway and WebSocket streaming channel. The analysis layer integrates time series prediction models and anomaly detection algorithm libraries, running on the Flink stream processing engine to achieve real-time insights; the visualization layer includes a three-dimensional rendering engine and an AR interactive interface, and the device digital twin integrates real-time sensor data and historical maintenance records.

Images