Scene perception information fusion decision-making method for industrial field multi-source heterogeneous data
By employing a three-tiered collaborative architecture and a dynamic iterative approach, the problem of high-precision real-time decision-making based on multi-source heterogeneous data in industrial settings was solved. This enabled efficient and reliable data processing and intelligent decision-making, adapting to dynamic changes in industrial settings and enhancing the model's adaptability and interpretability.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- HANGZHOU HOLLYSYS AUTOMATION
- Filing Date
- 2026-04-15
- Publication Date
- 2026-05-12
AI Technical Summary
Existing technologies struggle to achieve high-precision, real-time, and high-reliability decision-making when processing multi-source heterogeneous data from industrial sites. They also suffer from problems such as misjudgment, limitations of a single architecture, poor model interpretability, weak preprocessing targeting, and incomplete model training.
It adopts a three-level collaborative architecture, including the edge layer, cloud layer and interaction layer, and achieves efficient processing and intelligent decision-making of multi-source heterogeneous data through adaptive preprocessing, lightweight scene perception, causal reasoning fusion and dynamic model iteration.
It enables efficient processing and real-time response of multi-source heterogeneous data in industrial sites, improves decision-making accuracy and reliability, reduces implementation difficulty, and ensures that the model continuously adapts to dynamic changes in industrial sites through causal reasoning and model iterative optimization.
Smart Images
Figure CN122020577A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of industrial field data processing technology, specifically relating to a scene perception information fusion decision-making method for multi-source heterogeneous data in industrial fields. Background Technology
[0002] The operation of industrial sites generates massive amounts of heterogeneous data from multiple sources, including time-series sensor data, video surveillance data, PLC control signals, and text operation logs. This data is the core foundation for realizing industrial scene perception and intelligent decision-making.
[0003] Currently, the industrial sector mainly uses traditional data fusion methods combined with a single architecture deployment to process and integrate multi-source heterogeneous data, thereby enabling monitoring and decision-making in industrial scenarios.
[0004] In-depth analysis of existing technologies revealed the following core shortcomings, making it difficult to meet the high-precision, real-time, and high-reliability decision-making requirements of industrial settings: (1) Multi-source data fusion relies on statistical correlation, which is prone to misjudgment: Existing technologies only rely on statistical correlation between data for fusion, and cannot distinguish between causal correlation and false correlation.
[0005] For example, there may be a statistical correlation between industrial ambient humidity and equipment failure, but no direct causal relationship. Existing methods are prone to misjudging this as a valid association, leading to unnecessary downtime losses.
[0006] (2) The scene perception and decision-making architecture is simple and cannot balance real-time performance and accuracy: the full edge deployment architecture is limited by the computing power of edge devices and the model accuracy is low; the full cloud deployment architecture needs to transmit all the original data, and the network latency is high, which cannot meet the real-time response requirements of emergency scenarios in industrial sites.
[0007] (3) The decision results lack interpretability and are difficult to implement: Most existing AI fusion decision models are black box models, and the output results have no clear business explanation. Industrial operators find it difficult to understand and trust them, which limits the actual application of the technology in industrial sites.
[0008] (4) Data preprocessing is not targeted and has poor adaptability: Existing preprocessing methods require manual definition of data types and filtering parameters, which cannot adapt to the dynamically changing environment of industrial sites, resulting in poor preprocessing effect and affecting the accuracy of subsequent fusion decision-making.
[0009] (5) The AI model training process is incomplete and iterative optimization is lacking: Existing technologies do not clearly define the key contents of AI model training data, training steps, loss function design, etc., and lack a model iterative optimization mechanism based on actual decision feedback in industrial sites, making it difficult to continuously improve model performance.
[0010] Therefore, there is an urgent need to develop a scene perception information fusion decision-making method based on multi-source heterogeneous data from industrial sites to solve the problems in existing technologies. Summary of the Invention
[0011] To overcome the shortcomings of existing technologies, the present invention aims to provide a scene perception information fusion decision-making method for multi-source heterogeneous data in industrial sites, which can realize efficient processing and intelligent decision-making of multi-source heterogeneous data in industrial sites, and has a simple structure and is easy to use, so as to solve the problems mentioned in the background technology.
[0012] To achieve the above objectives, the present invention provides the following technical solution: The aforementioned scene perception information fusion decision-making method based on multi-source heterogeneous data in industrial settings comprises an edge layer, a cloud layer, and an interaction layer, forming a three-level collaborative architecture. The specific steps of the decision-making method are as follows: S1: The edge layer collects multi-source heterogeneous raw data from the industrial site and performs adaptive preprocessing to obtain cleaned multi-source feature data; S2: The edge layer processes multi-source feature data through a lightweight scene perception model and outputs preliminary scene judgment results. In emergency scenarios, local decision-making is triggered directly, while in non-emergency scenarios, the multi-source feature data and preliminary scene judgment results are uploaded to the cloud layer through the interaction layer. S3: The cloud layer constructs and updates the causal relationship graph based on industrial business knowledge and incremental data from the edge layer. It processes the data uploaded from the edge layer through a causal reasoning fusion model, outputs decision results, and distributes them through the interaction layer. S4: The cloud layer optimizes the parameters of the causal reasoning fusion model and the lightweight scene perception model based on the decision execution results fed back from the edge layer, and sends the optimized parameters down to the edge layer through the interaction layer to achieve dynamic iteration.
[0013] By adopting the above technical solutions, and through a coherent process of data acquisition and preprocessing, scene-aware decision-making, cloud-based precise reasoning, and dynamic model iteration, efficient processing, real-time response, and accurate decision-making of multi-source heterogeneous data in industrial sites are achieved, while ensuring continuous optimization of model performance.
[0014] As a further aspect of the present invention: the adaptive preprocessing in step S1 adopts a meta-learning driven processing method, which specifically includes three sub-steps: adaptive data type recognition, dynamic noise filtering, and scenario-based missing data completion, adapting to the dynamically changing environment of the industrial site without the need for manual intervention.
[0015] By adopting the above technical solutions, the preprocessing of multi-source heterogeneous data in industrial sites can be automated and adaptive. Data type identification, noise filtering, and missing data completion can be completed without manual intervention, adapting to the dynamically changing industrial environment, improving preprocessing efficiency and quality, and providing a reliable data foundation for subsequent scene perception.
[0016] As a further aspect of the present invention, the adaptive data type recognition includes the following sub-steps: S11: Extract the three core meta-features of the data: temporal, discrete, and spatial. S12: The recognition model is trained using a model-independent meta-learning algorithm, and new data types are quickly adapted using a pre-set subset of samples. S13: Input the meta-features of the data to be processed into the trained recognition model, and output one or more data types from the following categories: continuous time series, discrete time series, image type, and text type.
[0017] By adopting the above technical solutions, automatic and accurate identification of multi-source heterogeneous data types can be achieved. By extracting core meta-features and combining them with model-independent meta-learning algorithms, new data types can be quickly adapted using a preset subset of samples, ensuring the universality and efficiency of data type identification and providing a basis for differentiated preprocessing.
[0018] As a further aspect of the present invention, the dynamic noise filtering includes the following sub-steps: SQ11: For time-series sensor data, dynamically adjust the wavelet threshold based on the device's operating status to filter high-frequency components after wavelet decomposition and retain effective signals. SQ12: For video data, taking advantage of the fixed background characteristics of industrial scenes, it uses Gaussian mixture background modeling and foreground segmentation algorithms to distinguish moving targets from environmental noise; SQ13: For text logs, an industrial terminology dictionary matching and noise word filtering method is used to retain the core text containing operation instructions and alarm information.
[0019] By adopting the above technical solutions, targeted noise filtering is achieved for different types of data. Adaptive filtering strategies are applied to time-series sensor data, video data, and text logs to effectively remove various noise interferences, retain core valid data, and improve data purity.
[0020] As a further aspect of the present invention: the lightweight scene perception model is constructed using knowledge distillation technology, including two stages: cloud-based multimodal Transformer teacher model training and edge-layer lightweight student model distillation; after the teacher model is trained, the student model is obtained by reducing the number of encoder layers, reducing the number of attention heads, and using feature vector quantization, ensuring that the model is lightweight while retaining high accuracy.
[0021] By adopting the above technical solutions, a scene perception model that balances lightweight deployment and high-precision recognition is constructed. Knowledge distillation technology is used to achieve knowledge transfer from complex cloud models to lightweight edge models, thereby reducing the resource consumption of model deployment while ensuring scene perception accuracy and meeting the real-time processing requirements of the edge layer.
[0022] As a further aspect of the present invention: In step S2, the scene perception logic of the lightweight scene perception model includes: calculating the contribution of different modal features to scene recognition through a lightweight self-attention mechanism. The calculation method is to obtain the mutual information between each modal feature and the scene classification label. The greater the mutual information, the higher the weight. The weighted multimodal features are fused and input into the student model, and the preliminary scene judgment result containing scene type and confidence level is output.
[0023] By adopting the above technical solution, accurate weighted fusion of multimodal features and scene judgment can be achieved. The weight of each modal feature is dynamically allocated through the self-attention mechanism, which improves the pertinence and accuracy of scene perception and outputs the judgment result containing scene type and confidence level, providing a basis for subsequent differentiation between emergency and non-emergency scenes.
[0024] As a further aspect of the present invention: in step S3, the construction and updating of the causal relationship graph includes the following sub-steps: S31: Combine industrial business knowledge to define data features, equipment status, and scenario events as nodes in the cause-effect graph, sort out the causal relationships between nodes and connect them with directed edges to build an initial cause-effect graph; S32: Based on incremental data uploaded from the edge layer, causal discovery is performed through conditional independence test and causal direction determination, new causal relationship nodes and edges are added, and causal strength weights are updated to ensure the timeliness of the causal graph.
[0025] By adopting the above technical solutions, a causal relationship graph that fits the actual industrial scenario is constructed and dynamically updated. The causal nodes and relationships are continuously optimized by combining business knowledge and incremental data to ensure the timeliness and accuracy of the causal relationship graph, providing reliable support for filtering false associations in subsequent causal reasoning.
[0026] As a further aspect of the present invention, the processing procedure of the causal reasoning fusion model includes: filtering features that have no causal relationship with the decision target using a backdoor adjustment method based on the causal relationship graph; calculating the causal effect value of each feature node on the decision target node after filtering and using it as the fusion weight, and weighted fusion to obtain a fusion feature vector; constructing a Bayesian network reasoning model based on the fusion feature vector and the causal relationship graph, and outputting three-dimensional decision information including decision results, visual explanations, and business text explanations.
[0027] By adopting the above technical solutions, accurate and interpretable multi-source data fusion decision-making can be achieved. Irrelevant features are filtered out through causal reasoning, effective features are weighted and fused, and three-dimensional decision information is output by combining Bayesian network reasoning. This not only improves the accuracy of decision-making but also enhances the understandability of the decision results, making it easier to implement in industrial settings.
[0028] As a further aspect of the present invention: the dynamic iteration in step S4 includes the following sub-steps: S41: Label the decision execution results fed back from the edge layer, and construct reward and punishment signals based on the labeling results; S42: Combining reward and punishment signals with fused feature vectors, the gradient descent method is used to optimize the conditional probability parameters of the Bayesian network; S43: Retrain the cloud-based teacher model based on feedback data, and perform secondary distillation to update the parameters of the edge-layer student model; S44: The optimized model parameters are sent to the edge layer and cloud layer using incremental transmission to complete the update.
[0029] By adopting the above technical solutions, continuous iterative optimization of model performance is achieved. A reward and punishment mechanism is built based on decision execution feedback. The model parameters on the cloud and edge are optimized and incremental updates are completed respectively, so that the model can continuously adapt to changes in the industrial site and ensure steady improvement in decision performance.
[0030] As a further aspect of the present invention: In step S4, the core functions of the interaction layer include encrypted data transmission, incremental synchronization, and breakpoint resume transmission; encrypted transmission adopts a preset encryption algorithm and industrial Ethernet protocol, and the key is updated according to a preset cycle; during incremental synchronization, the edge layer only uploads preprocessed feature data, and in abnormal scenarios, additional key original data is uploaded, while the cloud layer only sends down the changed model parameters; breakpoint resume transmission is achieved through local caching at the edge and integrity verification in the cloud, and missing data is automatically resumed after the network is restored.
[0031] By adopting the above technical solutions, the security, efficiency and stability of data transmission between the edge layer and the cloud layer are ensured. Encrypted transmission and incremental synchronization reduce the amount of data transmission and security risks. Resuming interrupted transmission avoids data loss and ensures the smooth operation of the three-level collaborative architecture.
[0032] Compared with the prior art, the present invention has the following advantages: (1) This invention balances the real-time performance and accuracy of industrial site decision-making through a three-level collaborative architecture and lightweight model deployment, enabling rapid local response in emergency scenarios and accurate cloud-based reasoning in non-emergency scenarios. At the same time, it reduces network load by leveraging incremental transmission, thereby improving decision-making efficiency and reliability. (2) This invention is based on meta-learning adaptive preprocessing and causal reasoning fusion decision-making, which can adapt to dynamic industrial environments without human intervention, effectively filter false correlation features, improve data processing quality and decision accuracy, and combine dynamic iteration mechanism to ensure continuous model adaptation and reduce the difficulty of implementation.
[0033] Other features and advantages of the present invention will be disclosed in detail in the following detailed description and accompanying drawings. Attached Figure Description
[0034] Figure 1This is a flowchart illustrating the steps of a scene perception information fusion decision-making method based on multi-source heterogeneous data in an industrial setting, as described in this invention. Detailed Implementation
[0035] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.
[0036] Example 1 like Figure 1 As shown, this invention provides a scene-aware information fusion decision-making method for multi-source heterogeneous data in industrial settings. It employs a three-tiered collaborative architecture of edge layer, cloud layer, and interaction layer. Through a coherent process of data acquisition and preprocessing, scene-aware decision-making, cloud-based precise inference, and dynamic model iteration, it achieves efficient processing, real-time response, and accurate decision-making for multi-source heterogeneous data in industrial settings, while simultaneously ensuring continuous optimization of model performance. The specific steps of this invention are as follows: (a) S1: The edge layer collects multi-source heterogeneous raw data from the industrial site and performs adaptive preprocessing to obtain cleaned multi-source feature data.
[0037] The adaptive preprocessing adopts a meta-learning driven approach, which includes three sub-steps: adaptive data type recognition, dynamic noise filtering, and scenario-based missing data completion. It adapts to the dynamically changing environment of industrial sites without the need for manual intervention.
[0038] (1) Adaptive data type recognition includes the following sub-steps: S11: Extract the three core meta-features of the data: temporal, discrete, and spatial. S12: The recognition model is trained using a model-independent meta-learning algorithm, and new data types are quickly adapted using a pre-set subset of samples. S13: Input the meta-features of the data to be processed into the trained recognition model, and output one or more data types from the following categories: continuous time series, discrete time series, image type, and text type.
[0039] (2) Dynamic noise filtering includes the following sub-steps: SQ11: For time-series sensor data, dynamically adjust the wavelet threshold based on the device's operating status to filter high-frequency components after wavelet decomposition and retain effective signals. SQ12: For video data, taking advantage of the fixed background characteristics of industrial scenes, it uses Gaussian mixture background modeling and foreground segmentation algorithms to distinguish moving targets from environmental noise; SQ13: For text logs, an industrial terminology dictionary matching and noise word filtering method is used to retain the core text containing operation instructions and alarm information.
[0040] (ii) S2: The edge layer processes multi-source feature data through a lightweight scene perception model and outputs preliminary scene judgment results. In emergency scenarios, local decision-making is triggered directly, while in non-emergency scenarios, multi-source feature data and preliminary scene judgment results are uploaded to the cloud layer through the interaction layer.
[0041] The lightweight scene perception model is constructed using knowledge distillation technology, which includes two stages: cloud-based multimodal Transformer teacher model training and edge-layer lightweight student model distillation. After the teacher model is trained, the student model is obtained by reducing the number of encoder layers, reducing the number of attention heads, and using feature vector quantization, ensuring that the model is lightweight while retaining high accuracy.
[0042] The scene perception logic of the lightweight scene perception model includes: calculating the contribution of different modal features to scene recognition through a lightweight self-attention mechanism. The calculation method is to obtain the mutual information between each modal feature and the scene classification label. The greater the mutual information, the higher the weight. The weighted multimodal features are fused and input into the student model to output a preliminary scene judgment result containing scene type and confidence level.
[0043] By adopting the above technical solution, accurate weighted fusion of multimodal features and scene judgment can be achieved. The weight of each modal feature is dynamically allocated through the self-attention mechanism, which improves the pertinence and accuracy of scene perception and outputs the judgment result containing scene type and confidence level, providing a basis for subsequent differentiation between emergency and non-emergency scenes.
[0044] (III) S3: The cloud layer constructs and updates the causal relationship graph based on industrial business knowledge and incremental data from the edge layer. It processes the data uploaded from the edge layer through a causal reasoning fusion model, outputs decision results, and distributes them through the interaction layer.
[0045] (1) The construction and updating of the causal relationship diagram includes the following sub-steps: S31: Combine industrial business knowledge to define data features, equipment status, and scenario events as nodes in the cause-effect graph, sort out the causal relationships between nodes and connect them with directed edges to build an initial cause-effect graph; S32: Based on incremental data uploaded from the edge layer, causal discovery is performed through conditional independence test and causal direction determination, new causal relationship nodes and edges are added, and causal strength weights are updated to ensure the timeliness of the causal graph.
[0046] By adopting the above technical solutions, a causal relationship graph that fits the actual industrial scenario is constructed and dynamically updated. The causal nodes and relationships are continuously optimized by combining business knowledge and incremental data to ensure the timeliness and accuracy of the causal relationship graph, providing reliable support for filtering false associations in subsequent causal reasoning.
[0047] (2) The processing of the causal reasoning fusion model includes: filtering features that have no causal relationship with the decision target by using a backdoor adjustment method based on the causal relationship graph; calculating the causal effect value of each feature node on the decision target node after filtering and using it as the fusion weight, and weighted fusion to obtain the fusion feature vector; constructing a Bayesian network reasoning model based on the fusion feature vector and the causal relationship graph, and outputting three-dimensional decision information including decision results, visual explanations, and business text explanations.
[0048] By adopting the above technical solutions, accurate and interpretable multi-source data fusion decision-making can be achieved. Irrelevant features are filtered out through causal reasoning, effective features are weighted and fused, and three-dimensional decision information is output by combining Bayesian network reasoning. This not only improves the accuracy of decision-making but also enhances the understandability of the decision results, making it easier to implement in industrial settings.
[0049] (iv) S4: Based on the decision execution results fed back from the edge layer, the cloud layer optimizes the parameters of the causal reasoning fusion model and the lightweight scene perception model, and sends the optimized parameters down to the edge layer through the interaction layer to achieve dynamic iteration.
[0050] Dynamic iteration includes the following sub-steps: S41: Label the decision execution results fed back from the edge layer, and construct reward and punishment signals based on the labeling results; S42: Combining reward and punishment signals with fused feature vectors, the gradient descent method is used to optimize the conditional probability parameters of the Bayesian network; S43: Retrain the cloud-based teacher model based on feedback data, and perform secondary distillation to update the parameters of the edge-layer student model; S44: The optimized model parameters are sent to the edge layer and cloud layer using incremental transmission to complete the update.
[0051] By adopting the above technical solutions, continuous iterative optimization of model performance is achieved. A reward and punishment mechanism is built based on decision execution feedback. The model parameters on the cloud and edge are optimized and incremental updates are completed respectively, so that the model can continuously adapt to changes in the industrial site and ensure steady improvement in decision performance.
[0052] The core functions of the interaction layer of this invention include encrypted data transmission, incremental synchronization, and breakpoint resume. Encrypted transmission adopts a preset encryption algorithm and industrial Ethernet protocol, and the key is updated according to a preset period. During incremental synchronization, the edge layer only uploads preprocessed feature data, and in abnormal scenarios, it additionally uploads key original data. The cloud layer only sends down the changed model parameters. Breakpoint resume is achieved through local caching at the edge and integrity verification in the cloud. Missing data is automatically resumed after the network is restored.
[0053] Example 2 Continue reading Figure 1This invention discloses a scene perception information fusion decision-making method for multi-source heterogeneous data in industrial settings. It employs a three-tiered collaborative architecture of edge layer, cloud layer, and interaction layer. Through a full-link design involving: S1, adaptive preprocessing of multi-source data; S2, lightweight scene perception; S3, causal reasoning fusion decision-making; and S4, dynamic iterative optimization of the model, it achieves efficient processing and intelligent decision-making for multi-source heterogeneous data. The core design, non-public knowledge, and input / output logic of each layer are as follows: (a) Edge layer: Adaptive preprocessing and lightweight scene perception module Core functions: Real-time acquisition of multi-source data, adaptive cleaning, feature extraction and lightweight scene perception; triggering local decision-making in emergency scenarios; uploading feature data and preliminary judgment results to the cloud in non-emergency scenarios.
[0054] (1) Multi-source data acquisition unit Input: Raw data from industrial field time sensors, cameras, PLC controllers, logging systems, etc.; Processing procedure: Adapt to common industrial interfaces using an extensible interface, synchronously collect data from multiple devices and convert it into a unified structured format, including time-series data, video data, text logs, and PLC signals. The text logs are converted into vector sequences using an industrial-grade word segmentation dictionary. Output: Standardized multi-source heterogeneous data, transmitted to the adaptive preprocessing unit; Note: The industrial-specific word segmentation dictionary is not based on common knowledge. It is constructed based on industrial corpora and has a word segmentation accuracy of ≥98%, solving the problem of low recognition accuracy of industrial terms in general dictionaries.
[0055] (2) Adaptive preprocessing unit Input: Standardized multi-source heterogeneous data; Processing procedure: Adaptive Data Type Recognition: The recognition model is built based on meta-learning, eliminating the need for manual annotation. It extracts three core meta-features: temporal, discrete, and spatial. The MAML algorithm is used to train the model, with training data consisting of typical industrial data. The convergence condition is a recognition accuracy of ≥99%. Output types include continuous-time, discrete-time, image, and text. Dynamic noise filtering: Adaptive strategies are designed based on data type and operating conditions: Time-series sensor data is combined with equipment operating status to dynamically adjust wavelet thresholds to filter high-frequency noise; Gaussian mixture background modeling is used for video data to distinguish moving targets from environmental noise; Text logs are filtered for noise words by matching with an industrial terminology dictionary. Contextualized missing data completion: Prior knowledge of industrial scenarios is introduced to construct an LSTM time-series prediction and scenario-constrained completion model. When missing data is detected, the current scenario is first determined, then the missing value is predicted using LSTM, and finally the predicted value is constrained by the scenario data distribution range to ensure its reasonableness. Output: Cleaned multi-source feature data, including temporal feature vectors, video target feature vectors, and text keyword vectors; Input / output logic: After standardized data is input, cleaned feature data is output. The preprocessing delay is ≤30ms to meet real-time requirements.
[0056] (3) Lightweight scene perception unit Input: Cleaned multi-source feature data; Processing procedure: Model architecture design: A framework for transferring knowledge from the teacher model to the student model is constructed using knowledge distillation techniques. Teacher Model: The multimodal Transformer model is trained in the cloud. Multimodal features are input and mapped to a unified feature space. After processing by the Transformer encoder, the output scene classification results are given, including normal production, equipment malfunction, personnel violation, and emergency alarm. Teacher model training: Collect a large amount of historical multi-source data and divide it into training, validation, and test sets according to proportions. Use a combination loss function of cross-entropy loss and modality fusion loss, and train with the Adam optimizer. The convergence condition is that the accuracy on the validation set is ≥97%. Student model: edge deployment, achieving lightweight compression by reducing the number of encoder layers, reducing the number of attention heads, and quantizing feature vectors; Distillation training: A combined loss function of student classification loss and KL divergence loss is used for training with SGD optimizer. The convergence condition is that the difference in accuracy between the student and teacher models is ≤2%, achieving a parameter compression rate of ≥70% and an inference latency of ≤20ms. Scene perception logic: The mutual information between each modal feature and the scene classification label is calculated through a lightweight self-attention mechanism. Weights are assigned according to the size of the mutual information. After fusion, the information is input into the student model and the preliminary judgment result containing scene type and confidence level is output. Emergency Decision Response: Set confidence thresholds, with 0.9 for emergency alarms and 0.8 for other scenarios. In emergency scenarios where the confidence threshold is met, local decisions are triggered directly, including audible and visual alarms, power cut-off, and push alarms, without waiting for cloud feedback. Output: In emergency scenarios, local decision-making instructions are output; in non-emergency scenarios, multi-source feature data and preliminary judgment results are output and transmitted to the cloud. Input / output logic: After the cleaned feature data is input, the corresponding result is output, with a total delay of ≤50ms, which meets the real-time requirements.
[0057] (ii) Cloud layer: Fusion decision-making module driven by causal reasoning Core functionality: Receives data from non-urgent scenarios at the edge layer, filters out false correlations through causal reasoning to achieve accurate decision-making, and iteratively optimizes the model based on feedback. The core non-public common sense module is designed as follows: (1) Cause-effect graph construction unit Input: Industrial business knowledge and edge layer incremental data, where industrial business knowledge includes equipment manuals, process specifications, and historical failure cases; Processing procedure: Initial cause-effect graph construction: Define data features, device status, and scene events as core nodes, sort out causal relationships based on business knowledge and connect them through directed edges, assign causal strength weights, and generate an initial cause-effect graph; Dynamic update: Based on incremental data at the edge layer, potential causal relationships are mined through the PC algorithm, nodes and edges are supplemented, and causal strength weights are updated to ensure timeliness. The PC algorithm includes conditional independence test and causal direction determination. Output: Initial and updated causal relationship graph of the industrial scene; Input / output logic: After business knowledge and incremental data are input, a causal relationship graph is output to provide a foundation for subsequent reasoning.
[0058] (2) Causal reasoning fusion unit Input: edge layer non-emergency scenario data, preliminary judgment results, causal relationship graph; Processing steps: False association filtering: Based on the causal relationship graph, a backdoor adjustment method is used to analyze the causal path from features to the decision target, filtering features without causal paths and blocking interference from mixed nodes; Causal feature weighted fusion: The causal effect value of each feature on the decision target is calculated, normalized, and used as the fusion weight to obtain the fused feature vector; Inference model training: Based on the fused feature data and decision labels, a Bayesian network inference model is trained. The Bayesian network inference model uses the expectation-maximization algorithm to estimate parameters, with the convergence condition being the stability of the log-likelihood function, and a decision accuracy of ≥95% on the test set; Interpretable decision generation: Outputting three-dimensional information including decision results, visual explanations, and business-oriented text explanations. The text explanations are combined with industry standards to improve operator comprehension. Output: Accurate decision results and fused feature vectors for model iteration; Input / output logic: After the edge layer data and cause-effect graph are input, the decision result is output, with a decision delay of ≤200ms.
[0059] (3) Model optimization iteration unit Inputs: Edge layer decision execution feedback results, fused feature vectors; Processing steps: Feedback data annotation: Annotate results such as correct decision, incorrect decision, and mediocre processing effect; Reward and punishment mechanism construction: Construct reward and punishment signals based on annotation results; Model parameter update: Combine reward and punishment signals and fused feature vectors, and use gradient descent to optimize Bayesian network parameters; Retrain the cloud teacher model, and update the parameters of the marginal student model through secondary distillation; Parameter distribution: Use incremental transmission to distribute the optimized parameters to the edge layer and the corresponding cloud module; Output: Optimized causal inference model parameters, lightweight student model parameters; Input / output logic: After the feedback results and fusion features are input, the output optimization parameters are output, and the iteration cycle can be dynamically adjusted.
[0060] (III) Interaction Layer: Edge-Cloud Collaborative Interaction Module Core Functionality: Enables encrypted data transmission between the edge and cloud, incremental synchronization, and breakpoint resumption, ensuring secure, stable, and efficient transmission. A key, lesser-known aspect is the incremental data synchronization strategy. Input: Data uploaded from the edge layer, data distributed from the cloud layer; Processing steps: Incremental synchronization: The edge layer only uploads preprocessed feature data, and additionally uploads key raw data in abnormal scenarios. The cloud only sends down the changed model parameters to reduce the amount of data transmitted. Encrypted transmission: Pre-set encryption algorithms and industrial Ethernet protocols are used for transmission, and the key is updated regularly to prevent data tampering and theft. Resume interrupted transmission: Data to be uploaded is cached locally at the edge and automatically resumed after the network is restored. The cloud verifies the integrity of the data and triggers a retransmission when data is missing. Output: Encrypted transmitted data; Input / output logic: Edge and cloud data are transmitted after encryption and incremental synchronization, with a data transmission success rate of ≥99.9% and a transmission delay of ≤1s after network outage recovery.
[0061] Experimental data In one feasible embodiment, the edge layer includes an industrial edge gateway, CPU: Intel Core i5-1135G7, memory: 8GB, storage: 128GB SSD; sensors include temperature, pressure, and vibration sensors; and a high-definition camera with 1080P resolution and 30fps.
[0062] The cloud layer includes servers with a CPU of Intel Xeon Gold 6330, 128GB of RAM, an NVIDIA A100 40GB GPU, and 2TB of SSD storage.
[0063] The interaction layer includes industrial Ethernet with a bandwidth of 1Gbps and 5G industrial modules with a latency of ≤20ms.
[0064] Data preprocessing was performed using Python 3.8, PyTorch 1.12, and Scikit-learn 1.2.
[0065] The model was trained using PyTorch Lightning 1.9 and Transformers 4.26.
[0066] Causal inference was performed using PyTorch 1.12, CausalML 0.16, and pgmpy 0.1.23.
[0067] Data transmission uses the MQTT 3.1.1 protocol and AES-256 encryption algorithm.
[0068] The data sources include: one month of real-world operational data from an automotive parts processing plant, covering three core scenarios: normal production, equipment malfunction, and personnel violations, as well as emergency alarm scenarios such as equipment short circuits and high-temperature overloads.
[0069] Data composition: Time-series sensor data 100,000 / 15,000 / 15,000 Real-time monitoring data for temperature (-20~150℃), pressure (0~10MPa), vibration (0~5g), etc. Video data 500 hours / 50 hours / 50 hours Monitoring videos of equipment operation and personnel operation in the production workshop Text log 50,000 items / 8,000 items / 7,000 items PLC control commands, equipment alarm logs, and manual operation records (including industrial terminology accounting for ≥60%). Hybrid heterogeneous data 30,000 items / 5,000 items / 5,000 items The above-mentioned mixed samples of multiple data types were collected simultaneously. Experimental results: (1) Experimental results of the adaptive preprocessing module submodule Experimental indicators Experimental results Target value Does it meet the standard? Data type identification Overall recognition accuracy 99.3% ≥99% yes Accuracy of recognition of time-series continuous data 99.6% - - Text data recognition accuracy 99.1% - - Image data recognition accuracy 99.5% - - Dynamic noise filtering Improved SNR of time-series sensor data Improved from 25dB to 48dB ≥30dB yes Video object detection accuracy 98.7% ≥95% yes Text core information retention rate 99.2% ≥98% yes Contextual missing information completion Missing data completion accuracy 98.5% (when the missing data rate is ≤20%) ≥95% yes Overall preprocessing Preprocessing delay 22ms ≤30ms yes (2) Experimental results of lightweight scene perception model Experimental indicators Experimental results Target value Does it meet the standard? Cloud-based teacher model validation set accuracy 97.8% ≥97% yes Marginal student model test set accuracy 96.2% - - Difference in accuracy between teacher and student models 1.6% ≤2% yes Model parameter compression ratio 75.3% ≥70% yes Student model inference delay 15ms ≤20ms yes Total delay in emergency scenario decision-making 42ms ≤50ms yes Scene confidence assessment accuracy 99.0% (Threshold-triggered scenarios) ≥98% yes (3) Experimental results of the causal reasoning fusion module Experimental indicators Experimental results Target value Does it meet the standard? Timeliness of causal relationship diagram updates The update will be completed within 10 minutes after the incremental data is received. - - False association filtering rate 99.5% (excluding features without causal relationship) ≥99% yes Bayesian network inference accuracy 96.8% ≥95% yes Decision delay 168ms ≤200ms yes Decision outcome comprehensibility score 4.8 out of 5 (Engineer's rating) ≥4.5 points yes (4) Results of the interaction layer and model iteration experiment Experimental indicators Experimental results Target value Does it meet the standard? Data encryption transmission security No data breaches / tampering (72-hour continuous test) - - The percentage reduction in incremental synchronization data volume 82% (compared to full transmission) ≥70% yes Data transmission success rate 99.96% ≥99.9% yes Resuming interrupted downloads will restore latency 0.6s ≤1s yes Decision accuracy improved after 3 rounds of iteration From 96.8% to 98.2% ≥1% yes Model adaptation to new scenarios convergence speed The new scenario data converges within 24 hours after being integrated. - - Results analysis: Performance metrics verification: All core performance metrics met or exceeded the target values set in the technical solution, such as data type recognition accuracy of 99.3%, model parameter compression rate of 75.3%, and decision accuracy of 96.8%, confirming the feasibility of the technical approach.
[0070] This invention provides a scene perception information fusion decision-making method for multi-source heterogeneous data in industrial sites, which can achieve efficient processing and intelligent decision-making of multi-source heterogeneous data in industrial sites with high reliability.
[0071] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.
Claims
1. A scene perception information fusion decision-making method based on multi-source heterogeneous data from an industrial site, characterized in that, The architecture comprises an edge layer, a cloud layer, and an interaction layer, forming a three-tiered collaborative structure. The specific steps of the decision-making process are as follows: S1: The edge layer collects multi-source heterogeneous raw data from the industrial site and performs adaptive preprocessing to obtain cleaned multi-source feature data; S2: The edge layer processes multi-source feature data through a lightweight scene perception model and outputs preliminary scene judgment results. In emergency scenarios, local decision-making is triggered directly, while in non-emergency scenarios, the multi-source feature data and preliminary scene judgment results are uploaded to the cloud layer through the interaction layer. S3: The cloud layer constructs and updates the causal relationship graph based on industrial business knowledge and incremental data from the edge layer. It processes the data uploaded from the edge layer through a causal reasoning fusion model, outputs decision results, and distributes them through the interaction layer. S4: The cloud layer optimizes the parameters of the causal reasoning fusion model and the lightweight scene perception model based on the decision execution results fed back from the edge layer, and sends the optimized parameters down to the edge layer through the interaction layer to achieve dynamic iteration.
2. The scene perception information fusion decision-making method based on multi-source heterogeneous data in an industrial site according to claim 1, characterized in that, The adaptive preprocessing in step S1 adopts a meta-learning driven processing approach, which includes three sub-steps: adaptive data type recognition, dynamic noise filtering, and scenario-based missing data completion. It adapts to the dynamically changing environment of the industrial site and requires no manual intervention.
3. The scene perception information fusion decision-making method based on multi-source heterogeneous data in an industrial site according to claim 2, characterized in that, The adaptive data type recognition includes the following sub-steps: S11: Extract the three core meta-features of the data: temporal, discrete, and spatial. S12: The recognition model is trained using a model-independent meta-learning algorithm, and new data types are quickly adapted using a pre-set subset of samples. S13: Input the meta-features of the data to be processed into the trained recognition model, and output one or more data types from the following categories: continuous time series, discrete time series, image type, and text type.
4. The scene perception information fusion decision-making method based on multi-source heterogeneous data in an industrial site according to claim 2, characterized in that, The dynamic noise filtering includes the following sub-steps: SQ11: For time-series sensor data, dynamically adjust the wavelet threshold based on the device's operating status to filter high-frequency components after wavelet decomposition and retain effective signals. SQ12: For video data, taking advantage of the fixed background characteristics of industrial scenes, it uses Gaussian mixture background modeling and foreground segmentation algorithms to distinguish moving targets from environmental noise; SQ13: For text logs, an industrial terminology dictionary matching and noise word filtering method is used to retain the core text containing operation instructions and alarm information.
5. The scene perception information fusion decision-making method based on multi-source heterogeneous data in an industrial site according to claim 1, characterized in that, The lightweight scene perception model is constructed using knowledge distillation technology, which includes two stages: cloud-based multimodal Transformer teacher model training and edge-layer lightweight student model distillation. After the teacher model is trained, the student model is compressed by reducing the number of encoder layers, reducing the number of attention heads, and using feature vector quantization, ensuring that the model is lightweight while retaining high accuracy.
6. The scene perception information fusion decision-making method for multi-source heterogeneous data in industrial sites according to claim 5, characterized in that, In step S2, the scene perception logic of the lightweight scene perception model includes: calculating the contribution of different modal features to scene recognition through a lightweight self-attention mechanism. The calculation method is to obtain the mutual information between each modal feature and the scene classification label. The greater the mutual information, the higher the weight. The weighted multimodal features are fused and input into the student model, and the preliminary scene judgment result containing scene type and confidence level is output.
7. The scene perception information fusion decision-making method based on multi-source heterogeneous data in an industrial site according to claim 1, characterized in that, In step S3, the construction and updating of the causal relationship graph includes the following sub-steps: S31: Combine industrial business knowledge to define data features, equipment status, and scenario events as nodes in the cause-effect graph, sort out the causal relationships between nodes and connect them with directed edges to build an initial cause-effect graph; S32: Based on incremental data uploaded from the edge layer, causal discovery is performed through conditional independence test and causal direction determination, new causal relationship nodes and edges are added, and causal strength weights are updated to ensure the timeliness of the causal graph.
8. The scene perception information fusion decision-making method for multi-source heterogeneous data in industrial sites according to claim 1, characterized in that, The processing steps of the inference fusion model include: filtering features that have no causal relationship with the decision target using a backdoor adjustment method based on the causal relationship graph; calculating the causal effect value of each feature node on the decision target node after filtering and using it as the fusion weight, and weighted fusion to obtain the fusion feature vector; constructing a Bayesian network inference model based on the fusion feature vector and the causal relationship graph, and outputting three-dimensional decision information including decision results, visual explanations, and business text explanations.
9. The scene perception information fusion decision-making method based on multi-source heterogeneous data in an industrial site according to claim 1, characterized in that, The dynamic iteration in step S4 includes the following sub-steps: S41: Label the decision execution results fed back from the edge layer, and construct reward and punishment signals based on the labeling results; S42: Combining reward and punishment signals with fused feature vectors, the gradient descent method is used to optimize the conditional probability parameters of the Bayesian network; S43: Retrain the cloud-based teacher model based on feedback data, and perform secondary distillation to update the parameters of the edge-layer student model; S44: The optimized model parameters are sent to the edge layer and cloud layer using incremental transmission to complete the update.
10. The scene perception information fusion decision-making method based on multi-source heterogeneous data in an industrial site according to claim 1, characterized in that, In step S4, the core functions of the interaction layer include encrypted data transmission, incremental synchronization, and breakpoint resume transmission. Encrypted transmission uses a preset encryption algorithm and industrial Ethernet protocol, and the key is updated according to a preset cycle. During incremental synchronization, the edge layer only uploads preprocessed feature data, and in abnormal scenarios, it uploads additional key original data. The cloud layer only sends down the changed model parameters. Breakpoint resume transmission is achieved through local caching at the edge and integrity verification in the cloud. Missing data is automatically resumed after the network is restored.