Multi-source heterogeneous data flow real-time fusion analysis system

Abstract

The invention relates to the technical field of data processing, in particular to a multi-source heterogeneous data flow real-time fusion analysis system which comprises a data source interface unit, a streaming semantic alignment unit, a multi-layer fusion calculation unit, a lightweight knowledge evolution unit and a dynamic resource scheduling unit. The data source interface unit generates stream feature fingerprints and analysis rules through a protocol semantic self-recognition engine, non-preset interface data is analyzed in a zero configuration mode, the stream semantic alignment unit constructs three-dimensional semantic anchor points, multi-modal data soft synchronization is achieved by means of a dynamic alignment matrix, and multi-modal data soft synchronization is achieved. The multi-layer fusion calculation unit projects heterogeneous features and arbitrates conflict data through a domain-hierarchical architecture, the lightweight knowledge evolution unit incrementally updates a knowledge graph and performs closed-loop feedback, the dynamic resource scheduling unit guarantees real-time performance, data is transmitted through cross-domain channel encryption, and the real-time performance and accuracy of multi-source heterogeneous data stream fusion analysis are improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data processing, in particular to a multi-source heterogeneous data stream real-time fusion analysis system. BACKGROUND

[0002] Data processing is an important technology. Under the background of the current popularization of Internet of Things devices and the increasing complexity of data types, this technology is a key foundation for supporting smart industry, intelligent security, and smart transportation scenarios. It can not only break the island state of video, sensor, and text data, avoid decision bias caused by single data dimension analysis, but also ensure rapid conversion of data value through real-time fusion to meet the needs of timeliness and accuracy in industrial equipment fault real-time warning and security event immediate response scenarios, and promote data-driven decision-making from offline analysis to real-time intelligence upgrade. The existing multi-source heterogeneous data stream fusion analysis technology faces core problems such as poor adaptability, low synchronization accuracy, and insufficient fusion reliability in actual application. Traditional systems rely on manually preset interface protocols to parse data streams, lack self-recognition ability for new types of data without preset interfaces, and need manual reconfiguration of parsing rules when new Bluetooth sensors or special format video stream data are added, which not only prolongs the data access period but also easily interrupts existing data processing. At the same time, multi-modal data often appears out of sync due to different collection periods and spatial coordinate systems. Traditional fixed-period sampling methods can further amplify semantic distortion. For example, sensor data is collected once every second, and video is 33 milliseconds per frame. Direct alignment can cause data correlation errors. In addition, low-confidence data and conflict data are not effectively arbitrated during the fusion process, and only simple weighted fusion can easily lead to decision bias. Moreover, the system knowledge graph is fixed for a long time and cannot absorb new entities and relationships in real time. With the accumulation of data, the semantic understanding ability gradually decreases. These problems are layered and superimposed. Slow interface adaptation leads to data access lag, time and space out of sync leads to data correlation distortion, fusion arbitration deficiency leads to inaccurate decision-making, and knowledge is not updated, leading to declining system adaptability. Ultimately, the fusion analysis result cannot meet the accuracy and timeliness requirements of real-time scenarios. In order to solve this technical problem, we provide a multi-source heterogeneous data stream real-time fusion analysis system. SUMMARY

[0003] The purpose of the present application is to provide a multi-source heterogeneous data stream real-time fusion analysis system to solve the problems raised in the background art.

[0004] Since it depends on manual preset interfaces, new type data stream access is slow and easy to interrupt existing processing. Therefore, the protocol semantic self-recognition engine of the data source interface unit in this case dynamically generates stream feature fingerprints and parsing rules and loads them into the pluggable adapter, which can parse new data without configuration and without interrupting existing processing.

[0005] Due to the space-time asynchronization of multi-modal data, fixed sampling amplifies semantic distortion, therefore, the case builds three-dimensional semantic anchor points through a streaming semantic alignment unit, and a dynamic alignment matrix is generated by calculating the weight of space-time context module, so that data soft synchronization can be realized and semantic distortion can be reduced.

[0006] In order to achieve the above object, a multi-source heterogeneous data stream real-time fusion analysis system is provided, comprising: The data source interface unit is used for receiving multi-source heterogeneous data streams, and through the built-in protocol semantic self-recognition engine, the stream characteristic fingerprint is dynamically generated based on the message entropy value and transmission cycle statistical characteristics of the multi-source heterogeneous data streams, the analysis rules of the non-pre-set interface are automatically constructed, and the standardized data stream is output to the streaming semantic alignment unit; The streaming semantic alignment unit extracts the object bounding box coordinates of the video key frame through the cross-modal feature anchor point extractor, analyzes the space grid coordinates of the sensor data, and identifies the named entity of the text log, and constructs three-dimensional semantic anchor points; and by using the space-time context perception alignment module, the space-time correlation weight between the anchor points in the sliding time window is calculated to generate a dynamic alignment matrix to realize soft synchronization, and the space-time alignment feature vector is transmitted to the multi-layer fusion calculation unit to replace the semantic distortion caused by fixed cycle sampling; The multi-layer fusion calculation unit adopts a domain-classification dual-channel fusion architecture, wherein the domain channel projects the heterogeneous features to a unified semantic space through a real-time updated domain knowledge graph to generate an intermediate vector with domain labels, the classification channel performs tensor-level fusion on high-confidence intermediate vectors based on a streaming confidence evaluator, and triggers a reinforcement learning arbitrator for low-confidence and conflict data, the arbitrator dynamically adjusts the multi-source weight according to the feedback reward of the historical fusion result, and outputs the final fusion decision to the lightweight knowledge evolution unit; The lightweight knowledge evolution unit analyzes the new entities and relationships in the fusion decision in real time, reconstructs the knowledge graph topology in a micro-batch manner through an incremental graph embedding update engine, and feeds back the updated graph to the domain channel and anchor point extractor in real time to form a semantic understanding closed loop optimization.

[0007] Compared with the prior art, the beneficial effects of the present application are: Through the protocol semantic self-recognition engine of the data source interface unit, the cross-modal feature anchor point extractor and the space-time context perception alignment module of the streaming semantic alignment unit, the domain-classification dual-channel architecture of the multi-layer fusion calculation unit, and the incremental graph embedding update engine of the lightweight knowledge evolution unit, the effects of zero-configuration analysis of new type data stream, multi-modal data soft synchronization, improvement of fusion decision reliability and dynamic optimization of semantic understanding are achieved, the problems of poor adaptability and low synchronization accuracy of multi-source heterogeneous data stream fusion analysis are effectively solved, and the real-time and efficient fusion and accurate decision of the system on multi-source data are ensured. BRIEF DESCRIPTION OF DRAWINGS

[0008] Figure 1 is a whole block diagram of the present application.

[0009] The meanings of various labels in the figure are as follows: 1, data source interface unit; 2, stream semantic alignment unit; 3, multi-layer fusion computing unit; 4, lightweight knowledge evolution unit; 5, dynamic resource scheduling unit. DETAILED DESCRIPTION

[0010] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0011] The present application provides a multi-source heterogeneous data stream real-time fusion analysis system, please refer to Figure 1 as shown, comprising: The data source interface unit 1 is used for receiving multi-source heterogeneous data streams, and through the built-in protocol semantic self-recognition engine, the stream characteristic fingerprint is dynamically generated based on the message entropy value and transmission cycle statistical characteristics of the multi-source heterogeneous data streams, the analysis rules of the non-preset interface are automatically constructed, and the standardized data stream is output to the stream semantic alignment unit 2; The stream semantic alignment unit 2 extracts the object bounding box coordinates of the video key frame through the cross-modal feature anchor point extractor, analyzes the space grid coordinates of the sensor data, and identifies the named entity of the text log, and constructs a three-dimensional semantic anchor point; and using the spatio-temporal context perception alignment module, the spatio-temporal correlation weight between the anchor points is calculated in the sliding time window, a dynamic alignment matrix is generated to realize soft synchronization, and the spatio-temporal alignment feature vector is transmitted to the multi-layer fusion computing unit 3 to replace the semantic distortion caused by fixed cycle sampling; The multi-layer fusion computing unit 3 adopts a domain-level dual-channel fusion architecture, wherein the domain channel projects the heterogeneous features to a unified semantic space through a real-time updated domain knowledge graph to generate an intermediate vector with a domain label, the hierarchical channel performs tensor-level fusion on high-confidence intermediate vectors based on a stream confidence evaluator, and triggers a reinforcement learning arbitrator for low-confidence and conflict data, the arbitrator dynamically adjusts the multi-source weight according to the feedback reward of the historical fusion result, and outputs the final fusion decision to the lightweight knowledge evolution unit 4; The lightweight knowledge evolution unit 4 analyzes the new entities and relationships in the fusion decision in real time, restructures the knowledge graph topology in a micro-batch manner through an incremental graph embedding update engine, and feeds back the updated graph to the domain channel and anchor extractor in real time, forming a semantic understanding closed loop optimization.

[0012] The protocol semantic self-recognition engine comprises a flow fingerprint dynamic modeling module and a rule generation module. The flow fingerprint dynamic modeling module performs pattern mining on the entropy value and transmission cycle statistical characteristics of the multi-source heterogeneous data stream through an unsupervised clustering algorithm, generates a flow characteristic fingerprint with a unique identifier, and the rule generation module automatically constructs an analysis rule set including data field segmentation rules, checksum verification rules, etc. based on the flow characteristic fingerprint, using syntax tree deduction technology, and dynamically loads the analysis rule set to the pluggable protocol adapter to realize zero-configuration analysis of the non-preset interface data stream.

[0013] The cross-modal feature anchor extractor comprises a multi-modal feature mapping module. The module extracts the texture and geometric features of the object in the video key frame to generate a bounding box coordinate through a convolutional neural network with shared weights. At the same time, a spatial encoder is used to map the original coordinates of the sensor data to a standard grid coordinate system, and the text entity position information output by the named entity recognition model is combined to construct a three-dimensional semantic anchor with the spatial grid as the reference, the timestamp as the sequence axis, and the entity type as the semantic dimension, forming a unified space-time reference system for cross-modal data.

[0014] The space-time context-aware alignment module further comprises a space-time correlation weight calculator and a dynamic alignment matrix generator. The space-time correlation weight calculator calculates the space-time correlation weight according to the spatial distance, time interval and semantic similarity of adjacent data points in the three-dimensional semantic anchor within a sliding time window. The dynamic alignment matrix generator generates an alignment matrix carrying a confidence score by interpolating and compensating non-interval data streams based on the weight, and synchronously transmits the matrix and the space-time alignment feature vector to the multi-layer fusion calculation unit 3.

[0015] The domain knowledge graph of the domain channel comprises a real-time topology update layer and a semantic projection layer. The real-time topology update layer receives the graph structure change information fed back by the lightweight knowledge evolution unit 4, and dynamically adjusts the entity relationship network. The semantic projection layer embeds the heterogeneous feature vector into a unified semantic space with domain entities as nodes and relationships as edges through a graph neural network, generating an intermediate vector carrying entity type labels and relationship weights. The dimension of the intermediate vector is consistent with the entity dimension of the knowledge graph.

[0016] The reinforcement learning arbitrator comprises a conflict detector and a weight dynamic adjuster. The conflict detector identifies low-confidence or conflicting data based on the confidence scores output by the streaming confidence evaluator and the cosine similarity between the intermediate vectors. The weight dynamic adjuster dynamically updates the contribution weight of the multi-source data in the fusion decision through the Q-learning algorithm according to the reward signal generated by the accuracy feedback of the historical fusion decision in the actual business scenario, and outputs the arbitration result to the lightweight knowledge evolution unit 4.

[0017] The incremental graph embedding update engine includes a micro-batch topology reconfigurator and a closed-loop feedback controller. The micro-batch topology reconfigurator extracts new entity relationship triples from the fusion decision, compares the differences between the existing graph structure through negative sampling technology, and updates the entity embedding vector and the relationship matrix in a micro-batch incremental manner. The closed-loop feedback controller pushes the updated graph to the real-time topology update layer and the multi-modal feature mapping module of the domain channel in real time, optimizing the accuracy of the next semantic projection and anchor extraction.

[0018] Further comprising a dynamic resource scheduling unit 5, which includes a pipeline monitor and an elastic operator controller. The pipeline monitor collects the processing delay, data backlog and computing node load from the data source interface unit 1 to the lightweight knowledge evolution unit 4 in real time. The elastic operator controller predicts the data flow peak value based on the monitoring data, dynamically adjusts the thread parallelism and memory allocation ratio of the stream processing operator, and ensures that the end-to-end processing delay is stable below the set threshold.

[0019] The pluggable protocol adapter supports a protocol hot switching mechanism. When the protocol semantic self-recognition engine detects a new type of data stream, the protocol hot switching mechanism loads the newly generated parsing rule set to an idle protocol adapter instance without interrupting the existing data stream processing, realizing parallel and seamless switching of new and old data stream parsing.

[0020] The system deploys a cross-domain security isolation channel. The channel establishes an encrypted tunnel between the data source interface unit 1 and the stream semantic alignment unit 2, dynamically updates the encryption key through quantum key distribution technology, ensures the non-tamperability of standardized data streams during transmission, and the encrypted tunnel is transparent to the dynamic resource scheduling unit 5 to avoid affecting real-time performance.

[0021] It needs to be further explained that after the multi-layer fusion computing unit 3 outputs the final fusion decision, the lightweight knowledge evolution unit 4 needs to undertake the semantic update requirement in the decision, and its specific implementation first analyzes the new entity and relationship in the fusion decision in real time. The fusion decision contains the comprehensive judgment results of multiple modalities such as video, sensor and text. For example, on May 20, 2024, at 14:30, device C collected temperature 28℃ in area C, the system identified device C and area C as new entities from the text description through entity extraction algorithm (based on BERT lightweight model), determined the association between the two through relationship extraction rules, and verified the attributes of the new entity by combining the spatial coordinates in the sensor data and the bounding box information of the video key frame. Device C is a temperature sensor and area C is a 10m x 10m grid area, ensuring the authenticity of the new entity and relationship. Through the incremental graph embedding update engine, the knowledge graph topology structure is reconstructed in a micro-batch manner. The core of the incremental graph embedding update engine is to avoid the real-time decline caused by full update of the graph. The micro-batch manner means that each 100 fusion decisions are a micro-batch, considering the update efficiency and data integrity. The micro-batch topology reconstructor extracts new entity relationship triples (such as 〈device C, located in, area C>) from the micro-batch fusion decision, generates false triples (such as 〈device C, located in, area B>) through negative sampling technology, compares them with existing entity relationships in the knowledge graph (such as 〈device A, located in, area B>), and selects the real new triples. Then, the incremental graph embedding algorithm is used to update only the embedding vector of the new entity, that is, to map device C to a 128-dimensional semantic vector and a matrix of new relationships, and to update the weight matrix of the located relationship, rather than to reconstruct the entire graph, thereby greatly reducing the computational overhead. The updated graph is fed back to the domain channel and the anchor extractor in real time, forming a semantic understanding closed loop optimization. The updated graph contains the new entity device C and its associated relationship, which is pushed to the real-time topology update layer of the domain channel. When the domain channel processes the sensor data of device C next time, it can quickly project its features to the unified semantic space without relearning entity attributes. After being pushed to the multi-modal feature mapping module of the cross-modal feature anchor extractor, the next time the video key frame or text log is extracted, the features of device C can be directly identified, such as the appearance features of device C in the video and the named entity of device C in the text, improving the accuracy of anchor extraction and forming a closed loop of fusion decision → graph update → semantic understanding optimization → more accurate fusion decision. The data source interface unit 1 is the data entrance of the system, and the first problem to be solved is the analysis of multi-source heterogeneous data streams, which depends on the core capability of the protocol semantic self-recognition engine. The engine includes a stream fingerprint dynamic modeling module and a rule generation module. The stream fingerprint dynamic modeling module uses unsupervised clustering algorithm to mine the entropy value and transmission cycle statistical characteristics of multi-source heterogeneous data streams, and generates stream feature fingerprints with unique identification. The message entropy value is the degree of confusion of message data, and the transmission cycle statistical characteristic is the time interval of data transmission.The module adopts K-means unsupervised clustering algorithm, data streams with similar entropy values and transmission periods are classified into one class, for example, data streams with entropy value 1.2 and period 1 second are classified into industrial sensor class, a unique stream feature fingerprint is generated for each class of data stream, the fingerprint contains entropy value range, period range, data length core feature, the fast identification of data stream type is realized, the rule generation module automatically constructs the analysis rule set including data field segmentation rule, checksum verification rule based on stream feature fingerprint, using syntax tree derivation technology, for example, for industrial sensor class fingerprint F001, the module first determines the data format through packet sample analysis, then generates the field segmentation rule using syntax tree derivation technology, and the rule is divided by comma, the first field is Unix timestamp, the second is floating point temperature, the third is floating point humidity, and the fourth is 8-bit checksum, at the same time, the checksum verification rule is derived, the sum of the values of the first three fields is taken to the last 8 bits, which is compared with the fourth field, to form a complete analysis rule set, and the analysis rule set is dynamically loaded into the pluggable protocol adapter, realizing zero-configuration analysis of the data stream of the interface not preset, the pluggable protocol adapter supports multi-protocol parallel analysis, when loading, the rule set is compiled into executable code through the dynamic loading interface of the adapter, without restarting the adapter, for example, the rule set of F001 is loaded into adapter instance 1, when a new industrial sensor data stream is accessed, adapter 1 directly analyzes the data according to the rule, extracts the timestamp and temperature field, and outputs the standardized JSON format data stream, without manual configuration of interface parameters, realizing zero-configuration analysis, the stream semantic alignment unit 2 needs to solve the problem of time and space asynchronization of multi-modal data, the core relies on cross-modal feature anchor extractor, which contains a multi-modal feature mapping module that is the key to building a unified time and space reference system, the module extracts the texture and geometric features of the object in the key frame of the video through a shared weight convolutional neural network to generate a bounding box coordinate, the shared weight convolutional neural network adopts a lightweight ResNet-18 model, and the model weight is pre-trained on multi-modal data to ensure that the texture and geometric features of the object in the key frame of the video can be extracted at the same time, and the bounding box coordinate of the object is generated through a bounding box regression algorithm (such as (x1=100, y1=200, x2=300, y2=400), corresponding to the pixel coordinates of the video frame, accurately positioning the position of the object in the video, at the same time, a spatial encoder is used to map the original coordinates of the sensor data to a standard grid coordinate system, the original coordinates of the sensor data are, for example, the latitude and longitude (north latitude 30°, east longitude 120°) of GPS, and the three-dimensional coordinates (x=5m, y=3m, z=1m) of laser radar, the standard grid coordinate system is a 1m×1m×2m cubic grid preset by the system, the spatial encoder maps the original coordinates to grid coordinates through a coordinate conversion algorithm, such as (x=5, y=3, z=0), which corresponds to the 5th column, 3rd row and 0th layer grid, realizing the spatial standardization of sensor data, and combining with the text entity position information output by the named entity recognition model,A three-dimensional semantic anchor point is constructed, taking a spatial grid as a reference, a timestamp as a sequence axis, and an entity type as a semantic dimension, to form a unified space-time reference system for cross-modal data. A named entity recognition model adopts a BERT-tiny model to identify a device A5-3 grid entity from a text log and output entity position information. The 5-3 grid corresponds to grid coordinates (x=5, y=3). The module aligns the bounding box coordinates of the video, the grid coordinates of the sensor, and the entity position information of the text according to the timestamp, constructs a three-dimensional semantic anchor point, and formats it as <spatial grid coordinates (x, y, z), timestamp t, entity type>. For example, <(5, 3, 0), 1699999999876 ms, sensor device>. This makes multi-modal data share the same space-time reference system and avoids semantic bias. After constructing the three-dimensional semantic anchor point, the space-time context perception alignment module needs to further implement soft synchronization of non-interval data streams. The space-time correlation weight calculator and the dynamic alignment matrix generator work together. The space-time correlation weight calculator calculates the space-time correlation weight in a sliding time window according to the spatial distance, time interval, and semantic similarity of adjacent data points in the three-dimensional semantic anchor point. The sliding time window is set to 5 seconds, taking into account real-time performance and context integrity. To calculate the weight, all three-dimensional semantic anchor points in the window are first obtained. For example, anchor point A <(5, 3, 0), t1, device> and anchor point B <(5, 4, 0), t2, device>. The spatial distance is calculated according to the grid step. Anchor points A and B are the same in the x-axis and differ by 1 grid in the y-axis, with a distance of 1. The time interval is |t2-t1|. The semantic similarity is calculated according to the entity type (both devices, similarity 1.0) and the attribute (both temperature sensors, similarity 0.9). The weight formula integrates the influence of the three factors (weight=(1 / spatial distance)×(1 / time interval)× semantic similarity). For example, the weight of anchor points A and B is (1 / 1)×(1 / 0.5)×0.9=1.8. The higher the weight, the stronger the correlation between the two anchor points. The dynamic alignment matrix generator performs interpolation compensation on the non-interval data stream based on the weight to generate an alignment matrix carrying a confidence score. The non-interval data stream may include, for example, one anchor point per second for a sensor and one anchor point every 33 ms for a video. The matrix generator first divides the time slices uniformly according to the time axis. For time slices without original anchor points (e.g., t1+100 ms without a sensor anchor point), interpolation is performed according to the weight of adjacent anchor points (e.g., the weight of anchor points A and B is used to calculate the sensor anchor point of the time slice). Meanwhile, a confidence score is assigned to each matrix element (corresponding to the anchor point of the time slice-mode). The confidence score of the original anchor point is 0.9 (true data), and the confidence score of the interpolated anchor point is 0.7 (with an estimated error). The final alignment matrix has dimensions of time slice number×modality number. Each element contains anchor point features and a confidence score. The matrix is synchronized with the space-time alignment feature vector and transmitted to the multi-layer fusion calculation unit 3 to provide space-time synchronized and confidence-based feature data for subsequent fusion. The domain channel of the multi-layer fusion calculation unit 3 needs to rely on the domain knowledge graph to realize semantic unification of heterogeneous features.The graph includes a real-time topology update layer and a semantic projection layer. The real-time topology update layer receives graph structure change information fed back by the lightweight knowledge evolution unit 4, dynamically adjusts the entity relationship network, for example, the lightweight knowledge evolution unit 4 adds a new entity device C and a relationship device C-located-area C, after feeding back to the real-time topology update layer, the entity relationship manager in the layer first verifies the legality of the new entity (such as whether the device C has corresponding sensor data support), then adds the device C to the entity library, adds the edge between the device C and the area C in the relationship network, and updates the relationship weight of the associated entity (such as the original device of the area C is only device D, after adding device C, the relationship weight sum of area C-device is re-normalized), ensuring that the entity relationship network reflects the latest semantic association in real time. The semantic projection layer embeds the heterogeneous feature vector into a unified semantic space with domain entities as nodes and relationships as edges through a graph neural network, generating an intermediate vector carrying entity type labels and relationship weights. The dimension of the intermediate vector is consistent with the entity dimension of the knowledge graph. Heterogeneous feature vectors such as 256-dimensional visual features of video key frames, 64-dimensional numerical features of sensor data, and 128-dimensional language features of text logs; The graph neural network uses graph convolution network (GCN) to input heterogeneous feature vectors, and transfers feature information through the association relationship of entities in the graph (such as device A-located-area B), for example, mapping the 64-dimensional feature of the sensor data to a 128-dimensional vector consistent with the graph entity. The generated intermediate vector not only contains feature information, but also carries entity type labels (such as the label of device A is temperature sensor) and relationship weights (such as the weight of device A-area B is 0.8), ensuring that features of different modalities are comparable and integrable at the semantic level. The hierarchical channel of the multi-layer fusion calculation unit 3 needs to solve the problem of low confidence and conflict data through a reinforcement learning arbitrator. The arbitrator includes a conflict detector and a weight dynamic adjuster: the conflict detector identifies low-confidence or conflict data based on the confidence score output by the streaming confidence evaluator and the cosine similarity between intermediate vectors. The streaming confidence evaluator outputs a confidence score based on data integrity (such as whether the sensor data is missing fields) and consistency with historical data (such as whether the current temperature is within the historical fluctuation range). Generally, a score <0.6 is considered low-confidence data. The cosine similarity between intermediate vectors is used to identify conflicts. If the similarity between two intermediate vectors (from sensors and text respectively) of the same entity (such as device A) is <0.3, it indicates that there is a conflict (such as the sensor shows a temperature of 28°C and the text log records 25°C). The conflict detector marks low-confidence and conflict data as pending arbitration data, and the rest as directly integrable data. The weight dynamic adjuster dynamically updates the contribution weight of multi-source data in the fusion decision through the Q-learning algorithm, according to the correctness feedback of historical fusion decisions in the actual business scenario to generate a reward signal. For example, in a certain batch of fusion decisions, the weight of sensor data is 0.5 and the weight of text data is 0.5, and the actual business scenario verifies the correctness rate of 80%,A small positive reward (+1) is given, the sensor weight is adjusted to 0.6 and the text to 0.4 next time, the correct rate is increased to 90%, a large positive reward (+3) is given, and if the correct rate is reduced to 70%, a negative reward (-2) is given. The weight adjuster updates the Q table according to the reward signal, determines the optimal weight distribution next time, and outputs the arbitration result to the lightweight knowledge evolution unit 4. The arbitration result includes the final fusion weight and conflict processing logic, which provides the basis for graph update (such as frequently conflicting entity relationships that need to be reverified). The incremental graph embedding update engine is the core component of the lightweight knowledge evolution unit 4, which includes a micro-batch topology reconfigurator and a closed-loop feedback controller. The micro-batch topology reconfigurator extracts new entity relationship triples from the fusion decision, compares the differences with the existing graph structure through negative sampling technology, and updates the entity embedding vector and relationship matrix in a micro-batch incremental manner. When extracting triples, invalid data (such as repeated or contradictory triples) is filtered first, and then false triples similar to the true triple structure are generated (such as true <Device C, located in, Area C> and false <Device C, located in, Area B>). The true and false triples are input into the embedding update model (such as the incremental version of TransE), which only updates the embedding vector of the new entity (Device C) (so that it is close to Area C and far from Area B in the semantic space) and the matrix involved in the new relationship (updates the conversion matrix of the located relationship), avoiding the computational delay caused by full update. The size of the micro-batch is usually set to 100-200 triples, balancing the update efficiency and accuracy. The closed-loop feedback controller pushes the updated graph to the real-time topology update layer of the domain channel and the multi-modal feature mapping module in real time, optimizing the accuracy of the next semantic projection and anchor extraction. When pushing, the publish-subscribe mode is used, and the domain channel and the multi-modal feature mapping module act as the subscriber and receive the graph update notification in real time. After receiving the notification, the domain channel can directly call the embedding vector of the new entity during the next semantic projection without the need for retraining. After receiving the notification, the multi-modal feature mapping module can identify the features of the new entity (such as the appearance of Device C in the video and the name of Device C in the text) during the next anchor extraction, improving the recall rate and accuracy of anchor extraction, forming a positive cycle of semantic understanding. To ensure the real-time performance of the system when the data flow fluctuates, the dynamic resource scheduling unit 5 needs to optimize resource allocation in real time. It includes a pipeline monitor and an elastic operator controller. The pipeline monitor collects the processing delay, data backlog, and computing node load from the data source interface unit 1 to the lightweight knowledge evolution unit 4 in real time, with a collection frequency of 1 second per second. The processing delay includes the internal processing time of each unit (such as data source interface parsing delay and stream alignment delay) and the transmission delay between units. The data backlog is the number of unprocessed data in the input queue of each unit (such as 50 data backlog in the data source interface and 30 data backlog in the stream alignment). The computing node load includes CPU usage, memory occupancy, and network bandwidth usage (such as 75% CPU usage and 60% memory usage). The monitoring data is stored through the Prometheus time series database.Support real-time query and trend analysis, elastic operator controller based on monitoring data to predict data flow peak, through dynamic adjustment of flow processing operator thread parallelism and memory allocation ratio, to ensure that the end-to-end processing delay is stable below the set threshold, when predicting the data flow peak, using sliding window prediction method (such as predicting the peak of the next 5 minutes based on the average data volume of the past 5 minutes), when adjusting the thread parallelism, if the predicted peak increases by 2 times, the thread number of the flow processing operator (such as parsing operator, alignment operator) is adjusted from 4 to 8, when adjusting the memory allocation, if the data backlog exceeds 100, the operator memory is adjusted from 2GB to 4GB, to ensure that the data can be processed quickly, the set threshold is usually 500ms (to meet the needs of most real-time scenarios), the controller ensures that the end-to-end delay does not exceed the threshold through continuous monitoring and adjustment, avoiding system lag caused by data backlog. The pluggable protocol adapter needs to support protocol hot switching mechanism to cope with new types of data flow, the specific implementation is: When the protocol semantic self-recognition engine detects a new type of data flow, the engine compares the flow feature fingerprints (if the similarity between the new data flow fingerprint and the existing fingerprint is less than 0.5, it is determined as a new type), for example, detecting a Bluetooth format device data flow (fingerprint F003, which is different from the existing F001 and F002), immediately triggering the protocol hot switching process, the protocol hot switching mechanism loads the newly generated parsing rule set to the idle protocol adapter instance without interrupting the existing data flow processing. The system usually deploys multiple adapter instances (such as 3, supporting parallel processing of different protocols), when hot switching, first check the idle instance (such as instance 3 is idle), compile the parsing rule set of the new type of data flow (such as field segmentation, verification rules of Bluetooth data) into executable code for instance 3, inject instance 3 through the dynamic loading interface, the loading process does not affect the normal operation of instance 1 (processing industrial sensors) and instance 2 (processing video stream), realizing parallel and seamless switching of new and old data flow parsing, after loading is completed, the new type of data flow (Bluetooth data) is automatically routed to instance 3 for processing, and the standardized data flow is output, the existing data flow is still processed by the original instance, the whole process is uninterrupted and no data is lost, realizing seamless switching of new protocol without affecting existing business, improving the adaptability of the system to multi-source data flow, to ensure the safety of cross-unit data transmission, the system deploys cross-domain security isolation channel, the specific implementation is: The channel establishes an encryption tunnel between the data source interface unit 1 and the streaming semantic alignment unit 2, the encryption tunnel is constructed by using an IPsec protocol, VPN gateways are configured at both ends of the tunnel (the data source interface and the streaming semantic alignment), the standardized data stream output by the data source interface is first encrypted by the gateway (using an AES-256 encryption algorithm), and then transmitted to the gateway of the streaming semantic alignment unit 2 through the tunnel, and after decryption, the subsequent processing is entered, so that the data is not stolen in the transmission process, the encryption key is dynamically updated by using a quantum key distribution technology, so that the standardized data stream is not tampered with in the transmission process, a quantum key distribution (QKD) system is linked with the encryption tunnel, a new quantum key (with a key length of 256 bits) is generated every hour, is transmitted to the gateways at both ends through a special quantum channel, and the old key is automatically replaced, if the data transmitted through the tunnel is tampered with, the randomness of the quantum key will cause decryption failure, the system immediately triggers an alarm, and at the same time, the data transmission is suspended until the key is re-synchronized, so that the data integrity is fundamentally guaranteed, and the encryption tunnel is transparent to the dynamic resource scheduling unit 5, so as to avoid affecting the real-time performance, the transparent means that the dynamic resource scheduling unit 5 does not need to participate in the configuration and management of the encryption tunnel, when the scheduling unit collects and processes the delay and load data, the original monitoring data of each unit is directly read, and a decryption operation is not needed, the encryption / decryption process of the tunnel is processed by a hardware acceleration module, the delay is controlled to be less than 1 ms, far lower than the end-to-end delay threshold of the system, and the real-time performance of the resource scheduling is not affected, so that the balance between safety and real-time performance is realized.

[0022] In the application, the data source interface unit 1 generates a flow characteristic fingerprint and an analysis rule through a protocol semantic self-recognition engine, zero-configuration analysis is performed on preset interface data, the streaming semantic alignment unit 2 constructs three-dimensional semantic anchor points, realizes soft synchronization of multi-modal data by using a dynamic alignment matrix, the multi-layer fusion calculation unit 3 projects heterogeneous characteristics in a domain-grading architecture and arbitrates conflict data, the lightweight knowledge evolution unit 4 incrementally updates a knowledge graph and feeds back in a closed loop, and the dynamic resource scheduling unit 5 guarantees real-time performance, and the cross-domain channel encrypts and transmits data, so that the real-time performance and accuracy of multi-source heterogeneous data stream fusion analysis are improved.

[0023] The basic principle, main features and advantages of the application are shown and described above. It should be understood by those skilled in the art that the application is not limited by the above examples, the above examples and descriptions in the specification are only preferred examples of the application, and are not used to limit the application, various changes and improvements of the application fall within the scope of the application without departing from the spirit and scope of the application. The scope of protection of the application is defined by the appended claims and their equivalents.

Claims

1. A real-time fusion analysis system for multi-source heterogeneous data streams, characterized in that, include: The data source interface unit (1) is used to receive multi-source heterogeneous data streams and dynamically generate stream feature fingerprints based on the message entropy value and transmission cycle statistical characteristics of the multi-source heterogeneous data streams through the built-in protocol semantic self-recognition engine. It automatically constructs parsing rules for unpreset interfaces and outputs standardized data streams to the streaming semantic alignment unit (2). The streaming semantic alignment unit (2) extracts the bounding box coordinates of the video keyframes, parses the spatial grid coordinates of the sensor data, and identifies named entities of the text logs through the cross-modal feature anchor extractor to construct three-dimensional semantic anchors; and uses the spatiotemporal context-aware alignment module to calculate the spatiotemporal correlation weights between anchors within the sliding time window, generates a dynamic alignment matrix to achieve soft synchronization, and passes the spatiotemporal alignment feature vector to the multi-layer fusion computing unit (3) to replace the semantic distortion caused by fixed period sampling; The multi-layer fusion computing unit (3) adopts a domain-level dual-channel fusion architecture. The domain channel projects heterogeneous features onto a unified semantic space through a real-time updated domain knowledge graph to generate intermediate vectors with domain labels. The hierarchical channel performs tensor-level fusion on high-confidence intermediate vectors based on a streaming confidence evaluator and triggers a reinforcement learning arbitrator on low-confidence and conflicting data. The arbitrator dynamically adjusts the multi-source weights based on the feedback rewards of historical fusion results and outputs the final fusion decision to the lightweight knowledge evolution unit (4). The lightweight knowledge evolution unit (4) analyzes new entities and relationships in the fusion decision in real time. Through the incremental graph embedding update engine, it reconstructs the knowledge graph topology in a micro-batch manner and feeds the updated graph back to the domain channel and anchor extractor in real time, forming a semantic understanding closed-loop optimization.

2. A real-time fusion analysis system for multi-source heterogeneous data streams according to claim 1, characterized in that: The protocol semantic self-identification engine includes a flow fingerprint dynamic modeling module and a rule generation module. The flow fingerprint dynamic modeling module uses an unsupervised clustering algorithm to perform pattern mining on the entropy value and transmission cycle statistical characteristics of multi-source heterogeneous data streams, generating flow feature fingerprints with unique identifiers. Based on the flow feature fingerprints, the rule generation module uses syntax tree derivation technology to automatically construct a set of parsing rules, including data field segmentation rules, checksums, and verification rules, and dynamically loads the parsing rule set into the pluggable protocol adapter to achieve zero-configuration parsing of data streams without preset interfaces.

3. A real-time fusion analysis system for multi-source heterogeneous data streams according to claim 2, characterized in that: The cross-modal feature anchor extractor includes a multimodal feature mapping module. This module extracts the texture and geometric features of objects in video keyframes through a convolutional neural network with shared weights to generate bounding box coordinates. At the same time, it uses a spatial encoder to map the original coordinates of sensor data to a standard grid coordinate system. Combined with the text entity position information output by the named entity recognition model, it constructs a three-dimensional semantic anchor with the spatial grid as the reference, the timestamp as the sequence axis, and the entity type as the semantic dimension, forming a unified spatiotemporal reference system for cross-modal data.

4. A real-time fusion analysis system for multi-source heterogeneous data streams according to claim 3, characterized in that: The spatiotemporal context-aware alignment module further includes a spatiotemporal correlation weight calculator and a dynamic alignment matrix generator. The spatiotemporal correlation weight calculator calculates the spatiotemporal correlation weight based on the spatial distance, time interval and semantic similarity of adjacent data points in the three-dimensional semantic anchor within the sliding time window. The dynamic alignment matrix generator performs interpolation compensation on the non-interval data stream based on the weight, generates an alignment matrix carrying a confidence score, and synchronously transmits the matrix and the spatiotemporal alignment feature vector to the multi-layer fusion computing unit (3).

5. A real-time fusion analysis system for multi-source heterogeneous data streams according to claim 4, characterized in that: The domain knowledge graph of the domain-specific channel includes a real-time topology update layer and a semantic projection layer. The real-time topology update layer receives graph structure change information from the lightweight knowledge evolution unit and dynamically adjusts the entity relationship network. The semantic projection layer embeds heterogeneous feature vectors into a unified semantic space with domain entities as nodes and relationships as edges through a graph neural network, generating an intermediate vector carrying entity type labels and relationship weights. The dimension of the intermediate vector is consistent with the entity dimension of the knowledge graph.

6. A real-time fusion analysis system for multi-source heterogeneous data streams according to claim 5, characterized in that: The reinforcement learning arbitrator includes a conflict detector and a weight dynamic adjuster. The conflict detector identifies low-confidence or conflicting data based on the confidence score output by the streaming confidence evaluator and the cosine similarity between intermediate vectors. The weight dynamic adjuster generates a reward signal based on the feedback of the accuracy of historical fusion decisions in actual business scenarios through the Q-learning algorithm, dynamically updates the contribution weight of multi-source data in fusion decisions, and outputs the arbitration result to the lightweight knowledge evolution unit (4).

7. A real-time fusion analysis system for multi-source heterogeneous data streams according to claim 6, characterized in that: The incremental graph embedding update engine includes a micro-batch topology reconstructor and a closed-loop feedback controller. The micro-batch topology reconstructor extracts new entity relation triples from the fusion decision and compares the differences in the existing graph structure using negative sampling technology. It updates the entity embedding vectors and relation matrices in a micro-batch incremental manner. The closed-loop feedback controller pushes the updated graph to the real-time topology update layer and multimodal feature mapping module of the domain channel in real time to optimize the accuracy of the next semantic projection and anchor point extraction.

8. A real-time fusion analysis system for multi-source heterogeneous data streams according to claim 7, characterized in that: It further includes a dynamic resource scheduling unit (5), which includes a pipeline monitor and an elastic operator controller. The pipeline monitor collects the processing latency, data backlog and computing node load from the data source interface unit (1) to the lightweight knowledge evolution unit (4) in real time. The elastic operator controller predicts the peak value of the data flow based on the monitoring data and ensures that the end-to-end processing latency is consistently lower than the set threshold by dynamically adjusting the thread parallelism and memory allocation ratio of the stream processing operator.

9. A real-time fusion analysis system for multi-source heterogeneous data streams according to claim 8, characterized in that: The pluggable protocol adapter supports a protocol hot-switching mechanism. When the protocol semantic self-recognition engine detects a new type of data stream, the protocol hot-switching mechanism loads the newly generated parsing rule set into the idle protocol adapter instance without interrupting the processing of the existing data stream, thereby achieving parallel and seamless switching between parsing the old and new data streams.

10. A real-time fusion analysis system for multi-source heterogeneous data streams according to claim 8, characterized in that: The system deploys a cross-domain secure isolation channel. This channel establishes an encrypted tunnel between the data source interface unit (1) and the streaming semantic alignment unit (2). The encryption key is dynamically updated through quantum key distribution technology to ensure the immutability of the standardized data stream during transmission. At the same time, the encrypted tunnel is transparent to the dynamic resource scheduling unit (5) to avoid affecting real-time performance.

Images