A data governance method and system for distributed data architecture

Abstract

The present invention relates to the field of data processing technology and discloses a data governance method and system for a distributed data architecture. The method comprises performing spatiotemporal feature encoding on multimodal data generated by a device to obtain device data carrying the spatiotemporal feature encoding; constructing a multi-level association bitmap based on the device data carrying the spatiotemporal feature encoding; and realizing collaborative computing between edge physical nodes and central governance nodes of the distributed data architecture based on the constructed multi-level association bitmap. The present invention constructs a multi-level association bitmap through spatiotemporal feature encoding, thereby solving the problem of association analysis distortion caused by storage sharding of edge nodes and realizing efficient collaboration between data sharding storage of edge physical nodes and global association computing of central governance nodes.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and more specifically, to a data governance method and system for a distributed data architecture. Background Art

[0002] In today's digital age, the Industrial Internet and the Internet of Things are developing rapidly, and the amount of multimodal data generated by devices is exploding. Efficiently storing and analyzing this data within a distributed data architecture has become a critical issue that needs to be addressed.

[0003] In the prior art, a Chinese patent application with publication number CN116257509A discloses a method and apparatus for implementing data governance, which manages data through data access, integration, classification, and data model construction, thereby improving the value and utilization of data assets. However, this prior art does not address the problem of spatiotemporal correlation fragmentation caused by sharded storage of multimodal data on edge nodes under a distributed architecture. In actual application scenarios, when data is stored in a dispersed manner on multiple edge nodes, the correlation of the data in the time and space dimensions is broken, making it difficult to achieve accurate correlation analysis.

[0004] A Chinese patent application with publication number CN119829551A discloses a method for constructing a data index for distributed data storage. It focuses on index construction of distributed data storage systems, and uses high-order graph theory, consistent hashing algorithms, and other methods to optimize storage node load balancing and data storage strategies to ensure the availability, scalability, and reliability of the system. However, this prior art does not take into account the problem of broken cross-modal logical associations. In a multimodal data environment, such as in a smart factory where multiple modalities such as image data and sensor data of equipment coexist, the logical association between different modal data is crucial for in-depth analysis of the operating status of the equipment.

[0005] However, when processing multimodal data in a distributed data architecture, we currently face severe challenges such as the fragmentation of spatiotemporal associations, the breakdown of cross-modal logical associations, and the technical separation of data storage and associated computing. Summary of the Invention

[0006] This method is applicable to distributed monitoring scenarios with dense equipment in the Industrial Internet of Things (IIoT), such as equipment monitoring on smart factory production lines and status management of wind power equipment clusters. In these scenarios, multi-source heterogeneous devices generate massive amounts of multimodal data in real time, requiring efficient data governance to achieve early fault warning, locate anomaly propagation, and achieve system-level collaborative decision-making.

[0007] To overcome the aforementioned shortcomings of the prior art, the present invention provides a data governance method and system for a distributed data architecture. This method constructs a multi-level association bitmap through spatiotemporal feature encoding (hash spatiotemporal anchoring and dynamic time window slicing), integrating physical storage locations, cross-modal logical associations within devices, and topological relationships between devices to form a three-dimensional association network covering the physical, logical, and topological layers. This solution achieves efficient collaboration between sharded storage at edge physical nodes and global computation at central governance nodes, resolving the issues of data association discontinuity and analytical distortion in traditional architectures. It significantly improves the efficiency of association analysis for multimodal data, provides underlying governance support for device fault prediction, anomaly propagation location, and system-level collaborative decision-making, and overcomes the technical separation between storage and analysis in distributed architectures.

[0008] To achieve the above object, the present invention provides the following technical solutions:

[0009] A data governance method for a distributed data architecture, comprising:

[0010] Performing spatiotemporal feature encoding on the multimodal data generated by the device to obtain device data carrying the spatiotemporal feature encoding; the spatiotemporal feature encoding includes hash spatiotemporal anchoring and dynamic time window slicing;

[0011] Construct a multi-level correlation bitmap based on device data carrying spatiotemporal feature encoding;

[0012] Based on the constructed multi-level association bitmap, collaborative computing between edge physical nodes and central governance nodes of the distributed data architecture is achieved.

[0013] Furthermore, the device data carrying the spatiotemporal feature encoding is written in the form of data blocks, and the data blocks at least include vibration data blocks and thermal imaging data blocks.

[0014] Furthermore, the method for performing the hash time-space anchoring includes:

[0015] Get the device ID and generate a 128-bit hash value based on the device ID, which is recorded as the device hash value;

[0016] The lower 8 bits of the device hash value are intercepted as the physical node mapping identifier, which is recorded as the node mapping code;

[0017] The multimodal data generated by the same device within a preset period T are marked with the same node mapping code; the period T is the basic time unit for data governance.

[0018] Furthermore, the method for performing the dynamic time window slicing includes:

[0019] Taking the first activation moment of the device as the time origin, the device operation cycle is divided into multiple sliding time windows according to the preset period T;

[0020] When a data block generated by a device is written, the deviation Δτ between the timestamp of the data block and the end of the current time window is calculated. Based on the deviation Δτ, the time window to which the data block belongs is determined, and dynamic data sharding is performed.

[0021] Furthermore, the multi-level association bitmap includes a first-level bitmap, a second-level bitmap and a third-level bitmap; the first-level bitmap is a physical association bitmap, the second-level bitmap is a logical association bitmap, and the third-level bitmap is a topological association bitmap.

[0022] Furthermore, the method for constructing the first-level bitmap includes:

[0023] A two-dimensional bit matrix is ​​created for the device data carrying the spatiotemporal feature coding of each device, which is recorded as the device association matrix. According to the arrival time window of the device data carrying the spatiotemporal feature coding, the association status is marked at the corresponding row and column positions of the device association matrix to form a first-level bitmap.

[0024] Furthermore, the method for constructing the secondary bitmap includes:

[0025] Perform logical correlation judgment on vibration data blocks and thermal imaging data blocks within the same time window;

[0026] If the logical association determination result is that there is a logical association, a logical association channel is created in the corresponding time window row in the device association matrix;

[0027] The spatiotemporal coupling degree of the logically associated channels is calculated, and the corresponding bits of the logically associated channels are marked according to the spatiotemporal coupling degree to generate a secondary bitmap.

[0028] Furthermore, the method for determining the logical association between the vibration data block and the thermal imaging data block within the same time window includes:

[0029] Extracting vibration frequency features of vibration data blocks and temperature difference features of thermal imaging data blocks;

[0030] When writing data blocks, if the vibration frequency characteristics of the vibration data block in the same time window exceed the preset frequency threshold, and the temperature difference characteristics of the thermal imaging data block exceed the preset temperature difference threshold, it is determined that the vibration data block in the time window is logically associated with the thermal imaging data block.

[0031] Furthermore, the method for implementing collaborative computing between edge physical nodes and central governance nodes of a distributed data architecture includes:

[0032] Based on the secondary bitmap and the tertiary bitmap, a global anomaly propagation graph is constructed;

[0033] Based on the global anomaly propagation graph, distributed root cause location is achieved through graph database query;

[0034] Based on the root cause location results, the multimodal data and logically associated channels corresponding to the root cause location results are determined in the first-level bitmap and the second-level bitmap, and the data distributed in the edge physical nodes are aggregated to the central governance node;

[0035] The central governance node receives the aggregated data and performs unified correlation calculations.

[0036] A data governance system for a distributed data architecture, which is used to implement the above-mentioned data governance method for a distributed data architecture, comprises:

[0037] Data encoding module: used to perform spatiotemporal feature encoding on the multimodal data generated by the device to obtain device data carrying spatiotemporal feature encoding; the spatiotemporal feature encoding includes hash spatiotemporal anchoring and dynamic time window slicing;

[0038] Bitmap construction module: constructs a multi-level association bitmap based on device data carrying spatiotemporal feature encoding; the multi-level association bitmap includes a first-level bitmap, a second-level bitmap, and a third-level bitmap; the first-level bitmap is a physical association bitmap, the second-level bitmap is a logical association bitmap, and the third-level bitmap is a topological association bitmap;

[0039] Collaborative computing module: Based on the constructed multi-level association bitmap, it realizes the collaborative computing of the edge physical nodes and central governance nodes of the distributed data architecture.

[0040] Compared with the prior art, the present invention has the following beneficial effects:

[0041] The present invention effectively solves the problems of spatiotemporal correlation fragmentation and cross-modal logical correlation rupture caused by the shard storage of edge node data in a distributed architecture by encoding the spatiotemporal features of multimodal data and constructing a multi-level association bitmap. This method integrates multi-dimensional information such as physical storage location, cross-modal logical association within the device, and topological relationship between devices, and constructs a three-dimensional association network covering the physical layer, logical layer, and topological layer, breaking through the technical separation of data storage and association analysis in traditional distributed architectures. Based on the constructed multi-level association bitmap, the present invention realizes the efficient coordination of data shard storage of edge physical nodes and global association calculation of central governance nodes, ensuring the globality and accuracy of association analysis while ensuring load balancing of edge nodes. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0043] Figure 1 A flow chart of a data governance method for a distributed data architecture in the present invention;

[0044] Figure 2 A flow chart of a method for performing spatiotemporal hashing of multimodal data generated by a device in a data governance method for a distributed data architecture of the present invention;

[0045] Figure 3 This is a flow chart of the principle of determining the time window to which data belongs in an embodiment of the present invention;

[0046] Figure 4 A schematic diagram of the structure of a multi-level association bitmap in an embodiment of the present invention;

[0047] Figure 5 A flow chart of a method for constructing a global anomaly propagation graph in a data governance method for a distributed data architecture according to the present invention;

[0048] Figure 6 This is a functional module diagram of a data governance system with a distributed data architecture in the present invention. DETAILED DESCRIPTION

[0049] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0050] Example 1:

[0051] See also Figure 1 As shown, this embodiment provides a data governance method for a distributed data architecture, including:

[0052] Step S1000: performing spatiotemporal feature encoding on the multimodal data generated by the device to obtain device data carrying the spatiotemporal feature encoding; the spatiotemporal feature encoding includes hash spatiotemporal anchoring and dynamic time window slicing;

[0053] Furthermore, step S1000 includes:

[0054] Step S1100, performing hashing of the multimodal data generated by the device in time and space;

[0055] Furthermore, if Figure 2 As shown, step S1100 includes:

[0056] Step S1110: Obtain the device ID and generate a 128-bit hash value based on the device ID, which is recorded as the device hash value;

[0057] Step S1120: intercept the lower 8 bits of the device hash value as the physical node mapping identifier, recorded as the node mapping code;

[0058] Step S1130, marking the multimodal data generated by the same device within a preset period T with the same node mapping code; the period T is the basic time unit of data governance.

[0059] Specifically, the purpose of step S1100 is to solve the problems of data physical node mapping confusion and time dimension correlation break caused by identification differences and storage fragmentation of multi-source heterogeneous devices in the industrial Internet through standardized hash processing of device identification and data clustering within a time period.

[0060] Device IDs, as unique identifiers for devices, often come in forms like IMEI and MAC addresses, but their formats and lengths vary significantly. To ensure unified data processing and subsequent efficient operations, a hashing operation is used to convert device IDs of varying formats and lengths into a fixed-length 128-bit hash value. The hashing process uses a specific hashing algorithm to take the device ID as input, perform a series of mathematical transformations, and output a fixed-length hash value. For example, the IMEI number of a certain industrial device is "IMEI_00123456789." After hashing, the resulting hash value is a 128-bit hash value of "a1b2c3d4e5f6..." This approach aims to transform the uniqueness and diversity of device IDs into a unified, easy-to-process data format. This eliminates the need to consider differences in the original device ID format during subsequent data storage, querying, and correlation analysis, thereby improving the versatility and efficiency of data processing. Step S1110 resolves the data processing difficulties caused by differences in device ID format and length, simplifying the data processing process and enhancing the versatility of data processing. By standardizing the device ID format, subsequent operations can process data from different devices in a consistent manner, avoiding errors and complications caused by format differences. For example, when storing data, there's no need to design separate storage strategies for device IDs in different formats, reducing system design complexity and the probability of errors. The unified hash value format facilitates the subsequent extraction of fixed-length fields (such as the lower 8 bits) as a basis for physical node mapping, ensuring that data from different devices can be distributed to edge physical nodes according to unified rules, facilitating load balancing for distributed storage.

[0061] The distributed data architecture consists of several edge physical nodes and a central governance node. The edge physical nodes are the underlying data storage units of the distributed data architecture, directly connected to industrial equipment and responsible for real-time access and sharded storage of multimodal data. Each edge physical node corresponds to a physical storage entity (such as an industrial gateway or edge server). The central governance node is the high-level management unit of the distributed data architecture, responsible for integrating the fragmented data of edge physical nodes, building global correlation models, and performing complex analysis. The lower 8 bits of the device hash value are used as the physical node mapping identifier because they provide good hashing properties for node mapping. Hashability refers to the ability of hash values ​​to be relatively evenly distributed within a certain range. In this embodiment, this means that data from different devices can be evenly distributed to different physical nodes. For example, in a distributed system with eight edge physical nodes, when 100 devices generate data, by using the lower 8 bits of the device hash value as the node mapping code, each physical node receives roughly equal amounts of data, preventing any one node from receiving too much or too little data. The purpose of this approach is to achieve data load balancing at the physical node level, preventing uneven data distribution from causing overload on some nodes and impacting overall system performance. Step S1120 addresses the load imbalance that can occur when data is stored on edge physical nodes, improving the performance and stability of the distributed system. By balancing the load, each physical node can operate under a relatively stable load, reducing the risk of system failure due to overloaded nodes and improving the reliability and responsiveness of the entire distributed storage system. For example, when querying data, each node can respond to requests more quickly, reducing query latency. The node mapping code, as a direct mapping identifier between devices and physical nodes, determines the data storage location without complex routing algorithms, reducing communication overhead and coordination costs between edge physical nodes. Traditional distributed storage often uses static allocation or simple hashing (such as modulo operations), which can easily lead to node load imbalance due to uneven distribution of device IDs. Step S1120 leverages the strong hashing properties of cryptographic hashing to significantly improve mapping uniformity, addressing the storage inefficiency problem caused by "data sharding skew" in existing technologies.

[0062] Multimodal data includes vibration data, thermal imaging data, and text data. The operating status of industrial equipment requires comprehensive monitoring across three core dimensions: mechanical vibration, thermal characteristics, and historical records. Vibration data directly reflects the operating status of mechanical components (such as bearings and gears). Abnormal vibration frequencies often indicate faults such as component wear and imbalance, making it a key indicator for early warning of equipment failures. Thermal imaging data is used to monitor the thermal state of heat-generating components (such as motors and terminal blocks). Temperature differences can often indicate poor contact and heat dissipation failures. Thermal imaging data complements vibration data. For example, abnormal vibration accompanied by localized overheating can more accurately pinpoint the fault location (e.g., bearing wear leading to increased vibration and associated temperature rise). Text data (such as maintenance logs and sensor configuration parameters) provides historical maintenance information (e.g., water cleaning records, component replacement times) and static configuration (e.g., rated power). This provides contextual semantics for data analysis and provides background reference. For example, combined with maintenance logs, it can distinguish temporary anomalies (e.g., temporary load fluctuations) from persistent faults (e.g., long-term fouling leading to reduced efficiency).

[0063] The preset period T is the basic time unit for data governance. Its length is determined by the specific scenario and can usually be set to 5 to 15 minutes. Taking a factory's production equipment as an example, if T is set to 10 minutes, during these 10 minutes, the multimodal data generated by the equipment, such as vibration data, thermal imaging data, and text data, will all be marked with the same node mapping code. The purpose of this is to cluster data from the same device at the physical node level, facilitating subsequent association analysis and query. Because data generated by the same device in a short period of time often has strong correlations, clustering and storing them allows for faster access to relevant data during association analysis, improving analysis efficiency. For example, when predicting equipment failures, it is necessary to analyze multiple data features of the equipment over a period of time. Clustering and storing data can reduce the time it takes to search and read data, improving the accuracy and timeliness of fault predictions. Step S1130 solves the problem of difficulty in association analysis caused by the lack of reasonable organization of data during storage, thereby improving the efficiency of data association analysis and query. By clustering the data of the same equipment within a preset period, the scope and time of data retrieval are reduced, the efficiency of data processing is improved, and it helps to more accurately perform operations such as equipment status monitoring and fault prediction. For example, when diagnosing equipment faults, relevant data can be obtained more quickly for analysis, shortening the fault diagnosis time and reducing equipment downtime.

[0064] Step S1100 unifies the device ID format through hash operations to solve the data management chaos caused by the incompatibility of multi-source device identifiers. The hashing property of the lower 8 bits of the hash value is used to achieve uniform data sharding, avoiding overloading of a single node. Through the periodic T marking, it is ensured that multi-modal data within the same time period is associated at the physical storage layer, providing a basis for subsequent dynamic time window slicing. If S1110 is missing, heterogeneous device IDs cannot be unified, and the subsequent generation of node mapping codes loses standardized input, resulting in chaotic data sharding and unbalanced loads on edge nodes. If S1120 is missing, data cannot be evenly distributed to edge nodes, which may cause some nodes to be overloaded with storage and increased computing latency, violating the original intention of the distributed data architecture design. If S1130 is not implemented, multi-modal data within the same cycle is stored dispersedly, and when constructing the first-level bitmap in subsequent step S2000, the time window and data modality cannot be accurately associated, resulting in distorted spatio-temporal correlation analysis.

[0065] Step S1200 performs dynamic time window slicing on the multi-modal data generated by the device;

[0066] Furthermore, step S1200 includes:

[0067] Step S1210 takes the moment when the device is first activated as the time origin, divides the device operation cycle into multiple sliding time windows according to the preset period T; at the same time, an absolute time interval is generated for each time window;

[0068] Step S1220 writes the device data carrying spatio-temporal feature encoding in the form of data blocks. When a data block is written, calculate the deviation Δτ between the timestamp of the data block and the end of the current time window. Through the deviation Δτ, determine the time window to which the data block belongs and perform dynamic data sharding. The data blocks include vibration data blocks, thermal imaging data blocks, and text data blocks.

[0069] Furthermore, step S1220 includes:

[0070] Step S1221, if Δτ is less than or equal to the allowed time deviation threshold Δt, then force the data block to be written to the physical node corresponding to the current time window;

[0071] Step S1222, if Δt < Δτ < T, then start cross-window sharding compensation, split the data block into a main data block and a mirror data block carrying spatio-temporal coordinate metadata, write the main data block to the current physical node, and write the mirror data block to the physical node in the adjacent cycle;

[0072] Step S1223, if Δτ ≥ T, then determine the data block as an abnormal sharding, trigger abnormal sharding fusing, and perform hierarchical abnormal processing.

[0073] The method of the hierarchical abnormal processing includes:

[0074] When T≤Δτ<2T, the device clock automatic calibration request is triggered, and the data block is temporarily stored in the edge physical node. After calibration, the time window is remapped;

[0075] When Δτ≥2T, the data is transferred to the central governance node and marked as "serious clock anomaly", and the device offline alarm is triggered.

[0076] Specifically, the dynamic time window slicing of the multimodal data blocks generated by the device in step S1200 is a key link in the spatiotemporal feature encoding, which aims to solve the problem of spatiotemporal correlation fragmentation caused by device clock drift and inconsistent data arrival timing. For example, when monitoring the operating status of the device, if there is a slight deviation in the device clock, the data stored according to the fixed timestamp may be incorrectly classified, resulting in deviations in the subsequent analysis of the device operation trend and the inability to detect potential faults in time. By constructing a sliding time window system and implementing a dynamic slicing strategy, the continuity and correlation of multimodal data in the time dimension are ensured, providing a temporal basis for the subsequent construction of multi-level correlation bitmaps.

[0077] The sliding time window mechanism can break through the limitations of absolute timestamps and provide a unified time reference system. During the operation of the equipment, data is continuously generated. If only relying on absolute timestamps, when the device clock drifts, the time sequence and correlation of the data will become chaotic. The sliding time window mechanism can classify data generated at different times according to a fixed period T. No matter how the device clock drifts, as long as the data generation time is within a certain time window, it can be correctly classified and processed. The use of sliding time windows is in line with the characteristics of continuous generation of device data and sensitivity to time sequence in the industrial Internet scenario. It effectively solves the problem of data classification confusion caused by inconsistent time references, so that the data has a clear division and organization in the time dimension, providing an orderly basis for subsequent data correlation analysis, and helping to more accurately grasp the changes in the equipment operation status over time.

[0078] When a data block is written into a distributed system, the timestamp it carries reflects the time when the data was generated. By calculating the difference Δτ between this timestamp and the end of the current time window, the time window to which the data block belongs is determined. This is the core operation of dynamic data sharding. For example, Figure 3As shown in the figure, the current time window is [10, 19], and the corresponding absolute time interval is [2024-01-01 08:10:00, 2024-01-01 08:19:00]. If a data block has a timestamp of 2024-01-01 08:15:00, the deviation Δτ between the data block's timestamp and the end of the time window is calculated to be 4 minutes, indicating that the data block belongs to the current [10, 19] time window. This method of determining the time window to which data belongs based on timestamp deviation enables real-time and dynamic data sharding and storage. This adapts to the uncertainty of industrial equipment data generation time, avoids storage and analysis errors caused by data chronological disorder, and solves the problem of difficulty in establishing data associations due to chronological issues during storage. This provides an accurate data sharding foundation for subsequent time series-based data association analysis, helping to improve the accuracy and reliability of data analysis.

[0079] The permissible time deviation threshold, Δt, is set to tolerate a certain degree of clock drift. It's difficult to achieve completely accurate synchronization of device clocks, and some error will always exist. For example, if Δt is set to 30 seconds, when the deviation, Δτ, between the calculated data block timestamp and the end of the current time window is less than or equal to 30 seconds, the data block's time deviation is within an acceptable range, and the data block is written directly to the physical node corresponding to the current time window. This is because within this deviation range, the data block can still be considered generated within the current time window. Directly writing to the corresponding node ensures accurate data attribution, reduces unnecessary cross-window operations, and avoids the increased complexity of data storage and correlation analysis caused by cross-window operations. This processing method resolves the problem of data storage errors that may be caused by slight device clock drift, ensures data continuity and consistency in the time dimension, and enables time series-based data correlation analysis to more accurately reflect device operating status, improving the accuracy and stability of data processing. It also avoids unnecessary sharding and improves storage efficiency.

[0080] Δt < Δτ < T indicates that the time deviation of the data block exceeds the range that can be directly written into the current time window but does not belong to a serious deviation. At this time, the cross-window sharding compensation mechanism is started, and the data block is split into a main data block and a mirrored data block carrying spatio-temporal coordinate metadata. The spatio-temporal coordinate metadata includes the time information of the data block (such as the approximate time range when the data was generated) and the information about its association with other data blocks (such as the device ID it belongs to, its position in the original data block sequence, etc.). For example, the timestamp of a data block shows that the deviation between its generation time and the end of the current time window is 1 minute (assuming Δt is 30 seconds and T is 10 minutes), which exceeds Δt but is less than T. At this time, the data block is split into a main data block and a mirrored data block. The main data block is written to the current physical node, while the mirrored data block carrying the spatio-temporal coordinate metadata is written to the physical node in the adjacent cycle. This can ensure that data is not lost or misaligned due to timestamp deviation. At the same time, the coordinate metadata carried by the mirrored data block provides clues for subsequent data association. For example, when conducting equipment fault analysis later, through the spatio-temporal coordinate metadata of the mirrored data block, data that is actually related but in different time windows can be associated, so as to more comprehensively analyze the equipment operation status, improve the integrity and relevance of the data, solve the data processing problem caused by a large but not seriously abnormal timestamp deviation, and provide data support for more accurate equipment status monitoring and fault prediction. Equipment clock drift or data transmission delay may cause data blocks to cross the time window boundary. Traditional methods may discard or force attribution incorrectly, resulting in time series breaks. By storing both the main data block and the mirrored data block, it can ensure the integrity of the data in the current window and retain the cross-window association information.

[0081] If Δτ ≥ T, a serious issue may have occurred with the device clock. In this case, the data block is identified as an anomalous shard, and the anomalous shard circuit breaking mechanism is triggered. Anomalous shard circuit breaking involves taking special action on anomalous data blocks to prevent them from disrupting normal data processing. Specifically, the data block is transferred to a central governance node for hierarchical anomaly handling. This hierarchical anomaly handling occurs in two scenarios: When T ≤ Δτ < 2T, an automatic clock calibration request (e.g., NTP synchronization) is triggered, and the data block is temporarily stored at an edge physical node. After calibration, the time window is remapped. This is because the clock deviation in this case may be temporary. Automatic calibration can restore the device clock to normal, and data can be correctly processed after remapping the time window. For example, if the deviation Δτ is 12 minutes (T is 10 minutes), a clock calibration request is triggered. The data block is temporarily stored at an edge physical node while waiting for calibration. After calibration, the time window to which the data block belongs is reassigned based on the new time information. If Δτ ≥ 2T, the data block is transferred to the central governance node and marked as a "serious clock anomaly." A device offline alarm is also triggered. This is because the clock deviation is severe at this time, potentially impacting the normal operation of the equipment and requiring prompt notification to relevant personnel for resolution. Only extreme scenarios with "Δτ ≥ K × T" (K is an empirical threshold, such as 2) are retained for manual processing. This hierarchical processing strategy reduces manual intervention, improves the system's automated processing capabilities, and avoids overloading the central governance node, meeting the real-time and reliability requirements of industrial scenarios. This approach effectively isolates abnormal data, preventing it from disrupting the normal data sharding order, ensuring the stability and reliability of the entire data processing system and ensuring accurate analysis of equipment operating status.

[0082] The above-mentioned double encoding process is performed on the original multimodal data to obtain device data carrying spatiotemporal feature encoding. The device data carries the following spatiotemporal feature encoding to form a structured data unit, as shown in Table 1:

[0083] Table 1 Device data with spatiotemporal feature encoding

[0084]

[0085] Step S1200 ensures the continuity of data in the time dimension through dynamic time window and deviation classification processing, regardless of whether the device clock is fully synchronized. The cross-window sharding compensation mechanism retains the spatiotemporal context of the data to avoid association breaks caused by time deviation. Distinguishing between repairable and serious anomalies improves the fault tolerance of the system and meets the high reliability requirements of industrial scenarios. If the S1210 time window division is omitted, the data will rely on absolute timestamp storage. Time alignment between different devices will be difficult due to clock differences. When the first-level bitmap is subsequently constructed, the multimodal data time windows of the same device cannot be accurately associated, resulting in the failure of cross-modal analysis. If the S1220 dynamic sharding strategy is lacking, moderate time deviation data will be incorrectly attributed or discarded, forming data holes; severe abnormal data that is not isolated will pollute the edge physical nodes, resulting in errors in the construction of the associated bitmap, and ultimately affecting the accuracy of the global anomaly propagation graph.

[0086] Step S2000: constructing a multi-level association bitmap based on the device data carrying the spatiotemporal feature encoding, wherein the multi-level association bitmap includes a primary bitmap, a secondary bitmap, and a tertiary bitmap;

[0087] Furthermore, step S2000 includes:

[0088] Step S2100: constructing a first-level bitmap, where the first-level bitmap is a physical association bitmap;

[0089] Furthermore, step S2100 includes:

[0090] Step S2110: Create a two-dimensional bit matrix with M rows and N columns for the device data carrying the spatiotemporal feature encoding of each device, denoted as a device association matrix; the rows of the device association matrix represent sliding time windows, and the columns represent data modalities. The number of rows M corresponds to the number of sliding time windows, and the number of columns N corresponds to the type of data modality.

[0091] Step S2120 , marking the associated status at the corresponding row and column positions according to the arrival time window of the device data carrying the spatiotemporal feature encoding, to form a first-level bitmap;

[0092] The step of marking the associated state at the corresponding row and column position according to the arrival time window of the device data carrying the spatiotemporal feature encoding includes:

[0093] Set the column in the device association matrix corresponding to the row of the time window where the vibration data block arrives to 1, set the column in the device association matrix corresponding to the row of the time window where the thermal imaging data block arrives to 1, and set the column in the device association matrix corresponding to the row of the time window where the text data block arrives to 1.

[0094] Specifically, in a distributed data architecture, each device generates multimodal data during operation, including vibration data, thermal imaging data, and text data. These data require structured associations across time and modality. In step S2110, for each device, a two-dimensional bit matrix (DeviceMap) is created based on the spatiotemporal feature encoding generated in step S1000 (including the node mapping code for hashed spatiotemporal anchoring and the time window sequence number for dynamic time window slicing). The number of rows, M, in this matrix corresponds to the number of sliding time windows generated during device operation (e.g., if the device operates for 72 hours with a preset period of 10 minutes, M = 432). Each row uniquely corresponds to a time window (e.g., row i corresponds to time window [iT, (i+1)T)). The number of columns, N, corresponds to the number of data modality types (N = 3, representing vibration data, thermal imaging data, and text data, respectively, but this can be expanded), with each column representing a different data modality. Matrix elements use binary bits (0 or 1) to indicate whether data of that modality exists within the corresponding time window. Initially, all elements are set to 0. When a data block is written, they are set to 1 based on its time window and modality type. For example, if a wind turbine sensor device WT-001 collects both vibration data and thermal imaging data during the fifth time window (10:00-10:10), DeviceMap[5,1] and DeviceMap[5,2] are set to 1, and DeviceMap[5,3] remains 0 (no text log). This matrix maps the node mapping code of S1130 with the sliding time window of S1210, ensuring a one-to-one correspondence between the physical storage location (edge ​​physical node) of each data block, the time window, and the modal type. This solves the problem of invisible spatiotemporal distribution caused by the fragmentation of multimodal data in traditional distributed storage.

[0095] When device data carrying spatiotemporal feature codes is written to an edge physical node in the form of a data block, the system parses the spatiotemporal feature codes carried in the data block and extracts the time window sequence number W' and modal type C' (vibration = 1, thermal imaging = 2, text = 3). The matrix element at row W' and column C' in the DeviceMap is located and set to 1 to indicate the presence of the corresponding modal data within that time window. If a data block generates mirrored data blocks through cross-window sharding compensation, both the primary and mirrored data blocks must be marked in the rows corresponding to their respective time windows (the primary data block marks the current time window, and the mirrored data block marks the adjacent time window) to ensure traceability of the existence of time window deviation data. For example, if a vibration data block triggers cross-window sharding compensation due to a timestamp deviation of Δτ = 40 seconds (Δt = 30 seconds), the primary data block is written to the node corresponding to time window W' = 10, and the mirrored data block is written to the node corresponding to W' = 11. In this case, both DeviceMap[10,1] and DeviceMap[11,1] are set to 1, and the mirrored location is recorded in the metadata of the primary data block. This marking mechanism achieves efficient data status recording through atomic operations on binary bits, and supports rapid judgment of the integrity of multimodal data through bit operations (for example, whether vibration and thermal imaging data exist simultaneously in a certain time window can be directly calculated through DeviceMap[W',1]&DeviceMap[W',2]).

[0096] Traditional distributed storage manages multimodal data only at the file or database table level, lacking a unified index for spatial and temporal associations. For example, vibration data is sharded by sensor ID, and thermal imaging data is sharded by timestamp. This results in multimodal data from the same device and within the same time window being distributed across different nodes. Cross-modal association queries require traversing multiple shards, which is time-consuming. Furthermore, data time window misalignment caused by device clock drift further exacerbates data fragmentation, making it difficult for analysis systems to quickly locate complete multimodal data sets. Step S2110, through the structured design of the device association matrix, converts the time windows and modality types of multimodal data into computable matrix dimensions, forming an index of the physical presence of multimodal data within the device. This enables the distributed system to quickly obtain modal integrity information within any time window by locating the matrix rows and columns, improving query efficiency compared to traditional unstructured storage. Step S2120, through dynamic tagging of data arrival status, records the actual storage location and time window association of multimodal data in real time, resolving the problem of difficulty in tracing the presence of cross-modal data. Support edge physical nodes to quickly respond to data integrity verification requests to avoid distortion of correlation analysis caused by data block loss or misalignment.

[0097] The first-level bitmap, the core carrier of physical-layer associations, supports the logical association analysis of the second-level bitmap (providing the time window prerequisite for data existence) and connects to the S1000's spatiotemporal feature encoding (converting the physical storage location of data blocks into computable matrix elements), forming the underlying data path of "storage encoding-state recording-association analysis." By explicitly expressing the spatiotemporal distribution of multimodal data through a lightweight bit matrix, fragmented data in a distributed data architecture can be physically represented as "digital twins" on a device-by-device basis. This provides a reliable underlying data source for subsequent high-level analysis of logical and topological associations, ultimately achieving efficient collaboration between edge storage sharding and central association computing.

[0098] Step S2200: constructing a secondary bitmap, wherein the secondary bitmap is a logical association bitmap;

[0099] Furthermore, step S2200 includes:

[0100] Step S2210, performing logical correlation determination on the vibration data block and the thermal imaging data block within the same time window;

[0101] Furthermore, step S2210 includes:

[0102] Step S2211, extracting the vibration frequency feature F of the vibration data block vib and the temperature difference feature T of the thermal imaging data block her ;

[0103] Step S2212: When writing a data block, if the vibration frequency characteristic of the vibration data block in the same time window exceeds the preset frequency threshold F th , and the temperature difference feature of the thermal imaging data block exceeds the preset temperature difference threshold T th , it is determined that there is a logical association between the vibration data block and the thermal imaging data block within the time window.

[0104] Specifically, equipment failures are usually manifested as coordinated anomalies of multimodal data (such as the simultaneous occurrence of vibration frequency anomalies and local temperature rise). Therefore, it is necessary to further explore the logical associations of different modal data within the same time window based on the equipment association matrix. Step S2210 implements logical association determination for the vibration data block and the thermal imaging data block through feature extraction and threshold comparison. Frequency domain analysis (such as fast Fourier transform) is performed on the vibration data block to extract the main frequency component as the vibration frequency feature F vib , which reflects the periodic vibration intensity of the equipment during operation; for the thermal imaging data block, the difference between the maximum and minimum temperature of the region of interest (ROI) is calculated as the temperature difference feature T her , which reflects the degree of local heating of the equipment. For example, when a motor bearing fails, F vibA peak will appear near the bearing characteristic frequency, and the bearing seat thermal imaging temperature difference T her Significantly increased. Frequency threshold F th (such as the ±10% fluctuation range of the nominal value of the bearing characteristic frequency) and the temperature difference threshold T th (e.g. 2 times the maximum temperature difference during normal operation of the equipment) is determined based on the normal operating parameters of the equipment and historical fault data through data analysis and expert experience. When the data block is written, if the F vib More than F th And T her More than T th , it is determined that there is a logical correlation between the two, indicating that there may be a common cause of the failure (such as component wear leading to increased vibration and accompanied by frictional heat).

[0105] Step S2220: If the logical association determination result is that a logical association exists, a logical association channel is created in the corresponding time window row in the device association matrix;

[0106] Specifically, once a logical association is determined between vibration and thermal imaging data blocks within the same time window, a data structure needs to be established in the device association matrix to represent this association, essentially creating a logical association channel. In the device association matrix, each row represents a sliding time window, and each column represents a different data modality (e.g., vibration data, thermal imaging data, text data). For example, in the 10th time window for a particular device, where a logical association is determined between the vibration and thermal imaging data blocks, a connection is established between the columns corresponding to the vibration and thermal imaging data in the 10th row of the device association matrix using a data marker or pointer. An additional data field can be added to the matrix to record this logical association, such as setting a flag that, when set to 1, indicates a logical association between the two data sets within that time window. Alternatively, a pointer can be created to point to other data records associated with the two data sets to clarify their logical connection. By creating logical association channels, the originally scattered different modal data are connected at the logical level, solving the problem of difficulty in reflecting logical associations across modal data. It provides an intuitive data structure for subsequent calculation of logical association strength and construction of secondary bitmaps, making it possible to quickly locate and analyze logically associated data when analyzing the operating status of equipment, thereby improving the efficiency and accuracy of data analysis.

[0107] Step S2230 , calculating the spatiotemporal coupling degree C of the logically associated channels, marking the bits corresponding to the logically associated channels according to the spatiotemporal coupling degree C, and generating a secondary bitmap.

[0108] Furthermore, step S2230 includes:

[0109] Step S2231, obtaining the spatiotemporal coupling degree C according to the vibration frequency characteristics, the temperature difference characteristics and the deviation Δτ;

[0110] Step S2232, if , then mark the bit corresponding to the logically associated channel as 1, and the bit marked as 1 is a strongly associated bit; if , then it is marked as 0, and the bit marked as 0 is a weakly associated bit; otherwise, it is not marked; where, is the preset strong correlation threshold, is the preset weak correlation threshold.

[0111] Specifically, step S2230 calculates the spatiotemporal coupling degree C of the logically associated channels and marks the corresponding bits of the logically associated channels according to its value, thereby generating a secondary bitmap containing rich logical association information. The spatiotemporal coupling degree C is an indicator that comprehensively considers the vibration frequency characteristics, temperature difference characteristics and data block time deviation Δτ, and is used to measure the association strength of the logically associated channels. It is obtained by normalizing the vibration frequency characteristics, thermal imaging temperature difference characteristics and time deviation respectively and then multiplying them. Taking the vibration frequency characteristics as an example, the vibration frequency characteristics F are normalized by a specific normalization function. vib Convert it into a unified numerical range, so that the vibration frequency characteristics of different equipment or different working conditions are comparable; the thermal imaging temperature difference characteristic T her A similar normalization process is also performed. The time deviation Δτ is reverse-normalized (the smaller Δτ, the higher the normalized value, indicating that the data is closer to the ideal time window). These three normalized values ​​are then multiplied together to obtain the spatiotemporal coupling degree C. A larger value indicates a stronger anomaly coordination between the vibration and thermal imaging data. This calculation method comprehensively considers the impact of multiple factors on the strength of logical associations and more comprehensively reflects the degree of correlation between data. The actual operating state of equipment is complex and changeable, and a single feature cannot accurately measure the strength of correlation between data. By comprehensively considering multiple factors, the degree of logical correlation between different modal data within the equipment can be more accurately determined, providing a more reliable basis for subsequent labeling and analysis.

[0112] Strong correlation threshold (such as 0.8) and weak correlation threshold (such as 0.5) is determined based on the historical operation data and failure cases of the equipment after a large amount of data analysis and experiments. Taking a certain device as an example, if the calculated spatiotemporal coupling degree C is greater than or equal to the strong correlation threshold , indicating that the vibration data block and the thermal imaging data block represented by the logical association channel are closely associated. In the secondary bitmap, the bit corresponding to the logical association channel is marked as 1, representing a strong association. If C is within the weak association threshold and strong correlation threshold If C is less than , no marking is performed. This marking method intuitively reflects the correlation strength of different logical association channels in the secondary bitmap, facilitating subsequent analysis of the correlation propagation characteristics of fault symptoms within the device. This solves the problem of difficulty in quantifying and distinguishing different logical correlation strengths, provides more accurate data support for local root cause analysis, and can quickly locate key data and logical associations related to the fault.

[0113] Traditional distributed data management only stores the physical presence of multimodal data and lacks explicit expression of the logical relationships between data. For example, although vibration and thermal imaging data are stored on the same node, the analysis system must use complex algorithms to align timestamps and calculate correlations one by one, which is time-consuming and has low accuracy. In addition, time deviations caused by device clock drift further interfere with correlation judgments, increasing the rate of missed fault diagnosis. Step S2210 transforms multimodal signs of equipment failure into computable logical correlation judgment conditions through feature extraction and threshold comparison, solving the problem of difficulty in identifying abnormal coordination of cross-modal data. This enables the analysis system to quickly locate time windows with potential fault signs, reducing preprocessing time compared to traditional data block-by-data correlation calculation. Step S2220 establishes a direct correlation between vibration and thermal imaging data within the time window through explicit marking of logical correlation channels, solving the problem of difficult to trace cross-modal logical correlation relationships. It supports direct screening of time windows with correlations through bitwise operations, improving query efficiency. Step S2230 quantifies the strength of cross-modal associations by calculating spatiotemporal coupling and labeling them in a hierarchical manner, resolving the difficulty of quantifying association strength. This provides a weighted basis for subsequent anomaly propagation analysis (for example, strongly correlated channels are treated as high-weighted edges when constructing a local anomaly propagation subgraph), enabling the root cause location algorithm to prioritize high-confidence association paths.

[0114] The secondary bitmap, the core carrier of logical-layer associations, inherits the physical presence of multimodal data within devices (the primary bitmap) and supports the determination of topological associations between devices, forming a mid-level association network of "physical presence-logical association-topological propagation." Through feature threshold determination and coupling quantification, multimodal signs of device failures are transformed into computable and traceable logical association markers. This enables cross-modal data within a distributed data architecture to construct a "digital thread" of logical associations centered around the failure, providing a critical mid-level index for subsequent system-level anomaly propagation analysis. Ultimately, this represents a breakthrough from fragmented data storage to semantic association.

[0115] Step S2300: constructing a three-level bitmap, wherein the three-level bitmap is a topological association bitmap;

[0116] Furthermore, step S2300 includes:

[0117] Step S2310: construct a global spatiotemporal correlation matrix, where the rows of the global spatiotemporal correlation matrix represent all devices and the columns represent all time windows;

[0118] Step S2320: Traverse the secondary bitmaps of each pair of devices. If the time windows divided by the two devices in each pair overlap in absolute time intervals, and strong correlation bits appear in the overlapping area, and the spatial distance between the two devices does not exceed a preset distance threshold, then set the same columns of the corresponding rows of the global spatiotemporal correlation matrix of the two devices to 1, thus forming a cross-device correlation tunnel.

[0119] Step S2330: Calculate the topological association strength W of the cross-device association tunnel, where the topological association strength W is inversely proportional to the spatial distance D between the two devices.

[0120] Step S2340: quantize the topology association strength W into L levels, each level corresponding to a level code; fill the quantized level code into the bit corresponding to the device association tunnel to generate a three-level bitmap.

[0121] Specifically, the global spatiotemporal correlation matrix is ​​a data structure used to describe the relationships between all devices within all time windows. Its rows represent all devices, and its columns represent all time windows. The global spatiotemporal correlation matrix transcends the intra-device limitations of the device correlation matrix and extends correlation analysis to inter-device contexts. For example, if a system has 10 devices and 50 time windows are divided in step S1210, the resulting global spatiotemporal correlation matrix is ​​a 10-row, 50-column matrix. To construct the global spatiotemporal correlation matrix, all devices and time windows are first traversed, reserving storage space for each device's correlation status in each time window. By establishing a unified matrix structure, devices are associated with time windows, enabling subsequent analysis of inter-device relationships from a global perspective. In an industrial internet environment, where the number of devices is large and data generated is time-series, this matrix structure effectively integrates and manages inter-device correlation information, providing a foundation for subsequent discovery of logical relationships and abnormal propagation paths between devices. This solves the problem of fragmented inter-device correlation information, making unified management and analysis difficult, and provides a global framework for device correlation analysis, facilitating a holistic understanding of inter-device relationships.

[0122] After constructing the global spatiotemporal correlation matrix, the logical associations between different devices need to be determined. This is achieved by traversing the secondary bitmaps of each pair of devices. The distance threshold can be set based on the maximum functional impact range within the device deployment area. For example, in an industrial workshop, the maximum logical impact distance between adjacent functionally related devices typically does not exceed 10 meters, so the distance threshold can be set to 10 meters. When the time windows assigned to each pair of devices overlap in absolute time intervals, strong correlation bits appear in the overlapping region, and the spatial distance between the two devices does not exceed the preset distance threshold, the same column in the corresponding row of the global spatiotemporal correlation matrix is ​​set to 1 for both devices, thus forming a cross-device correlation tunnel. For example, if devices A and B overlap in time windows [10, 20) and [15, 25), and strong correlation bits appear in the secondary bitmaps of both devices within this overlapping region, and their spatial distance does not exceed the preset distance threshold, then the corresponding rows of devices A and B and the corresponding columns of the overlapping time windows in the global spatiotemporal correlation matrix are set to 1. The spatial distance between the two devices is calculated using the Euclidean formula based on their location coordinates. This process builds on the intra-device logical association information mined from the secondary bitmap and further extends it to analyzing inter-device logical associations. In industrial production, potential logical associations often exist between adjacent or functionally related devices. This method accurately identifies these associations, resolving the difficulty of discovering and locating inter-device logical associations. It also reveals the spatial distribution patterns of these associations, providing key information for analyzing the mutual impact of devices.

[0123] Cross-device association tunnels establish logical associations between devices, but the strength of this association needs to be quantified, namely by calculating the topological association strength W. The topological association strength W is inversely proportional to the spatial distance D between the two devices, meaning that closer devices have a higher topological association strength. To calculate this, the spatial distance D between the two devices is first determined, and then the topological association strength W is calculated based on this inverse relationship. For example, the spatial distance between devices A and B is obtained using their location information. The topological association strength between them is then calculated using a predefined inverse proportional calculation rule (e.g., a distance of 1 equals a strength of 10, a distance of 2 equals a strength of 5, and so on). This calculation method reflects the reality of inter-device interaction in industrial scenarios: devices with close proximity are more likely to influence each other. It addresses the difficulty of quantifying the strength of logical associations between devices, more accurately describing the propagation of these inter-device logical associations in a spatial network, and providing a quantitative basis for subsequent analysis of anomaly propagation.

[0124] To more efficiently store and analyze topological association strength, the topological association strength W is quantized into L levels, with each level corresponding to a level code. The quantized level codes are then populated into the bits corresponding to the device association tunnels to generate a three-level bitmap. For example, the value range of the topological association strength W is divided into five levels: a strength between 0 and 2 is level 1, corresponding to the code 001; a strength between 3 and 4 is level 2, corresponding to the code 010, and so on. These codes are then populated into the global spatiotemporal association matrix to form a three-level bitmap. This quantization and encoding method can describe complex spatiotemporal association structures with extremely low storage overhead, solving the problem of difficult storage of complex association structures. While ensuring the integrity of the association information, it reduces the use of storage resources and improves the efficiency of data storage and processing.

[0125] In traditional distributed data architectures, topological association analysis between devices is often insufficiently thorough and comprehensive. On the one hand, due to a lack of adequate consideration of the spatial relationships between devices, it's impossible to accurately identify pairs of devices that are spatially close and have strong data associations. On the other hand, the lack of quantification and encoding of the strength of data associations between devices makes effective management and analysis difficult. For example, in a large industrial park, numerous devices are distributed across diverse areas. Traditional methods may be unable to accurately identify which devices are associated with each other and the strength of these associations. This prevents the full utilization of inter-device association information during fault diagnosis, performance optimization, and other tasks, resulting in low efficiency and accuracy. Step S2300 calculates the spatial distances between device pairs and selects pairs whose distances fall within a preset threshold. This allows subsequent topological association analysis to focus on pairs of devices that are spatially close, improving the relevance and effectiveness of the analysis. By constructing a global association matrix and identifying pairs of devices that share strong association markers within a common time window, this provides an accurate basis for establishing device association tunnels and clearly represents the topological associations between devices. By establishing a device association tunnel and calculating the spatial distance between devices, the data association strength between devices can be quantified through spatial distance, providing strong support for subsequent analysis and management.

[0126] The single-dimensional limitation of traditional indexing makes it difficult to fully and accurately display the relationship between data, thus affecting the in-depth understanding and effective use of data. Step S2000 of the present invention successfully breaks through the limitations of traditional indexing by constructing a multi-level association bitmap to form a physical-logical-topological three-dimensional association network. The multi-level association bitmap makes data query more efficient. Figure 4As shown, the multi-level association bitmap includes a first-level bitmap, a second-level bitmap, and a third-level bitmap. The first-level bitmap can quickly determine the time window and modality of the data. The second-level bitmap can further filter out data with logical associations. The third-level bitmap can help find other devices and their data associated with a specific device. This hierarchical query method helps reduce the scope of data search and improve query speed. For example, when querying the relevant data of a device within a specific time period, traditional methods may require traversing a large number of files or database records. However, the multi-level association bitmap can directly locate the relevant data blocks, shortening the query time from several minutes to seconds, greatly improving work efficiency.

[0127] Step S3000, based on the constructed multi-level association bitmap, realize the collaborative computing of the edge physical nodes and the central governance nodes of the distributed data architecture.

[0128] Furthermore, step S3000 includes:

[0129] Step S3100: At the edge physical node level, a device-node routing table is established based on the first-level bitmap and the node mapping code;

[0130] Specifically, in a distributed data architecture, edge physical nodes are responsible for sharding and storing multimodal data. Rapidly locating data storage locations is fundamental to efficient data governance. Step S3100 integrates the first-level bitmap with node mapping codes to construct a device-node routing table, addressing the inefficiency of traditional edge physical node data mapping. The device-node routing table stores device IDs, node mapping codes, and their corresponding relationships with physical nodes. To implement this, the system traverses the first-level bitmap, extracting the device ID and corresponding node mapping code for each device. For example, in an industrial scenario with 100 devices and 10 edge physical nodes, the first-level bitmap records the data storage and corresponding node mapping code for each device within different time windows. For example, suppose device A has a device ID of "001" and its data corresponds to a node mapping code of "10101010" (obtained through hashing and spatiotemporal anchoring) across multiple time windows. This node mapping code corresponds to edge physical node 3. The system organizes this information and constructs a device-node routing table. By solidifying the correspondence between the device ID and the node mapping code in the first-level bitmap into a routing table, the tedious process of recalculating the mapping relationship every time the device data is accessed is avoided.

[0131] Step S3200: At the central governance node level, a global anomaly propagation graph is constructed based on the secondary bitmap and the tertiary bitmap;

[0132] Furthermore, if Figure 5 As shown, step S3200 includes:

[0133] Step S3210: Construct a local anomaly propagation subgraph: Identify the logical association channels in the secondary bitmap of each device, and add directed edges between the two end nodes of the logical association channel. The direction of the edge is the time-increasing direction, and the weight of the edge is the spatiotemporal coupling degree C of the logical association channel.

[0134] Step S3220, constructing a global anomaly propagation backbone graph: identifying cross-device association tunnels in the three-level bitmap, adding undirected edges between the two end devices of the cross-device association tunnels, with the edge weight being the topological association strength W;

[0135] Step S3230: connect the local anomaly propagation subgraphs of all devices in parallel to obtain a complete intra-device anomaly propagation graph, and then superimpose the global anomaly propagation backbone graph connecting all devices to form a global anomaly propagation graph.

[0136] Specifically, existing technologies often lack comprehensive and effective methods when analyzing the propagation of anomalies in distributed systems, making it difficult to accurately grasp the propagation patterns of anomalies across different devices and time dimensions. For example, it may only be possible to perform a simple analysis of the anomalies of a single device, and it is impossible to consider how the anomalies spread across multiple devices from the perspective of the entire system. This results in an inability to quickly locate the root cause of the fault and predict the development trend of the fault when dealing with complex industrial system failures. Step S3200 aims to construct a global anomaly propagation graph based on the secondary bitmap and the tertiary bitmap to achieve effective analysis and location of anomaly propagation in distributed systems, providing a decision-making basis for root cause analysis and predictive maintenance.

[0137] Taking a device's time window as a node, each node contains the existence status of multimodal data within that time window (from the first-level bitmap) and a logical association tag (from the second-level bitmap). Step S3210 establishes directed edges between sequential abnormal state nodes from a single device perspective, forming a local abnormality propagation subgraph. This subgraph depicts the abnormality development and evolution within a single device. During industrial equipment operation, abnormal states within different time windows often have logical connections. By establishing directed edges and assigning weights, these connections and the strength of the associations can be intuitively displayed. Traditional time series analysis relies on direct timestamp comparison. Step S3210, by explicitly tagging logical association channels, transforms abnormality propagation within the device into edge weights within a directed graph, enabling quantitative analysis of the abnormal state's temporal evolution. Step S3210 addresses the difficulty of visually presenting and analyzing the abnormality development process within a single device. It provides an intuitive data structure for local root cause analysis, facilitates in-depth study of abnormality evolution patterns within a single device, and helps quickly locate the root cause of internal faults.

[0138] When constructing a global anomaly propagation backbone graph, it is necessary to identify cross-device association tunnels in the three-level bitmap. For example, in an industrial production line consisting of multiple devices, devices A and B may have a cross-device association tunnel in the three-level bitmap. This means that they have a strong logical association within certain time windows and their spatial distance meets a preset distance threshold. The system adds an undirected edge between the two devices at the ends of the cross-device association tunnel, with the edge weight being the topological association strength W. Assuming the topological association strength W between devices A and B is 0.6, when constructing the global anomaly propagation backbone graph, an undirected edge with a weight of 0.6 is added between the nodes corresponding to devices A and B. In this way, each cross-device association tunnel is abstracted as an undirected edge in the backbone graph, connecting the device nodes corresponding to the two ends of the association tunnel, forming a backbone graph that reflects the global anomaly propagation path. This backbone graph reveals the spatial topology of anomaly propagation between different devices. Multiple devices are often interconnected, and anomalies may propagate between them. The global anomaly propagation backbone graph can demonstrate the anomaly propagation path and strength at the system level, providing a macro perspective for analyzing fault propagation across the entire system. When a device experiences an anomaly, the global anomaly propagation backbone diagram allows for rapid identification of potentially affected devices and the extent of the impact, providing a basis for implementing appropriate preventive and remedial measures. Step S3220 addresses the difficulty of analyzing the anomaly propagation paths and intensity between different devices at a system level. It reveals the spatial topology of the global anomaly propagation path, providing a crucial basis for system-level fault analysis and prevention.

[0139] After constructing the local anomaly propagation subgraph and the global anomaly propagation backbone graph, the system connects the local anomaly propagation subgraphs for all devices in parallel to form a complete intra-device anomaly propagation graph. This step integrates the anomaly propagation process within each device, forming a graph structure that covers anomalies within all devices. The complete intra-device anomaly propagation graph is then superimposed with the global anomaly propagation backbone graph to form the global anomaly propagation graph. The global anomaly propagation graph logically reflects the spatiotemporal evolution of anomalies within the distributed system. This approach integrates anomaly propagation information within and between devices, providing a comprehensive decision-making basis for root cause identification and predictive maintenance. In real-world industrial scenarios, it is necessary to comprehensively understand the propagation of anomalies throughout the entire distributed system. The global anomaly propagation graph integrates local and global anomaly propagation information to provide a comprehensive perspective. When an anomaly occurs in a distributed system, the global anomaly propagation graph can quickly locate the root cause, predict its development trend, and formulate appropriate maintenance strategies to reduce losses caused by equipment failures. Step S3230 solves the problem of being unable to fully analyze the spatiotemporal evolution of anomalies within a distributed system, provides a comprehensive and intuitive decision-making basis for root cause location and predictive maintenance, and greatly improves the system's ability to respond to abnormal situations.

[0140] Step S3300: Based on the global anomaly propagation graph, distributed root cause location is achieved through graph database query;

[0141] Specifically, in the distributed data architecture of the Industrial Internet, when a system anomaly occurs, accurately identifying the root cause is crucial for quickly resolving the issue and minimizing losses. Step S3300 leverages the query capabilities of a graph database, using the constructed global anomaly propagation graph, to achieve distributed root cause location. A graph database is a database specifically designed for processing graph data structures. It excels at storing and querying graph-structured data composed of nodes and edges, making it ideally suited for handling the complex device associations and anomaly propagation relationships in this solution. In actual implementation, the system stores the global anomaly propagation graph in the graph database. For example, if a critical device experiences an anomaly in a production system with multiple devices, the system inputs information related to the anomaly, such as the time range and the device ID, through the graph database's query interface. Based on the structure of the global anomaly propagation graph and the stored association information, the graph database traverses and analyzes the edges in the graph (each edge represents an anomaly association between devices, with its weight indicating the strength of the association) starting from the device node where the anomaly occurred. By comparing the association strength and anomaly propagation logic along different paths, the system identifies the device or event most likely to have caused the anomaly. In the Industrial Internet, devices are numerous and complexly interconnected, making it difficult to quickly locate the root cause of anomalies using traditional search methods. However, a global anomaly propagation graph integrates the logical connections between devices and anomaly propagation information. Graph databases can efficiently process this graph-structured data and quickly identify the path and root cause of anomaly propagation. For example, on the production line of an automobile manufacturing plant, numerous devices work together. When an anomaly occurs in a particular production link, querying the global anomaly propagation graph in the graph database can quickly pinpoint the cause of an upstream device failure that triggered the subsequent chain reaction, rather than blindly troubleshooting all devices. This step solves the problem of quickly and accurately locating the root cause of anomalies in distributed systems. The beneficial effect is that the cause of the anomaly can be quickly determined, providing a basis for subsequent targeted measures, reducing the impact of equipment failures on production, and improving production efficiency.

[0142] Step S3400: Based on the root cause location result, determine the multimodal data and logically associated channels corresponding to the root cause location result in the primary bitmap and the secondary bitmap, and aggregate the data distributed on the edge physical nodes to the central governance node;

[0143] Specifically, after root cause location is determined, related data must be accurately aggregated to avoid communication bottlenecks caused by moving all data. This step achieves targeted data aggregation through the spatiotemporal indexing of a multi-level association bitmap. The first-level bitmap records the distribution characteristics of the device's multimodal data in the temporal and spatial dimensions, while the second-level bitmap overlays the logical association information of cross-modal data. Based on the root cause location results obtained in step S3300, the system searches the first-level bitmap for multimodal data generated by the device related to the anomaly within different time windows. For example, if root cause location determines that the abnormal vibration data of a device within a specific time window is one of the causes of the system problem, the system will find the location information of the vibration data block of the device in the corresponding time window in the first-level bitmap. At the same time, in the second-level bitmap, based on the information of the logically associated channels, other data blocks that are logically associated with the vibration data block are determined, such as thermal imaging data blocks or text data blocks that may have logical associations. After determining the required data, the spatiotemporal index information provided by the multi-level association bitmap is used to accurately obtain the data fragments related to the query. Since the data is stored on distributed edge physical nodes, these index information can be used to avoid full data migration, and only data related to anomalies can be aggregated to the central governance node. In actual operation, data is read from the corresponding edge physical node based on the node mapping code and time window information. In the industrial Internet scenario, the amount of data is huge and distributed across multiple edge physical nodes. If full data migration is performed, extremely high communication costs and computing overhead will be incurred, seriously affecting system performance. By utilizing the spatiotemporal index information of the multi-level association bitmap, data can be aggregated in a targeted manner, which can reduce resource consumption while ensuring the acquisition of key data. This step solves the problem of excessive communication and computing overhead when obtaining relevant data under a distributed data architecture, reduces the waste of data transmission and computing resources, and improves data processing efficiency.

[0144] Step S3500: The central governance node receives the aggregated data and performs unified correlation calculations.

[0145] Specifically, the central governance node integrates fragmented data from edge physical nodes and utilizes a global anomaly propagation graph to perform complex correlation calculations, overcoming the computing power limitations of edge physical nodes. Correlation calculations are a series of graph computations performed by the central governance node based on aggregated data and the global anomaly propagation graph. Their objectives are to reveal correlations between devices, estimate the impact of anomalies, and predict anomaly trends. The central governance node first integrates and preprocesses the aggregated data to ensure consistency and availability. Then, the data is analyzed in conjunction with the global anomaly propagation graph. For example, by analyzing data changes before and after an anomaly occurs and their correlations in the global anomaly propagation graph, correlations between devices can be revealed. To estimate the impact of an anomaly, the global anomaly propagation graph shows the path and intensity of anomaly propagation, combined with aggregated data, to determine which devices are likely to be affected by the anomaly and to what extent. To predict anomaly trends, historical data and current anomaly conditions are analyzed, using graph computation methods to predict whether the anomaly will spread further, as well as the direction and speed of spread.

[0146] This centralized correlation calculation approach at the central governance node is adopted because edge physical nodes typically have limited computing power, making them unable to handle complex correlation calculation tasks. Central governance nodes, on the other hand, possess greater computing power and can centrally process large amounts of data. In industrial Internet scenarios, it is necessary to analyze relationships and anomalies between devices from a global perspective. Centralized computing at the central governance node can better achieve this goal. This step solves the problem of insufficient computing power at edge physical nodes, which prevents complex correlation calculations and the difficulty of analyzing device relationships and anomalies from a global perspective. It overcomes the computing power bottleneck of edge physical nodes, achieves global optimization of computing resources, and enables more comprehensive and in-depth analysis of relationships and anomalies between devices, providing stronger support for decision-making.

[0147] Example 2:

[0148] This embodiment provides a data governance system with a distributed data architecture based on the first embodiment. Figure 6 Shown, including:

[0149] Data encoding module: used to perform spatiotemporal feature encoding on the multimodal data generated by the device to obtain device data carrying spatiotemporal feature encoding; the spatiotemporal feature encoding includes hash spatiotemporal anchoring and dynamic time window slicing;

[0150] Bitmap construction module: constructs a multi-level association bitmap based on device data carrying spatiotemporal feature encoding; the multi-level association bitmap includes a first-level bitmap, a second-level bitmap, and a third-level bitmap; the first-level bitmap is a physical association bitmap, the second-level bitmap is a logical association bitmap, and the third-level bitmap is a topological association bitmap;

[0151] Collaborative computing module: Based on the constructed multi-level association bitmap, it realizes the collaborative computing of the edge physical nodes and central governance nodes of the distributed data architecture.

[0152] In the data encoding module, the method for performing the hash time-space anchoring includes:

[0153] Step S1110: Obtain the device ID and generate a 128-bit hash value based on the device ID, which is recorded as the device hash value;

[0154] Step S1120: intercept the lower 8 bits of the device hash value as the physical node mapping identifier, recorded as the node mapping code;

[0155] Step S1130, marking the multimodal data generated by the same device within a preset period T with the same node mapping code; the period T is the basic time unit of data governance.

[0156] In the bitmap construction module, the multi-level associated bitmap includes a first-level bitmap, a second-level bitmap, and a third-level bitmap;

[0157] The method for constructing the three-level bitmap includes:

[0158] Step S2310: construct a global spatiotemporal correlation matrix, where the rows of the global spatiotemporal correlation matrix represent all devices and the columns represent all time windows;

[0159] Step S2320: Traverse the secondary bitmaps of each pair of devices. If the time windows divided by the two devices in each pair overlap in absolute time intervals, and strong correlation bits appear in the overlapping area, and the spatial distance between the two devices does not exceed a preset distance threshold, then set the same columns of the corresponding rows of the global spatiotemporal correlation matrix of the two devices to 1, thus forming a cross-device correlation tunnel.

[0160] Step S2330: Calculate the topological association strength W of the cross-device association tunnel, where the topological association strength W is inversely proportional to the spatial distance D between the two devices.

[0161] Step S2340: quantize the topology association strength W into L levels, each level corresponding to a level code; fill the quantized level code into the bit corresponding to the device association tunnel to generate a three-level bitmap.

[0162] In the collaborative computing module, the method for implementing collaborative computing between edge physical nodes and central governance nodes of a distributed data architecture based on the constructed multi-level association bitmap includes:

[0163] Step S3100: At the edge physical node level, a device-node routing table is established based on the first-level bitmap and the node mapping code;

[0164] Step S3200: At the central governance node level, a global anomaly propagation graph is constructed based on the secondary bitmap and the tertiary bitmap;

[0165] Step S3300: Based on the global anomaly propagation graph, distributed root cause location is achieved through graph database query;

[0166] Step S3400: Based on the root cause location result, determine the multimodal data and logically associated channels corresponding to the root cause location result in the primary bitmap and the secondary bitmap, and aggregate the data distributed on the edge physical nodes to the central governance node;

[0167] Step S3500: The central governance node receives the aggregated data and performs unified correlation calculations.

[0168] The methods and systems of the present application may be implemented in many ways. For example, the methods and systems of the present application may be implemented using software, hardware, firmware, or any combination of software, hardware, and firmware. The above order of steps used in the method is for illustration only, and the steps of the method of the present application are not limited to the order specifically described above unless otherwise specified.

[0169] In addition, the parts of the above technical solutions provided in the embodiments of the present application that are consistent with the implementation principles of the corresponding technical solutions in the prior art are not described in detail to avoid excessive redundancy.

[0170] The above-described specific embodiments further illustrate the objectives, technical solutions, and beneficial effects of the present invention. It should be understood that the above description is merely a specific embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention shall be included within the scope of protection of the present invention.

Claims

1. A data governance method for a distributed data architecture, characterized in that: The method comprises: Performing spatiotemporal feature encoding on multimodal data generated by the device to obtain device data carrying the spatiotemporal feature encoding; the spatiotemporal feature encoding includes hash spatiotemporal anchoring and dynamic time window slicing; the device data carrying the spatiotemporal feature encoding is written in the form of data blocks, the data blocks including at least a vibration data block and a thermal imaging data block; The method for performing the hash spatiotemporal anchoring includes: obtaining a device ID, generating a 128-bit hash value based on the device ID, and recording it as the device hash value; intercepting the lower 8 bits of the device hash value as a physical node mapping identifier, and recording it as a node mapping code; marking the multimodal data generated by the same device within a preset period T with the same node mapping code; the period T is the basic time unit of data governance; Based on the device data carrying the spatiotemporal feature encoding, a multi-level association bitmap is constructed; the multi-level association bitmap includes a first-level bitmap, a second-level bitmap, and a third-level bitmap; the first-level bitmap is a physical association bitmap, the second-level bitmap is a logical association bitmap, and the third-level bitmap is a topological association bitmap; The first-level bitmap construction method includes: creating a two-dimensional bit matrix for each device's device data carrying the spatiotemporal feature encoding, recorded as a device association matrix; marking the association status at the corresponding row and column positions of the device association matrix according to the arrival time window of the device data carrying the spatiotemporal feature encoding, thereby forming a first-level bitmap; The method for constructing a secondary bitmap includes: performing a logical association determination on the vibration data block and the thermal imaging data block within the same time window; if the logical association determination result indicates that a logical association exists, creating a logical association channel within the corresponding time window row in the device association matrix; calculating the spatiotemporal coupling degree of the logical association channel, marking the corresponding bits of the logical association channel according to the spatiotemporal coupling degree, and generating a secondary bitmap; The method for constructing a three-level bitmap includes: constructing a global spatiotemporal correlation matrix, wherein the rows of the global spatiotemporal correlation matrix represent all devices and the columns represent all time windows; traversing the second-level bitmap of each pair of devices, if the time windows divided by the two devices in each pair of devices have overlapping areas in the absolute time interval, and strong correlation bits appear in the overlapping area, and the spatial distance between the two devices does not exceed a preset distance threshold, then setting the same columns of the corresponding rows of the global spatiotemporal correlation matrix of the two devices to 1 to form a cross-device correlation tunnel; calculating the topological correlation strength W of the cross-device correlation tunnel; quantizing the topological correlation strength W into L levels, each level corresponding to a level code; filling the quantized level code into the bit corresponding to the device correlation tunnel to generate a three-level bitmap; Based on the constructed multi-level association bitmap, collaborative computing between edge physical nodes and central governance nodes of the distributed data architecture is achieved.

2. The data governance method of a distributed data architecture according to claim 1, characterized in that: The method for performing the dynamic time window slicing includes: Taking the first activation moment of the device as the time origin, the device operation cycle is divided into multiple sliding time windows according to the preset period T; When a data block generated by a device is written, the deviation Δτ between the timestamp of the data block and the end of the current time window is calculated. Based on the deviation Δτ, the time window to which the data block belongs is determined, and dynamic data sharding is performed.

3. The data governance method of a distributed data architecture according to claim 2, characterized in that: The method for performing logical association determination on the vibration data block and the thermal imaging data block within the same time window includes: Extracting vibration frequency features of vibration data blocks and temperature difference features of thermal imaging data blocks; When writing data blocks, if the vibration frequency characteristics of the vibration data block in the same time window exceed the preset frequency threshold, and the temperature difference characteristics of the thermal imaging data block exceed the preset temperature difference threshold, it is determined that the vibration data block in the time window is logically associated with the thermal imaging data block.

4. The data governance method of a distributed data architecture according to claim 3, characterized in that: The method for implementing collaborative computing between edge physical nodes and central governance nodes of a distributed data architecture includes: Based on the secondary bitmap and the tertiary bitmap, a global anomaly propagation graph is constructed; Based on the global anomaly propagation graph, distributed root cause location is achieved through graph database query; Based on the root cause location results, the multimodal data and logically associated channels corresponding to the root cause location results are determined in the first-level bitmap and the second-level bitmap, and the data distributed in the edge physical nodes are aggregated to the central governance node; The central governance node receives the aggregated data and performs unified correlation calculations.

5. A data governance system of a distributed data architecture, which is used to implement a data governance method of a distributed data architecture according to any one of claims 1 to 4, characterized in that: The system comprises: Data encoding module: used to perform spatiotemporal feature encoding on the multimodal data generated by the device to obtain device data carrying spatiotemporal feature encoding; the spatiotemporal feature encoding includes hash spatiotemporal anchoring and dynamic time window slicing; Bitmap construction module: constructs a multi-level association bitmap based on device data carrying spatiotemporal feature encoding; the multi-level association bitmap includes a first-level bitmap, a second-level bitmap, and a third-level bitmap; the first-level bitmap is a physical association bitmap, the second-level bitmap is a logical association bitmap, and the third-level bitmap is a topological association bitmap; Collaborative computing module: Based on the constructed multi-level association bitmap, it realizes the collaborative computing of the edge physical nodes and central governance nodes of the distributed data architecture.

Images