A digital twin system based on time sequence big data

Abstract

The application relates to the technical field of computer data processing, and particularly discloses a digital twin system based on time sequence big data. The system comprises a physical perception layer, a data stream processing layer, a twin state calculation layer, a state storage and index layer and an application service layer. Through the cooperation of a late data management module and an incremental correction engine, only the local recalculation of the affected state is triggered for the late data; combined with the mixed storage of a time sequence state library and a causal graph library, the traceability and deterministic playback of the state are realized. The application improves the ability of the system to process out-of-order data, and guarantees the smoothness of state evolution and the reliability of historical analysis.

Description

Technical Field

[0001] This invention belongs to the field of computer data processing technology, and specifically relates to a digital twin system based on time-series big data. Background Technology

[0002] Digital twin technology, as a key enabling technology for the deep integration of the physical world and virtual space, is being widely applied in complex systems fields such as intelligent manufacturing, smart cities, and the industrial internet. Its core lies in constructing a virtual model that is dynamically synchronized with and highly faithfully mapped to the physical entity, supporting simulation, prediction, monitoring, and optimization. Among these, digital twin systems based on time-series big data aim to drive the evolution and updating of the virtual model by continuously collecting massive amounts of time-series data generated during the operation of the physical entity, forming the foundation for dynamic twinning and intelligent decision-making.

[0003] Existing technologies typically use time-series databases to store devices or business data and update the twin model through scheduled tasks or batch processing, employing static parameters or simple rules for state inference within the model. This approach suffers from the following problems: twin state updates are lagging, failing to reflect the true real-time state; late data is often ignored or recalculated entirely, leading to unsafe data correction and potentially causing aberrant twin state transitions; furthermore, the twin state lacks traceability, making it impossible to explain which historical data the current state is derived from, resulting in unstable historical playback results, with playback states at the same point in time potentially differing due to different data processing orders. As data volume continues to grow, these problems further exacerbate system scalability, making it difficult to maintain twin state consistency under data deluge.

[0004] Therefore, how to build a digital twin system that can process time-series big data in real time, accurately and traceably, and ensure strong consistency and high scalability has become a technical challenge that urgently needs to be solved in this field. Summary of the Invention

[0005] The purpose of this invention is to provide a digital twin system based on time-series big data to solve the technical contradictions in the prior art, such as delayed twin state updates, improper handling of late data leading to state jumps, unstable historical playback results, and poor system scalability.

[0006] This invention provides a digital twin system based on time-series big data, comprising:

[0007] The physical sensing layer is used to continuously collect multi-source heterogeneous time-series data that characterizes the operating status of physical entities through sensor networks deployed on physical entities and business system interfaces. The collected raw time-series data is encapsulated into standard data units with globally unique incremental timestamps and data source identifiers and asynchronously pushed to the data stream processing layer.

[0008] The data stream processing layer is used to receive and process standard data unit streams from the physical sensing layer.

[0009] The twin state computation layer is used to drive the state evolution of the twin virtual model based on time-series data;

[0010] The state storage and indexing layer is used to persistently store twin state sequences and support efficient multi-dimensional queries;

[0011] The application service layer is used to provide twin state query services, historical playback services, and cause analysis services to the outside world based on the state and causal data provided by the lower layer.

[0012] Preferably, the data stream processing layer includes a distributed stream processing engine and a late data management module;

[0013] The distributed stream processing engine is used to distribute data units to the downstream twin state computation layer according to their timestamp order.

[0014] The late data management module is used to continuously monitor the data stream. When a late data unit with a timestamp earlier than the current processing level is detected, the timestamp is routed to a dedicated late data buffer queue, and a notification signal carrying the range of late timestamps is sent to the twin state computing layer.

[0015] Preferably, the twin state calculation layer is constructed based on event temporal semantics and includes a main state calculation engine and an incremental correction engine;

[0016] The main state calculation engine is used to respond to the standard data unit stream from the data stream processing layer, and continuously calculate and output the twin state sequence in an incremental manner according to the preset twin model calculation logic.

[0017] The incremental correction engine is specifically designed to respond to notification signals from the late data management module and performs the following operations:

[0018] Based on the range of late timestamps in the notification signal, the existing twin state sequence fragments affected by the range of late timestamps are accurately retrieved from the state storage and index layer.

[0019] Based on the late data units in the late data buffer queue, only the retrieved state sequence fragments are locally recalculated; the corrected state sequence obtained from the local recalculation is then synchronized to the state storage and index layer in an incremental update manner.

[0020] Preferably, the state storage and index layer adopts a hybrid storage architecture, specifically including a time-series state library and a causal graph library;

[0021] The time-series state library is used to store complete, linear twin state sequences output by the twin state computation layer in time partitions. Each state entry contains a precise timestamp, a state value, and a reference identifier pointing to the corresponding entry in the causal graph library.

[0022] The causal graph library is used to store the inferred causal relationships between twin states in a graph structure. Each causal graph node represents a twin state. The node attributes record the set of original data unit identifiers and their computational logic summary on which the twin state is generated. The directed edges between nodes represent the inference order between states.

[0023] Preferably, the historical playback service is used to first obtain the basic state of the historical time point from the time series state library according to the historical time point specified by the user, and then trace and obtain all upstream data and intermediate states that affect the basic state according to the causal graph library, thereby assembling a deterministic and repeatable playback view.

[0024] The causal analysis service allows users to directly obtain the complete data dependency chain and computational path leading to the final state by querying the causal graph library for any twin state.

[0025] Preferably, the twin model computation logic in the twin state computation layer adopts a computation framework based on state transition functions and observation functions;

[0026] The state transition function defines the mapping relationship from the twin state at the previous time step and the input data at the current time step to the predicted state at the current time step;

[0027] The observation function defines how to fuse the predicted state with the actual observation data to output the final corrected twin state.

[0028] Both the main state calculation engine and the incremental correction engine reuse the same set of state transition functions and observation functions.

[0029] Preferably, the local recalculation process of the incremental correction engine follows the principle of causal dependency propagation;

[0030] When processing late data, the incremental correction engine first determines the set of initial states that are directly affected by the late data;

[0031] Then, based on the state dependencies in the twin model's computational logic, iteratively calculate all subsequent states affected by the initial state change until the impact propagates to the current time level or the state change converges.

[0032] Preferably, the set of original data unit identifiers associated with nodes in the causal graph library is compressed and stored using a Bloom filter data structure;

[0033] When it is necessary to determine whether a certain data unit affects a specific state, a Bloom filter can be used for quick retrieval.

[0034] For positive results returned by the Bloom filter, further confirmation is performed by accessing the metadata index.

[0035] Preferably, the time-series state library of the state storage and index layer adopts a multi-level sharding and compression strategy;

[0036] The time frame is divided into first-level segments, and within each time segment, it is further divided into second-level segments according to the different state dimensions of the twin model.

[0037] For the state sequence within each dimension slice, a combination of incremental encoding and lossy compression is used for compression and storage.

[0038] The compressed storage process includes: incrementally encoding the changes in state values, and then applying a time-series-based compression algorithm to the encoded sequence.

[0039] Preferably, the distributed stream processing engine of the data stream processing layer adopts an event-time-based windowing processing mechanism;

[0040] The processing water level is dynamically advanced based on the timestamps of the received data, and the maximum out-of-order time tolerance threshold can be configured.

[0041] Late data arriving within the tolerance threshold can still be included in the correct calculation window;

[0042] Extremely late data exceeding the tolerance threshold will be processed by the late data management module according to the aforementioned process.

[0043] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0044] 1. This invention achieves precise and localized processing of late data by designing a collaborative mechanism between the data stream processing layer and the twin state computation layer, particularly by introducing a late data management module and an incremental correction engine based on event temporal semantics. The system can safely integrate late data into existing state sequences, performing incremental correction only on affected state segments. This avoids the huge overhead and state jumps caused by traditional overall recalculation, ensuring the smoothness and eventual consistency of twin state evolution, and significantly improving the system's ability to handle out-of-order and delayed real-world data.

[0045] 2. This invention creatively constructs a hybrid storage architecture comprising a temporal state library and a causal graph library, storing the complete data dependencies and computational logic associations between twin states and generated states. This design ensures complete traceability for any twin state. During historical playback or causal analysis, the system can accurately reproduce the entire process of state generation through the causal graph, obtaining definite and stable results. This completely resolves the ambiguity in historical playback caused by the uncertainty of data processing order, providing a reliable basis for in-depth analysis and fault diagnosis.

[0046] 3. The system architecture of this invention adopts a layered decoupling and microservice design, with each layer having clearly defined functions and being independently scalable. Key components such as data stream processing, state computation, and storage indexing all possess distributed processing capabilities, and throughput can be linearly increased by adding the number of instances. Combining a multi-level sharding compression strategy for state storage with an optimized storage structure for causal graphs, the system demonstrates significant advantages in computing performance, storage efficiency, and system scalability when dealing with massive amounts of time-series big data, and can support large-scale, high-concurrency digital twin application scenarios. Attached Figure Description

[0047] Figure 1 This is a schematic diagram of the overall technical solution architecture of the present invention;

[0048] Figure 2 This is a schematic diagram of the core principle framework of the collaborative processing mechanism for late data management and incremental correction in this invention;

[0049] Figure 3 This is a flowchart of the state evolution logic of the twin state calculation layer based on event temporal semantics in this invention.

[0050] Figure 4 This is a schematic diagram of the hybrid storage architecture of state storage and index layer and the causal tracing relationship in this invention;

[0051] Figure 5 This is a schematic diagram illustrating the principle of the application service layer based on causal graph-based historical playback and causal analysis service in this invention. Detailed Implementation

[0052] refer to Figures 1 to 5 The digital twin system based on time-series big data proposed in this invention consists of five main parts: a physical perception layer, a data stream processing layer, a twin state calculation layer, a state storage and indexing layer, and an application service layer. These layers communicate loosely through standardized interfaces and high-throughput message channels, forming an end-to-end, scalable, and highly consistent state evolution and traceability system. The following will, with reference to the accompanying drawings, elaborate on the specific implementation mechanism, internal module composition, data flow logic, and key technical details of each layer.

[0053] First, the physical sensing layer, serving as the system's data entry point, is responsible for continuously acquiring multi-source heterogeneous time-series data characterizing the operational state of entities from the physical world. This layer is deployed around or embedded within the target physical entity and includes several types of sensor devices (such as temperature sensors, pressure sensors, vibration sensors, and position sensors) as well as data interface modules that interface with the business information system. These sensors and interfaces collect raw data at fixed or dynamic sampling frequencies, including but not limited to analog quantities, digital quantities, event logs, and control command feedback. All raw data is uniformly formatted and encapsulated into standard data units in a local preprocessing unit.

[0054] Each standard data unit contains three core fields: a globally unique, incrementing timestamp, a data source identifier, and the original observation or structured data payload. The globally unique, incrementing timestamp is generated by a time synchronization service deployed at the physical sensing layer. This service achieves microsecond-level time alignment based on network time protocols or precision time protocols, ensuring comparability and consistency of data from different data sources across time dimensions. The data source identifier employs a hierarchical namespace structure, encoding metadata such as device type, installation location, and subsystem affiliation, facilitating subsequent data routing and traceability.

[0055] The encapsulated standard data units are pushed to the data stream processing layer in an asynchronous, non-blocking manner. The push process uses a reliable transmission channel based on the transmission control protocol to ensure that the data is not lost or duplicated.

[0056] Please refer to the attached document. Figure 1 The data stream processing layer receives high-speed standard data unit streams from the physical sensing layer and undertakes key tasks such as data cleaning, out-of-order tolerance, watermark management, and late data identification. The core components of this data stream processing layer are a distributed stream processing engine and a late data management module. The distributed stream processing engine is built based on event-time semantics and internally maintains a dynamically advancing processing watermark. This processing watermark does not simply follow the system clock, but is calculated by subtracting a preset maximum out-of-order time tolerance threshold (e.g., 500 milliseconds) from the maximum timestamp among the received data units.

[0057] Newly arrived data units are considered valid if their timestamps are greater than or equal to the current watermark, and are distributed to the twin state computation layer in timestamp order. If the timestamp is less than the current watermark but still within the tolerance threshold, it is still included in the corresponding computation window for processing. If the timestamp is much earlier than the watermark and exceeds the tolerance threshold, it is judged as extremely late data and handled by the late data management module. This mechanism ensures robustness to unavoidable data delays and out-of-order phenomena in the real world while maintaining processing efficiency.

[0058] The late data management module continuously monitors for abnormal timestamp patterns in the data stream. When it detects one or more late data units with timestamps earlier than the current processing level, the module performs the following operations: First, it writes these late data units into a dedicated late data buffer queue. This buffer queue employs a first-in, first-out (FIFO) strategy and has persistence capabilities to prevent data loss due to system failures. Second, it analyzes the timestamp range of these late data units and extracts the minimum timestamp. With the maximum timestamp This forms continuous or discrete time intervals; finally, a late notification signal is sent to the twin state computation layer, carrying... and This parameter indicates the timeframe within which local corrections are needed. It's important to note that this module does not trigger a global state recalculation; instead, it only marks the affected areas, providing precise input for subsequent incremental corrections.

[0059] Please refer to the attached document. Figure 2 The twin state computation layer is the core computational hub of the entire system. Its design goal is to efficiently handle state correction requirements caused by delayed data while ensuring the real-time performance and accuracy of state evolution. This twin state computation layer comprises two cooperating sub-engines: a main state computation engine and an incremental correction engine. The main state computation engine continuously receives valid data unit streams from the data stream processing layer and drives the virtual model's state evolution incrementally according to the preset twin model computation logic. The twin model computation logic adopts a framework based on state transition functions and observation functions. State transition function... Defined from Twin state of time and Input data at any time arrive Predicting the state at any time The mapping relationship is as follows:

[0060] ;

[0061] Observation function Then used for fusing and predicting states Compared with actual observation data The final corrected twin state is output. Under ideal conditions without external interference, the final corrected twin state is... However, in the presence of noise or model bias, the observation function can incorporate Kalman filtering, weighted averaging, or other fusion strategies to improve the robustness of state estimation. The main state calculation engine calls [the appropriate function] after receiving each new data unit. and The function completes a state update and sends the final corrected twin state along with its precise timestamp. It is output to the state storage and index layer together.

[0062] Upon receiving a late notification signal from the late data management module, the incremental correction engine immediately initiates a local recalculation process. This local recalculation process strictly adheres to the principle of causal dependency propagation, and the specific steps are as follows:

[0063] The first step is to determine the timestamp range from the notification signal. It initiates a query request to the state storage and index layer to accurately retrieve all twin state sequence fragments stored within this time range;

[0064] The second step is to extract all late data units within the corresponding timestamp range from the late data buffer queue.

[0065] The third step is to re-execute the state transition function using these late data units as input. With observation function The retrieved state sequence fragments are locally recalculated to generate a corrected state sequence.

[0066] Fourth, during the recalculation process, the engine dynamically tracks the dependencies between states—if the recalculated state... Compared with the original state If discrepancies exist, further calculations are required. The subsequent states affected by the change continue until the change in state is less than a preset convergence threshold (e.g., 0.001) or propagates to the current processing level. The fifth step involves writing the final determined corrected state sequence into the state storage and index layer using atomic transactions, ensuring that other read / write operations are isolated during the writing process, thereby maintaining state consistency and integrity. The entire process involves only a limited number of state nodes on the causal chain, avoiding the high overhead of global recalculation.

[0067] Please refer to the attached document. Figure 3 The above state evolution logic clearly demonstrates the state continuity guarantee mechanism under the event temporal semantics. Regardless of whether the data arrives on time, the system always uses the actual time of the event as the benchmark for state deduction, and dynamically corrects the state deviation caused by data delay through the incremental correction engine, thereby achieving the dual goals of "final correctness" and "process smoothness".

[0068] The state storage and indexing layer is responsible for persistently storing all twin states and their generation contexts, and supports efficient querying and tracing. For example... Figure 4As shown, the state storage and index layer adopts a hybrid storage architecture, consisting of a time-series state library and a causal graph library. The time-series state library employs a multi-level storage strategy combining time partitioning and dimensional sharding. The first-level partitioning is divided according to fixed time windows (e.g., hourly or daily), with each time partition corresponding to an independent storage bucket. Within each time partition, secondary sharding is performed based on the state dimensions of the twin model (e.g., location, velocity, temperature, load rate, etc.), with each dimensional shard storing the complete state time series under that dimension.

[0069] For the state sequence within each dimension slice, the system employs a combination of incremental encoding and lossy compression for compression. Specifically, the state value sequence is first differentially processed to record the changes between adjacent states. ; for Predict the state at any time, and then... Sequences are compressed using time-series compression algorithms (such as Gorilla compression or Delta-of-Delta coding), leveraging the high autocorrelation of time-series data to significantly reduce storage size. The compressed data is stored in a columnar format, facilitating efficient scanning by time range or dimension.

[0070] The causal graph library stores the derived causal relationships between states using a directed graph structure. Each node in the graph represents a twin state, and node attributes include: a state timestamp, a state value, a set of original data unit identifiers that the generated state depends on, and a summary of the computational logic used (such as a function). and (Version number or hash value). Directed edges between nodes represent the order of state derivation, i.e., from... point to .

[0071] The original data unit identifier set is compressed and stored using a Bloom filter. A Bloom filter is a probabilistic data structure that can efficiently determine whether an element belongs to a set with minimal space. When it is necessary to verify a data unit... Does it affect the status? When the system first queries the Bloom filter corresponding to the original data unit identifier set, if it returns "not found", it can be quickly excluded; if it returns "may exist", it further accesses the metadata index for precise matching. This design reduces storage overhead to less than 10% of the original set while ensuring causal tracing functionality.

[0072] When the incremental correction engine performs local recalculation, the corresponding nodes and their associated edges in the causal graph library will be updated synchronously. For example, if Replaced the original In the atlas The nodes will reconstruct the set of data unit identifiers (including late data unit identifiers) and re-establish edge connections with the predecessor and successor states. Simultaneously, old nodes and their associated edges are marked as historical versions and preserved in the graph to support complete historical backtracking. This versioned graph design ensures that the state at any historical moment can be accurately reproduced.

[0073] The application service layer, based on the state and causal data provided by the lower layer, offers three core services: real-time twin state query, historical playback, and causal analysis. The real-time query service responds to user requests for the current or specified point-in-time twin state, directly retrieving the corresponding state value from the time-series state database and returning it. The historical playback service is more complex, such as... Figure 5 As shown, its workflow is as follows:

[0074] When the user specifies a historical time point At that time, the service first retrieves historical time points from the time-series state database. status As a base state; subsequently, tracing back through the causal graph library The system retrieves the complete data dependency chain, recursively querying its predecessor nodes until the initial state or external input data; during this process, the system collects all influencing factors. The service identifies the original data unit, intermediate states, and computation logic; finally, based on this information, the service can replay the entire computation process in an isolated sandbox environment, generating a replay view that is completely consistent with the original computation.

[0075] Because the replay process strictly relies on the deterministic dependencies recorded in the causal graph, the results remain stable regardless of when it is executed, completely eliminating replay ambiguity caused by the uncertainty of the data processing order.

[0076] The cause analysis service allows users to explore the causes of any twin state. By traversing the path back to the root node from the twin state in the causal graph library, the service extracts the complete data dependency chain and computational path, presenting it in the form of a visual graph or structured report. For example, if a device experiences an abnormally high temperature, cause analysis can reveal whether the condition is caused by abnormal data from specific sensors, control command delays, or model parameter deviations, providing direct evidence for fault diagnosis and root cause localization.

[0077] The entire system runs on a containerized microservice architecture. The physical awareness layer, data stream processing layer, twin state computation layer, state storage and indexing layer, and application service layer are all encapsulated as independent Docker containers and orchestrated and managed using Kubernetes. The microservices at each layer communicate asynchronously through the Apache Kafka messaging middleware, ensuring high throughput and decoupling.

[0078] The service interface adopts a combination of RESTful API and gRPC protocol. The former is used for management configuration and status query, while the latter is used for high-performance data stream transmission. The system supports horizontal scaling: when data throughput increases, the processing capacity can be linearly increased by adding the number of instances of the data stream processing layer or the twin state computation layer; when storage pressure increases, the number of shards in the time-series state library or the graph database nodes in the causal graph library can be dynamically expanded.

[0079] In addition, each layer has built-in health checks, automatic failover and load balancing mechanisms to ensure the stability and availability of the system under high concurrency and long-term operation scenarios.

[0080] In summary, this embodiment constructs a highly consistent, highly traceable, and highly scalable digital twin system through five core technologies: layered architecture, event-temporal semantics, incremental correction mechanism, hybrid storage model, and causal graph tracing. This system not only reflects the state of physical entities in real time and accurately, but also maintains the smoothness and ultimate correctness of state evolution when facing the data disorder and latency problems prevalent in the real world, while providing a solid data foundation for in-depth analysis and decision-making.

[0081] Based on the aforementioned embodiments, this embodiment further refines the specific implementation of the state transition function and observation function in the twin state computation layer, and introduces a dynamic model switching mechanism to adapt to the changes in the behavior patterns of physical entities under different operating conditions.

[0082] Specifically, the computational logic of the twin model no longer uses a single, fixed approach. and Instead of functions, the system maintains a model library containing multiple pre-trained combinations of state transition functions and observation functions for different operating conditions (such as startup, steady-state operation, fault modes, and maintenance states). Each combination has undergone offline training and online validation, possessing clearly defined applicability boundaries. During operation, the system uses an operating condition identification module to analyze the statistical characteristics of the current input data in real time (such as mean, variance, spectral characteristics, and density of abrupt change points), and matches them with feature templates for each operating condition, dynamically selecting the most suitable model combination for the current state calculation.

[0083] For example, when a physical entity is in the startup phase, its state changes drastically and is significantly nonlinear. In this case, the system selects a nonlinear state-space model. Functions and Particle-Based Filtering Once the system enters steady-state operation, the state changes tend to be stable, and the system switches to a lightweight model combination based on linear Kalman filtering to reduce computational overhead. The model switching process is uniformly managed by the scheduler inside the twin-state computation layer, ensuring the continuity of the state before and after the switch—that is, the initial state of the new model is derived from the final state of the old model through coordinate transformation or state mapping function, avoiding state jumps caused by model switching.

[0084] Furthermore, the incremental correction engine must also consider the history of model switching when processing late data. When recalculation spans multiple operating condition time periods, the engine will call the corresponding model combination for segmented recalculation based on the operating condition label corresponding to each sub-time period, and perform state connection verification at the operating condition switching point. If the connection error is found to be greater than the threshold, a secondary correction or alarm will be triggered, indicating that there may be a problem with the model boundary definition.

[0085] The dynamic model mechanism enhances the system's adaptability to complex and changing physical environments, enabling the digital twin not only to "synchronize" with physical entities but also to "understand" their behavioral patterns. This allows it to play a greater role in advanced application scenarios such as predictive maintenance and energy efficiency optimization. The introduction of this dynamic model mechanism does not compromise the integrity of the original architecture. All model metadata, operating condition labels, and switching logs are fully recorded in the causal graph library as part of the state computation context, ensuring that the accuracy of historical playback and causal analysis remains unaffected.

[0086] This embodiment focuses on optimizing the state storage and indexing layer, especially enhancing the storage and query performance of the causal graph library in ultra-large-scale deployment scenarios.

[0087] In the original design, the causal graph library stored all state nodes and their dependencies using a single graph database instance. When the number of twin models managed by the system reached tens of thousands and the total number of state nodes exceeded tens of billions, the single-instance graph database faced problems of increased query latency and decreased write throughput. Therefore, this embodiment introduces a graph sharding and index layering strategy.

[0088] First, the causal graph is partitioned into primary shards based on twin model identifiers, with each model having an independent subgraph that is physically isolated from others. Second, within each subgraph, secondary sharding is performed based on time windows; for example, a causal chain of states every 7 days constitutes a subgraph shard. Shards are connected via cross-shard pointers, supporting causal tracing across time periods. All shards are distributed across a distributed graph database cluster, which uses a consistent hashing algorithm for data routing to ensure load balancing.

[0089] Secondly, for high-frequency query patterns (such as "all direct predecessors of a certain state" or "all states affected by a certain data unit"), the system constructs a multi-level index. The first-level index is a mapping from state timestamps to graph nodes, stored in a high-performance key-value store; the second-level index is an inverted index from the original data unit identifier to the list of affected states, stored using a compressed bitmap or Roaring Bitmap structure, supporting fast set operations; the third-level index is a composite index of meta-attributes such as working condition labels and model versions, used to accelerate conditional filtering queries.

[0090] In addition, the system introduces a graph caching layer to cache recently active causal subgraphs (such as state dependency chains from the last 24 hours) in an in-memory graph database, reducing the access pressure on the underlying persistent storage. The cache adopts an LRU eviction policy and maintains eventual consistency with the underlying storage through asynchronous synchronization via change logs.

[0091] Through the above optimizations, the causal graph library can still guarantee millisecond-level single-point query response and second-level complex path traversal performance even at a scale of hundreds of billions of nodes, providing solid support for real-time analysis and decision-making in large-scale digital twin systems.

[0092] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus.

[0093] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A digital twin system based on time-series big data, characterized in that, include: The physical sensing layer is used to continuously collect multi-source heterogeneous time-series data that characterizes the operating status of physical entities through sensor networks deployed on physical entities and business system interfaces. The collected raw time-series data is encapsulated into standard data units with globally unique incremental timestamps and data source identifiers and asynchronously pushed to the data stream processing layer. The data stream processing layer is used to receive and process standard data unit streams from the physical sensing layer. The twin state computation layer is used to drive the state evolution of the twin virtual model based on time-series data; The state storage and indexing layer is used to persistently store twin state sequences and support efficient multi-dimensional queries; The application service layer is used to provide twin state query services, historical playback services, and cause analysis services to the outside world based on the state and causal data provided by the lower layer. The data stream processing layer includes a distributed stream processing engine and a late data management module; The distributed stream processing engine is used to distribute data units to the downstream twin state computation layer according to their timestamp order. The late data management module is used to continuously monitor the data stream. When a late data unit with a timestamp earlier than the current processing level is detected, the timestamp is routed to the late data buffer queue, and a notification signal carrying the range of late timestamps is sent to the twin state computing layer. The twin state calculation layer is constructed based on event temporal semantics and includes a main state calculation engine and an incremental correction engine. The main state calculation engine is used to respond to the standard data unit stream from the data stream processing layer, and continuously calculate and output the twin state sequence in an incremental manner according to the preset twin model calculation logic. The incremental correction engine is used to perform the following operations in response to a notification signal from the late data management module: Based on the range of late timestamps in the notification signal, the existing twin state sequence fragments affected by the range of late timestamps are accurately retrieved from the state storage and index layer. Based on the late data units in the late data buffer queue, only the retrieved state sequence fragments are locally recalculated; the corrected state sequence obtained from the local recalculation is synchronized to the state storage and index layer in an incremental update manner. The state storage and index layer adopts a hybrid storage architecture, specifically including a time-series state library and a causal graph library; The time-series state library is used to store complete, linear twin state sequences output by the twin state computation layer in time partitions. Each state entry contains a precise timestamp, a state value, and a reference identifier pointing to the corresponding entry in the causal graph library. The causal graph library is used to store the inferred causal relationships between twin states in a graph structure. Each causal graph node represents a twin state. The node attributes record the set of original data unit identifiers and their computational logic summary on which the twin state is generated. The directed edges between nodes represent the inference order between states. The historical playback service is used to first obtain the basic state of the historical time point specified by the user from the time series state library, and then trace and obtain all upstream data and intermediate states that affect the basic state according to the causal graph library, thereby assembling a playback view. The causal analysis service allows users to directly obtain the complete data dependency chain and computational path leading to the final state by querying the causal graph library for any twin state.

2. The digital twin system based on time-series big data according to claim 1, characterized in that, The twin model computation logic in the twin state computation layer adopts a computation framework based on state transition functions and observation functions; The state transition function defines the mapping relationship from the twin state at the previous time step and the input data at the current time step to the predicted state at the current time step; The observation function defines how to fuse the predicted state with the actual observation data to output the final corrected twin state. Both the main state calculation engine and the incremental correction engine reuse the same set of state transition functions and observation functions.

3. A digital twin system based on time-series big data according to claim 2, characterized in that, The local recalculation process of the incremental correction engine follows the principle of causal dependency propagation. When processing late data, the incremental correction engine first determines the set of initial states that are directly affected by the late data; Then, based on the state dependencies in the twin model's computational logic, iteratively calculate all subsequent states affected by the initial state change until the impact propagates to the current time level or the state change converges.

4. A digital twin system based on time-series big data according to claim 3, characterized in that, The set of original data unit identifiers associated with nodes in the causal graph library is compressed and stored using a Bloom filter data structure; When it is necessary to determine whether a certain data unit affects the state, a quick retrieval can be performed by querying a Bloom filter. For positive results returned by the Bloom filter, further confirmation is performed by accessing the metadata index.

5. A digital twin system based on time-series big data according to claim 4, characterized in that, The time-series state library of the state storage and index layer adopts a multi-level sharding and compression strategy. The time frame is divided into first-level segments, and within each time segment, it is further divided into second-level segments according to the different state dimensions of the twin model. For the state sequence within each dimension slice, a combination of incremental encoding and lossy compression is used for compression and storage. The compressed storage process includes: incrementally encoding the changes in state values, and then applying a time-series-based compression algorithm to the encoded sequence.

6. A digital twin system based on time-series big data according to claim 5, characterized in that, The distributed stream processing engine of the data stream processing layer adopts an event-time-based windowing processing mechanism. The processing water level is dynamically advanced based on the timestamps of the received data, and the maximum out-of-order time tolerance threshold can be configured. Late data arriving within the tolerance threshold is still included in the correct calculation window; Extremely late data exceeding the tolerance threshold will be processed by the late data management module according to the established procedures.

Images