A digital twin distributed data storage and computing platform for a logistics transfer yard

Abstract

The present invention discloses a digital twin distributed data storage and computing platform for a logistics transfer yard, including: a distributed data storage module, which is used to design a data partitioning strategy, a redundancy backup and fault tolerance mechanism; perform real-time preprocessing and short-term caching, and achieve long-term persistence; a real-time computing module, which constructs a parallel computing architecture to process batch and streaming data; designs a global state update model; a collaborative simulation module, which constructs a high-precision digital twin model of the physical transfer yard; the cloud generates scheduling decisions and correction amounts according to the global simulation results, and feeds them back to the edge nodes through secure communication, and the edge immediately responds to adjust the local data processing and device control parameters, so as to achieve the closed-loop consistency between the real scenario and the digital twin model. The present invention constructs a scheduling architecture for a logistics sorting digital twin system supported by distributed data storage, edge-cloud collaboration, real-time computing and collaborative simulation.

Description

Technical Field

[0001] The present invention relates to the field of logistics, and particularly to a digital twin distributed data storage and computing platform for a logistics transfer yard. Background Art

[0002] With the rapid development of industrial Internet and Internet of Things, the data sources in traditional logistics transfer yards have become increasingly diverse, including sensor data, video surveillance, manually input data, etc. By using digital twin technology, a real-time digital model of a physical transfer yard can be constructed in a virtual environment, and through high-fidelity simulation means, the prediction, monitoring, and scheduling optimization of equipment status, logistics flow, and environmental parameters can be realized. However, traditional centralized storage systems have the following deficiencies in processing massive data:

[0003] 1. High-concurrency data writing and reading bottlenecks: Centralized systems are prone to single-point failures and are difficult to meet the real-time access requirements of massive heterogeneous data generated by equipment, sensors, etc. in logistics transfer yards.

[0004] 2. Limited real-time computing power: Batch processing systems cannot analyze data immediately and cannot meet real-time warning and scheduling feedback.

[0005] 3. Insufficient collaborative simulation accuracy: There is a lack of efficient coupling between discrete data and simulation models, and synchronous physical and virtual environment feedback cannot be achieved. Summary of the Invention

[0006] The purpose of the present invention is to overcome the deficiencies of the prior art and provide a digital twin distributed data storage and computing platform for a logistics transfer yard, which constructs a scheduling architecture for a logistics sorting digital twin system based on distributed data storage, edge-cloud collaboration, real-time computing, and collaborative simulation support. A data partitioning and real-time status update model is established for the sorting scenario of the logistics transfer yard, and the balanced distribution and fault tolerance recovery of data are realized through the consistent hashing algorithm, virtual nodes, and Raft protocol.

[0007] The purpose of the present invention is achieved through the following technical solutions: A digital twin distributed data storage and computing platform for a logistics transfer yard, comprising:

[0008] A distributed data storage module, which is used to design a data partitioning strategy to evenly distribute massive heterogeneous data in a distributed environment; achieve high data availability through redundant backup and fault tolerance mechanisms; perform real-time preprocessing and short-term caching on the logistics sorting site at the edge layer side, and then upload it to the cloud asynchronously for replication, and use a cloud distributed file system or object storage to achieve long-term persistence;

[0009] A real-time computing module, used to build a parallel computing architecture to process batch and streaming data; design a global state update model, enabling each computing node to cooperate on local states and utilize a high-performance big data platform for global data fusion to obtain the global state.

[0010] A co-simulation module, used to build a high-precision digital twin model of the physical transfer yard; use continuous time and discrete event simulation technologies to achieve multi-node co-simulation, ensure the consistency between the real scenario and the digital twin model, the cloud generates scheduling decisions and correction amounts based on the global simulation results, and feeds them back to the edge nodes through secure communication. The edge immediately responds to adjust the local data processing and device control parameters, realizing the closed-loop consistency between the real scenario and the digital twin model.

[0011] The beneficial effects of the present invention are as follows: The present invention constructs a scheduling architecture for a logistics sorting digital twin system supported by distributed data storage, edge-cloud cooperation, real-time computing, and co-simulation. It establishes a data partitioning and real-time state update model for the sorting scenario of the logistics transfer yard, and realizes the balanced distribution and fault tolerance recovery of data through the consistent hashing algorithm, virtual nodes, and the Raft protocol. At the same time, real-time data collection, preprocessing, and short-term caching of on-site data are completed at the edge layer, while the cloud is responsible for data aggregation, persistent storage, and global state fusion, and uses the MapReduce and stream computing frameworks to achieve distributed parallel computing. Further combined with the digital twin model and the closed-loop feedback control mechanism (using the PID algorithm), it realizes the precise monitoring and dynamic verification of global scheduling decisions and on-site device states, effectively reconciling the balance among data consistency, system throughput, response speed, and scheduling accuracy. Thus, the present invention provides a new theoretical basis and technical path for the intelligent scheduling and efficient decision-making of the logistics sorting system, effectively realizes the dynamic collaborative optimization of key performance indicators, and lays a solid foundation for the subsequent research and application of the intelligent logistics digital twin system. Description of the Drawings

[0012] Figure 1 It is a schematic diagram of the principle of the present invention. Detailed Embodiments

[0013] The technical solution of the present invention will be further described in detail below with reference to the drawings, but the protection scope of the present invention is not limited to the following.

[0014] As Figure 1 shown, a digital twin distributed data storage and computing platform for a logistics transfer yard includes:

[0015] 1. Distributed data storage module, which is used to design a data partitioning strategy to evenly distribute massive heterogeneous data in a distributed environment; achieve high data availability through redundancy backup and fault tolerance mechanisms; perform real-time preprocessing and short-term caching on the logistics sorting site on the edge layer side, and then upload it to the cloud through asynchronous replication, and use the cloud distributed file system or object storage to achieve long-term persistence;

[0016] 1.1 Data Partitioning and Load Balancing: The consistent hashing algorithm is used to evenly distribute the current logistics transfer yard sorting data items to each storage node of the digital twin system. Specifically, it includes: assuming that the current digital twin system includes storage nodes, and multiple virtual nodes are configured on each node to improve the balance; define the hash function , and map the sorting data item to the interval ; among them, represents the set of sorting data items , i = 1, 2,.., I , where I represents the number of sorting data items; assume that the positions of all virtual nodes are located on the hash ring, and the values are between 0 and 1; the virtual node position of the storage node is , where is the number of virtual nodes;

[0017] Then the main storage node selection rule for the sorting data item is:

[0018]

[0019] That is, find all the values of j that satisfy , and take the j value among them; represents mapping the sorting data item to the interval , and getting the hash value;

[0020] If all do not satisfy , then ;

[0021] 1.2 Redundancy Backup and Fault Tolerance Mechanism: In addition to the main storage node for each data item, the system will sequentially select the next one in the clockwise direction of the main node A storage node serves as a replica storage point. To ensure data state consistency between the primary node and replicas, the system adopts the distributed consensus protocol Raft, specifically including: in the current cluster of the primary node and replicas, the Raft protocol elects a leader responsible for scheduling all write operations; the remaining nodes act as follower nodes; when a sorting data write request arrives, the leader first writes the sorting data write request into its own log and then replicates the sorting data write request to other follower nodes; when more than half of the total number of nodes in the cluster confirm receiving the sorting data write request and successfully write it into their own logs, the leader and follower nodes write the sorting data and update the node state.

[0022] Through replicas, when the primary node fails, the system can elect a new primary node from the remaining replicas, quickly complete fault recovery, ensure data is not lost, and business operations continue without interruption.

[0023] 1.3 Edge-Cloud Hierarchical Storage Architecture: To meet the dual requirements of low-latency response and big data deep processing in the logistics sorting scenario, this solution adopts an edge-cloud collaborative architecture, including edge layer storage, cloud storage, data synchronization, and consistency.

[0024] 1.3.1 Edge Layer Storage, which is concentrated on edge servers or intelligent gateways deployed near the logistics sorting site. Its main functions include:

[0025] Real-time data collection and preprocessing, that is, directly collecting sensor sorting data on-site, including sorting speed, equipment status, video, RFID information, and performing filtering and normalization processing on the original sorting data using a sliding window algorithm to obtain:

[0026]

[0027] where is the sliding window size; is the current data processing time, is the offset step within the sliding window, and its value range is , representing the historical time point traced back from the current moment; represents at time the original data value;

[0028] Short-term caching and local analysis, that is, caching the sorting data within the last few seconds to minutes in local memory or flash memory, and using a local state update model to achieve real-time updates:

[0029]

[0030] Among them, represents the state vector of the edge node. The initial value of the vector is usually zero, which dynamically reflects the local evaluation of the edge node in dimensions such as sorting state recognition and operation stability. represents the time step of state update, with the unit of second, which is specifically determined by the system sampling frequency or the processing period of the edge node; is the edge - side state mapping function, which is used to estimate its change rate based on the current state and input data. The specific definition is

[0031]

[0032] Among them, is the weight matrix, is the bias vector, is the ReLU activation function, and the input is the concatenation of . The output result is the estimated value of the state change rate. After multiplying by the time step , the state vector is updated.

[0033] The training of the weight and bias vectors is as follows:

[0034] The input feature pair z(t) = , the true state change rate (obtained from historical collected data)

[0035] ;

[0036] The definition of the loss function: ;

[0037] Based on the loss function, use Adam to iteratively update the parameters until convergence (that is, the loss function is less than the set threshold).

[0038] 1.3.2 Cloud storage is responsible for aggregating and persistently storing the data from each edge node, and storing the global state after global update of the real - time computing module.

[0039] 1.3.3 Data synchronization and consistency. Upload to the cloud through asynchronous replication and message queues to ensure the consistency of sorting data upload, and use the consistent hashing algorithm and the Raft fault - tolerance mechanism mentioned in 1.2 to ensure the synchronization of edge and cloud data versions.

[0040] 2. Real - time computing module is used to build a parallel computing architecture to process batch and streaming data; design a global state update model so that each computing node can cooperate to utilize local states and perform global data fusion on a high - performance big data platform to obtain the global state;

[0041] The real-time computing module adopts a parallel computing architecture and uses MapReduce and a stream computing framework to process the batch and streaming data uploaded from the edge, which is used to support the decentralized computing and global collaborative update of data:

[0042] Based on the edge-cloud hierarchical storage architecture, local calculations are first performed separately at each edge node in the Map stage to obtain local states.

[0043] Then, in the Reduce stage, the global state is updated using the local states uploaded by each edge node.

[0044]

[0045] Among them, represents the global state vector, and the initial value of the vector is zero; represents the global input:

[0046] is a preset weight, represents the original sorting data of the i-th edge node The result obtained after filtering and normalization through the sliding window algorithm;

[0047] is a fusion function, defined as:

[0048]

[0049] Among them, is the weight matrix, is the bias vector, is the ReLU activation function. The weights and bias parameters are trained as follows:

[0050] Input feature pair = , the target output of the real global state change rate (obtained from historical collected data):

[0051]

[0052] Function:

[0053] Based on the loss function, Adam is used to iteratively update the parameters until convergence (i.e., the loss function is less than the set threshold).

[0054] 2.2 Global state update: First, each edge node updates the local state according to the real-time data Then, the cloud fuses the states from each node, corrects errors, and performs in-depth data analysis, and finally updates the global digital twin state to achieve seamless collaboration.

[0055] 3. The co-simulation module is used to build a high-precision digital twin model of the physical transfer yard; it uses continuous-time and discrete-event simulation technologies to achieve multi-node co-simulation, ensuring the consistency between the real scenario and the digital twin model. The cloud generates scheduling decisions and correction amounts based on the global simulation results and feeds them back to the edge nodes through secure communication. The edge nodes immediately respond to adjust the local data processing and device control parameters, realizing the closed-loop consistency between the real scenario and the digital twin model.

[0056] 3.1 Data coordination and synchronization of the high-precision digital twin model of the physical transfer yard: It includes simulating the dynamic changes of sorting equipment during sorting tasks in continuous-time simulation, and using differential equations to describe the equipment state :

[0057]

[0058] where represents the local sorting status information of the sorting equipment. And in discrete-event simulation, key events such as scheduling decisions and abnormal events are described, and the state is corrected through event triggering.

[0059] 3.2 Multi-node co-simulation: Combine the states of all key scheduling nodes into a global state vector :

[0060]

[0061] where is the scheduling and control signal to ensure the coordinated operation of each node.

[0062] 3.3 Coordinated feedback and closed-loop control: To ensure the real-time synchronization of cloud scheduling decisions and on-site control, a closed-loop feedback mechanism is established, including global error calculation and feedback generation and immediate response of edge nodes.

[0063] 3.3.1 Global error calculation and feedback generation: According to the global state updated at each moment , determine the desired state through historical sorting data, and calculate the global error:

[0064]

[0065] Based on the error , use the PID algorithm to calculate the feedback correction amount:

[0066]

[0067] where is the proportional gain, directly amplifying the current error; is the integral gain, used to eliminate the steady-state error; is the differential gain, which is used to predict the change of error and reduce system oscillation.

[0068] 3.3.2 Edge Node Instantaneous Response: As the part closest to the on-site physical environment, the main task of the edge side is to receive feedback control information from the cloud and timely adjust local parameters or device operating parameters. Specifically, it includes using the edge node to maintain a persistent connection with the cloud using MQTT and receiving feedback messages; and verifying the received messages, using the check bits to verify the message integrity, and confirming the timeliness and version consistency of the messages. After the edge node receives the cloud feedback, update the processing parameters:

[0069]

[0070] Among them, is the original sorting data after filtering and normalization by the sliding window algorithm, is the updated parameter. At the same time, the edge side can configure a local buffering mechanism: before receiving new feedback, continue to use the previous parameter, temporarily store the key data, and wait for subsequent updates to ensure that the system will not fall into an unstable state due to communication delays.

[0071] The updated parameters are directly applied to the control logic of the edge device. For example, automatically adjusting the camera sampling rate, controlling the running speed of the sorting device, correcting the threshold value collected by the sensor, etc.; the edge node also feeds back the updated local state to the cloud for subsequent global fusion and iterative improvement, so as to form a closed-loop control and achieve continuous update of the global and local states of the system and continuously approach the optimal equilibrium state.

[0072] The overall working principle of the present invention is as follows: It consists of three modules: data storage, real-time computing, and collaborative simulation, forming a tightly linked hierarchical closed-loop structure with each other: 1) The data storage module first collects on-site sorting data at the edge side, performs preprocessing and short-term caching, and uploads the data to the cloud through the consistent hashing and redundant backup mechanisms to achieve efficient partitioning, distributed storage, and version consistency guarantee of multi-source heterogeneous data; 2) The real-time computing module directly calls the structured data and state vectors at the edge and in the cloud, and uses the MapReduce and stream computing frameworks for batch processing and incremental analysis, and updates the local state at the edge and the global state in the cloud in real time to provide dynamic data support for subsequent simulation and scheduling; 3) The collaborative simulation module runs continuous-time and discrete-event simulation models based on the state information output by the real-time computing module, constructs a digital twin picture of the global logistics sorting scenario, and calculates the global error by comparing with the historical model; 4) Subsequently, a feedback control quantity is generated through the PID algorithm and transmitted to the edge node via the message middleware to be used for real-time adjustment of the device operation parameters and local processing strategies, and the updated local state is then fed back to the storage and computing modules to achieve dynamic closed-loop control with edge-cloud collaboration and data-model linkage and efficient coupling of multiple modules.

[0073] The above is the preferred embodiment of the present invention. It should be understood that the present invention is not limited to the form disclosed herein, and should not be regarded as excluding other embodiments, but can be used in other combinations, modifications, and environments, and can be changed within the scope of the concept described herein through the above teachings or the technology or knowledge in related fields. Any changes and variations made by those skilled in the art without departing from the spirit and scope of the present invention shall fall within the protection scope of the appended claims of the present invention.

Claims

1. A digital twin distributed data storage and computing platform for a logistics transfer yard, characterized in that: Including: A distributed data storage module, which is used to design a data partitioning strategy, add redundant backup and fault tolerance mechanisms, perform real-time preprocessing and short-term caching on the logistics sorting site at the edge layer side, and then upload it to the cloud asynchronously for long-term persistence using the cloud distributed file system or object storage; The distributed data storage module includes a data partitioning and load balancing sub-module, a redundant backup and fault tolerance mechanism sub-module, and an edge-cloud hierarchical storage architecture: The data partitioning and load balancing sub-module evenly distributes the current logistics transfer yard sorting data items to each storage node of the digital twin system through the consistent hashing algorithm: Suppose the current digital twin system includes storage nodes, and multiple virtual nodes are configured on each node to improve balance; define a hash function , and map the sorting data item to the interval ; where represents the set of sorting data items , i = 1, 2,.., I , where I represents the number of sorting data items; assume that the positions of all virtual nodes are on the hash ring and take values between 0 and 1; the virtual node position of the storage node is , where is the number of virtual nodes. Then, for the sorting data item the main storage node selection rule is as follows: That is, find all the values of j that satisfy and take the value of j among them; Indicates mapping the sorting data item to the interval to obtain the hash value; If all do not meet , then ; For the redundant backup and fault tolerance mechanism sub-module, outside the primary storage node for each data item, the next storage node is sequentially selected in the clockwise direction of the primary node as the replica storage point; to ensure the consistency of the data status between the primary storage node and the replica storage points, the distributed consensus protocol Raft is adopted, which specifically includes: For ensuring the consistency of the data status between the primary storage node and the replica storage points, the distributed consensus protocol Raft is adopted, which specifically includes: In the cluster of the current primary storage node and replica storage points, the Raft protocol will elect a leader, and the remaining nodes will be follower nodes; when the sorting data write request arrives, the leader first writes the sorting data write request into its own log, and then copies the sorting data write request to other follower nodes; when more than half of the total number of nodes in the cluster confirm that they have received the sorting data write request and successfully written it into their own logs, the leader and follower nodes will write and update the sorting data and update the node status; Via copies, when the primary node fails, a new primary storage node is elected from the remaining copy storage points to quickly complete the fault recovery, ensuring that data is not lost and the business continues without interruption; The edge-cloud hierarchical storage architecture: is used to implement edge storage and cloud storage, and ensure data synchronization and consistency; A real-time computing module, which is used to build a parallel computing architecture to process batch and streaming data; design a global state update model so that each computing node can cooperate with local states to perform global data fusion using a high-performance big data platform to obtain the global state; A collaborative simulation module, which is used to build a high-precision digital twin model of the physical transfer yard; use continuous time and discrete event simulation technologies to achieve multi-node collaborative simulation to ensure the consistency between the real scenario and the digital twin model. The cloud generates scheduling decisions and correction amounts based on the global simulation results and feedbacks them to the edge nodes through secure communication. The edge immediately responds to adjust the local data processing and device control parameters to achieve the closed-loop consistency between the real scenario and the digital twin model.

2. The digital twin distributed data storage and computing platform for a logistics transfer yard according to claim 1, wherein: The edge-cloud hierarchical storage architecture includes an edge layer storage unit, a cloud storage unit, and a data synchronization and consistency unit; The edge layer storage unit is concentrated on edge servers or intelligent gateways deployed near the logistics sorting site, and its functions include: Real-time data acquisition and preprocessing, that is, directly collecting sensor sorting data on-site, including sorting speed, equipment status, video, and RFID information, and for the original sorting data using the sliding window algorithm for filtering and normalization to obtain: Among them, is the sliding window size; is the current data processing time, is the offset step within the sliding window, and the value range is , indicating the historical time point traced back from the current moment; represents at time the original data value at that point; Short-term caching and local analysis, that is, caching the sorting data in the local memory or flash memory in the recent period of time, and using the local state update model to achieve real-time update: Among them, represents the state vector of the edge node. The initial value of the vector is zero, and it dynamically reflects the local evaluation of the edge node in the dimensions of sorting state recognition and operation stability; is the edge - side state mapping function, which is used to estimate its change rate based on the current state and input data, and is specifically defined as Among them, is the weight matrix, is the bias vector, is the ReLU activation function; represents the time step for state update, with the unit of seconds; The cloud storage unit is responsible for aggregating and persistently storing the data from each edge node, and storing the global state after the global update of the real-time computing module; The data synchronization and consistency unit uses the consistent hashing algorithm and the Raft fault tolerance mechanism to ensure the data version synchronization between the edge and cloud storage units.

3. A digital twin distributed data storage and computing platform for a logistics transfer yard according to claim 2, characterized in that: The real-time computing module adopts a parallel computing architecture, and uses the MapReduce and stream computing frameworks to process the batch and streaming data uploaded from the edge to support the decentralized computing and global collaborative update of the data: Based on the edge-cloud hierarchical storage architecture, first, each edge node performs local computing separately in the Map stage to obtain the local state; Then, in the Reduce stage, the global state is updated using the local states uploaded by each edge node, and the updated data is uploaded to the cloud storage unit for storage. Among them, represents the global state vector, and the initial value of the vector is zero; represents the global input: is a preset weight, represents the original sorting data of the i-th edge node is the result obtained by filtering and normalizing through the sliding window algorithm; is the fusion function, defined as: Among them, is the weight matrix, is the bias vector, is the ReLU activation function.

4. A digital twin distributed data storage and computing platform for a logistics transfer yard according to claim 3, wherein: The co-simulation module includes: Multi-high-precision physical transfer field digital twin model data collaboration and synchronization sub-module: including simulating the dynamic changes of sorting equipment in sorting tasks through continuous-time simulation, and using differential equations to describe the equipment state : Among them represents the local sorting status information of the sorting equipment, and describes the key events of scheduling decisions and abnormal events in discrete event simulation, and the status is corrected by event triggering; Multi-node collaborative simulation sub-module, which combines the states of all key scheduling nodes into a global state vector : Among them, is a scheduling and control signal to ensure the coordinated operation of each node; Cooperative feedback and closed-loop control sub-module: To ensure real-time synchronization between cloud scheduling decisions and on-site control, a closed-loop feedback mechanism is established, including global error calculation and feedback generation, and immediate response of edge nodes.

5. A digital twin distributed data storage and computing platform for a logistics transfer yard according to claim 4, characterized in that: The cooperative feedback and closed-loop control sub-module includes: Global error calculation and feedback generation unit: Based on the global state updated at each moment , determine the desired state through historical sorting data , and calculate the global error: Based on the error Use the PID algorithm to calculate the feedback correction amount: Among them, is the proportional gain, which directly amplifies the current error; is the integral gain, which is used to eliminate the steady-state error; is the derivative gain, which is used to predict the error change and reduce system oscillation; Immediate response unit of edge nodes: As the part closest to the on-site physical environment, the main task of the edge side is to receive feedback control information from the cloud and timely adjust local parameters or device working parameters, specifically including: Using the edge node and the cloud to maintain a persistent connection via MQTT to receive feedback messages; and verifying the received messages, using the check bits to verify the message integrity, and confirming the timeliness and version consistency of the messages. After the edge node receives the cloud feedback, update the processing parameters: Among them, is the original sorting data after filtering and normalization by the sliding window algorithm, is the updated parameter; meanwhile, a local buffering mechanism is configured on the edge side: before new feedback is received, continue to use the previous parameter, temporarily store the key data at the same time, and wait for subsequent updates to ensure that the system will not be in an unstable state due to communication delays; The updated parameters are directly applied to the control logic of the edge device, and the edge node simultaneously feeds back the updated local state to the cloud for subsequent global fusion and iterative improvement, thus forming a closed-loop control to achieve continuous update of the global and local states of the system and continuously approach the optimal equilibrium state.

Images