Distributed intelligent measurement data synchronization management method and system

Abstract

The invention discloses a distributed intelligent metering data synchronous management method and system, and relates to the related field of metering data management, and the method comprises the steps: caching metering data obtained by connecting an intelligent metering system through an edge node; carrying out batch differential synchronization on the measurement data according to a time window to obtain transmission data; performing synchronous transmission of the transmission data through the transmission channel to generate a message queue; and carrying out equipment data analysis on the message queue based on the central node, and constructing a distributed database for distributed query. The technical problems that existing metering data management is low in synchronization efficiency and accuracy and insufficient in safety and reliability are solved, and the technical effects of improving the synchronization efficiency and accuracy of the metering data and providing safe and reliable data support are achieved.

Description

Technical Field

[0001] This application relates to the field of measurement data management, and in particular, to a distributed intelligent measurement data synchronization management method and system. Background Art

[0002] The accuracy and real-time nature of measurement data are crucial for an enterprise's operation decision-making. With the sharp increase in the number of measurement devices and the explosive growth of data volume, how to efficiently and accurately synchronize and manage this measurement data has become an urgent problem to be solved. Currently, the main method to solve this problem is to adopt a centralized data management method, that is, all measurement data is first sent to the central server for storage and processing. However, when faced with massive data, on the one hand, due to the huge amount of data, the processing capacity and storage space of the central server are easily reached to the limit, resulting in problems such as data synchronization delay and data loss; on the other hand, there are also significant challenges in data security and reliability in the centralized data management method.

[0003] In the related technologies at the present stage, there are technical problems in measurement data management such as low synchronization efficiency and accuracy, and insufficient security and reliability. Summary of the Invention

[0004] This application provides a distributed intelligent measurement data synchronization management method and system. By using edge nodes to cache measurement data and performing batch differential synchronization according to time windows, synchronously transmitting the transmission data to the central node through a transmission channel, and performing device data analysis on the message queue based on the central node and constructing a distributed database and other technical means, the technical effects of improving the synchronization efficiency and accuracy of measurement data and providing secure and reliable data support are achieved.

[0005] This application provides a distributed intelligent measurement data synchronization management method, including: using edge nodes to cache the measurement data obtained by connecting to an intelligent measurement system; performing batch differential synchronization on the measurement data according to time windows to obtain transmission data; synchronously transmitting the transmission data through a transmission channel to generate a message queue; performing device data analysis on the message queue based on a central node and constructing a distributed database for distributed query.

[0006] In a possible implementation, performing batch differential synchronization on the measurement data according to time windows to obtain transmission data, the following processing is performed: respectively setting time windows of distributed devices based on service requirements, transmission delay requirements, system load, and data change frequency; comparing the current data within the time window with the synchronization record, and determining the changed data with a change threshold; triggering synchronization with the changed data to obtain the transmission data.

[0007] In a possible implementation, the change data is used to trigger synchronization to obtain the transmission data, and the following processing is performed: a unique identification code is generated based on the change data; a recent difference value is generated according to the difference value between the unique identification code and the most recent record in the synchronization record to identify the change data; and the recent difference value is used to trigger synchronization to obtain the transmission data.

[0008] In a possible implementation, the synchronous transmission of the transmission data is performed through a transmission channel to generate a message queue, and the following processing is performed: a communication protocol is selected to determine the transmission channel of the transmission data; a time synchronization mechanism is introduced to perform the transmission of the transmission data, and transmission confirmation is performed based on the transmission channel to obtain the message queue.

[0009] In a possible implementation, a time synchronization mechanism is introduced, and the following processing is performed: a synchronization fault tolerance time of the distributed device is set; the clock drift of the distributed device is verified based on the synchronization fault tolerance time, and a time source is started based on the degree of clock drift; and a healthy time source in the time sources is selected to configure the distributed device.

[0010] In a possible implementation, transmission confirmation is performed based on the transmission channel, and the following processing is performed: the distributed device is used as a message producer, and the edge node is used as a message consumer. The partition message obtained by the message producer is transmitted to the message consumer through the transmission channel; the partition message is read by the message consumer and a processing receipt is sent. If it is determined that the processing fails according to the processing receipt, the message retry of the partition message is performed until it is determined that the processing in the processing receipt is successful; and the dead letter queue is extracted based on the message retry for error handling.

[0011] In a possible implementation, device data analysis of the distributed device is performed on the message queue based on the central node to construct a distributed database, and the following processing is performed: device data analysis of the distributed device is performed on the message queue to obtain device data; and the device data is stored in the distributed database.

[0012] The present application further provides a distributed intelligent metering data synchronization management system, including: a metering data caching module, configured to cache the metering data obtained by connecting to the intelligent metering system by using an edge node; a batch differential synchronization module, configured to perform batch differential synchronization on the metering data according to a time window to obtain transmission data; a message queue generation module, configured to perform synchronous transmission of the transmission data through a transmission channel to generate a message queue; and a distributed database construction module, configured to perform device data analysis of the distributed device on the message queue based on a central node to construct a distributed database for distributed query.

[0013] A distributed intelligent metering data synchronization management method and system proposed in this application first caches the metering data obtained by connecting to the intelligent metering system using edge nodes, then performs batch differential synchronization on the metering data according to a time window to obtain transmission data, then synchronously transmits the transmission data through a transmission channel to generate a message queue, and finally analyzes device data based on a central node for the message queue to construct a distributed database for distributed query. It achieves the technical effects of improving the synchronization efficiency and accuracy of metering data and providing safe and reliable data support. Brief Description of the Drawings

[0014] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings of the embodiments of the present invention will be briefly introduced below. Flowcharts are used in this application to illustrate the operations performed by the system according to the embodiments of this application. It should be understood that the operations in the front or below do not necessarily need to be executed precisely in sequence. On the contrary, according to needs, they can be executed in reverse order or simultaneously. At the same time, other operations can also be added to these processes, or one or several steps can be removed from these processes.

[0015] Figure 1 It is a schematic flowchart of a distributed intelligent metering data synchronization management method provided by an embodiment of this application.

[0016] Figure 2 It is a schematic structural diagram of a distributed intelligent metering data synchronization management system provided by an embodiment of this application.

[0017] Description of the reference numerals: Metering data caching module 10, batch differential synchronization module 20, message queue generation module 30, distributed database construction module 40. Detailed Description of the Embodiments

[0018] The above description is only an overview of the technical solutions of this application. In order to be able to understand the technical means of this application more clearly, it can be implemented according to the content of the specification. And in order to make the above and other purposes, features and advantages of this application more obvious and understandable, the specific embodiments of this application are specifically given below.

[0019] In order to make the purpose, technical solutions and advantages of this application clearer, the present application will be further described in detail below in conjunction with the drawings. The described embodiments should not be regarded as limitations of this application. All other embodiments obtained by those of ordinary skill in the art without creative efforts fall within the scope of protection of this application.

[0020] In the following description, reference is made to "some embodiments", which describe a subset of all possible embodiments. However, it is understood that "some embodiments" may be the same subset or different subsets of all possible embodiments, and may be combined with each other without conflict. The terms "comprising" and "having", and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product or server that comprises a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or modules not clearly listed or inherent to these processes, methods, products or devices. Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the technical field to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application.

[0021] An embodiment of the present application provides a distributed intelligent metering data synchronization management method, as Figure 1 shown, the method includes:

[0022] Step S100, using edge nodes to cache the metering data obtained by connecting to the intelligent metering system.

[0023] Specifically, edge nodes are deployed at various key positions in the intelligent metering system. These edge nodes are microcomputers or embedded devices located near the data generation sources (such as sensors and meters in thermal power plants) and having data processing and storage capabilities. The intelligent metering system is a system integrating sensors, communication technologies and data analysis capabilities, and is used to monitor and record energy consumption data in real time. The edge nodes are connected to the intelligent metering system through preset interfaces or protocols, and collect metering data from different metering devices in real time, including power generation, coal consumption, water consumption, steam flow, etc. The collected metering data is temporarily stored in the local storage medium (such as memory or hard disk) of the edge nodes. The caching strategy may include timestamp marking, data priority sorting, etc., to ensure that important data is not overwritten. For example, cache strategies such as First In First Out (FIFO) or Least Recently Used (LRU) can be used to manage the cached data.

[0024] Step S200, performing batch differential synchronization on the metering data according to a time window to obtain transmission data.

[0025] Specifically, according to the system requirements and the frequency of data synchronization, time is divided into fixed time windows (such as every minute, every hour, etc.). At the end of each time window, the edge node performs differential processing on the cached measurement data, that is, compares the data within the current time window with the data within the previous time window through methods such as data hashing and timestamp comparison to identify newly added or updated data. The differentially processed data is packaged into transmission data, including information such as the data itself, timestamp, device ID, etc., and is prepared for synchronous transmission through the transmission channel. The packaging process can include operations such as data compression, encryption, and formatting.

[0026] In a possible implementation, batch differential synchronization of the measurement data is performed according to time windows to obtain transmission data. Step S200 further includes step S210 of setting the time windows of the distributed devices respectively based on service requirements, transmission delay requirements, system load, and data change frequency. Specifically, through historical data analysis, the change frequency and transmission delay of different measurement data are determined, and the time window settings are adjusted according to the operation requirements and system load conditions. Specifically, according to operation requirements, such as the need to monitor key parameters such as temperature, pressure, and flow in real time, and the need to generate operation reports regularly, the time windows of different measurement data are set. For example, for real-time monitoring data, it needs to be synchronized once per second or per minute; for the cumulative data required for generating reports, it can be set to be synchronized once per hour or per day. Considering the network delay and communication bandwidth during data transmission, a reasonable time window is set to avoid transmission delays caused by data accumulation. For example, if the network condition is poor, the time window can be appropriately extended to reduce the number of data transmissions, thereby reducing the demand for network resources. According to the processing capabilities of the edge node and the central node, the time windows are reasonably allocated to avoid system overload. If the system load is high, the system pressure can be reduced by increasing the time window or reducing the synchronization frequency. Analyze the change patterns of the measurement data, set a shorter time window for data with frequent changes to ensure data real-time performance; for data with slower changes, a longer time window can be set to reduce the data transmission volume.

[0027] Step S220, compare the current data within the time window with the synchronization record, and determine the changed data with a change threshold. Specifically, at the end of each time window, the current measurement data is compared with the data synchronized last time. A change threshold is set to determine whether the data has changed significantly. If the data change exceeds the threshold, the data is considered to be changed data.

[0028] Step S230: Trigger synchronization with the changed data to obtain the transmitted data. Specifically, when changed data is detected, a synchronization mechanism is triggered to package the changed data into transmitted data. Through a preset transmission channel, the transmitted data is sent to the central node for further processing. This implementation method can significantly reduce unnecessary data transmission by setting a time window and a change threshold, thereby improving data transmission efficiency. By reasonably allocating the time window and reducing the synchronization frequency, the processing pressure on the edge node and the central node can be reduced, avoiding system overload. For important data changes, synchronization can be triggered in real time to ensure the timeliness and accuracy of the data.

[0029] In a possible implementation method, to trigger synchronization with the changed data to obtain the transmitted data, step S230 further includes step S231: Generate a unique identification code based on the changed data. Specifically, apply a hash function to the changed data to generate a hash value with a fixed length as the unique identification code (UID). This hash value can uniquely represent the changed data. Even if the data content changes slightly, the hash value will be different. Store the generated UID in association with the changed data for subsequent use.

[0030] Step S232: Identify the difference value of the changed data based on the unique identification code and the most recent record in the synchronization record to generate the most recent difference value. Specifically, search for the most recent record in the synchronization record that matches the UID of the current changed data. If a matching record is found, calculate the difference value between the current changed data and the most recent record. This difference value can be a binary difference, a text difference, or a difference representation under a specific business logic. Store the calculated difference value in association with the UID as part of the transmitted data.

[0031] Step S233: Trigger synchronization with the most recent difference value to obtain the transmitted data. Specifically, package the calculated difference value, UID, and necessary metadata (such as timestamp, data source identifier, etc.) into a transmission data packet. Through a preset transmission channel, send the packaged transmission data packet to the central node for further processing. This implementation method significantly reduces the amount of data transmitted and the network bandwidth consumption and transmission delay by calculating the difference value and only transmitting these difference values. Since only the difference values are transmitted, the central node can update the database faster after receiving the data, improving the synchronization efficiency. Reducing the amount of data transmitted and the synchronization time can reduce the processing pressure on the edge node and the central node, optimizing resource utilization.

[0032] Step S300: Perform synchronous transmission of the transmitted data through a transmission channel to generate a message queue.

[0033] Specifically, a reliable transmission channel is established between the edge node and the central node, such as using a wired network, a wireless network, or a dedicated data communication link, etc. After each time window ends, the edge node sends the packaged transmission data to the central node through the transmission channel. After receiving the transmission data, the central node puts it into a message queue and waits for further processing. A message queue is a first-in-first-out data structure used to manage the data packets to be processed, which can be implemented using in-memory data structures (such as linked lists, queues, etc.) or built based on a message queue system (such as RabbitMQ, Kafka, etc.).

[0034] In a possible implementation, the synchronous transmission of the transmission data is performed through the transmission channel to generate a message queue. Step S300 further includes step S310 of selecting a communication protocol to determine the transmission channel of the transmission data. Specifically, the characteristics of the transmission data are evaluated, including data type, size, real-time requirement, etc. According to the evaluation results, the most suitable protocol is selected from the available communication protocols. For example, for data that requires high real-time performance, the TCP / IP protocol can be used because it has a reliable connection mechanism and error checking function; for batch data with low real-time requirements, a more efficient UDP protocol or HTTP protocol can be selected for transmission. The corresponding communication channels are configured between the edge node and the central node to ensure smooth data transmission.

[0035] Step S320, introducing a time synchronization mechanism to perform the transmission of the transmission data, and performing transmission confirmation based on the transmission channel to obtain the message queue. Specifically, the time synchronization mechanism is the key to ensuring the time consistency of each node in a distributed system. During the data transmission process, a time synchronization mechanism is introduced to ensure the accuracy of the data sending and receiving timestamps, thereby avoiding data chaos or loss. Before data transmission, through NTP (Network Time Protocol) or other time synchronization mechanisms, ensure that the time of the edge node and the central node is consistent. According to the selected communication protocol, the measurement data is batch-differentially synchronized by time window and sent to the central node through the transmission channel. After receiving the data, the central node sends a feedback confirmation message to the edge node to confirm the integrity and successful reception of the data. At the central node, the received data is organized into a message queue and waits for subsequent device data analysis. This implementation method ensures the efficient and accurate synchronization of a large amount of measurement data between multiple systems or nodes through the time synchronization mechanism and transmission confirmation.

[0036] In a possible implementation, a time synchronization mechanism is introduced, and step S320 further includes step S321 of setting the synchronization fault tolerance time for the distributed devices. Specifically, in a computer system, each distributed device (such as a sensor, a data collector, etc.) has its own internal clock. Due to factors such as hardware differences and network latency, these clocks will gradually drift, that is, there will be slight differences in time. To handle this drift, it is first necessary to set a reasonable synchronization fault tolerance time for each device. This fault tolerance time defines the maximum deviation range within which the device clock is considered to be "synchronized". For example, each distributed device in a thermal power plant generates measurement data once per second. Considering the time requirements for data transmission and processing, setting the synchronization fault tolerance time to ±5 milliseconds can ensure that the timestamps of the data remain consistent within an acceptable range.

[0037] Step S322, verifying the clock drift of the distributed devices with the synchronization fault tolerance time, and starting the time source based on the degree of clock drift. Specifically, the system regularly (such as every minute) checks the difference between the clock of each device and the system standard time (such as the time provided by an NTP server). If the clock drift of a certain device exceeds the synchronization fault tolerance time, it is determined that the device clock needs to be calibrated. Among them, clock drift refers to the change in the deviation of the device clock relative to the system standard time over time. The time source refers to a device or network service that provides time synchronization services, such as an NTP (Network Time Protocol) server.

[0038] Step S323, selecting a healthy time source from the time sources to configure the distributed devices. Specifically, the system maintains one or more time sources and selects the best time source according to its health status (such as response time, stability, etc.). Once a healthy time source is determined, the system will use this time source to calibrate the drifted device clock. For example, if the system has two NTP servers as time sources, the system will select the best time source for clock calibration according to their response times in the past period (for example, a server with a response time less than 10 milliseconds is considered healthy). This implementation can regularly calibrate the device clock through the time synchronization mechanism, greatly reducing data errors caused by clock desynchronization, ensuring that the data of all devices are recorded and analyzed under a unified time frame, and ensuring the timing consistency and reliability of the transmitted data.

[0039] In a possible implementation, transmission confirmation is performed based on the transmission channel. Step S320 further includes step S324. Taking the distributed device as the message producer and the edge node as the message consumer, the partitioned messages obtained by the message producer are transmitted to the message consumer through the transmission channel. Specifically, distributed devices (such as sensors, data collectors, etc.) act as message producers and are responsible for generating and sending metering data. These data are divided into different partitioned messages, and each partition represents a different data type or time window. The edge node acts as the message consumer and is responsible for receiving and processing these partitioned messages. For example, a thermal power plant has 100 distributed devices, and each device generates 1 piece of metering data per second. To optimize data transmission, these data can be divided into multiple partitions according to device ID or data type, and each partition contains messages from specific devices or data types. The message producer (distributed device) uses a selected communication protocol (such as MQTT, HTTP, etc.) to send the partitioned messages to the transmission channel. The edge node, as the message consumer, subscribes to these partitioned messages and is ready to receive them.

[0040] Step S325, the message consumer reads the partitioned message and sends a processing receipt. If it is determined that the processing fails according to the processing receipt, message retry of the partitioned message is performed until it is determined that the processing in the processing receipt is successful. Specifically, after receiving the partitioned messages, the edge node will attempt to process these messages (such as storing them in the local cache, etc.). After the processing is completed, the edge node will send a processing receipt to the message producer, indicating that the message has been successfully processed. If the processing fails (such as due to network problems, device failures, etc.), the edge node will perform message retry, that is, re-attempt to process the partitioned message. When processing the partitioned messages, the edge node will maintain a processing status table to record the processing results of each message. If the processing fails, the edge node will re-attempt to process the message according to the configured number of retries and update the processing status table after each retry. If the number of retries reaches the upper limit and still fails, the message will be transferred to the dead letter queue for error handling. For example, if the success rate of the edge node in processing partitioned messages is 99%, then for the remaining 1% of the failed messages, the edge node will perform message retry. The number of retries can be configured according to the actual situation, such as 3 times, 5 times, etc.

[0041] Step S326: Based on the message, retry extracting the dead letter queue for error handling. Specifically, for partition messages that still cannot be processed after multiple retries, the edge node will transfer these messages to the dead letter queue. The dead letter queue is a queue specifically used to store messages that cannot be processed or have failed to be processed. When the edge node transfers a message to the dead letter queue, it will record relevant error information (such as error codes, error descriptions, etc.). System administrators can view this information through monitoring tools or log systems, analyze the reasons for failures, and take corresponding corrective measures (such as fixing device failures, optimizing processing logic, etc.). This implementation method ensures the integrity and accuracy of metering data during transmission through the mechanisms of message producers, message consumers, and partition messages. Through the mechanisms of processing acknowledgments and message retries, errors during transmission can be detected and handled in a timely manner, improving the reliability and stability of the system. Through the mechanism of the dead letter queue, messages that cannot be processed can be centrally managed, facilitating problem troubleshooting and error handling by system administrators, thereby improving the maintainability and scalability of the system.

[0042] Step S400: Based on the central node, perform device data analysis on the message queue and construct a distributed database for distributed query.

[0043] Specifically, the central node (the core computing device responsible for data processing, storage, and query) extracts data packets from the message queue and parses and analyzes them, including operations such as data cleaning (removing outliers, filling missing values, etc.), format conversion (such as unit conversion, format conversion, etc.), and statistical analysis (such as calculating the average value, total, etc.). The analyzed data is integrated according to keyword fields such as device ID and timestamp to form a complete device data set. The integrated device data set is stored in the distributed database. A distributed database is a database system that can store and query data across multiple physical nodes and has high availability and scalability. The distributed database system can be built based on NoSQL databases (such as MongoDB, Cassandra, etc.) or relational databases (such as MySQL Cluster, PostgreSQLXL, etc.). Indexes and query mechanisms are established in the distributed database to support efficient distributed query operations, allowing users to access and query data concurrently from different nodes, improving query efficiency and response speed. Through technical means such as using edge nodes to cache metering data, performing batch differential synchronization according to time windows, synchronously transmitting transmission data to the central node through the transmission channel, and performing device data analysis on the message queue based on the central node to construct a distributed database, the embodiments of this application achieve the technical effects of improving the synchronization efficiency and accuracy of metering data and providing secure and reliable data support.

[0044] In a possible implementation, the message queue is analyzed for device data based on a central node to construct a distributed database, and step S400 further includes step S410, in which the message queue is used to analyze the device data of distributed devices to obtain device data. Specifically, a specific parsing library or tool is used to parse each message in the message queue using a predefined message format or protocol to extract the device data therein, such as power generation, device temperature, pressure, flow, and other parameters. The parsed device data is cleaned to remove invalid or abnormal data. For example, values that are clearly beyond a reasonable range are considered abnormal data and are removed. The cleaned device data is integrated according to dimensions such as device or time for subsequent analysis.

[0045] Step S420, storing the device data in the distributed database. Specifically, the central node uses a specific database connection library or driver to establish a connection with the distributed database, and uses SQL statements or specific database operation APIs to insert the cleaned and integrated device data into the distributed database according to a predetermined data model or table structure. In order to improve query efficiency, an index is created for the device data in the distributed database. An index is a data structure used to speed up data retrieval. In this implementation, the distributed database can provide high availability and data consistency guarantees, and can still ensure data integrity and availability even when some nodes fail. In addition, the distributed database system is easy to expand and maintain, and nodes can be added or reduced according to actual needs to adjust the performance and capacity of the system.

[0046] In the above, refer to Figure 1 A distributed intelligent metering data synchronization management method according to an embodiment of the present invention is described in detail. Figure 2 A distributed intelligent metering data synchronization management system according to an embodiment of the present invention is described.

[0047] A distributed intelligent metering data synchronization management system according to an embodiment of the present invention is used to solve the technical problems of low synchronization efficiency and accuracy and insufficient security and reliability in existing metering data management, so as to achieve the technical effect of improving the synchronization efficiency and accuracy of metering data and providing safe and reliable data support. A distributed intelligent metering data synchronization management system includes: a metering data cache module 10, a batch differentiation synchronization module 20, a message queue generation module 30, and a distributed database construction module 40.

[0048] The measurement data caching module 10 is used to cache the measurement data obtained by connecting to the intelligent metering system using edge nodes; the batch differential synchronization module 20 is used to perform batch differential synchronization on the measurement data according to a time window to obtain transmission data; the message queue generation module 30 is used to perform synchronous transmission of the transmission data through a transmission channel to generate a message queue; the distributed database construction module 40 is used to perform device data analysis on the message queue based on a central node to construct a distributed database for distributed query.

[0049] Next, the specific configuration of the batch differential synchronization module 20 will be described in detail. As described above, batch differential synchronization is performed on the measurement data according to a time window to obtain transmission data. The batch differential synchronization module 20 may further include: a time window setting unit for setting the time window of distributed devices based on service requirements, transmission delay requirements, system load, and data change frequency; a changed data determination unit for comparing the current data within the time window with the synchronization record to determine the changed data with a change threshold; a synchronization trigger unit for triggering synchronization with the changed data to obtain the transmission data.

[0050] Among them, triggering synchronization with the changed data to obtain the transmission data, the synchronization trigger unit may further include: a unique identification code generation subunit for generating a unique identification code based on the changed data; a difference value identification subunit for identifying the difference value of the changed data according to the unique identification code and the most recent record in the synchronization record to generate the most recent difference value; a transmission data acquisition subunit for triggering synchronization with the most recent difference value to obtain the transmission data.

[0051] Next, the specific configuration of the message queue generation module 30 will be described in detail. As described above, synchronous transmission of the transmission data is performed through a transmission channel to generate a message queue. The message queue generation module 30 may further include: a transmission channel determination unit for selecting a communication protocol to determine the transmission channel of the transmission data; a data transmission unit for introducing a time synchronization mechanism to perform the transmission of the transmission data, and performing transmission confirmation based on the transmission channel to obtain the message queue.

[0052] Among them, introducing a time synchronization mechanism, the data transmission unit may further include: a synchronization fault tolerance time setting subunit for setting the synchronization fault tolerance time of distributed devices; a time source start subunit for checking the clock drift of the distributed devices with the synchronization fault tolerance time and starting the time source based on the degree of clock drift; a distributed device configuration subunit for selecting a healthy time source from the time sources to configure the distributed devices.

[0053] Among them, for transmission confirmation based on the transmission channel, the data transmission unit may further include: a transmission subunit for using the distributed device as a message producer and the edge node as a message consumer, and transmitting the partitioned message obtained by the message producer to the message consumer through the transmission channel; a processing receipt sending subunit for reading the partitioned message by the message consumer and sending a processing receipt, and if it is determined that the processing fails according to the processing receipt, performing message retry on the partitioned message until it is determined that the processing in the processing receipt is successful; an error processing subunit for extracting a dead letter queue based on the message retry for error processing.

[0054] Next, the specific configuration of the distributed database construction module 40 will be described in detail. As described above, based on the central node for device data analysis of the message queue, a distributed database is constructed. The distributed database construction module 40 may further include: a device data acquisition unit for performing device data analysis on distributed devices with the message queue to acquire device data; a data storage unit for storing the device data into the distributed database.

[0055] A distributed intelligent metering data synchronization management system provided by an embodiment of the present invention can execute a distributed intelligent metering data synchronization management method provided by any embodiment of the present invention, and has functional modules and beneficial effects corresponding to the execution of the method.

[0056] Although the present application makes various references to certain modules in the system according to the embodiments of the present application, however, any number of different modules can be used and run on the user terminal and / or the server. The included respective units and modules are only divided according to functional logic, but are not limited to the above division, as long as the corresponding functions can be achieved; in addition, the specific names of the respective functional units are only for the convenience of mutual distinction and do not limit the protection scope of the present invention.

[0057] The above specific embodiments do not constitute a limitation to the protection scope of the present application. Those skilled in the art should understand that various modifications, combinations and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principle of the present application shall be included within the protection scope of the present application. In some cases, the actions or steps recorded in the present application can be executed in a different order from that in the embodiments and still achieve the desired results. Additionally, the processes depicted in the drawings do not necessarily require the specific order or sequential order shown to achieve the desired result. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

Claims

1. A distributed intelligent metering data synchronization management method, characterized in that, Including: Using edge nodes to cache the measurement data obtained by connecting to the intelligent metering system; Performing batch differential synchronization on the measurement data according to time windows to obtain transmission data; Performing synchronous transmission of the transmission data through a transmission channel to generate a message queue; Based on a central node, performing device data analysis on the message queue and constructing a distributed database for distributed query.

2. The distributed intelligent metering data synchronization management method according to claim 1, wherein Performing batch differential synchronization on the measurement data according to time windows to obtain transmission data, including: Based on service requirements, transmission delay requirements, system load, and data change frequency, respectively setting time windows for distributed devices; Comparing the current data within the time window with the synchronization record and determining changed data based on a change threshold; Triggering synchronization with the changed data to obtain the transmission data.

3. The distributed intelligent metering data synchronization management method according to claim 2, wherein Triggering synchronization with the changed data to obtain the transmission data, including: Generating a unique identification code based on the changed data; Identifying the difference value of the changed data according to the unique identification code and the nearest record in the synchronization record to generate a nearest difference value; Triggering synchronization with the nearest difference value to obtain the transmission data.

4. A distributed intelligent metering data synchronization management method according to claim 1, characterized in that, Performing synchronous transmission of the transmission data through a transmission channel to generate a message queue, including: Selecting a communication protocol to determine the transmission channel of the transmission data; Introducing a time synchronization mechanism to perform the transmission of the transmission data, performing transmission confirmation based on the transmission channel, and obtaining the message queue.

5. A distributed intelligent metering data synchronization management method according to claim 4, characterized in that, Introducing a time synchronization mechanism, including: Setting the synchronization fault tolerance time of distributed devices; Checking the clock drift of the distributed devices with the synchronization fault tolerance time and starting a time source based on the degree of clock drift; Selecting a healthy time source from the time sources to configure the distributed devices.

6. The distributed intelligent metering data synchronization management method according to claim 4, characterized in that, Performing transmission confirmation based on the transmission channel, including: Using a distributed device as a message producer, using the edge node as a message consumer, and transmitting the partition message obtained by the message producer to the message consumer through the transmission channel; Reading the partition message by the message consumer and sending a processing receipt. If it is determined that the processing fails according to the processing receipt, performing message retry on the partition message until it is determined that the processing in the processing receipt is successful; Extracting a dead letter queue based on the message retry for error handling.

7. A distributed intelligent metering data synchronization management method according to claim 1, characterized in that Based on a central node, performing device data analysis on the message queue and constructing a distributed database, including: Performing device data analysis of distributed devices with the message queue to obtain device data; Storing the device data in the distributed database.

8. A distributed intelligent metering data synchronization management system, characterized in that, The system is used to implement a distributed intelligent metering data synchronization management method according to any one of claims 1-7. The system includes: A metering data caching module for using edge nodes to cache the measurement data obtained by connecting to the intelligent metering system; A batch differential synchronization module for performing batch differential synchronization on the measurement data according to time windows to obtain transmission data; A message queue generation module for performing synchronous transmission of the transmission data through a transmission channel to generate a message queue; A distributed database construction module is used to perform device data analysis on the message queue based on a central node, construct a distributed database for distributed query.

Images