Power monitoring system historical data dual-computer dual-network synchronization method and system
Through the formatting processing of historical data and the dual-network synchronization mechanism, the problems of insufficient database log heterogeneity and dual-network support in traditional methods are solved, and the efficient and reliable historical data management and fault tolerance of the power monitoring system are realized. .
Patent Information
- Application Number
- CN202410922287.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-10
- Publication Date
- 2024-11-15
AI Technical Summary
The traditional dual-machine synchronization method of historical data has problems such as different database log file formats and synchronization mechanisms, no dual network configuration, and high complexity, which is difficult to meet the needs of power monitoring systems.
By obtaining historical data for formatting, saving it as an XML file, and redundantly stored in two directories on the local disk, the inbound service program is used to monitor directory changes, realize synchronization of dual network segment databases, and use thread switching and retry The mechanism ensures the reliability of the data.
Significantly improves the efficiency and reliability of the historical data management of the power monitoring system, realizes compatibility of multiple database types, ensures that data can be switched to another machine for synchronization when one machine fails, reduces the risk of data loss and simplifies Operation and maintenance process.
Smart Images

Figure CN118964477A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of power system automation, and in particular to a method and system for synchronizing historical data of a power monitoring system with two machines and two networks. Background Art
[0002] In recent years, with the rapid progress of computer technology, the application of autonomous and controllable intelligent power monitoring systems in the power industry and related fields has become increasingly widespread. These systems have not only greatly improved the level of intelligence in power management, but also put forward higher requirements for the stability of system operation and data security. As a core component of the power monitoring system, the historical database carries key information such as historical operation data and fault alarms of the power system, providing important support for advanced functions such as system analysis, trend prediction and intelligent alarm.
[0003] In order to ensure that the power monitoring system can maintain the integrity of historical data and the continuity of application when facing a single point of failure, the dual-machine synchronization technology of historical data is particularly important. The traditional dual-machine synchronization method of historical data mainly relies on the operation log file of the database, that is, recording all the addition, deletion, and modification operations that occur in the database, and storing these operations on another machine to achieve data synchronization. However, this method has obvious limitations: first, the log file format and synchronization mechanism of different database systems are different, resulting in the need to develop specific synchronization strategies for each database, which greatly increases the complexity and maintenance cost of the system; second, the synchronization service provided by some database systems may not support dual-network configuration, which is a necessary configuration requirement in the power monitoring system, and therefore cannot fully meet the needs of actual applications; finally, some database systems may not provide ready-made dual-machine synchronization components, requiring additional development, further increasing the difficulty and complexity of technical implementation. Summary of the invention
[0004] The purpose of the present invention is to provide a method and system for synchronizing historical data of a power monitoring system using two machines and two networks in order to solve at least one of the above technical problems.
[0005] The present invention provides a method for synchronizing historical data of a power monitoring system with two machines and two networks, comprising the following steps:
[0006] Acquire historical data, and format the historical data;
[0007] The formatted historical data is saved in the first directory and the second directory of the local disk at the same time;
[0008] The warehousing service program cyclically traverses and monitors the first directory and the second directory;
[0009] When new data is written to the first directory and the second directory, the warehousing service program reads the new data, writes the data read from the first directory into the first database, and writes the data read from the second directory into the second database; the warehousing service program can connect to the corresponding databases through two network segments.
[0010] Furthermore, the historical data is formatted, including:
[0011] The historical data is saved as an XML file in a preset format; the historical data includes historical storage data and event storage data;
[0012] The saved content includes the data source corresponding to the current XML file, data generation time, table name, column name and type attributes corresponding to historical data, and data records.
[0013] Furthermore, the formatted historical data is saved simultaneously in the first directory and the second directory of the local disk, including:
[0014] Name the XML file in a fixed format, which is "database name_table name_timestamp_nanosecond.xml";
[0015] Query whether the first directory and the second directory exist on the local disk; if not, create the first directory and the second directory;
[0016] Check whether corresponding historical data folders exist in the first directory and the second directory; if not, create the historical data folders in the first directory and the second directory;
[0017] Check whether there are Error folders and Input folders under each historical data folder; if not, create the Error folder and the Input folder; wherein the Error folder is used to store XML files that failed to be stored, and the Input folder is used to store XML files named in a fixed format;
[0018] Store the XML files named in a fixed format in the corresponding folder.
[0019] Furthermore, the number and naming of the historical data folders in the first directory and the second directory are determined by a database in the current data source for storing historical data.
[0020] Furthermore, the thread corresponding to each directory periodically reads the corresponding Error folder and retry to store the data that failed to be stored.
[0021] Furthermore, the warehousing service program cyclically traverses and monitors the first directory and the second directory, including:
[0022] The warehousing service program starts a first thread and a second thread; the first thread and the second thread are used to cyclically traverse and monitor the first directory and the second directory respectively.
[0023] Furthermore, when the first thread and the second thread detect that there is a newly added xml file in the corresponding folder, the newly added xml file is parsed, and the historical data of the newly added xml file is cached in the memory, and the table name, column name and type, and historical record of the newly added xml file are obtained;
[0024] Query the database to see whether a table corresponding to the newly added xml file exists; if so, store the historical data corresponding to the newly added xml file into the corresponding database table; if not, generate a corresponding table creation statement based on the table name, column name and type of the newly added xml file, according to the type of database and data type mapping table, execute the table creation operation, and then store the historical data corresponding to the newly added xml file into the corresponding database table.
[0025] Furthermore, the type mapping table includes database types and built-in data type mapping relationships of each database.
[0026] Furthermore, the warehousing service program queries whether the database handle connected to the current thread is normal; if normal, use the current network to connect to the corresponding database; if abnormal, switch to another network to connect to the corresponding database;
[0027] If both network segments cannot be connected, wait for the preset time and try to connect again.
[0028] The present invention also provides a dual-machine dual-network synchronization system for historical data of an electric power monitoring system, comprising:
[0029] An acquisition module, used for acquiring historical data and formatting the historical data;
[0030] A processing module, used for saving the formatted historical data to a first directory and a second directory of a local disk at the same time;
[0031] A monitoring module, wherein the warehousing service program cyclically traverses and monitors the first directory and the second directory;
[0032] The warehousing module, when new data is written to the first directory and the second directory, the warehousing service program reads the new data, writes the data read from the first directory into the first database, and writes the data read from the second directory into the second database; the warehousing service program can be connected to the corresponding database through two network segments.
[0033] The beneficial effects of the present invention are:
[0034] The present invention significantly improves the efficiency and reliability of historical data management in the power monitoring system; by being compatible with multiple database types, the present invention realizes flexible data storage; at the same time, the dual-network synchronization technology is adopted to ensure that even if a network segment fails, it can quickly switch to another network segment for synchronization, greatly improving the fault tolerance of the system. In addition, for data that fails to be stored, the present invention intelligently saves it and automatically retries after a delay, effectively reducing the risk of data loss.
[0035] The present invention not only ensures the consistency and integrity of the dual-database data, but also significantly improves the operational stability of the power monitoring system, greatly simplifies the workflow of the on-site monitoring system installation and operation and maintenance personnel, and ensures that when any data host fails, historical data can still be stored normally and related applications can continue to operate stably. BRIEF DESCRIPTION OF THE DRAWINGS
[0036] Figure 1 A flow chart of a method for synchronizing historical data of a power monitoring system using two machines and two networks according to one embodiment of the present invention;
[0037] Figure 2 This is a general architecture diagram of a method for synchronizing historical data of a power monitoring system using two machines and two networks according to one embodiment of the present invention;
[0038] Figure 3 A schematic diagram of local caching of historical data in one embodiment of the present invention;
[0039] Figure 4 A schematic diagram of a cache XML file format according to one embodiment of the present invention;
[0040] Figure 5 This is a schematic diagram of the structure of a dual-machine dual-network synchronization system for historical data of an electric power monitoring system according to one embodiment of the present invention. DETAILED DESCRIPTION
[0041] The content of the present invention will now be discussed with reference to exemplary embodiments. It should be understood that the embodiments discussed are only to enable those skilled in the art to better understand and thus implement the content of the present invention, rather than implying any limitation on the scope of the present invention.
[0042] As used herein, the term “including” and variations thereof are to be interpreted as open-ended terms meaning “including, but not limited to.” The term “based on” is to be interpreted as “based, at least in part, on.” The terms “one embodiment” and “an embodiment” are to be interpreted as “at least one embodiment.”
[0043] Embodiment 1
[0044] Figure 1 This is a flow chart of a method for synchronizing historical data of a power monitoring system using two machines and two networks in accordance with one embodiment of the present invention. Figure 1 As shown, according to an embodiment of the present invention, a method for synchronizing historical data of a power monitoring system with two machines and two networks includes the following steps:
[0045] Step S102: Acquire historical data and format the historical data;
[0046] Historical data is collected from various parts of the power monitoring system. The acquired historical data may have different formats and structures, so it needs to be formatted for subsequent storage and query.
[0047] Step S104: saving the formatted historical data to the first directory and the second directory of the local disk at the same time;
[0048] By saving the data to two different directories (ie, the first directory and the second directory) at the same time, redundant backup of the data can be ensured. If one of the directories (or disks) fails, the data in the other directory is still available.
[0049] Step S106: the warehousing service program cyclically traverses and monitors the first directory and the second directory;
[0050] The import service program is a background service that is specifically responsible for importing data from the local directory into the database. It runs continuously and monitors file changes in the directory. When the import service program detects new files or file updates, it triggers subsequent data import operations.
[0051] Step S108: When new data is written into the first directory and the second directory, the warehousing service program reads the new data, writes the data read from the first directory into the first database, and writes the data read from the second directory into the second database; the warehousing service program can connect to the corresponding databases through two network segments.
[0052] When a new file is detected, the import service program will read the data in the file and generate the corresponding database import command (such as SQL statement); then, the import service program will import the data into the two databases through the network connection. In this way, even if one of the databases fails or data is lost, the data in the other database can still be used as a backup or recovery source. The data in the first directory is stored in the first database, and the data in the second directory is stored in the second database.
[0053] The warehousing service program has two network connection modes, network segment A and network segment B. The warehousing service program can be connected to the first database or the second database through network segment A, or through network segment B. When one of the networks fails, the other network can still ensure the normal import of data.
[0054] The present invention significantly improves the fault tolerance and reliability of data storage and transmission by storing data in two independent directories and two independent databases at the same time and connecting them with dual network segments, ensuring that historical data can still be safely and reliably stored and queried even in the event of hardware or network failures, thus providing a strong guarantee for the stable operation of the power monitoring system.
[0055] According to one embodiment of the present invention, step S102 includes:
[0056] Save the historical data as an XML file in a preset format; the historical data includes historical storage data and event storage data;
[0057] The saved content includes the data source corresponding to the current XML file, data generation time, table name, column name and type attributes corresponding to historical data, and data records.
[0058] In step S102, the historical data of the power monitoring system is converted into a preset XML file format for storage. These historical data include: historical storage data and event storage data. This formatting process ensures the structuring and readability of the data. The saved content not only contains specific data records, but also attaches important metadata information, such as: the data source corresponding to the current XML file, the data generation time, and the table name, column name and data type and other attributes corresponding to these historical data in the database. This information is of great significance for subsequent data query, import and maintenance, and can greatly improve the efficiency and accuracy of data management.
[0059] According to one embodiment of the present invention, step S104 includes:
[0060] Name the XML file in a fixed format, which is "database name_table name_timestamp_nanosecond.xml";
[0061] Check whether the first directory and the second directory exist on the local disk; if not, create the first directory and the second directory;
[0062] Check whether corresponding historical data folders exist in the first directory and the second directory; if not, create corresponding historical data folders in the first directory and the second directory;
[0063] Check whether there are Error folders and Input folders under each historical data folder; if they do not exist, create Error folders and Input folders; the Error folder is used to store XML files that failed to be uploaded, and the Input folder is used to store XML files named in a fixed format;
[0064] Store the XML files named in a fixed format in the corresponding folder.
[0065] The XML file of historical data is named in a fixed format of "database name_table name_timestamp_nanosecond.xml", which clearly identifies the source of the file content and the time of generation, which is helpful for subsequent data management and query. Before saving the file, check whether the preset first and second directories exist on the local disk, check whether there are corresponding historical data folders under the first and second directories, and check whether there are Error folders and Input folders under each historical data folder; if the corresponding folder does not exist at the same level, create the corresponding folder;
[0066] The first directory and the second directory are used to store independent locations of historical data to achieve redundant backup of data; each historical data folder (such as hds, alarm, etc.) classifies and stores data from different tables for easy management and query; preferably, the number and naming of historical data folders under the first directory and the second directory are determined by the database used to store historical data in the current data source; the Error folder is used to store XML files that fail to be stored due to various reasons (such as data format errors, database connection problems, etc.) for subsequent error troubleshooting and processing; the Input folder is used to store XML files named in a fixed format that are ready and waiting to be stored;
[0067] Store the formatted and named xml files in the corresponding Input folders. The warehousing service program can read files from these Input folders during subsequent traversal and perform data import operations. When the import fails, the data is stored in the Error folder corresponding to the Input folder. Preferably, the thread corresponding to each directory periodically reads the corresponding Error folder, and retries the data that failed to be imported. The time for the periodic reading is set according to actual needs.
[0068] The present invention ensures that the XML file of historical data can be stored in a local disk clearly and orderly, laying a solid foundation for subsequent data import and query.
[0069] According to an implementation of the present invention, step S106 includes: the warehousing service program starts a first thread and a second thread; the first thread and the second thread are used to loop through and monitor the first directory and the second directory respectively.
[0070] Step S108 includes: when the first thread and the second thread detect that there is a new XML file in the corresponding folder, parsing the new XML file, caching the historical data of the new XML file into the memory, and obtaining the table name, column name and type, and historical record of the new XML file;
[0071] Check the database to see if there is a table corresponding to the newly added XML file; if so, store the historical data corresponding to the newly added XML file into the corresponding database table; if not, generate the corresponding table creation statement based on the table name, column name and type of the newly added XML file, the type of database and the data type mapping table, and execute the table creation operation, and then store the historical data corresponding to the newly added XML file into the corresponding database table. When creating a table, you need to specify the column type, and the built-in type names of each database are different, so type mapping is required.
[0072] The warehousing service program starts the first thread and the second thread. The first thread is used to monitor the first directory and execute the warehousing task of the historical data in the folders under the first directory. The second thread is used to monitor the second directory and execute the warehousing task of the historical data in the folders under the second directory. When the first thread or the second thread detects that there are newly added XML files in the folders they monitor, they will start to process these files. The thread will parse the newly added XML files, extract the historical data therein, and cache these data in the memory for subsequent processing. At the same time, the thread will also obtain the metadata such as the table name, column name, data type and historical record of the XML file.
[0073] The data type mapping table is a table that stores the built-in data type mapping relationships of different database types. Before importing XML data into the database, the thread will check whether the table corresponding to the newly added XML file exists in the database. If so, the historical data in the XML file will be imported into the corresponding database. If not, the thread will generate a table creation statement based on the data type mapping table according to the metadata such as the table name, column name, and data type of the XML file, and import the XML data into the database after executing the table creation operation.
[0074] The warehousing service process starts two threads to monitor two directories respectively, and each thread performs the warehousing task of historical data in its own directory. When a new xml file is detected in the corresponding directory, the xml file is parsed, and the historical data is cached in the memory to obtain the table name, column name and type, and historical record. According to the characteristics of the current database, the corresponding SQL statement is generated. The column type needs to be specified in the table creation statement, and the built-in type names of each database are also different, so type mapping is required. If the table does not exist in the current database, the table creation SQL statement is generated according to the table information obtained above and the type mapping method of each database, and the table creation operation is performed. According to the historical records obtained above and the characteristics of the SQL statements of each database, the historical record storage SQL statement is generated to insert the historical data into the corresponding table of the database.
[0075] The present invention realizes efficient processing of newly added XML files by starting two threads to monitor different directories respectively; when a newly added file is detected, the thread can quickly parse the XML file, extract historical data and cache it to the memory, and obtain the metadata of the file at the same time; through the built-in data type mapping table, the thread can automatically generate the corresponding table creation statement and execute the table creation operation when the database table does not exist, thereby realizing the rapid import of XML data into the database.
[0076] According to an embodiment of the present invention, the warehousing service program queries whether the database handle connected to the current thread is normal; if normal, the current network is used to connect to the corresponding database; if abnormal, another network is switched to connect to the corresponding database;
[0077] If both network segments cannot be connected, wait for the preset time and try to connect again.
[0078] The warehousing service program has two network connection modes, network segment A and network segment B. The warehousing service program can be connected to the first database or the second database through network segment A, or through network segment B. When one of the networks fails, the other network can still ensure the normal import of data.
[0079] When performing data transmission, the entry service program will query whether the database handle connected to the current thread is normal; the database handle is the connection identifier established between the entry service program and the database, which allows the entry service program to communicate with the database. If the database handle is normal, the entry service program will try to use the current network environment to establish a connection with the corresponding database; if the connection with the database cannot be established in the current network environment (possibly due to network failure, database server unavailability or other reasons), the entry service program will try to switch to another network environment to establish a connection with the database. If both network segments cannot connect to the database (possibly due to a complete network outage, database server failure, etc.), the service program will wait for a preset time (such as a few minutes or hours) and then try to connect again; this retry mechanism can improve the fault tolerance and availability of the system, especially when the network is unstable or the database server occasionally fails.
[0080] The dual-network setting of the present invention improves the robustness and fault tolerance of the warehousing service program. The warehousing service program can maintain connection with the database when one of the networks fails or the database server is unavailable, thereby ensuring the continuous warehousing of historical data; the waiting and retrying mechanism further enhances the reliability and availability of the system.
[0081] Embodiment 2
[0082] Figure 2 This is a general architecture diagram of a method for synchronizing historical data of a power monitoring system using two machines and two networks according to one embodiment of the present invention; Figure 3 A schematic diagram of local caching of historical data in one embodiment of the present invention; Figure 4 This is a schematic diagram of a cache XML file format according to one embodiment of the present invention. Figure 2-4 As shown, according to one embodiment of the present invention, the present invention provides a method for synchronizing historical data of a power monitoring system with two machines and two networks, comprising the following steps:
[0083] Step 1: Obtain the historical data generated by the historical data generation modules such as historical storage and event storage, save it as an XML file in a specific format through the historical data formatting cache module and save it to the local disk;
[0084] The specific storage content includes: the data source corresponding to the current XML file, the data generation time, the table name, column name and type attribute corresponding to the historical data, data records, etc. For the specific format, see Figure 4 shown.
[0085] Step 2: Write the XML file that stores historical data to two directories, server1 and server2. Figure 3 As shown, the specific method is as follows:
[0086] (1) Historical data is written to the memory buffer pool.
[0087] (2) The memory buffer pool consists of a table information array. Each array element records the data table information corresponding to the element. This includes the table name, column name, column type attribute, table record size, and the memory address pointing to the actual record of the table. The records corresponding to the same table can exist in multiple array elements at the same time. When executing the data cache operation, if the data corresponding to the current array element is performing the disk operation, the data will be cached in the newly created element space. At the same time, the table structure information corresponding to the current data is recorded.
[0088] (3) Create new folders server1 and server2.
[0089] (4) Create new historical data folders (such as hds, alarm, etc.) in the folders server1 and server2 respectively. The number and names of the newly created folders in this level directory are determined by the number and names of the newly created databases for storing historical data in the current data source.
[0090] (5) Create folders Error and Input in each historical data folder (such as server1 / hds). Input is used to store XML files that store historical data, and Error is used to store XML files that failed to be stored.
[0091] (6) The thread scheduling module starts two threads, corresponding to directory server1 and directory server2. Each thread is responsible for the data storage operation of the corresponding directory.
[0092] (7) After the corresponding record in the memory buffer is successfully written to the disk by two threads at the same time, the corresponding record is deleted and marked accordingly for next use.
[0093] Step 3: If Figure 2 As shown, the historical data storage service program cyclically monitors directories server1 and server2. When an XML file is detected, it reads the XML file and writes the historical data in the XML file into the corresponding database server1 and server2. Directories server1 and server2 mean that the historical data stored in directory server1 needs to be written into the data source server1, and the historical data stored in directory server2 needs to be written into the data source server2. The specific method is as follows:
[0094] (1) The historical data storage service program starts two threads, Thread1 and Thread2.
[0095] (2) Thread1 monitors directory server1, and Thread2 monitors directory server2. When XML data is detected to be written, the monitoring thread of the corresponding directory reads the corresponding XML file and executes the storage operation.
[0096] (3) First, check whether the table corresponding to the data to be inserted exists in the database. If it does, the storage operation is performed immediately; if it does not exist, parse the table information corresponding to the data stored in the XML file. Generate a table creation statement through the mapping table of each database type and perform the table creation operation. After the table is successfully created, insert the data into the corresponding table.
[0097] (4) If data insertion fails, move the erroneous XML file to the Error directory at the same level as the directory where it is located.
[0098] (5) The thread corresponding to each directory periodically reads the Error directory in step (4) above, and retry to store the data that failed to be stored.
[0099] Step 4: The historical data storage service program selects a network segment with normal database connection according to the current machine network environment status and performs the storage operation. The specific method is as follows:
[0100] (1) Before executing the storage operation, check whether the database handle connected to the current thread is normal (the default is network segment A). If it is not normal, try to connect to the database from network segment B.
[0101] (2) If both network segments A and B cannot connect to the database normally, wait for 5 seconds and try to connect again.
[0102] The present invention adjusts the data warehousing process of the historical data generation module, from directly entering the historical database to converting the historical data into a fixed format and writing it into a local disk; writing to two directories server1 and server2 at the same time, meaning that the data in the directory server1 needs to be written into the data source 1, and the data in the directory server2 needs to be written into the data source 2; starting the warehousing service program, traversing and monitoring server1 and server2, reading and parsing the historical data file, and inserting the data into the corresponding database; when there is a fault in the A network segment, it will switch to the B network segment for warehousing operation; when both the AB network segments fail, the warehousing will be retried with a delay; the present invention guarantees the consistency and integrity of the historical data of the dual libraries to the greatest extent, improves the operational stability of the electric power monitoring system, and simplifies the work flow of the installation and operation personnel of the on-site monitoring system.
[0103] The present invention is compatible with a variety of database types and supports dual-network synchronization. The AB network is mutually redundant. When one of the network segments fails, it can quickly switch to the other network segment for synchronization. The present invention saves data that failed to be stored in the database and tries to store it again after a delay. The method adopted by the present invention guarantees the consistency and integrity of the dual database data to the greatest extent, improves the operational stability of the power monitoring system, and simplifies the workflow of the installation and operation personnel of the on-site monitoring system.
[0104] Embodiment 3
[0105] Figure 5 FIG. 1 is a schematic diagram of a historical data dual-machine dual-network synchronization system of an electric power monitoring system according to one embodiment of the present invention. Figure 5 As shown, a dual-machine dual-network synchronization system for historical data of an electric power monitoring system includes:
[0106] The acquisition module 10 is used to acquire historical data and format the historical data;
[0107] The processing module 20 is used to save the formatted historical data to the first directory and the second directory of the local disk at the same time;
[0108] Monitoring module 30, the warehousing service program cyclically traverses and monitors the first directory and the second directory;
[0109] The warehousing module 40, when new data is written into the first directory and the second directory, the warehousing service program reads the new data, writes the data read from the first directory into the first database, and writes the data read from the second directory into the second database; the warehousing service program can connect to the corresponding database through two network segments.
[0110] The method for synchronizing historical data of a power monitoring system with two machines and two networks provided by the present invention can realize simultaneous writing of two independent directories by caching and formatting historical data into XML files, and can monitor and store data using a multi-threaded service program to ensure reliable synchronization of data in a dual-machine and dual-network environment. When the network environment is abnormal, the system can automatically switch network segments to try to connect, ensuring the continuity and integrity of data storage. The present invention effectively improves the security and reliability of historical data of the power monitoring system, and reduces the risk of data loss through a dual-machine and dual-network synchronization mechanism.
[0111] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the system described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.
[0112] The above description is only a preferred embodiment of the present application and an explanation of the technical principles used. Those skilled in the art should understand that the scope of the invention involved in the present application is not limited to the technical solution formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the inventive concept. For example, the above features are replaced with (but not limited to) technical features with similar functions disclosed in the present application.
[0113] It should be understood that the size of the serial numbers of each step in the content of the invention and the embodiments of the present invention does not absolutely mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.
Claims
1. A method for synchronizing historical data of a power monitoring system using two machines and two networks. It is characterized in that The steps include: Acquire historical data, and format the historical data; The formatted historical data is saved in the first directory and the second directory of the local disk at the same time; The warehousing service program cyclically traverses and monitors the first directory and the second directory; When new data is written to the first directory and the second directory, the warehousing service program reads the new data, writes the data read from the first directory into the first database, and writes the data read from the second directory into the second database; the warehousing service program can connect to the corresponding databases through two network segments.
2. According to the method for synchronizing historical data of a power monitoring system using two machines and two networks as claimed in claim 1, It is characterized in that The historical data is formatted, including: The historical data is saved as an XML file in a preset format; the historical data includes historical storage data and event storage data; The saved content includes the data source corresponding to the current XML file, data generation time, table name, column name and type attributes corresponding to historical data, and data records.
3. The method for synchronizing historical data of a power monitoring system using two machines and two networks according to claim 1, It is characterized in that The formatted historical data is saved to the first and second directories of the local disk at the same time, including: Name the XML file in a fixed format, which is "database name_table name_timestamp_nanosecond.xml"; Query whether the first directory and the second directory exist on the local disk; if not, create the first directory and the second directory; Check whether corresponding historical data folders exist in the first directory and the second directory; if not, create the historical data folders in the first directory and the second directory; Check whether there are Error folders and Input folders under each historical data folder; if not, create the Error folder and the Input folder; wherein the Error folder is used to store XML files that failed to be stored, and the Input folder is used to store XML files named in a fixed format; Store the XML files named in a fixed format in the corresponding folder.
4. The method for synchronizing historical data of a power monitoring system using two machines and two networks according to claim 3, Features: The number and naming of the historical data folders in the first directory and the second directory are determined by the database used to store historical data in the current data source.
5. The method for synchronizing historical data of a power monitoring system using two machines and two networks according to claim 3, Features: The thread corresponding to each directory periodically reads the corresponding Error folder and tries to reload the data that failed to be loaded into the database.
6. The method for synchronizing historical data of a power monitoring system using two machines and two networks according to claim 1, It is characterized in that The warehousing service program cyclically traverses and monitors the first directory and the second directory, including: The warehousing service program starts a first thread and a second thread; the first thread and the second thread are used to cyclically traverse and monitor the first directory and the second directory respectively.
7. The method for synchronizing historical data of a power monitoring system using two machines and two networks according to claim 6, Features: When the first thread and the second thread detect that there is a newly added xml file in the corresponding folder, the newly added xml file is parsed, and the historical data of the newly added xml file is cached in the memory, and the table name, column name and type, and historical record of the newly added xml file are obtained; Query the database to see whether a table corresponding to the newly added xml file exists; if so, store the historical data corresponding to the newly added xml file into the corresponding database table; if not, generate a corresponding table creation statement based on the table name, column name and type of the newly added xml file, according to the type of database and data type mapping table, execute the table creation operation, and then store the historical data corresponding to the newly added xml file into the corresponding database table.
8. The method for synchronizing historical data of a power monitoring system using two machines and two networks according to claim 7, Features: The data type mapping table includes database types and built-in data type mapping relationships of each database.
9. The method for synchronizing historical data of a power monitoring system using two machines and two networks according to claim 1, Features: The warehousing service program queries whether the database handle connected to the current thread is normal; if normal, use the current network to connect to the corresponding database; if abnormal, switch to another network to connect to the corresponding database; If both network segments cannot be connected, wait for the preset time and try to connect again.
10. A dual-machine dual-network synchronization system for historical data of power monitoring system, It is characterized in that include: An acquisition module, used for acquiring historical data and formatting the historical data; A processing module, used for saving the formatted historical data to a first directory and a second directory of a local disk at the same time; A monitoring module, wherein the warehousing service program cyclically traverses and monitors the first directory and the second directory; The warehousing module, when new data is written to the first directory and the second directory, the warehousing service program reads the new data, writes the data read from the first directory into the first database, and writes the data read from the second directory into the second database; the warehousing service program can be connected to the corresponding database through two network segments.
Citation Information
Patent Citations
Data transmission method between heterogeneous systems and data transmission system
CN101754221A
File backup method and device based on iscsi volumes
CN103077098A
Server data synchronization method and device and computer readable storage medium
CN111526198A
Automated record attribute value merging from multiple directory servers
US20080208797A1
