Data synchronization method and system for local database of hydraulic power plant

Abstract

The invention discloses a hydraulic power plant local database data synchronization method and system, and relates to the technical field of hydraulic power plant local data management, and the method comprises the steps: configuring the connection between a local database and a remote monitoring system database; setting a synchronous triggering mechanism; sending a query instruction to a remote monitoring system database through the established database connection, and extracting updated data from last synchronization; performing data deduplication and formatting processing on the extracted updated data in the local cache; inserting the preprocessed update data into a local database; and after data insertion and updating are completed, data consistency verification is executed. According to the hydraulic power plant local database data synchronization method provided by the invention, data transmission between the local database and the remote monitoring system database is ensured to be safe and reliable through an encrypted communication protocol and a database authentication mechanism, and the problem of data loss or transmission interruption is avoided.

Description

Technical Field

[0001] The present invention relates to the technical field of local data management of hydropower plants, and in particular to a method and system for synchronizing data of a local database of a hydropower plant. Background Art

[0002] With the digital transformation of the power industry, the operating efficiency and safety of power plants are increasingly dependent on efficient data synchronization and monitoring systems. As an important part of energy production, the accurate collection and synchronization of data such as the speed regulator operating parameters and generator set status parameters of hydropower plants is the key to ensuring the stable operation of the power system. However, traditional data synchronization methods have certain limitations when dealing with data consistency between the remote monitoring system database and the local database. For example, existing synchronization technologies are prone to data loss, duplication, or inaccurate verification when the network connection is unstable or the database load is too high, resulting in incomplete or inconsistent synchronization between the local database and the remote database.

[0003] In addition, many existing systems lack automated deduplication and abnormal data processing mechanisms during data processing, resulting in the accumulation of a large amount of redundant and abnormal data, which affects subsequent data analysis and decision support. Therefore, how to ensure efficient and accurate synchronization between the local database of the hydropower plant and the remote monitoring system database in a complex database environment has become a technical problem that needs to be solved urgently. Summary of the invention

[0004] In view of the above-mentioned problems, the present invention is proposed.

[0005] Therefore, the technical problems solved by the present invention are: the lack of an effective data synchronization mechanism in the prior art leads to inconsistency between the remote monitoring system database and the local database data, which is prone to data loss or redundancy problems; when the network connection is unstable or the database load is high, the existing synchronization scheme cannot ensure the stability and security of data transmission; the existing data synchronization scheme lacks an automated processing mechanism when processing duplicate data, fuzzy data or abnormal data, resulting in low data processing efficiency and affecting data consistency verification; there is a lack of integrity and consistency verification mechanism for synchronized data, and the accuracy of data synchronization cannot be effectively verified, affecting subsequent data analysis and decision-making.

[0006] In order to solve the above technical problems, the present invention provides the following technical solutions: a method for synchronizing data of a local database of a hydropower plant, comprising:

[0007] Configuring a connection between a local database and a remote monitoring system database, wherein the connection is established through an encrypted communication protocol, and configuring database connection parameters, including the IP address, port number, database name, and user credentials of the remote database;

[0008] Setting a synchronization trigger mechanism, which is based on changes in fixed time intervals or real-time monitoring data, and triggers a synchronization task when the governor operating parameters and generator set status parameters in the remote monitoring system database are updated;

[0009] Sending a query command to the remote monitoring system database through the established database connection to extract updated data since the last synchronization, wherein the updated data includes governor operating parameters and generator set status information;

[0010] De-duplicate and format the extracted update data in the local cache to remove duplicate data and unify the data fields in different formats;

[0011] Insert the preprocessed update data into the local database. If the same primary key record exists in the local database, perform the update operation. If not, perform the insert operation.

[0012] After completing data insertion and updating, perform data consistency check to ensure that the synchronized data between the local database and the remote monitoring system database are consistent.

[0013] As a preferred solution of the method for synchronizing data of a local database of a hydropower plant described in the present invention, wherein: the configuration of the connection between the local database and the remote monitoring system database includes inputting the IP address, port number, database name and user credentials of the remote monitoring system database on the local server;

[0014] Use encrypted communication protocols to conduct an initial handshake with the remote monitoring system database, generate and exchange session keys, and ensure the establishment of a secure communication channel;

[0015] Verify the legitimacy of user credentials and confirm the connection identity through the database authentication mechanism;

[0016] Verify the database connection status. If the connection fails, start the reconnection process and record the connection status.

[0017] As a preferred solution of the method for synchronizing data in the local database of a hydropower plant described in the present invention, wherein: the setting of the synchronization trigger mechanism includes monitoring the frequency response of the speed regulator, and when the change of the frequency response exceeds a preset frequency change threshold, triggering the synchronization task;

[0018] Monitor the guide vane opening changes in real time. When the guide vane opening changes exceed the preset opening change threshold, the synchronization task is triggered.

[0019] When the frequency response and the guide vane opening do not reach the triggering condition, the timing synchronization is performed through the preset time interval threshold, and the triggering time of the synchronization is recorded.

[0020] As a preferred solution of the method for synchronizing data of a local database of a hydropower plant described in the present invention, wherein: the extracting updated data since the last synchronization includes extracting the governor load, frequency response and generator set operating status data since the last synchronization from the remote monitoring system database through an SQL query statement;

[0021] Sort the extracted data in ascending order according to the timestamp of the data to ensure the time continuity of the data during synchronization;

[0022] For multiple data records with the same timestamp, the latest record is extracted and the old records are marked as historical data and stored in the local cache.

[0023] As a preferred solution of the method for synchronizing data in the local database of a hydropower plant described in the present invention, the data deduplication and formatting process includes using a hash algorithm to calculate a hash value for each row of data, marking data with the same hash value as that in the local database as duplicate records, and avoiding duplicate data from entering the synchronization process;

[0024] Use regular expressions to format date and time fields from different sources;

[0025] Ambiguous or non-deduplicated data is marked as abnormal data and stored in the local cache for subsequent analysis.

[0026] As a preferred solution of the method for synchronizing data in a local database of a hydropower plant described in the present invention, wherein: inserting the pre-processed updated data into the local database includes calling a database transaction management function before executing the insert or update operation, and placing the data change operation into the same transaction to ensure data consistency;

[0027] If there are data records with the same primary key in the local database, the changed fields are updated by comparing them field by field, and the unchanged fields remain unchanged;

[0028] For data records that do not have the same primary key, a new primary key value is generated and a new record is inserted, and the primary key index is updated for subsequent operation calls.

[0029] As a preferred solution of the method for synchronizing data of a local database of a hydropower plant described in the present invention, wherein: the execution of data consistency check includes using a hash algorithm to calculate hash values ​​for each piece of data in the remote monitoring system database and the local database respectively;

[0030] Compare the hash values ​​in the remote and local databases. If any inconsistency is found, generate a difference report to record the inconsistent data entries and corresponding fields.

[0031] In subsequent synchronization tasks, data entries in the difference report are processed first until the data consistency check passes.

[0032] Another object of the present invention is to provide a local database data synchronization system for a hydropower plant, which can solve the problem that data transmission in existing distribution network operations needs to ensure security and confidentiality by building a remote secure communication system for distribution network operations, and that using a third-party platform is not conducive to data confidentiality.

[0033] In order to solve the above technical problems, the present invention provides the following technical solutions: a local database data synchronization system for a hydropower plant, comprising: a connection configuration module, a trigger mechanism module, a data extraction module, a data processing module, a data writing module and a consistency verification module; the connection configuration module is used to configure the connection between the local database and the remote monitoring system database, the connection is established through an encrypted communication protocol, and the database connection parameters are configured, including the IP address, port number, database name and user credentials of the remote database; the trigger mechanism module is used to set a synchronization trigger mechanism, the trigger mechanism is based on a fixed time interval or real-time monitoring data changes, when the speed regulator in the remote monitoring system database When the operating parameters and generator set status parameters are updated, the synchronization task is triggered; the data extraction module is used to send a query instruction to the remote monitoring system database through the established database connection to extract the updated data since the last synchronization, and the updated data includes the governor operating parameters and the generator set status information; the data processing module is used to perform data deduplication and formatting processing on the extracted updated data in the local cache, remove duplicate data, and uniformly format data fields of different formats; the data writing module is used to insert the pre-processed updated data into the local database, and if the same primary key record exists in the local database, the update operation is performed, and if not, the insert operation is performed;

[0034] The consistency check module is used to perform data consistency check after completing data insertion and update to ensure that the synchronization data of the local database and the remote monitoring system database are consistent.

[0035] A computer device comprises a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the above-mentioned method for synchronizing data of a local database of a hydropower plant when executing the computer program.

[0036] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the above-mentioned method for synchronizing data in a local database of a hydropower plant.

[0037] Beneficial effects of the present invention: The data synchronization method for the local database of a hydropower plant provided by the present invention ensures that the data transmission between the local database and the remote monitoring system database is safe and reliable through an encrypted communication protocol and a database authentication mechanism, thereby avoiding the problem of data loss or transmission interruption; a hash algorithm and a regular expression are introduced to automatically process data deduplication and formatting operations, and at the same time, an exception handling mechanism is established for fuzzy data and data that cannot be deduplicated, thereby greatly improving data processing efficiency; through a data consistency verification mechanism, the data inconsistency problem between the remote monitoring system database and the local database can be automatically detected and processed to ensure data integrity and synchronization accuracy; a synchronization trigger mechanism is set up to support real-time triggering of data synchronization based on time intervals or data changes, flexibly respond to different synchronization requirements, and ensure real-time data updates; the method can adapt to database environments of different sizes, and maintain the stability of the synchronization process under high load or network instability, and is suitable for various complex application scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

[0039] Figure 1 An overall flow chart of a method for synchronizing data in a local database of a hydropower plant provided by an embodiment of the present invention.

[0040] Figure 2 An overall structural diagram of a hydropower plant local database data synchronization system provided for each embodiment of the present invention. DETAILED DESCRIPTION

[0041] In order to make the above-mentioned purposes, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are described in detail below in conjunction with the drawings of the specification. Obviously, the described embodiments are part of the embodiments of the present invention, but not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary persons in the art without creative work should fall within the scope of protection of the present invention.

[0042] In the following description, many specific details are set forth to facilitate a full understanding of the present invention, but the present invention may also be implemented in other ways different from those described herein, and those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0043] Example 1

[0044] Reference Figure 1 , as an embodiment of the present invention, provides a method for synchronizing data in a local database of a hydropower plant, comprising:

[0045] S1 configures the connection between the local database and the remote monitoring system database, the connection is established through an encrypted communication protocol, and configures database connection parameters, including the IP address, port number, database name and user credentials of the remote database;

[0046] S2 sets a synchronization trigger mechanism, which is based on a fixed time interval or a change in real-time monitoring data. When the governor operating parameters and generator set status parameters in the remote monitoring system database are updated, the synchronization task is triggered;

[0047] S3 sends a query command to the remote monitoring system database through the established database connection to extract updated data since the last synchronization, wherein the updated data includes governor operating parameters and generator set status information;

[0048] S4 performs data deduplication and formatting on the extracted update data in the local cache, removes duplicate data, and formats the data fields of different formats uniformly;

[0049] S5 inserts the preprocessed update data into the local database. If the local database has a record with the same primary key, an update operation is performed; if not, an insert operation is performed;

[0050] After S6 completes data insertion and updating, it performs data consistency check to ensure that the synchronized data between the local database and the remote monitoring system database are consistent.

[0051] The configuring the connection between the local database and the remote monitoring system database includes inputting the IP address, port number, database name and user credentials of the remote monitoring system database on the local server;

[0052] Use encrypted communication protocols (such as SSL / TLS) to perform an initial handshake with the remote monitoring system database, generate and exchange session keys, and ensure the establishment of a secure communication channel;

[0053] Verify the legitimacy of user credentials and confirm the connection identity through the database authentication mechanism;

[0054] Verify the database connection status. If the connection fails, start the reconnection process and record the connection status.

[0055] The setting of the synchronization trigger mechanism includes monitoring the frequency response of the speed regulator, and triggering the synchronization task when the change of the frequency response exceeds a preset frequency change threshold;

[0056] Monitor the guide vane opening changes in real time. When the guide vane opening changes exceed the preset opening change threshold, the synchronization task is triggered.

[0057] When the frequency response and the guide vane opening do not reach the triggering condition, the timing synchronization is performed through the preset time interval threshold, and the triggering time of the synchronization is recorded.

[0058] It should be noted that the frequency response monitoring of the speed regulator is set. When the change of the frequency response exceeds ±0.03Hz, the synchronization task is automatically triggered; the change of the guide vane opening is monitored in real time, and when the change of the guide vane opening exceeds 5%, the synchronization task is triggered; if none of the monitoring parameters meet the triggering conditions, the synchronization task is executed at a fixed interval of 30 minutes, and the synchronization timestamp is recorded.

[0059] The extracting of updated data since the last synchronization includes extracting the governor load, frequency response and generator set operating status data since the last synchronization from the remote monitoring system database through an SQL query statement;

[0060] Sort the extracted data in ascending order according to the timestamp of the data to ensure the time continuity of the data during synchronization;

[0061] For multiple data records with the same timestamp, the latest record is extracted and the old records are marked as historical data and stored in the local cache.

[0062] The data deduplication and formatting process includes using a hash algorithm to calculate a hash value for each row of data, marking data with the same hash value as that in the local database as duplicate records, and preventing duplicate data from entering the synchronization process;

[0063] Use regular expressions to format date and time fields from different sources, and format the date and time fields uniformly as "YYYY-MM-DD HH:MM";

[0064] Ambiguous or non-deduplicated data is marked as abnormal data and stored in the local cache for subsequent analysis.

[0065] Fuzzy data refers to records whose authenticity cannot be determined due to incompleteness, inaccuracy or inconsistent format. For example: wrong or incomplete timestamp: the timestamp format of some data does not conform to the standardized format (such as "YYYY-MM-DDHH:MM"), or the timestamp is empty. Inconsistent field values: multiple fields of the same record are logically contradictory, or field values ​​are abnormal (for example, the generator set status is inconsistent with the governor frequency response data). Numerical deviation: data outside the normal range, such as the value of the frequency response exceeds the normal range.

[0066] Data that cannot be deduplicated refers to data records that have the same primary key or the same hash value in hash processing or other deduplication algorithms, but cannot be determined which data is the correct data record. For example: Same primary key but different field values: Two or more records have the same primary key (such as timestamp or ID), but the contents of other fields are different, resulting in the inability to directly deduplicate. Same hash value but from different sources: Records from different sources generate the same hash value after hash processing, but the content or source is different, making it difficult to determine which record is valid.

[0067] Fuzzy data processing includes checking the integrity of the timestamp for records marked as fuzzy data. If the timestamp is missing or the format is incorrect, the records with adjacent timestamps are used for comparison to try to correct the timestamp; if the field values ​​are inconsistent, the fields are corrected by calculating the average of adjacent records or using default values; the corrected fuzzy data records are checked for integrity again, and if the correction is successful, they are reinserted into the synchronization process for insertion or update operations; if the correction fails, the data record is marked as requiring manual processing and included in the abnormal data list.

[0068] The processing of data that cannot be deduplicated includes, for records with the same primary key or the same hash value but different field values, first analyzing the differences through the field comparison algorithm, trying to determine which field is closest to the normal range; if the field comparison cannot come to a clear conclusion, it is corrected according to the credibility of the source data (for example, giving priority to the use of credible data from a certain source); the corrected records are verified again through the deduplication algorithm, and if the deduplication is successful, the corrected records are reinserted into the synchronization process, and the insertion or update operation continues; if the correction cannot be made, it is included in the list of data that needs manual review for subsequent manual intervention. After the corrected abnormal data passes the verification, it will be re-added to the data synchronization process and participate in the next round of data consistency verification.

[0069] Inserting the pre-processed updated data into the local database includes calling a database transaction management function before performing the insert or update operation to put the data change operation into the same transaction to ensure data consistency;

[0070] If there are data records with the same primary key in the local database, the changed fields are updated by comparing them field by field, and the unchanged fields remain unchanged;

[0071] For data records that do not have the same primary key, a new primary key value is generated and a new record is inserted, and the primary key index is updated for subsequent operation calls.

[0072] The performing of data consistency check includes using a hash algorithm to calculate hash values ​​for each piece of data in the remote monitoring system database and the local database respectively;

[0073] Compare the hash values ​​in the remote and local databases. If any inconsistency is found, generate a difference report to record the inconsistent data entries and corresponding fields.

[0074] In subsequent synchronization tasks, data entries in the difference report are processed first until the data consistency check passes.

[0075] It should be noted that the security and integrity of data transmission is ensured by entering the IP address, port number, database name and user credentials of the remote monitoring system database on the local server and using the encrypted communication protocol to establish a connection. The encrypted communication protocol prevents possible malicious interception and tampering during data transmission by encrypting the data stream and session key, while verifying the user credentials and connection status to ensure that only authorized systems can access the remote database. This mechanism improves the security of the system, especially when dealing with network attacks or data leakage risks, and can effectively ensure the stability of the system and the secure transmission of data.

[0076] The synchronization trigger mechanism ensures that the synchronization task can respond to changes in the operating status of power plant equipment in real time by monitoring the frequency response of the speed regulator and the change in the guide vane opening. When the frequency response change exceeds the preset threshold, data synchronization is triggered immediately, effectively avoiding data update delays caused by system response lags. The real-time monitoring trigger mechanism ensures the timeliness and accuracy of data synchronization, especially when the operating conditions of the generator set change frequently, to ensure that the data is always up to date. At the same time, the timed synchronization mechanism with a preset time interval can regularly update data without triggering conditions, ensuring the integrity of system data and avoiding omissions.

[0077] Updated data is extracted through SQL query statements and sorted in ascending order according to the data's timestamp, ensuring the temporal continuity of the data during the synchronization process. In particular, when multiple data records have the same timestamp, the latest record is extracted and the old record is marked as historical data, avoiding interference from duplicate or outdated data. This not only ensures the accuracy and timeliness of the data, but also reduces the waste of storage resources to a certain extent, ensuring the efficient operation of database synchronization operations. This technical solution ensures data consistency and smooth processing flow when processing complex time series data.

[0078] The introduction of hash algorithms can efficiently identify and remove duplicate data, reducing the time and computing resources required for field-by-field comparison in traditional methods. At the same time, the use of regular expressions further improves the efficiency of formatting data from different sources, especially the standardization of date and time fields, ensuring data format consistency and facilitating subsequent processing and analysis. In addition, fuzzy or non-deduplicated data is marked as abnormal data and stored to ensure that this data does not enter the synchronization process, avoiding possible erroneous data from affecting the consistency and integrity of the database. This data preprocessing process significantly improves the accuracy and stability of data synchronization.

[0079] The database transaction management function is called before performing an insert or update operation to ensure data consistency. Even if errors or system interruptions occur during multiple data write operations, the integrity of the database can be guaranteed through a rollback mechanism. The field-by-field comparison and update mechanism ensures accurate updates of existing data in the local database, and for data records that do not have the same primary key, new primary keys are generated and new records are inserted, further ensuring data integrity and conflict-free updates. Through this transaction management mechanism, the system demonstrates higher stability and reliability during large-scale data synchronization, avoiding the generation of inconsistent or conflicting data.

[0080] By using the hash algorithm to perform consistency check on the data of the remote monitoring system database and the local database, data inconsistencies between the two can be detected quickly and effectively. Unlike traditional field-by-field comparison, the hash algorithm greatly improves the verification efficiency, especially when processing large-scale data sets, it can significantly reduce the calculation time. Generating a difference report to record inconsistent data entries and fields provides a clear correction path for subsequent synchronization tasks, and ensures the final data consistency by giving priority to processing difference data. This verification mechanism greatly improves the system's ability to verify data synchronization results and ensures the high accuracy and reliability of synchronized data.

[0081] Example 2

[0082] Reference Figure 2 , as an embodiment of the present invention, provides a hydropower plant local database data synchronization system, including:

[0083] Connection configuration module 100, trigger mechanism module 200, data extraction module 300, data processing module 400, data writing module 500 and consistency verification module 600;

[0084] The connection configuration module 100 is used to configure the connection between the local database and the remote monitoring system database, the connection is established through an encrypted communication protocol, and the database connection parameters are configured, including the IP address, port number, database name and user credentials of the remote database;

[0085] The trigger mechanism module 200 is used to set a synchronization trigger mechanism, which is based on a fixed time interval or a change in real-time monitoring data. When the governor operating parameters and generator set status parameters in the remote monitoring system database are updated, the synchronization task is triggered;

[0086] The data extraction module 300 is used to send a query instruction to the remote monitoring system database through the established database connection to extract the updated data since the last synchronization, wherein the updated data includes the governor operating parameters and the generator set status information;

[0087] The data processing module 400 is used to perform data deduplication and formatting processing on the extracted update data in the local cache, remove duplicate data, and uniformly format data fields of different formats;

[0088] The data writing module 500 is used to insert the pre-processed update data into the local database. If the same primary key record exists in the local database, an update operation is performed; if not, an insert operation is performed;

[0089] The consistency check module 600 is used to perform data consistency check after completing data insertion and update to ensure that the synchronization data of the local database and the remote monitoring system database are consistent.

[0090] Example 3

[0091] An embodiment of the present invention is different from the first two embodiments in that:

[0092] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc., which can store program codes.

[0093] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in conjunction with such instruction execution systems, devices or apparatuses. For the purposes of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in conjunction with such instruction execution systems, devices or apparatuses.

[0094] More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or more wires (electronic device), a portable computer disk case (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be a paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, deciphering or, if necessary, processing in another suitable manner, and then stored in a computer memory.

[0095] It should be understood that the various parts of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, a plurality of steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0096] Example 4

[0097] As an embodiment of the present invention, a method for synchronizing data of a local database of a hydropower plant is provided. In order to verify the beneficial effects of the present invention, a scientific demonstration is carried out through simulation experiments.

[0098] The database of the simulated remote power plant control center contains the governor operating parameters and generator set status parameters. The local database of the hydropower plant, data is extracted and updated from the remote monitoring system database through a synchronization mechanism.

[0099] Traditional methods do not use encrypted communication protocols, lack hash algorithm deduplication and regular expression formatting, and data consistency verification is performed line by line by comparing the fields of each record, and there is no automated abnormal data processing.

[0100] The method of the present invention uses an encrypted communication protocol, combines a hash algorithm to remove duplicates, formats data with a regular expression, and uses a hash check to check data consistency, while processing abnormal data.

[0101] The total amount of data simulated for synchronization is 10,000 data items, of which 500 are fuzzy data items and 200 cannot be deduplicated.

[0102] Each time 10,000 data items are synchronized, the traditional method directly synchronizes through SQL query, while the method of the present invention transmits data through an encrypted communication protocol and verifies user credentials during the transmission process. The accuracy of data synchronization is recorded, that is, the amount of data successfully synchronized.

[0103] 10,000 pieces of data were deduplicated, including 500 pieces of ambiguous data and 200 pieces of data that could not be deduplicated. The traditional method deduplicated data by simple field matching, while the method of the present invention deduplicated data by hash algorithm. The time required for deduplication and the amount of duplicate data after processing were recorded.

[0104] Among the 10,000 data, there are 500 fuzzy data and 200 data that cannot be deduplicated. Traditional methods cannot handle these abnormal data, but the method of the present invention marks and stores the abnormal data, and performs automatic correction or manual processing. The time of processing the abnormal data and the amount of abnormal data that are successfully processed are recorded.

[0105] Data consistency check was performed by comparing 10,000 data in the remote monitoring system database and the local database. The traditional method checks consistency field by field, while the method of the present invention checks by hash algorithm. The time required for verification and the verification success rate were recorded. The experimental results are shown in Table 1.

[0106] Table 1 Comparison of experimental results

[0107]

[0108]

[0109] The use of encrypted communication protocols combined with user credential verification not only ensures the security of data transmission, but also enhances the stability of data transmission under network fluctuations. Traditional synchronization methods rarely consider transmission security issues, but the present invention introduces encryption mechanisms into the synchronization process, so that the integrity and reliability of data transmission can be guaranteed even in complex network environments.

[0110] The present invention uses a hash algorithm for data deduplication and combines regular expressions to unify the data format, effectively improving the deduplication efficiency and data consistency. The hash algorithm is usually used in the encryption field. The present invention innovatively introduces it into the data processing link, which can quickly and accurately identify and remove duplicate data. At the same time, the use of regular expressions further ensures the consistency of the data format before synchronization, avoiding the processing problems caused by the inconsistency of data formats from different sources in traditional methods.

[0111] Abnormal data processing achieves efficient abnormal data correction through automated timestamp comparison and field correction mechanisms. Compared with the traditional method that relies on manual intervention, the automated processing of the present invention not only reduces human errors in operation, but also improves the overall accuracy of data synchronization. Through this mechanism, the system can automatically identify and process ambiguous or non-deduplicated data, ensuring the integrity of the data synchronization process.

[0112] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.

Claims

1. A method for synchronizing data in a local database of a hydropower plant, characterized in that: include: Configuring a connection between a local database and a remote monitoring system database, wherein the connection is established through an encrypted communication protocol, and configuring database connection parameters, including the IP address, port number, database name, and user credentials of the remote database; Setting a synchronization trigger mechanism, which is based on fixed time intervals or real-time monitoring data changes, and triggering a synchronization task when the governor operating parameters and generator set status parameters in the remote monitoring system database are updated; Sending a query command to the remote monitoring system database through the established database connection to extract updated data since the last synchronization, wherein the updated data includes governor operating parameters and generator set status information; De-duplicate and format the extracted update data in the local cache to remove duplicate data and unify the data fields in different formats; Insert the preprocessed update data into the local database. If the same primary key record exists in the local database, perform the update operation. If not, perform the insert operation. After completing data insertion and updating, perform data consistency check to ensure that the synchronized data between the local database and the remote monitoring system database are consistent.

2. The method for synchronizing data in a local database of a hydropower plant according to claim 1, characterized in that: The configuring the connection between the local database and the remote monitoring system database includes inputting the IP address, port number, database name and user credentials of the remote monitoring system database on the local server; Use encrypted communication protocols to conduct an initial handshake with the remote monitoring system database, generate and exchange session keys, and ensure the establishment of a secure communication channel; Verify the legitimacy of user credentials and confirm the connection identity through the database authentication mechanism; Verify the database connection status. If the connection fails, start the reconnection process and record the connection status.

3. The method for synchronizing data in a local database of a hydropower plant as claimed in claim 2, characterized in that: The setting of the synchronization trigger mechanism includes monitoring the frequency response of the speed regulator, and triggering the synchronization task when the change of the frequency response exceeds a preset frequency change threshold; Monitor the guide vane opening changes in real time. When the guide vane opening changes exceed the preset opening change threshold, the synchronization task is triggered. When the frequency response and the guide vane opening do not reach the triggering condition, the timing synchronization is performed through the preset time interval threshold, and the triggering time of the synchronization is recorded.

4. The method for synchronizing data in a local database of a hydropower plant as claimed in claim 3, characterized in that: The extracting of updated data since the last synchronization includes extracting the governor load, frequency response and generator set operating status data since the last synchronization from the remote monitoring system database through an SQL query statement; Sort the extracted data in ascending order according to the timestamp of the data to ensure the temporal continuity of the data during synchronization; For multiple data records with the same timestamp, the latest record is extracted and the old records are marked as historical data and stored in the local cache.

5. The method for synchronizing data in a local database of a hydropower plant as claimed in claim 4, characterized in that: The data deduplication and formatting process includes using a hash algorithm to calculate a hash value for each row of data, marking data with the same hash value as that in the local database as duplicate records, and preventing duplicate data from entering the synchronization process; Use regular expressions to format date and time fields from different sources; Ambiguous or non-deduplicated data is marked as abnormal data and stored in the local cache for subsequent analysis.

6. The method for synchronizing data in a local database of a hydropower plant as claimed in claim 5, characterized in that: Inserting the pre-processed updated data into the local database includes calling a database transaction management function before performing the insert or update operation to put the data change operation into the same transaction to ensure data consistency; If there are data records with the same primary key in the local database, the changed fields are updated by comparing them field by field, and the unchanged fields remain unchanged; For data records that do not have the same primary key, a new primary key value is generated and a new record is inserted, and the primary key index is updated for subsequent operation calls.

7. The method for synchronizing data in a local database of a hydropower plant as claimed in claim 6, characterized in that: The performing of data consistency check includes using a hash algorithm to calculate hash values ​​for each piece of data in the remote monitoring system database and the local database respectively; Compare the hash values ​​in the remote and local databases. If any inconsistency is found, generate a difference report to record the inconsistent data entries and corresponding fields. In subsequent synchronization tasks, data entries in the difference report are processed first until the data consistency check passes.

8. A system using the method for synchronizing data in a local database of a hydropower plant as claimed in any one of claims 1 to 7, characterized in that: include: A connection configuration module (100), a trigger mechanism module (200), a data extraction module (300), a data processing module (400), a data writing module (500) and a consistency verification module (600); The connection configuration module (100) is used to configure the connection between the local database and the remote monitoring system database, the connection is established through an encrypted communication protocol, and the database connection parameters are configured, including the IP address, port number, database name and user credentials of the remote database; The trigger mechanism module (200) is used to set a synchronization trigger mechanism, which is based on a fixed time interval or a change in real-time monitoring data, and triggers a synchronization task when the governor operating parameters and generator set status parameters in the remote monitoring system database are updated; The data extraction module (300) is used to send a query instruction to the remote monitoring system database through the established database connection to extract the updated data since the last synchronization, wherein the updated data includes the governor operating parameters and the generator set status information; The data processing module (400) is used to perform data deduplication and formatting processing on the extracted update data in the local cache, remove duplicate data, and uniformly format data fields in different formats; The data writing module (500) is used to insert the pre-processed update data into the local database, and if the same primary key record exists in the local database, an update operation is performed; if not, an insert operation is performed; The consistency check module (600) is used to perform data consistency check after completing data insertion and updating, so as to ensure that the synchronization data of the local database and the remote monitoring system database are consistent.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method for synchronizing data of a local database of a hydropower plant according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method for synchronizing data of a local database of a hydropower plant according to any one of claims 1 to 7 are implemented.

Images