Data synchronization method and system, terminal equipment and computer readable storage medium

Abstract

The invention is applicable to the technical field of cross-security-zone data synchronization of databases of an electric power system, and provides a data synchronization method and system, terminal equipment and a computer readable storage medium. Obtaining first data corresponding to the current data updating and a first updating mode corresponding to the first data; detecting second data matched with the field type of the first data in the first database; and if it is detected that the second data matched with the field type of the first data exists in the first database, updating the second data according to an updating mode corresponding to the first data. According to the method, the synchronization of the homologous asynchronous database can be realized without changing a service code, and the cost brought by database adaptation can be reduced.

Description

Technical Field

[0001] The present application belongs to the technical field of data synchronization across security zones of a power system database, and in particular, relates to a data synchronization method, system, terminal device, and computer-readable storage medium. Background Art

[0002] As 5G technology matures and artificial intelligence becomes more popular, substation systems are becoming more interoperable and interconnected, and there are more and more monitoring, operation and maintenance scenarios across security zones. Therefore, the cross-security database synchronization mechanism is particularly urgent.

[0003] Existing related methods cannot achieve synchronization of heterogeneous databases with the same source. In the scenario of synchronizing asynchronous databases with the same source, a large amount of code needs to be modified to adapt the source database and the target database, and the mapping relationship between the fields of the target database and the source database needs to be pre-configured, resulting in high development costs. Summary of the invention

[0004] The embodiments of the present application provide a data synchronization method, a synchronization system, a terminal device and a computer-readable storage medium, which can realize the synchronization of asynchronous databases of the same source without changing the business code or configuring the mapping relationship between the destination database and the source database, thereby reducing development costs.

[0005] In a first aspect, an embodiment of the present application provides a data synchronization method, including:

[0006] When it is detected that data update occurs in the second database, first data corresponding to the data update and an update method corresponding to the first data are obtained;

[0007] Detecting second data in the first database that matches the field type of the first data;

[0008] If it is detected that there is second data in the first database that matches the field type of the first data, the second data is updated according to the update method corresponding to the first data.

[0009] In the embodiment of the present application, when the first database detects that the data in the second database is updated, in the first database, the system will search for data records that match the field type of the first data (i.e., the data updated in the second database). If matching field type data is found, the system will update the corresponding data in the first database according to the update method in the second database. This is equivalent to the first database automatically matching the field type according to the updated data. After the match is successful, the first data is corrected to the first database according to the update method, thus realizing data synchronization between databases of different structures. There is no need to modify the mapping relationship between the data in the first database and the second database in advance, which can reduce development costs.

[0010] In a possible implementation manner of the first aspect, obtaining first data corresponding to the current data update and a first update manner corresponding to the first data includes:

[0011] Obtaining a first file generated by the second database, where the first file is a file generated by the second database based on encryption of the first data;

[0012] Decrypting the first file to obtain a decrypted second file;

[0013] The first data and a first updating method corresponding to the first data are obtained according to the decrypted second file.

[0014] In the embodiment of the present application, the security and integrity of the synchronization process can be guaranteed by encrypting and decrypting the data.

[0015] In a possible implementation manner of the first aspect, updating the second data according to a first update manner corresponding to the first data includes:

[0016] If the first updating mode indicates data insertion, the first data is inserted into the position corresponding to the second data.

[0017] In the embodiment of the present application, by inserting the first data into the second database, it can be ensured that the data remains consistent between different systems or databases, which is very important for maintaining the integrity and accuracy of the data.

[0018] In a possible implementation manner of the first aspect, the data synchronization method further includes:

[0019] If the first updating mode indicates data updating, the second data is replaced by the first data to update the second data.

[0020] In a possible implementation manner of the first aspect, the method further includes:

[0021] If the first update mode indicates data deletion, obtaining a first identifier corresponding to the first data;

[0022] searching the first database for a second identifier that matches the first identifier;

[0023] The data corresponding to the second identifier in the first database is deleted.

[0024] In the embodiment of the present application, by obtaining the first identifier corresponding to the first data, and then searching and deleting the data corresponding to the matching second identifier in the first database, the process is clear and unambiguous, reducing the possibility of misoperation.

[0025] In a possible implementation manner of the first aspect, the data synchronization method further includes:

[0026] When data in the first database is updated, the data table name of the third data corresponding to the data update and the synchronization mode corresponding to the third data are obtained;

[0027] Determine whether this data update needs to be synchronized to the third database based on the data table name and synchronization mode of the third data;

[0028] If the data update needs to be synchronized to the third database, the second update method corresponding to the third data is obtained;

[0029] Generate a third file according to the second update method corresponding to the third data and the third data, where the third file is a file generated by the first database based on encryption of the first data;

[0030] The third file is sent to a third database.

[0031] In an embodiment of the present application, by determining whether data synchronization is required and selecting an appropriate synchronization method, the resource utilization of the power system can be optimized and the efficiency of data processing can be improved. During the data synchronization process, the data is encrypted, which can effectively prevent the leakage of sensitive information during the transmission process and ensure the data security of the system.

[0032] In a possible implementation of the first aspect, determining whether the current data update needs to be synchronized to the third database according to the data table name and synchronization mode of the third data includes:

[0033] Obtain a preset data table, wherein the data table includes the name of the data table to be synchronized to the third database;

[0034] Query whether the preset data table includes the data table name of the third data and the synchronization mode corresponding to the third data;

[0035] If the preset data table includes the third data, the data table name and the synchronization method corresponding to the third data, it is determined that the data update needs to be synchronized to the third database.

[0036] In the embodiment of the present application, data synchronization is performed only when the preset data table includes the data table name of the third data and the corresponding synchronization method. This can avoid processing data that does not need to be synchronized and improve the efficiency of the synchronization operation.

[0037] In a second aspect, an embodiment of the present application provides a data synchronization system, including: the data synchronization system includes a plurality of databases;

[0038] Wherein, each database is used to execute the data synchronization method as described in any one of the first aspects above.

[0039] In a third aspect, an embodiment of the present application provides a terminal device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, a data synchronization method as described in any one of the first aspects above is implemented.

[0040] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it implements the data synchronization method as described in any one of the first aspects above.

[0041] In a fifth aspect, an embodiment of the present application provides a computer program product, which, when executed on a terminal device, enables the terminal device to execute any one of the data synchronization methods in the first aspect.

[0042] It can be understood that the beneficial effects of the second to fifth aspects mentioned above can be found in the relevant description of the first aspect mentioned above, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0044] Figure 1 is a schematic diagram of a data synchronization system provided in an embodiment of the present application;

[0045] Figure 2 It is a flowchart of a data synchronization method provided in an embodiment of the present application;

[0046] Figure 3 is a flowchart of obtaining first data and a first updating method provided in an embodiment of the present application;

[0047] Figure 4 is a flowchart of a data synchronization process provided by an embodiment of the present application;

[0048] Figure 5 is a schematic block diagram of a data synchronization method provided in an embodiment of the present application;

[0049] Figure 6 It is a schematic diagram of the structure of the terminal device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0050] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application may also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present application.

[0051] It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of described features, wholes, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or combinations thereof.

[0052] It should also be understood that the term “and / or” used in the specification and appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0053] As used in the specification and appended claims of this application, the term "if" can be interpreted as "when" or "uponce" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is detected" can be interpreted as meaning "uponce it is determined" or "in response to determining" or "uponce [described condition or event] is detected" or "in response to detecting [described condition or event]", depending on the context.

[0054] In addition, in the description of the present application specification and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.

[0055] References to "one embodiment" or "some embodiments" etc. described in the specification of this application mean that one or more embodiments of the present application include specific features, structures or characteristics described in conjunction with the embodiment. Therefore, the phrases "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. appearing in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways.

[0056] As 5G technology matures and artificial intelligence becomes more popular, substation systems are becoming more interoperable, and there are more and more scenarios for monitoring and operation and maintenance across security zones. According to the provisions of the "Overall Plan for the Safety Protection of the National Power Secondary System", the power system is divided into multiple security zones according to the importance of the business and the degree of impact on the primary system. Among them, Security Zone 1 is the most critical and highest-security part, responsible for real-time monitoring of the power production process. The systems within it have very high real-time requirements and usually strictly control data access; Zone 2 (non-control zone, or non-real-time zone): includes systems with low real-time requirements, such as dispatcher training simulation system, reservoir dispatching automation system, fault recording information management system, electric energy metering system, real-time and next-day power market operation system, etc.; Zone 3 (management information zone): includes a collection of power enterprise management business systems outside the production control zone, such as dispatching production management system, administrative telephone network management system, power enterprise data network management system, etc.

[0057] In the power system, achieving the synchronization of homogeneous and heterogeneous data between Zone 2 and Zone 3 means that the data in the safety Zone 2 (such as operating data, equipment status, etc.) needs to be synchronized to the database or file system in the safety Zone 3, or the data in Zone 3 needs to be synchronized to Zone 2 at the same time, while maintaining the consistency and integrity of the data. The purpose is to ensure that systems in different areas can obtain data of corresponding levels as needed, thereby improving the operational efficiency and management level of the entire power system. However, there is a high difficulty coefficient for data across regions. Therefore, in order to cope with the security isolation requirements and data exchange complexity between the security zones of the power dispatching automation system, it is particularly urgent to establish a cross-security database synchronization mechanism.

[0058] Existing related methods cannot achieve synchronization of heterogeneous databases with the same source. In the scenario of synchronizing asynchronous databases with the same source, a large amount of code needs to be modified to adapt the source database and the target database, and the mapping relationship between the fields of the target database and the source database needs to be pre-configured, resulting in high development costs.

[0059] In order to solve the problems in the above-mentioned related technologies, the embodiment of the present application provides a data synchronization method. In the present application, the data synchronization configuration process is simplified by configuring templates to reduce manual operations. The database interface adapter solves the problem of inconsistent types of heterogeneous databases, ensuring that data synchronization between different types of databases can be achieved without changing the code in homogeneous heterogeneous scenarios. The data writer submodule in the operation module combines the homology comparison algorithm with the primary key conflict strategy library to realize automatic matching and identification of database field types and operation types, without the need to pre-configure the mapping relationship. The above method greatly reduces maintenance costs and reduces development costs.

[0060] See also Figure 1, is a schematic diagram of a data synchronization system provided in an embodiment of the present application, the data synchronization system includes multiple databases (the figure shows 2 databases), and in the present application, two databases are taken as an example, namely a first database 11 and a second database 12, wherein the first database 11 can be a data storage library for a second safety zone in the power system, and the second database 12 can be a data storage library for a third safety zone, and multiple sub-modules are configured for each database system, and the sub-modules operate in coordination to achieve synchronization of homogeneous and heterogeneous data between the second safety zone and the third safety zone.

[0061] Multiple modules are configured in the first database 11 and the second database 12, including a configuration module, a monitoring module, an operation module, and a database interface adapter. The operation module is composed of multiple sub-modules, including a data collection perception engine, a data collector, a file security manager, a data writing perception engine, and a data writer. The operation module is mainly responsible for overall scheduling and coordination of the operation of each sub-module to ensure the smooth progress of the data synchronization process.

[0062] It should be noted that in the power system, synchronization of homogeneous and heterogeneous data between the safety zone 2 and the safety zone 3 is usually achieved.

[0063] See also Figure 2 , is a flowchart of a data synchronization method provided in an embodiment of the present application. As an example but not a limitation, the method may include the following steps:

[0064] S101, when it is detected that data update occurs in the second database, first data corresponding to the data update and a first update method corresponding to the first data are obtained.

[0065] In an embodiment of the present application, the first database may be a database corresponding to the second safety zone in the power system, which is used to store and manage power system operation data, and may include information about the power system status, equipment configuration, real-time monitoring data, etc. The second database may be a database corresponding to the third safety zone in the power system, which is used to store sensitive or critical data, which may include control system configuration, user authority information, etc.

[0066] When the data in the second database is updated, it may be caused by the monitoring system, maintenance operations or other management tasks in the power system. In order to update the updated data to the first database, the first database needs to determine which data in the second database has been updated, as well as the detailed information of these data, and determine how the data in the second database (first data) is updated (for example, whether to insert a new data record, modify an existing data record, or delete a data record), that is, the first update method.

[0067] In one embodiment, see Figure 3, is a flowchart of obtaining the first data and the first updating method provided in an embodiment of the present application, such as Figure 3 As shown, step S101 includes:

[0068] S201, obtaining a first file generated by a second database, where the first file is a file generated by the second database based on encryption of first data.

[0069] In an embodiment of the present application, data synchronization information between the first database and the second database is transmitted through the isolation device directory, where the isolation device directory is an information management system for storing and managing data about isolation devices.

[0070] Specifically, when the second database updates the source data, the data perception engine in the second database will send an event to the data collector. The data collector can extract the source data, i.e., the second data, through the data interface adapter, and then pass the second data and detailed information of the data through the security manager to package and encrypt them using the national encryption algorithm, and then serialize the encrypted ciphertext into the E file (first file), and finally move the first file to the isolation device sending directory of the second database for sending to the isolation device receiving directory of the first database. The data write perception engine in the first database can periodically poll and scan the isolation device receiving area directory in the first database until the first file is detected.

[0071] Among them, the above-mentioned data interface adapter supports multiple heterogeneous data sources and provides data query, deletion and update interfaces for data collectors and data writers to call. The adapter automatically loads the driver of the corresponding data source according to the type of the current security zone data source. There is no need to change the code, and it is ready to use, which greatly reduces the maintenance cost.

[0072] S202, decrypt the first file to obtain a decrypted second file.

[0073] In the embodiment of the present application, after the first database detects the first file, it uses an appropriate decryption algorithm and key to process the E file and convert its content from an encrypted state back to an original state. The purpose of the decryption process is to allow users or systems with access rights to read and process the data in the file.

[0074] Once the E-file is decrypted, a decrypted file is obtained, usually called the second file. This file contains the original first data, now in an unencrypted form and can be further processed or analyzed.

[0075] S203: Obtain first data and a first updating method corresponding to the first data according to the decrypted second file.

[0076] In the embodiment of the present application, the first data is identified and extracted from the decrypted second file. This may involve reading the file content and finding the location of the first data according to a certain data structure or mark. Once the first data is extracted, it is necessary to determine how the data is updated. This may include understanding the change history of the data, the frequency of updates, and the conditions for updates.

[0077] Exemplarily, each source record row in the first data is extracted through the second file, wherein the source record row is the update data obtained through the database interface adapter, and each source record carries the update method (first update method) of the data corresponding to the source record row. If the update method is combined with the source data in the form of an ot field, the ot field corresponding to each source record row can be identified to obtain the first update method corresponding to the first data.

[0078] In the above method, by encrypting and decrypting the data, the security and integrity of the synchronization process can be guaranteed.

[0079] S102, detecting second data in the first database that matches the field type of the first data.

[0080] In an embodiment of the present application, a specific field type in the "first data" is identified, and then a query condition is constructed to find out whether there is a field (second data) matching the first database in the first database. Finally, based on the query results, the matching data is further analyzed, operated or integrated.

[0081] Specifically, the data writer in the first database can deserialize the source record into a memory object through a homology comparison algorithm. The memory object contains multiple fields, each field contains a type and a value. The first database matches and searches for each field of the memory object according to the number and type of fields of the table cached in the memory during initialization. If found, the type conversion is performed to finally obtain a memory object that is adapted to the target data. The memory object can be updated and stored using the current database driver.

[0082] S103: If it is detected that there is second data in the first database that matches the field type of the first data, the second data is updated according to the update method corresponding to the first data.

[0083] In one embodiment, step S103 includes:

[0084] S301: If the first updating method indicates data insertion, insert the first data into the position corresponding to the second data.

[0085] In an embodiment of the present application, the data writer of the first database obtains the ot field corresponding to the first data. If ot=1, that is, the source record (first data) is an insert operation, the data writer calls the database interface adapter to insert the data. For the primary key conflict that may occur in the insert operation, the data writer will handle it according to the following three strategies: two areas cover three areas, three areas cover two areas, and merge when conflicts occur. Two areas cover three areas means that when a primary key conflict occurs, the data from the second area will cover the data from the third area; three areas cover two areas means that when a primary key conflict occurs, the data from the third area will cover the data from the second area; conflict merging means that when a primary key conflict occurs, the record will be inserted into the database table in a primary key self-incrementing manner. In this process, the database interface adapter matches and converts the database field type of the source records to ensure that the data is written to the target database according to the expected format and strategy.

[0086] For example, suppose there is state data of a power system (first data), and its corresponding primary key is state_id. Now two different regions (region 2 and region 3) try to insert data into this table, and a primary key conflict may occur. The processing method is as follows:

[0087] Zone 2 covers Zone 3:

[0088] Assume that zone 2 needs to insert a record with state_id 100 at the second data in the first database, and zone 3 also needs to insert a record with state_id 100 at the second data in the first database. According to this strategy, the data in zone 2 will overwrite the data in zone 3. In practice, this means that the record with state_id 100 in zone 3 will not be inserted into the first database, or may be deleted or modified to avoid conflicts.

[0089] Zone 3 covers Zone 2:

[0090] This strategy is the opposite of the above. Assume that zone 2 needs to insert a record with state_id 100 at the second data in the first database, and zone 3 also needs to insert a record with state_id 100 at the second data in the first database. According to this strategy, the data in zone 3 will overwrite the data in zone 3. In actual operation, this means that the record with state_id 100 in zone 2 will not be inserted into the first database, or may be deleted or modified to avoid conflicts.

[0091] Conflicted merge:

[0092] When a primary key conflict occurs, the conflict merge strategy will try to insert records in a way that the primary key is automatically incremented. For example, if a record with state_id 100 is inserted into zone 2, and zone 3 also tries to insert a record with state_id 100, the database may automatically set the state_id of the record in zone 3 to 101 (if 100 is already in use) and then insert it into the database. In this way, the data in each zone is retained and no data is overwritten or deleted.

[0093] In the above method, by inserting the first data into the second database, it can be ensured that the data remains consistent between different systems or databases, which is very important for maintaining the integrity and accuracy of the data.

[0094] In one embodiment, step S103 further includes:

[0095] If the first updating mode indicates data updating, the second data is replaced by the first data to update the second data.

[0096] In the embodiment of the present application, the first database data writer obtains the ot field in the source record. If ot=0, that is, the source record is an update operation, the data writer calls the database interface adapter to update the data. In this process, the database interface adapter matches and converts the database field type of the source record to ensure that the data is written to the target database according to the expected format and strategy.

[0097] Exemplarily, assuming that a data table name of a source record in the first data is named PowerGrid, and contains the following fields: transformer_id (transformer ID), voltage_level (voltage level), power_output (power generation) and ot (operation type), that is, a record of the first data is:

[0098] transformer_id: 200

[0099] voltage_level: 220

[0100] power_output: 500

[0101] ot: 0

[0102] This means that this is an update operation, that is, the operating data of a transformer in the power grid needs to be updated. After receiving this record, the data writer of the first database will call the database interface adapter to update the data, which can be:

[0103] The adapter obtains the data type of the second data that matches the first data in the first database. If the data structure corresponding to the second data is a floating-point structure, the data interface adapter will ensure that the data types of fields such as voltage_level and power_output are consistent with the field types in the database table. For example, it will convert the value 500 of the power_output field to a floating-point type so that the corresponding record in the database table can be correctly updated. The adapter uses the API or SQL statement provided by the database to update the record (second data) with transformer_id 200 in the PowerGr id table in the first database, that is, replace the record corresponding to the second data with a voltage level of 220 and a power generation of 500.

[0104] In one embodiment, step S103 further includes:

[0105] S401: If the first update mode indicates data deletion, obtain a first identifier corresponding to the first data.

[0106] In an embodiment of the present application, the data writer obtains the synchronization table name based on the E file name. If the synchronization table is the delete_records table, that is, the source record is a deletion operation, the data writer obtains the data table name corresponding to the source record (first data) and the first identifier corresponding to the first data (such as a unique identifier of the record such as an ID or a primary key value). This identifier is used to determine which specific data record in the first data table name needs to be deleted.

[0107] It should be noted that the data table name and the first identifier corresponding to the first data are recorded in the delete_records table, and the data table name and the primary key value (first identifier) ​​to be deleted can be extracted through the delete_records table.

[0108] S402: Searching a first database for a second identifier that matches the first identifier.

[0109] In an embodiment of the present application, the data interface adapter can quickly match the data table corresponding to the first data table name and the second identifier corresponding to the data table (the data primary key value corresponding to the first database) from the first database.

[0110] S403: Delete the data corresponding to the second identifier in the first database.

[0111] In an embodiment of the present application, a database interface adapter is called to delete the data corresponding to the second identifier in the first database.

[0112] In the above method, during the process of inserting, updating or deleting the update data, the data collection perception engine calls the data collector at a configured time (default 3 seconds) to detect whether there is any update failure record, and retries if there is any.

[0113] In one embodiment, see Figure 4 , is a flowchart of a data synchronization process provided by an embodiment of the present application, such as Figure 4 As shown, the data synchronization method also includes:

[0114] S501, when data is updated in the first database, obtain the data table name of the third data corresponding to the data update and the synchronization mode corresponding to the third data.

[0115] In an embodiment of the present application, when the data of the first database is updated (third data), the data perception engine of the first database uses the database's own trigger to work, and the trigger will record the primary key and change type of the changed row in the first database table to the update_records table and the delete_records table.

[0116] Specifically, the first database data collection perception engine calls the data collector to scan the update_records table and the delete_records table at a configured time (default 3 seconds). When the scan finds all the records in the update_records table and the delete_records table, the source table name (the name of the third data table), the data table change type, etc. are queried.

[0117] S502: Determine whether the current data update needs to be synchronized to the third database according to the data table name and synchronization mode of the third data.

[0118] In one embodiment, the implementation of step S502 includes:

[0119] S601, obtaining a preset data table, wherein the preset data table includes a name of a data table to be synchronized to a third database.

[0120] In an embodiment of the present application, the preset data table is also called a synchronization data table, which is configured through the configuration module of the first database during the initialization phase. The workspace directory, the database tables to be synchronized, the primary key conflict strategy for each table, the database type for each table, and whether to synchronize deletion operations are set using the user interface or manually modifying the configuration file, and saved as a synchronization configuration file. After the system is initialized, the configuration file will be loaded into the memory.

[0121] Specifically, when the data in the first database is updated, the data collector in the first database will call the synchronization data table (preset data table) in the memory to determine whether the updated third data needs to be synchronized to the third database, where the third database is a database other than the first database.

[0122] S602: Query whether the preset data table includes the data table name of the third data and the synchronization method corresponding to the third data.

[0123] In the embodiment of the present application, in the power system, the preset data table refers to a defined database table, such as Generators, Transformers, Lines, etc., which is used to store data of various power system equipment, and the data corresponding to these database tables is used for synchronization. If the third data updated above is the status data of a specific generator, including the power, speed, temperature, etc. of the generator, then the name of the third data table is generator. At this time, it is queried from the preset data table whether the name of the data table named generator is included, as well as the synchronization strategy of the generator data. According to the synchronization direction strategy of the source table (the data table corresponding to the third data), it is determined whether the change of the source table should be synchronized to the destination area. The synchronization strategy includes two-way synchronization and one-way synchronization.

[0124] S603: If the preset data table includes the data table name of the third data and the synchronization method corresponding to the third data, it is determined that the data update needs to be synchronized to the third database.

[0125] In an embodiment of the present application, as in the above example, if the name of the third data table such as the generator name is found to be included in the preset data table, and the query strategy corresponding to the data is found, it is determined that the third data updated this time needs to be synchronized to the third database.

[0126] In the above method, data synchronization is performed only when the preset data table includes the data table name of the third data and the corresponding synchronization mode, which can avoid processing data that does not need to be synchronized and improve the efficiency of the synchronization operation.

[0127] S503: If the data update needs to be synchronized to the third database, a second update method corresponding to the third data is obtained.

[0128] In the embodiment of the present application, if it is determined that the data is more detailed, that is, the third data needs to be synchronized to the third database, the data collector will obtain the second update method corresponding to the third data through the update_records table or the delete_records table.

[0129] Specifically, the primary key value corresponding to the third data updated this time is recorded in the update_records table by the trigger, and the specific update method corresponding to the third data (such as data insertion or data update) is obtained through the update_records table. If the primary key value corresponding to the third data updated this time is recorded in the delete_records table by the trigger, the second update method is represented by data deletion.

[0130] S504, generating a third file according to the second updating method corresponding to the third data and the third data, where the third file is a file generated by encrypting the first database according to the first data.

[0131] In an embodiment of the present application, if the primary key and other information corresponding to the third data updated this time is recorded in the update_records table by the trigger, the database interface adapter is called through the table name and primary key in the trigger record to obtain the synchronization table record, which is called the source record here, that is, the third data, and then the source record in the third data is checked for synchronization direction and the synchronization record is reordered to sort the records according to the time before and after the update.

[0132] In one implementation, the trigger record, i.e., the update_records table, contains the update method of the data update in the third data, recorded as the ot field value, and the ot field value in the trigger record is merged into the source record. The source record is grouped according to the synchronization table and transmitted to the file security manager. The file security manager will encrypt each group of source records using the national secret encryption algorithm and write them into an E file (the third file). An E file is generated for each group of source records, which can effectively avoid the generation of invalid files on the source side and reduce resource consumption. After completing all the above processes, the data collector will delete the above trigger record from the update_records table and send the generated E file to the sending directory of the source isolation device.

[0133] In another implementation method, if delete_records is scanned to have data (such as the primary key value corresponding to the third data), multiple source records corresponding to the third data in the table are extracted. They can be extracted in batches, such as 100 records per batch, and the source records are passed to the file security manager. The file security manager will encrypt the source records using the national secret algorithm and write them to an E file (the third file). This can effectively avoid the generation of invalid files on the source side and reduce resource consumption. After completing all the above processes, the data collector will delete the above source records from the delete_records table.

[0134] It should be noted that the format of the E file name is [millisecond timestamp. library type. table name.sy], for example, 1719384192635.cfg.user.sy. By generating such a time sequence file, the consistency of data in time is guaranteed. In addition, the file security manager provides encryption and decryption interfaces for data collectors and data writers, and adopts the national secret encryption algorithm to ensure the confidentiality and integrity of data during transmission and prevent data leakage or tampering.

[0135] S505: Send the third file to the third database.

[0136] In the embodiment of the present application, the first database sends the generated E file to the sending directory of the source isolation device, and the write perception engine of the third database periodically scans the isolation device receiving area directory to obtain the E file through a 3-second (configurable) periodic polling method. After obtaining the E file, the system sends the E file to the data writer through event notification and triggers the writing process.

[0137] After receiving the E file, the write executor of the third database can quickly match the field type and value of the changed data through the homology comparison algorithm and the primary key conflict strategy library, without the need to pre-configure the mapping relationship. After the data writer receives the E file reception event, it first moves the E file to the temporary directory to be written WRITE, where WRITE is a sub-directory under the workspace directory configured in the configuration file. The data writer traverses the E files under the WRITE directory, sorts them in positive order according to the file name, and then parses the E files through the file security manager to obtain the source record. The data writer is responsible for executing data deletion, update and insertion operations on the source record.

[0138] Among them, when the write executor executes the writing process, the monitoring module in the database will inject the embedding point into the write executor execution process, collect the data of each link and summarize and analyze it. It monitors the operation status of each stage in real time, and provides warnings and diagnostic information when anomalies occur to ensure the accuracy and timeliness of data synchronization. The monitoring content that can be provided includes: forward isolation transmission channel status, reverse isolation transmission channel status, the latest 100 sending records, the latest 100 receiving records, the latest 100 abnormal logs, the sending area file accumulation summary and the receiving area file accumulation summary. When a problem occurs, it can quickly locate the problem and trace the source of the problem.

[0139] See also Figure 5 , is a schematic block diagram of a data synchronization method provided in an embodiment of the present application, such as Figure 5As shown (two databases are given as an example), each database is a database corresponding to a safety zone in the power system, and the system configuration in each database is the same, including configuration module, monitoring module, operation module, database interface adapter, and the component is directly integrated and installed in the dispatching center or substation by other systems. A method and system for synchronizing homogeneous and heterogeneous databases across safety zones for power dispatching. Through the collaborative work of each module in the component, the system can realize the full process management from data collection, processing, transmission to the final writing into the target database, ensuring the efficiency, security and accuracy of the data synchronization process. It consists of the following steps:

[0140] When data in the first database is updated and the updated data is synchronized to the second database, the synchronization steps are as follows:

[0141] 1) Configuration module in the first database configures the synchronization data table

[0142] Use the user interface or manually modify the configuration file to set the workspace directory, database tables that need to be synchronized, the primary key conflict strategy for each table, the database type for each table, and whether to synchronize deletion operations, and save it as a synchronization configuration file (preset data table). After the system is initialized, the configuration file will be loaded into the memory, see the above step S601.

[0143] 2) System initialization

[0144] The data collection perception engine in the running module will create an update trigger (update_on_table name) and an insert trigger (insert_on_table name) for each table that needs to be synchronized according to the configuration file. If the on_delete field in the configuration file is true, a delete trigger (delete_on_table name) will also be created; the monitoring module will inject tracking points during the synchronous data sending and writing process, after which the system initialization is completed.

[0145] 3) Data collector collects update data

[0146] The data collection perception engine starts to periodically poll the delete_records and update_records tables. When the data in the source data table changes, the trigger inserts data into these two tables. When the data collection perception engine scans and finds data in these two tables, it sends an event to the data collector, and the data collector extracts the source records (third data). See the above step S501.

[0147] 4) Security Manager generates encrypted files

[0148] The data collector sends the third data to the file security manager, and the file security manager encrypts and packages the source records into the E file (the third file), see the above step S504.

[0149] 5) Send the encrypted file to the isolation device sending area directory

[0150] The file manager sends the encrypted file to the isolation device sending directory. After completing the above steps, the data collector will delete these trigger records from the delete_records and update_records tables, see the above step S505.

[0151] 6) The second database obtains the encrypted file

[0152] The data write perception engine scans the isolation device receiving area directory through periodic polling and finds that an E file has been generated. The data write perception engine sends the E file to the data writer through event notification and triggers the writing process.

[0153] 7) Security Manager Decryption

[0154] The security manager of the second database decrypts the received E file using the national encryption algorithm to obtain the source data record in the encrypted file and the update method corresponding to the source data record, see the above steps S201-S202.

[0155] 8) The writer writes the data to the second database

[0156] The data writer ensures that the data is written into the target database according to the expected format and strategy by calling the database interface adapter, see the above step S103.

[0157] It should be noted that the above steps are the process of synchronizing the updated data of the first database to the second database. Figure 5 The steps shown by the solid line are the same as those described above, such as the steps shown by the dotted line in FIG5 , when data update occurs in the second database and needs to be synchronized to the first database.

[0158] The data synchronization method and synchronization system provided by the present application solve the current problem of low reliability of cross-security zone database synchronization, and can significantly improve the consistency, integrity, and security of existing cross-security zone database synchronization methods. The present invention ensures the consistency of synchronized data in space and time through synchronization direction verification, synchronization record reordering, and timing file generation. The security and integrity of the synchronization process are ensured by encrypting the transmitted data content using the SM2 national secret encryption algorithm. The integrity of the synchronization process is ensured through a failed record retry mechanism, and synchronized data will not be lost due to abnormalities. After the above improvements, cross-security zone database synchronization becomes very safe and reliable.

[0159] The problem of high maintenance cost of existing cross-security zone database synchronization is solved by automatic adaptation of homogeneous heterogeneous databases, homogeneous comparison algorithms and primary key conflict strategy libraries. The present invention uses an abstract database interface layer to automatically load the underlying database driver according to the database type, so that direct synchronization of homogeneous heterogeneous databases can be achieved without changing the business code, significantly reducing the cost of database adaptation. The data writing submodule uses the homogeneous comparison algorithm and primary key conflict strategy library to accurately identify the type and value of the changed data, and there is no need to pre-set the automatic mapping relationship between the source database and the destination database, thereby reducing the cost of use.

[0160] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0161] The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.

[0162] Figure 6 Schematic diagram of the structure of the terminal device provided in the embodiment of the present application. Figure 6 As shown, the terminal device 6 of this embodiment includes: at least one processor 60 ( Figure 6 Only one is shown in the figure) a processor, a memory 61, and a computer program 62 stored in the memory 61 and executable on at least one processor 60, and when the processor 60 executes the computer program 62, the steps in any of the above-mentioned data synchronization method embodiments are implemented.

[0163] The terminal device may be a computing device such as a desktop computer, a notebook, a PDA, or a cloud server. The terminal device may include, but is not limited to, a processor and a memory. Those skilled in the art will understand that Figure 6It is only an example of the terminal device 6 and does not constitute a limitation on the terminal device 6. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, it may also include input and output devices, network access devices, etc.

[0164] The processor 60 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0165] In some embodiments, the memory 61 may be an internal storage unit of the terminal device 6, such as a hard disk or memory of the terminal device 6. In other embodiments, the memory 61 may also be an external storage device of the terminal device 6, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the terminal device 6. Further, the memory 61 may also include both an internal storage unit and an external storage device of the terminal device 6. The memory 61 is used to store an operating system, an application program, a boot loader (BootLoader), data, and other programs, such as program codes of a computer program. The memory 61 may also be used to temporarily store data that has been output or is to be output.

[0166] The embodiment of the present application further provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments can be implemented.

[0167] An embodiment of the present application provides a computer program product. When the computer program product runs on a terminal device, the terminal device can implement the steps in the above-mentioned method embodiments when executing the computer program product.

[0168] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, which can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may at least include: any entity or device that can carry the computer program code to the device / terminal device, a recording medium, a computer memory, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), an electric carrier signal, a telecommunication signal, and a software distribution medium. For example, a USB flash drive, a mobile hard disk, a magnetic disk or an optical disk. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electric carrier signals and telecommunication signals.

[0169] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0170] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0171] In the embodiments provided in the present application, it should be understood that the disclosed devices / terminal equipment and methods can be implemented in other ways. For example, the device / terminal equipment embodiments described above are only schematic, for example, the division of modules or units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0172] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0173] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. A data synchronization method, characterized in that: Applied to the first database, the method comprises: When it is detected that data update occurs in the second database, first data corresponding to the data update and a first update method corresponding to the first data are obtained; Detecting second data in the first database that matches the field type of the first data; If it is detected that the first database contains second data that matches the field type of the first data, the second data is updated according to an update method corresponding to the first data.

2. The data synchronization method according to claim 1, characterized in that: The obtaining of first data corresponding to the current data update and a first updating method corresponding to the first data includes: Acquire a first file generated by the second database, where the first file is a file generated by the second database based on encryption of the first data; Decrypting the first file to obtain a decrypted second file; The first data and a first updating method corresponding to the first data are obtained according to the decrypted second file.

3. The data synchronization method according to claim 1, characterized in that: The updating of the second data according to the first updating mode corresponding to the first data includes: If the first updating mode indicates data insertion, the first data is inserted into the position corresponding to the second data.

4. The data synchronization method according to claim 3, characterized in that: The method further comprises: If the first updating mode indicates data updating, the second data is replaced by the first data to update the second data.

5. The data synchronization method according to claim 3, characterized in that: The method further comprises: If the first update mode indicates data deletion, obtaining a first identifier corresponding to the first data; searching the first database for a second identifier that matches the first identifier; Delete the data corresponding to the second identifier in the first database.

6. The data synchronization method according to claim 1, characterized in that: The method further comprises: When data in the first database is updated, the data table name of the third data corresponding to the data update and the synchronization mode corresponding to the third data are obtained; Determine whether the current data update needs to be synchronized to the third database according to the data table name of the third data and the synchronization mode; If the data update needs to be synchronized to the third database, then the second update method corresponding to the third data is obtained; Generate a third file according to the second update mode corresponding to the third data and the third data, wherein the third file is a file generated by encrypting the first database according to the first data; The third file is sent to the third database.

7. The data synchronization method according to claim 6, characterized in that: The determining whether the current data update needs to be synchronized to the third database according to the data table name of the third data and the synchronization mode includes: Obtain a preset data table, wherein the preset data table includes a name of a data table to be synchronized to the third database; Query whether the preset data table includes the data table name of the third data and the synchronization mode corresponding to the third data; If the preset data table includes the data table name of the third data and the synchronization mode corresponding to the third data, it is determined that the data update needs to be synchronized to the third database.

8. A data synchronization system, characterized in that: The data synchronization system includes multiple databases; Wherein, each of the databases is used to execute the data synchronization method as described in any one of claims 1 to 7.

9. A terminal device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the method according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images