Historical library synchronization method and system based on power grid platform

Abstract

The invention provides a historical library synchronization method and system based on a power grid platform. The method comprises the following steps: a synchronous sending server receives sql data sent by a data change source through a message bus, and forwards the data to a write archive log thread and a TCP sending thread; the archive log writing thread stores the data into an archive log, and the TCP sending thread sends the data to a synchronous receiving server through a TCP link; the synchronous receiving server receives the data, forwards the data to a standby database writing and archiving log thread and a database interface, executes a storage operation and stores the data into an archiving log; when the synchronous receiving server side recovers from the fault state, a data state recovery message is sent to the synchronous sending server side, the synchronous sending server side retrieves unsynchronized data from the archived log and sends the unsynchronized data to the synchronous receiving server side through the TCP link, and the synchronous receiving server side stores the archived log and executes storage operation. According to the invention, the real-time performance, the reliability and the safety of the data are ensured.

Description

Technical Field

[0001] The present invention relates to the technical field of power system monitoring platform, and in particular to a historical database synchronization method and system based on a power grid platform. Background Art

[0002] At present, most of the database synchronization of power grid platforms relies on the database synchronization mechanism of the database itself, which is insufficient in economic operation. A small part relies on the platform synchronization system to operate, but the communication of platform synchronization is operated in the form of files. There are files in the synchronization process, and file reading and writing are the bottleneck of program operation; if the backup database fails, the database server of the main database will continue to generate small files. After a long time, there will be too many system files and the system will crash; the main and backup database servers will always think that database A is the main database and database B is the backup database. If database A fails, the main database will not be transferred to database B, but only database B is guaranteed to be the backup of database A. Based on the historical database synchronization method and system of the power grid platform, file reading and writing will only be stored in the archive log, and the communication depends on TCP, which has a stable long link feature. During the operation, the system will only generate archive logs according to the date, or form archive logs according to the maximum file size, and will not generate multiple small files. If there is a problem in the main database, the system can be transferred to the backup database, and the main database and backup database are submitted in real time. Summary of the invention

[0003] The purpose of this section is to summarize some aspects of embodiments of the present invention and briefly introduce some preferred embodiments. Some simplifications or omissions may be made in this section and the specification abstract and the invention title of this application to avoid blurring the purpose of this section, the specification abstract and the invention title, and such simplifications or omissions cannot be used to limit the scope of the present invention.

[0004] In view of the above existing problems, the present invention is proposed.

[0005] Therefore, the present invention provides a historical database synchronization method and system based on a power grid platform, which can solve the problems mentioned in the background technology.

[0006] In order to solve the above technical problems, the present invention provides the following technical solutions:

[0007] In a first aspect, the present invention provides a historical database synchronization method based on a power grid platform, which includes: a synchronous sending server receives sql data sent by a data change source through a message bus, and forwards the received sql data to a write archive log thread and a TCP sending thread;

[0008] The write archive log thread stores the received SQL data in the form of objects into the queue, and regularly stores the data in the queue into the archive log. At the same time, the TCP send thread stores the received SQL data in the form of objects into the queue, and regularly sends the data in the queue to the synchronous receiving server through the TCP link;

[0009] The synchronous receiving server receives the SQL data sent by the TCP sending thread through the TCP link, and forwards the received SQL data to the standby database archive log writing thread and the database interface. The database interface executes the SQL data storage operation, and the standby database archive log writing thread stores the SQL data in the archive log.

[0010] When the synchronous receiving server recovers from the fault state to the normal state, it sends a data recovery status message to the synchronous sending server. The synchronous sending server responds to the data recovery status message, retrieves the data that has not been synchronized to the synchronous receiving server from the archive log, and sends the data to the synchronous receiving server through the TCP link. The synchronous receiving server stores the received data in the archive log and performs the storage operation.

[0011] As a preferred solution of the historical database synchronization method based on the power grid platform described in the present invention, wherein: the synchronous sending service end includes a COptListen listening class, a COptRead archive log reading class, a COptRecv message receiving class, a COptWrite archive log writing class, a CSocketTcp client class and a CTransData data transmission subclass;

[0012] The COptListen listening class is used to inherit the QObject class, and has a built-in member variable CTcpServer object to complete client listening, new link establishment, and abnormal situation processing;

[0013] The COptRead archive log reading class is used to design a slot function slotReadData(). When the standby machine changes from a faulty state to a normal state, the slot function is triggered to read the logs in the local log and the standby database, and the newly added SQL statements. Each time a CTransData object is read, a read completion signal is sent after the reading is complete, so that the service returns to a normal state.

[0014] The COptRecv message receiving class is used to inherit the QObject class, rewrite the timing task, and the message bus receives the data of the db_commit submission service, the sql_sp_server direct sql service, and the model_modify modification service, saves it to the CTransData object, and passes the object to the CSocketTcp object through a signal;

[0015] The COptWrite archive log writing class is used to rewrite the scheduled task, and a slot function slotRecvData() is designed. Each time the COptRecv message receiving class sends a message, the data is stored in the queue, and the scheduled task reads the CTransData object from the queue, and the archive log is written to the disk;

[0016] The CSocketTcp client class is used to inherit the QObject class, and has a built-in member variable QTcpSocket object, which completes the client connection, data sending, data receiving and abnormal situation processing; data sending and data receiving are used to push the CTransData data transmission subclass object to the qlist queue;

[0017] The member variables of the CTransData data transmission subclass include database type, data source type, sql type, sql statement, and sqlID; the database type is set with different values ​​to represent the model library, alarm library, and history library; the data source type includes the message bus and reading from the file; the Sql type includes ordinary sql statements and bind form sql statements; SqlID is a hexadecimal integer calculated based on the timestamp.

[0018] As a preferred solution of the historical database synchronization method based on the power grid platform described in the present invention, the write archive log thread stores the received SQL data in the queue in the form of CTransData objects, and regularly stores the data in the queue into the archive log.

[0019] As a preferred solution of the historical database synchronization method based on the power grid platform described in the present invention, wherein: the TCP sending thread stores the received sql data in the form of a CTransData object into a queue, and regularly sends the data in the queue to the synchronous receiving server through a TCP link; the first byte of the communication message is EB, the second byte is 02, indicating the meaning of the function code, the third is int type data, indicating the total length of the entire message, the fourth is int type data, indicating the database type, the fifth is 8-byte int type data, indicating sqlID, and the sixth is a sql statement, indicating length = total length - all the above lengths.

[0020] As a preferred solution of the historical database synchronization method based on the power grid platform described in the present invention, wherein: the synchronization receiving service end includes COptReadState timed reading standby database status class, COptSaveData database operation storage class, COptWrite archive log writing class and CTransData class;

[0021] The COptReadState class for reading the status of the standby database at a scheduled time is used to inherit the QObject class, rewrite the scheduled task, regularly read the status information of the standby database, and write the status to the global instance;

[0022] The COptSaveData database operation storage class is used to generate three thread objects, which respectively operate on three databases, namely the model library, the alarm library, and the history library.

[0023] As a preferred solution of the historical database synchronization method based on the power grid platform described in the present invention, wherein: the synchronous receiving server receives the encrypted sql data sent by the TCP sending thread through the TCP link, uses RSA asymmetric decryption technology to decrypt it, and restores the original sql data. The decryption method adopts RSA asymmetric decryption, and the private key decrypts the data encrypted by the public key, and the decrypted sql data is stored in the archive log and the storage operation is performed.

[0024] As a preferred solution of the historical database synchronization method based on the power grid platform of the present invention, when the synchronization receiving service end recovers from the fault state to the normal state, the synchronization receiving service end performs the following steps, including:

[0025] Read the master and standby status information of the historical database, confirm that the current database is in the standby state, and create another thread to read the master and standby status of the historical database separately. When the historical database is read from the fault state to the standby state, the latest sqlID of the archive log of the current machine needs to be sent to the synchronization sending server;

[0026] Send the recovery data status message to the synchronous sending server;

[0027] After receiving the recovery data status message, the synchronous sending server matches the sqlID in the archive log and stores all updated sqlIDs in the synchronous sending server memory;

[0028] The synchronous sending server temporarily saves the SQL statements received by the message bus in the memory and no longer archives them. When all the SQL statements for updating the archive log are sent, the SQL archive log received by the message bus is sent via TCP.

[0029] The synchronous sending server retrieves the data that has not been synchronized to the synchronous receiving server from the archive log, and sends the data to the synchronous receiving server through the TCP link;

[0030] After receiving the recovery data sent from the synchronous sending server, the synchronous receiving server stores it in the archive log and performs the storage operation;

[0031] After completing the sending of the recovery data, the synchronous sending server archives the SQL statement received by the message bus and sends it to the synchronous receiving server through the TCP link to resume the normal synchronization process.

[0032] In a second aspect, the present invention provides a historical database synchronization system based on a power grid platform, which includes: a data receiving module, a data archiving module, a data synchronization module and a fault recovery module;

[0033] The data receiving module is used to synchronously send the sql data sent by the data change source through the message bus, and forward the received sql data to the archive log writing thread and the TCP sending thread;

[0034] The data archiving module is used to write the archive log thread to store the received SQL data in the queue in the form of an object, and regularly store the data in the queue in the archive log. At the same time, the TCP sending thread stores the received SQL data in the queue in the form of an object, and regularly sends the data in the queue to the synchronous receiving server through the TCP link;

[0035] The data synchronization module is used to synchronously receive the sql data sent by the TCP sending thread through the TCP link at the server end, forward the received sql data to the standby database archive log writing thread and the database interface, the database interface executes the sql data storage operation, and the standby database archive log writing thread stores the sql data in the archive log;

[0036] The fault recovery module is used to send a data recovery status message to the synchronous sending server when the synchronous receiving server recovers from a faulty state to a normal state. The synchronous sending server responds to the data recovery status message, retrieves the data that has not been synchronized to the synchronous receiving server from the archive log, and sends the data to the synchronous receiving server via the TCP link. The synchronous receiving server stores the received data in the archive log and performs a warehousing operation.

[0037] In a third aspect, the present invention provides a computer device, comprising a memory and a processor, wherein the memory stores a computer program, wherein: when the processor executes the computer program, the steps of a historical library synchronization method based on a power grid platform are implemented.

[0038] In a fourth aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein: when the computer program is executed by a processor, the steps of a historical library synchronization method based on a power grid platform are implemented.

[0039] Compared with the prior art, the present invention has the advantages of efficient data synchronization, which utilizes the stable long link characteristics of TCP to ensure the reliability and timeliness of data synchronization; optimizes storage space, generates archive logs according to date or file size, avoids generating multiple small files, and saves storage space; ensures failover, and real-time submission of the primary and standby databases. Even if there is a problem with the primary database, it can be quickly transferred to the standby database to ensure data security and availability; and simplifies file reading and writing, as file reading and writing will only be written to the disk in the archive log, which simplifies the data management process. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0041] Figure 1 A historical database synchronization architecture diagram of a historical database synchronization method and system based on a power grid platform provided by one embodiment of the present invention;

[0042] Figure 2 An internal structural diagram of a computer device of a historical database synchronization method and system based on a power grid platform provided by one embodiment of the present invention;

[0043] Figure 3 A transfer_recovery interface diagram of a historical database synchronization method and system based on a power grid platform provided by an embodiment of the present invention;

[0044] Figure 4 An archive log diagram of a historical database synchronization method and system based on a power grid platform provided by an embodiment of the present invention;

[0045] Figure 5 A message format diagram of a historical database synchronization method and system based on a power grid platform provided by one embodiment of the present invention;

[0046] Figure 6 An online data recovery diagram of a historical database synchronization method and system based on a power grid platform provided by an embodiment of the present invention;

[0047] Figure 7 An offline data recovery diagram of a history library synchronization method and system based on a power grid platform is provided in one embodiment of the present invention. DETAILED DESCRIPTION

[0048] In order to make the above-mentioned purposes, features and advantages of the present invention more understandable, the specific implementation methods of the present invention are described in detail below in conjunction with the drawings of the specification. Obviously, the described embodiments are part of the embodiments of the present invention, but not all of them. Based on the embodiments of the present invention, all other embodiments obtained by ordinary persons in the art without creative work should fall within the scope of protection of the present invention.

[0049] In the following description, many specific details are set forth to facilitate a full understanding of the present invention, but the present invention may also be implemented in other ways different from those described herein, and those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0050] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The term "in one embodiment" that appears in different places in this specification does not necessarily refer to the same embodiment, nor does it refer to a separate or selective embodiment that is mutually exclusive with other embodiments.

[0051] Example 1

[0052] Reference Figure 1-Figure 7 , which is the first embodiment of the present invention, and provides a historical database synchronization method based on a power grid platform, comprising:

[0053] This application provides a method that can effectively solve the above-mentioned problems. Next, we will combine multiple embodiments to explain in detail how to implement the historical database synchronization method based on the power grid platform;

[0054] Figure 1 A method flow chart of a historical database synchronization method and system based on a power grid platform is shown, including:

[0055] S1: The synchronous sending server receives the SQL data sent by the data change source through the message bus, and forwards the received SQL data to the archive log writing thread and the TCP sending thread;

[0056] Furthermore, the synchronous sending server transfer_out receives the sql data sent by the db_commit submission service, sql_sp_server direct sql service, and model_modify modification service through the message bus, where the sql data includes two formats: pure sql statement and bind statement. The bind statement is mainly sent by the db_commit submission service and is suitable for batch submission of data; the message receiving class COptRecv rewrites the scheduled task, and the message bus receives the data of the db_commit submission service, sql_sp_server direct sql service, and model_modify modification service, saves it to the CTransData object, and passes the object to the CSocketTcp object through a signal; through multi-source data reception, the real-time and accuracy of the data are ensured, and the flexibility and adaptability of the system are improved.

[0057] Furthermore, transfer_out forwards the received SQL data to the archive log writing thread and updates the archive log synchronously. The archive log writing thread stores the data in the queue in the form of objects and regularly stores the data in the queue into the archive log. The COptWrite archive log writing class rewrites the scheduled task and designs a slot function slotRecvData(), which realizes that every time the COptRecv message receiving class sends a message, the data can be stored in the queue. The scheduled task reads the CTransData object from the queue and the archive log is written to the disk. Through the double backup of the archive log writing thread, the reliability and recoverability of the data are ensured, and the risk of data loss is avoided.

[0058] Furthermore, transfer_out forwards the received SQL data to the TCP sending thread and completes the data sending through the TCP link. The TCP sending thread stores the data in the queue in the form of an object, and periodically sends the data in the queue through the TCP link. The CSocketTcp client class inherits the QObject class and has a built-in member variable QTcpSocket object. The CSocketTcp client class completes the client link, data sending, data receiving, and exception handling. Data sending and data receiving mainly push the CTransData data transmission simple small class object into the qlist queue, but the source data of the two are inconsistent. One is received from the message bus, and the other is the SQL read from the archived log. Real-time transmission through the TCP link ensures efficient data synchronization and improves the real-time and reliability of the system.

[0059] Furthermore, before sending the SQL statement, transfer_out uses RSA asymmetric encryption technology to encrypt the SQL statement, generate a 2048-bit RSA key pair, export the private key and public key, and call the encryption interface to encrypt the SQL statement. Through data encryption, the security and privacy of data during transmission are ensured, data is prevented from being stolen or tampered with, and the overall security of the system is improved.

[0060] S2: The write archive log thread stores the sql data received in S1 into the queue in the form of objects, and regularly stores the data in the queue into the archive log; at the same time, the TCP send thread stores the sql data received in S1 into the queue in the form of objects, and regularly sends the data in the queue to the synchronous receiving server transfer_in through the TCP link;

[0061] Furthermore, the archive log writing thread stores the received sql data into the queue in the form of CTransData objects. The CTransData object includes member variables: database type, data source type, sql type, sql statement, and sqlID. The database type is set with different values ​​to represent the model library, alarm library, and history library. The data source type includes the message bus and reading from the file. The Sql type includes ordinary sql statements and bind form sql statements. SqlID is a hexadecimal integer calculated based on the timestamp. Through queue management, efficient storage and management of data are ensured, avoiding data loss and duplication.

[0062] Furthermore, the archive log writing thread periodically reads CTransData objects from the queue and stores them in the archive log. There are two major principles for generating archive logs. One is to generate files with dates according to the system time, such as "2024-08-05". If a single file exceeds 1G, a "2024-08-05(2)" log is generated. Through regular archiving and file size management, efficient storage and management of archive logs are ensured, avoiding the generation of multiple small files and saving storage space.

[0063] Furthermore, the TCP sending thread stores the sql data received from the synchronous sending server in the queue in the form of CTransData objects. The CTransData object includes member variables: database type, data source type, sql type, sql statement, and sqlID. Through queue management, efficient data transmission and management are ensured, avoiding data loss and duplication.

[0064] Furthermore, the TCP sending thread periodically reads the CTransData object from the queue, and sends the encrypted sql data to the synchronous receiving server transfer_in through the TCP link. The first byte of the communication message is EB, the second byte is 02, indicating the meaning of the function code, the third is int type data, which is the total length of the entire message, the fourth is int type data, which is the database type, the fifth is 8-byte int type data, which is sqlID, and the sixth is the sql statement, length = total length - all the above lengths. The stable long connection characteristics of the TCP link ensure the reliability and timeliness of data transmission, and at the same time, data encryption ensures data security.

[0065] S3: The synchronous receiving server transfer_in receives the SQL data sent by the TCP sending thread in S2 through the TCP link, and forwards the received SQL data to the standby database archive log writing thread and the database interface. The database interface performs the SQL data storage operation, and the standby database archive log writing thread stores the SQL data in the archive log.

[0066] Furthermore, the synchronous receiving server transfer_in receives the encrypted sql data sent by the TCP sending thread through the TCP link. The first byte of the communication message is EB, the second byte is 02, indicating the meaning of the function code, the third is int type data, which is the total length of the entire message, the fourth is int type data, which is the database type, the fifth is 8-byte int type data, which is sqlID, and the sixth is sql statement, length = total length - all the above lengths; through the stable long connection characteristics of the TCP link, the reliability and timeliness of data transmission are ensured.

[0067] Furthermore, transfer_in uses RSA asymmetric decryption technology to decrypt the received encrypted SQL data and restore the original SQL data. The decryption method uses RSA asymmetric decryption, and the private key decrypts the data encrypted by the public key. Through data decryption, the security and privacy of the data are ensured, and the data is prevented from being stolen or tampered with during transmission.

[0068] Furthermore, transfer_in stores the decrypted sql data into the queue in the form of CTransData object. The CTransData object includes member variables: database type, data source type, sql type, sql statement, and sqlID. Through queue management, efficient data processing and management are ensured, avoiding data loss and duplication.

[0069] Furthermore, transfer_in forwards the CTransData object in the queue to the backup database archive log writing thread and database interface. The database interface provides the ExecuteCommandWithCommit interface and the BindExecuteCommandWithCommit interface respectively to ensure that both SQL files can be stored in the database through the interface. Through multiple database interface operations, efficient storage and consistency of data are ensured, improving the flexibility and reliability of the system.

[0070] Furthermore, the backup database archive log writing thread periodically reads CTransData objects from the queue and stores them in the archive log. There are two major principles for generating archive logs. One is to generate a file with a date according to the system time, such as "2024-08-05". If a single file exceeds 1G, a "2024-08-05(2)" log is generated. Through regular archiving and file size management, efficient storage and management of archive logs are ensured, avoiding the generation of multiple small files and saving storage space.

[0071] Furthermore, the database interface periodically reads CTransData objects from the queue and executes the sql data storage operation. Through the regular storage operation, the efficient processing and consistency of data are ensured, and the reliability and performance of the system are improved.

[0072] S4: When the synchronous receiving server transfer_in recovers from the fault state to the normal state, it sends a data recovery status message to the synchronous sending server transfer_out. transfer_out responds to the data recovery status message, retrieves the data that has not been synchronized to transfer_in from the archive log, and sends the data to transfer_in through the TCP link. transfer_in stores the received data in the archive log and performs the warehousing operation.

[0073] Furthermore, when the synchronous receiving server transfer_in recovers from the fault state to the normal state, it reads the active and standby state information of the historical database, confirms that the current state is the standby database, and creates another thread to read the active and standby state of the historical database separately. When it is read that the historical database changes from the fault state to the standby state, it is necessary to send the latest sqlID of the archive log of the current machine to the synchronous sending server transfer_out; COptReadState regularly reads the standby database state class inherits the QObject class, rewrites the scheduled task, regularly reads the standby database state information, and writes the state to the global instance;

[0074] transfer_in sends a data recovery status message to the synchronous sending server transfer_out. The message contains the latest sqlID of the current machine archive log. The first byte of the communication message is EB, the second byte is 01, indicating the meaning of the function code, the third is int type data, which is the total length of the entire message, the fourth is int type data, which is the active and standby status of the machine, and the fifth is 8 bytes of int type data, which is sqlID;

[0075] After receiving the recovery data status message, the synchronous sending server transfer_out matches the sqlID in the archive log and stores all the updated sqlIDs in the transfer_out memory; COptRead reads the archive log class

[0076] A slot function slotReadData() is designed. When the standby machine changes from a faulty state to a normal state, the slot function is triggered to read the logs in the local log and the standby database, and the newly added SQL statements. Each time a CTransData object is read, a read completion signal is sent after the read is complete, so that the service returns to the normal state.

[0077] transfer_out temporarily stores the SQL statements received by the message bus in memory and no longer archives them. When all SQL statements for updating the archive log are sent, the SQL archive log received by the message bus is sent via TCP.

[0078] transfer_out retrieves the data that has not been synchronized to transfer_in from the archive log, and sends the data to transfer_in through the TCP link. The first byte of the communication message is EB, the second byte is 02, indicating the meaning of the function code, the third is int type data, which is the total length of the entire message, the fourth is int type data, which is the database type, the fifth is 8-byte int type data, which is sqlID, and the sixth is the sql statement, length = total length - all the above lengths;

[0079] After transfer_in receives the restored data sent from transfer_out, it stores it in the archive log and performs the storage operation;

[0080] After transfer_out completes sending the recovery data, it archives the SQL statements received by the message bus and sends them to transfer_in through the TCP link, resuming the normal synchronization process.

[0081] Furthermore, this embodiment also provides a historical database synchronization system based on a power grid platform, including:

[0082] Data receiving module, data archiving module, data synchronization module and fault recovery module;

[0083] The data receiving module is used to synchronously send the sql data sent by the data change source through the message bus, and forward the received sql data to the archive log writing thread and TCP sending thread;

[0084] The data archiving module is used to write the archive log thread to store the received SQL data in the queue in the form of an object, and regularly store the data in the queue in the archive log. At the same time, the TCP sending thread stores the received SQL data in the queue in the form of an object, and regularly sends the data in the queue to the synchronous receiving server through the TCP link;

[0085] The data synchronization module is used to synchronously receive the SQL data sent by the TCP sending thread from the server through the TCP link, and forward the received SQL data to the standby database archive log writing thread and the database interface. The database interface executes the SQL data storage operation, and the standby database archive log writing thread stores the SQL data in the archive log.

[0086] The fault recovery module is used to send a data recovery status message to the synchronous sending server when the synchronous receiving server recovers from a faulty state to a normal state. The synchronous sending server responds to the data recovery status message, retrieves the data that has not been synchronized to the synchronous receiving server from the archive log, and sends the data to the synchronous receiving server via the TCP link. The synchronous receiving server stores the received data in the archive log and performs the warehousing operation.

[0087] This embodiment also provides a computer device, which may be a terminal, and its internal structure diagram may be as shown in FIG. Figure 2 As shown. The computer device includes a processor, a memory, a communication interface, a display screen and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be achieved through WIFI, an operator network, NFC (near field communication) or other technologies. When the computer program is executed by the processor, a historical library synchronization method based on a power grid platform is implemented. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covered on the display screen, or a button, trackball or touchpad set on the computer device housing, or an external keyboard, touchpad or mouse, etc.

[0088] This embodiment also provides a computer-readable storage medium having a computer program stored thereon, which implements the following steps when executed by a processor: a synchronous sending server receives sql data sent by a data change source via a message bus, and forwards the received sql data to a write archive log thread and a TCP sending thread;

[0089] The write archive log thread stores the received SQL data in the form of objects into the queue, and regularly stores the data in the queue into the archive log. At the same time, the TCP send thread stores the received SQL data in the form of objects into the queue, and regularly sends the data in the queue to the synchronous receiving server through the TCP link;

[0090] The synchronous receiving server receives the SQL data sent by the TCP sending thread through the TCP link, and forwards the received SQL data to the standby database archive log writing thread and the database interface. The database interface executes the SQL data storage operation, and the standby database archive log writing thread stores the SQL data in the archive log.

[0091] When the synchronous receiving server recovers from a faulty state to a normal state, it sends a data recovery status message to the synchronous sending server. The synchronous sending server responds to the data recovery status message, retrieves the data that has not yet been synchronized to the synchronous receiving server from the archive log, and sends the data to the synchronous receiving server via the TCP link. The synchronous receiving server stores the received data in the archive log and performs the warehousing operation.

[0092] Example 2

[0093] Reference Figure 3 - Figure 7 , which is the second embodiment of the present invention, and this embodiment provides a historical library synchronization method based on a power grid platform. In order to verify the beneficial effects of the present invention, scientific demonstration is carried out through economic benefit calculation and simulation experiments.

[0094] In order to verify the effectiveness and superiority of the historical database synchronization method based on the power grid platform, this embodiment is tested in detail in a simulated power grid environment. The test environment includes a synchronous sending server and a synchronous receiving server, and the two servers transmit data through a TCP link. The data change source sends SQL data through the message bus to simulate the data changes in the actual power grid platform.

[0095] Hardware environment: The synchronous sending server and the synchronous receiving server are each equipped with a high-performance server, which is configured with an Intel Xeon E5-2690 v4 processor, 64GB RAM, and 1TB SSD.

[0096] Software environment: The operating system is CentOS 7.9, the database is MySQL 8.0, the programming language is C++, and the Qt framework is used for development.

[0097] Data preparation: simulate data changes in the power grid platform and generate different types of SQL data, including insert, update and delete operations of the model library, alarm library and history library.

[0098] Data reception and forwarding: The synchronous sending server receives the SQL data sent by the data change source through the message bus. The specific steps are as follows:

[0099] Start the COptListen listening class, listen for client connection requests, and establish a TCP connection.

[0100] Use the COptRecv message receiving class to receive data from the db_commit submission service, the sql_sp_server direct sql service, and the model_modify modification service, save it to the CTransData object, and pass the object to the CSocketTcp object through a signal.

[0101] Forward the received SQL data to the archive log writing thread and the TCP sending thread.

[0102] Data archiving and sending: The archive log writing thread stores the received SQL data in the queue as an object, and regularly stores the data in the queue in the archive log. The TCP sending thread stores the received SQL data in the queue as an object, and regularly sends the data in the queue to the synchronous receiving server through the TCP link.

[0103] Data synchronization and storage: The synchronous receiving server receives the SQL data sent by the TCP sending thread through the TCP link, and forwards the received SQL data to the standby database archive log writing thread and the database interface. The database interface executes the storage operation of the SQL data, and the standby database archive log writing thread stores the SQL data in the archive log.

[0104] Simulate the synchronous receiving server to recover from the fault state to the normal state. The specific steps are as follows:

[0105] Read the master and backup status information of the historical database to confirm that the current database is in the backup state.

[0106] Create another thread to read the active and standby status of the historical database separately. When it is read that the historical database changes from a faulty state to a standby state, send the latest sqlID of the archive log of the current machine to the synchronization sending server.

[0107] After receiving the recovery data status message, the synchronous sending server matches the sqlID in the archive log and stores all updated sqlIDs in the synchronous sending server memory.

[0108] The synchronous sending server temporarily stores the SQL statements received by the message bus in the memory and no longer archives them. When all the SQL statements for updating the archive log are sent, the SQL archive log received by the message bus is sent via TCP.

[0109] The synchronous sending server retrieves the data that has not been synchronized to the synchronous receiving server from the archive log, and sends the data to the synchronous receiving server through the TCP link.

[0110] After receiving the recovery data sent from the synchronous sending server, the synchronous receiving server stores it in the archive log and performs the storage operation.

[0111] After completing the sending of the recovery data, the synchronous sending server archives the SQL statement received by the message bus and sends it to the synchronous receiving server through the TCP link to resume the normal synchronization process.

[0112] Table 1: Data transmission performance table of synchronous sending server and synchronous receiving server

[0113]

[0114] Table 2: Fault recovery performance test table

[0115]

[0116]

[0117] Through the data in the above two tables, we can clearly see the performance and advantages of the historical database synchronization method based on the power grid platform in practical applications.

[0118] As the amount of data increases, the average transmission time increases linearly, indicating that the method of the present invention has good scalability and stability when processing large-scale data. For example, it increases from 2.5 seconds for 1,000 data items to 242.8 seconds for 100,000 data items, with a steady growth trend.

[0119] Even in the case of a large amount of data transmission, the success rate is still maintained above 98.7%, indicating that the method of the present invention performs well in terms of data transmission reliability.

[0120] The data loss rate is between 0.2% and 0.7%, which is much lower than the data loss rate in the prior art, thanks to the dual-channel data processing mechanism and the double backup of the data archive log of the present invention.

[0121] As the amount of data increases, the size of the archive log also increases accordingly, but the increase is reasonable and does not cause excessive pressure on storage resources.

[0122] Memory usage increases as the amount of data increases, but within a reasonable range and will not cause system crashes or performance degradation.

[0123] The fault recovery time increases with the increase of fault duration, but the recovery time is relatively short, indicating that the method of the present invention is more efficient in fault recovery. For example, even if the fault lasts for 1200 seconds, the recovery time is only 200 seconds.

[0124] The amount of unsynchronized data increases with the duration of the fault, but the amount is controllable, indicating that the method of the present invention can effectively reduce data loss during the fault period.

[0125] The synchronization success rate is above 99.4%, indicating that the method of the present invention has high reliability in restoring data synchronization. The data loss rate after recovery is between 0.1% and 0.6%, which is much lower than the data loss rate in the prior art, thanks to the archive log mechanism and data encryption technology of the present invention. The memory usage after recovery is within a reasonable range and will not have a negative impact on system performance.

[0126] In summary, the historical database synchronization method based on the power grid platform of the present invention shows significant advantages in data transmission performance and fault recovery performance. Compared with the prior art, the method of the present invention not only improves the efficiency and reliability of data synchronization, but also enhances the security and flexibility of the system, achieving unique and irreplaceable beneficial effects. In particular, in terms of processing large-scale data and fault recovery, the method of the present invention shows obvious creativity and novelty.

[0127] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.

[0128] Those skilled in the art will appreciate that the embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the present application can adopt the form of complete hardware embodiments, complete software embodiments, or embodiments in combination with software and hardware. Moreover, the present application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code. The scheme in the embodiments of the present application can be implemented in various computer languages, for example, object-oriented programming language Java and literal scripting language JavaScript, etc.

[0129] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0130] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0131] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0132] Although the preferred embodiments of the present application have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications falling within the scope of the present application.

[0133] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. A historical database synchronization method based on a power grid platform, characterized in that: Including, the synchronous sending server receives the sql data sent by the data change source through the message bus, and forwards the received sql data to the archive log writing thread and the TCP sending thread; The write archive log thread stores the received SQL data in the form of objects into the queue, and regularly stores the data in the queue into the archive log. At the same time, the TCP send thread stores the received SQL data in the form of objects into the queue, and regularly sends the data in the queue to the synchronous receiving server through the TCP link; The synchronous receiving server receives the SQL data sent by the TCP sending thread through the TCP link, and forwards the received SQL data to the standby database archive log writing thread and the database interface. The database interface executes the SQL data storage operation, and the standby database archive log writing thread stores the SQL data in the archive log. When the synchronous receiving server recovers from a faulty state to a normal state, it sends a data recovery status message to the synchronous sending server. The synchronous sending server responds to the data recovery status message, retrieves the data that has not yet been synchronized to the synchronous receiving server from the archive log, and sends the data to the synchronous receiving server via the TCP link. The synchronous receiving server stores the received data in the archive log and performs the warehousing operation.

2. The historical database synchronization method based on the power grid platform according to claim 1, characterized in that: The synchronous sending service end includes a COptListen listening class, a COptRead archive log reading class, a COptRecv message receiving class, a COptWrite archive log writing class, a CSocketTcp client class, and a CTransData data transmission subclass; The COptListen listening class is used to inherit the QObject class, and has a built-in member variable CTcpServer object to complete client listening, new link establishment, and abnormal situation processing; The COptRead archive log reading class is used to design a slot function slotReadData(). When the standby machine changes from a faulty state to a normal state, the slot function is triggered to read the logs in the local log and the standby database, and the newly added SQL statements. Each time a CTransData object is read, a read completion signal is sent after the reading is complete, so that the service returns to a normal state. The COptRecv message receiving class is used to inherit the QObject class, rewrite the timing task, and the message bus receives the data of the db_commit submission service, the sql_sp_server direct sql service, and the model_modify modification service, saves it to the CTransData object, and passes the object to the CSocketTcp object through a signal; The COptWrite archive log writing class is used to rewrite the scheduled task, and a slot function slotRecvData() is designed. Each time the COptRecv message receiving class sends a message, the data is stored in the queue, and the scheduled task reads the CTransData object from the queue, and the archive log is written to the disk; The CSocketTcp client class is used to inherit the QObject class, and has a built-in member variable QTcpSocket object, which completes the client connection, data sending, data receiving and abnormal situation processing; data sending and data receiving are used to push the CTransData data transmission subclass object to the qlist queue; The member variables of the CTransData data transmission subclass include database type, data source type, sql type, sql statement, and sqlID; the database type is set with different values ​​to represent the model library, alarm library, and history library; the data source type includes the message bus and reading from the file; the Sql type includes ordinary sql statements and bind form sql statements; SqlID is a hexadecimal integer calculated based on the timestamp.

3. The historical database synchronization method based on the power grid platform according to claim 2, characterized in that: The write archive log thread stores the received SQL data in the form of CTransData objects into a queue, and periodically stores the data in the queue into an archive log.

4. The historical database synchronization method based on the power grid platform according to claim 3, characterized in that: The TCP sending thread stores the received sql data in the form of CTransData objects in the queue, and regularly sends the data in the queue to the synchronous receiving server through the TCP link; the first byte of the communication message is EB, the second byte is 02, indicating the meaning of the function code, the third is int type data, indicating the total length of the entire message, the fourth is int type data, indicating the database type, the fifth is 8-byte int type data, indicating sqlID, and the sixth is an sql statement, indicating length = total length - all the above lengths.

5. The historical database synchronization method based on the power grid platform according to claim 4, characterized in that: The synchronous receiving server includes COptReadState class for timing reading of standby database status, COptSaveData class for database operation storage, COptWrite class for writing archive logs, and CTransData class; The COptReadState class for reading the status of the standby database at a scheduled time is used to inherit the QObject class, rewrite the scheduled task, regularly read the status information of the standby database, and write the status to the global instance; The COptSaveData database operation storage class is used to generate three thread objects, which respectively operate on three databases, namely the model library, the alarm library, and the history library.

6. The historical database synchronization method based on the power grid platform according to claim 5, characterized in that: The synchronous receiving server receives the encrypted SQL data sent by the TCP sending thread through the TCP link, decrypts it using RSA asymmetric decryption technology, and restores the original SQL data. The decryption method adopts RSA asymmetric decryption, and the private key decrypts the data encrypted by the public key, and stores the decrypted SQL data in the archive log and executes the warehousing operation.

7. The historical database synchronization method based on the power grid platform according to claim 6, characterized in that: When the synchronous receiving server recovers from the fault state to the normal state, the synchronous receiving server performs the following steps: include, Read the master and standby status information of the historical database, confirm that the current database is in the standby state, and create another thread to read the master and standby status of the historical database separately. When the historical database is read from the fault state to the standby state, the latest sqlID of the archive log of the current machine needs to be sent to the synchronization sending server; Send the recovery data status message to the synchronous sending server; After receiving the recovery data status message, the synchronous sending server matches the sqlID in the archive log and stores all updated sqlIDs in the synchronous sending server memory; The synchronous sending server temporarily saves the SQL statements received by the message bus in the memory and no longer archives them. When all the SQL statements for updating the archive log are sent, the SQL archive log received by the message bus is sent via TCP. The synchronous sending server retrieves the data that has not been synchronized to the synchronous receiving server from the archive log, and sends the data to the synchronous receiving server through the TCP link; After receiving the recovery data sent from the synchronous sending server, the synchronous receiving server stores it in the archive log and performs the storage operation; After completing the sending of the recovery data, the synchronous sending server archives the SQL statement received by the message bus and sends it to the synchronous receiving server through the TCP link to resume the normal synchronization process.

8. A historical database synchronization system based on a power grid platform, based on the historical database synchronization method based on a power grid platform according to any one of claims 1 to 7, characterized in that: Including, data receiving module, data archiving module, data synchronization module and fault recovery module; The data receiving module is used to synchronously send the sql data sent by the data change source through the message bus, and forward the received sql data to the archive log writing thread and the TCP sending thread; The data archiving module is used to write the archive log thread to store the received SQL data in the queue in the form of an object, and regularly store the data in the queue in the archive log. At the same time, the TCP sending thread stores the received SQL data in the queue in the form of an object, and regularly sends the data in the queue to the synchronous receiving server through the TCP link; The data synchronization module is used to synchronously receive the sql data sent by the TCP sending thread through the TCP link at the server end, forward the received sql data to the standby database archive log writing thread and the database interface, the database interface executes the sql data storage operation, and the standby database archive log writing thread stores the sql data in the archive log; The fault recovery module is used to send a data recovery status message to the synchronous sending server when the synchronous receiving server recovers from a faulty state to a normal state. The synchronous sending server responds to the data recovery status message, retrieves the data that has not been synchronized to the synchronous receiving server from the archive log, and sends the data to the synchronous receiving server via the TCP link. The synchronous receiving server stores the received data in the archive log and performs a warehousing operation.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the historical database synchronization method based on the power grid platform are implemented in any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the historical database synchronization method based on the power grid platform according to any one of claims 1 to 7 are implemented.

Images