Data processing method, device and equipment based on PostgreSQL database and medium

Abstract

The application discloses a data processing method and device based on a PostgreSQL database, equipment and a medium, and relates to the technical field of PG databases. The method comprises the following steps: creating a preset data processing object in a to-be-processed PostgreSQL database to obtain a target database; the preset data processing object is used for collecting, analyzing and recording data definition languages in the database and is used for specifying a specified library that needs to be subjected to data processing; when it is monitored that a data manipulation language exists and is used for operating the target database, incremental change data generated by the operation is published, and a backup tool deployed on a target equipment is used for backing up the incremental change data according to a preset backup strategy to obtain a backup database; and the target database is recovered by a recovery tool deployed on the target equipment based on the backup database to obtain a recovery database. Through the technical scheme, remote data backup of the specified database can be safely and efficiently performed.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of PG database, and particularly relates to a data processing method and device based on a PostgreSQL database, equipment and medium. BACKGROUND

[0002] At present, with the rapid development of Internet user business, its data center network and system are increasingly large. Under the circumstances of increasing types of business and increasing business traffic, in order to ensure the safe, stable and controllable operation of various systems of users, for each application system, user data is the most core and important wealth of an enterprise, but due to various irresistible forces, it may cause irreversible damage to data. Only a perfect, efficient and flexible backup and recovery scheme can ultimately guarantee data security and business continuity.

[0003] PostgreSQL (PG for short) is a powerful open source object relational database system, which uses and extends the SQL language and combines many secure storage and extended most complex data workload functions. PostgreSQL has won a good reputation because of its proven architecture, reliability, data integrity, powerful feature set, scalability and the open source community behind the software that is committed to consistently providing high performance and innovative solutions. PostgreSQL runs on all mainstream operating systems and has powerful additional components, such as the popular PostGIS geospatial database extension.

[0004] In the same database system, according to business scenarios and business isolation and other factors, there are generally multiple databases of different businesses. Different business databases can set different backup and recovery strategies according to the importance of their data and the requirements of business continuity. The traditional PostgreSQL database incremental backup and recovery method only supports local backup of the database, and the backup data is large, which cannot backup specific databases according to business needs, resulting in a large amount of waste of data storage space. At the same time, the incremental backup data is stored in a physical replication manner, which cannot realize backup and recovery when different PostgreSQL database versions and different database types such as MySQL are crossed, and has many limitations. For PostgreSQL database, only the records corresponding to the data manipulation language (DML) are distributed to the replication slot in logical replication, and the operations corresponding to the data definition language (DDL) are not distributed. This results in that the traditional data backup tool cannot directly support the DDL incremental data synchronization of the PostgreSQL database.

[0005] In summary, how to flexibly specify the backup strategy of different databases, perform remote data backup on the specified database safely and efficiently, and ensure data integrity and security are problems to be solved at present. SUMMARY

[0006] Therefore, the purpose of the present application is to provide a data processing method, device and equipment based on PostgreSQL database and medium, which can flexibly specify the backup strategy of different databases, perform remote data backup on the specified database safely and efficiently, and ensure data integrity and security. The specific scheme is as follows:

[0007] In a first aspect, the present application discloses a data processing method based on PostgreSQL database, comprising:

[0008] determining a to-be-processed PostgreSQL database, and creating a preset data processing object in the to-be-processed PostgreSQL database to obtain a target database; wherein the preset data processing object is used to collect, analyze and record the data definition language in the to-be-processed PostgreSQL database, and the preset data processing object is used to specify the specified library in the to-be-processed PostgreSQL database that needs to be processed;

[0009] when it is monitored that there is a data manipulation language to operate the target database, publishing the incremental change data generated in the target database, and based on a preset backup strategy, backing up the incremental change data through a backup tool deployed on the target device to obtain a backup database;

[0010] based on the backup database, recovering the target database through a recovery tool deployed on the target device to obtain a recovery database.

[0011] Optionally, the step of creating a preset data processing object in the to-be-processed PostgreSQL database to obtain a target database comprises:

[0012] creating an operation record table for recording the data definition language in the to-be-processed PostgreSQL database, creating a statement capture function for capturing the data definition language and an event trigger for capturing the execution completion of the statement capture function, creating a logical slot for copying the data of the specified library, and creating user information for accessing the logical slot to obtain a target database.

[0013] Optionally, the step of publishing the incremental change data generated in the target database when it is monitored that there is a data manipulation language to operate the target database comprises:

[0014] continuously invoke the statement capture function based on the event trigger to capture the data definition language;

[0015] insert the captured data information of the data definition language into the operation record table, and count the number of records in the operation record table, so as to update the operation record table according to the number of records by using a preset update rule to obtain a current operation record table;

[0016] When it is monitored that the data manipulation language is used to operate on the current operation record table, the prewrite log is parsed by the mounted logical decoding plug-in through the logical slot;

[0017] The parsed data manipulation language in the prewrite log is filtered, and then the data manipulation language is published.

[0018] Optionally, the backup database obtained based on the pre-device backup strategy is obtained by the backup tool deployed on the target device according to the incremental change data, and the backup database comprises:

[0019] The backup period, data compression option and data encryption option of the target database are set based on the pre-device backup strategy, and the backup period comprises a full backup period and an incremental backup period;

[0020] The data backup instruction is called to remotely backup the data of the target database, and the incremental change data is cyclically obtained by the backup tool deployed on the target device according to the full backup period;

[0021] It is judged whether the operation table using the data manipulation language to operate on the target database is the operation record table in the target database, if not, the data manipulation language is directly appended to the incremental change data, if yes, it is judged whether the data manipulation language is to delete the record of the operation record table;

[0022] If the data manipulation language is to delete the record of the operation record table, the data manipulation language is ignored, if the data manipulation language is not to delete the record of the operation record table, the data definition language contained in the data manipulation language is extracted, and then the data definition language is appended to the incremental change data;

[0023] The incremental change data is stream-compressed according to the data compression option, and then the backup compressed file obtained after compression is stored by encryption according to the data encryption option, so as to obtain the backup database by using the backup compressed file stored by encryption.

[0024] Optionally, in the process of cyclically obtaining the incremental change data by the backup tool deployed on the target device according to the full backup period, the process further comprises:

[0025] establishing a first incremental data backup file according to the incremental change data, and monitoring a backup duration of the first incremental data backup file;

[0026] when the backup duration reaches the incremental backup period, closing the first incremental data backup file by the backup tool, and creating a second incremental data backup file based on the incremental change data;

[0027] compressing the first incremental data backup file, and cleaning up the first incremental data backup file when a storage time limit of the first incremental data backup file reaches a preset threshold.

[0028] Optionally, the restoring the target database based on the backup database by the recovery tool deployed on the target device to obtain a restored database comprises:

[0029] checking connectivity with the target database by the recovery tool deployed on the target device, and collecting first basic information of the backup database;

[0030] comparing the first basic information with second basic information of a preset database pre-constructed on the target device, and judging whether a comparison result satisfies a preset restoration condition;

[0031] if the comparison result satisfies the preset restoration condition, restoring the target database to obtain a restored database;

[0032] if the comparison result does not satisfy the preset restoration condition, returning incompatible error information of the backup database and the preset database.

[0033] Optionally, the restoring the target database based on the backup database by the recovery tool deployed on the target device to obtain a restored database comprises:

[0034] obtaining a backup directory of the backup database by the recovery tool deployed on the target device according to the backup database;

[0035] filtering out a backup file closest to a time of the timestamp according to the timestamp in the backup directory;

[0036] judging whether the preset database already exists on the target device;

[0037] if the preset database already exists on the target device, directly covering data in the preset database with data in the backup file in time sequence to obtain a restored database;

[0038] If the preset database does not exist on the target device, the preset database is created, and data in the backup file is sequentially overwritten on data in the preset database in chronological order to obtain a restored database.

[0039] In a second aspect, the present application discloses a data processing device based on a PostgreSQL database, comprising:

[0040] A preset data processing object creation module is configured to determine a PostgreSQL database to be processed, and create a preset data processing object in the PostgreSQL database to be processed to obtain a target database; wherein the preset data processing object is configured to collect, analyze and record data definition language in the PostgreSQL database to be processed, and the preset data processing object is configured to specify a specified library in the PostgreSQL database to be processed that needs data processing;

[0041] A database backup module is configured to publish incremental change data generated in the target database when it is monitored that there is data manipulation language to operate on the target database, and backup the incremental change data based on a preset backup strategy through a backup tool deployed on the target device to obtain a backup database;

[0042] A database recovery module is configured to recover the target database based on the backup database through a recovery tool deployed on the target device to obtain a restored database.

[0043] In a third aspect, the present application discloses an electronic device, comprising a processor and a memory; wherein the memory is configured to store a computer program, and the computer program is loaded and executed by the processor to implement the data processing method based on the PostgreSQL database as described above.

[0044] In a fourth aspect, the present application discloses a computer readable storage medium for storing a computer program; wherein the computer program is executed by a processor to implement the data processing method based on the PostgreSQL database as described above.

[0045] In the present application, firstly, a to-be-processed PostgreSQL database is determined, and a preset data processing object is created in the to-be-processed PostgreSQL database to obtain a target database; wherein the preset data processing object is used to collect, analyze and record the data definition language in the to-be-processed PostgreSQL database, and the preset data processing object is used to specify the specified library in the to-be-processed PostgreSQL database that needs to be processed; when it is monitored that there is a data manipulation language to operate the target database, the incremental change data generated in the target database is published, and based on the backup strategy, the backup tool deployed on the target device is used to backup according to the incremental change data to obtain a backup database; the target database is recovered based on the backup database through the recovery tool deployed on the target device to obtain a recovery database. It can be seen that the preset data processing object is created in the to-be-processed PostgreSQL database, which realizes the collection, analysis and record support of the data definition language, supports the backup of the data definition language such as the database table structure change, has no limitation of only supporting the data manipulation language backup, has strong practicability and innovation. At the same time, the method can flexibly specify a specific database for backup according to business needs, and different databases can set different backup strategies, compared with the existing database backup method, the backup data storage space occupation is small, the backup data collection and storage safety is high, effectively solves the problems of large backup storage space occupation and many irrelevant data in the traditional backup. Secondly, the incremental backup method is used to store the backup data, which can safely and efficiently collect the incremental change data of the data and store it in the storage device, greatly reducing the storage space occupation of the backup data. The backup and recovery of the target database are realized through the backup tool and the recovery tool of the target device, which realizes the support of remote access, and the method can be compatible with different versions of PostgreSQL, and can be conveniently recovered to other types of relational databases. BRIEF DESCRIPTION OF DRAWINGS

[0046] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or the prior art description. Obviously, the drawings in the following description are only embodiments of the present application, and for those skilled in the art, other drawings can be obtained without creative labor on the basis of the provided drawings.

[0047] Figure 1 A data processing method flow chart based on a PostgreSQL database is disclosed in the present application.

[0048] Figure 2A specific PostgreSQL database-based data processing method flowchart disclosed in the present application;

[0049] Figure 3 A log submission schematic diagram disclosed in the present application;

[0050] Figure 4 A incremental backup data publishing flowchart disclosed in the present application;

[0051] Figure 5 A specific PostgreSQL database-based data processing method flowchart disclosed in the present application;

[0052] Figure 6 A incremental backup data receiving flowchart disclosed in the present application;

[0053] Figure 7 A specific PostgreSQL database-based data processing method flowchart disclosed in the present application;

[0054] Figure 8 A incremental backup data recovery flowchart disclosed in the present application;

[0055] Figure 9 A PostgreSQL database-based data processing device structure schematic diagram disclosed in the present application;

[0056] Figure 10 An electronic device structure diagram disclosed in the present application. DETAILED DESCRIPTION

[0057] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0058] Currently, in the same database system, according to business scenarios and business isolation and other factors, there are generally multiple databases of different businesses. Different business databases can set different backup and recovery strategies according to the importance of their data and the requirements of business continuity. The physical replication method has many limitations and cannot realize cross-version backup and recovery. The logical replication method causes the traditional data backup tool to be unable to directly support DDL incremental data synchronization of the PostgreSQL database.

[0059] To this end, the application provides a data processing scheme based on a PostgreSQL database, which can flexibly specify backup strategies of different databases, perform remote data backup on the specified databases safely and efficiently, and ensure data integrity and security.

[0060] The embodiment of the application discloses a data processing method based on a PostgreSQL database, as shown in the figure, the method comprises the steps of: Figure 1

[0061] Step S11: determining a PostgreSQL database to be processed, and creating a preset data processing object in the PostgreSQL database to be processed to obtain a target database; wherein the preset data processing object is used for collecting, analyzing and recording data definition languages in the PostgreSQL database to be processed, and the preset data processing object is used for specifying a specified library in the PostgreSQL database to be processed that needs data processing.

[0062] It can be understood that, since physical replication requires complete consistency of master and standby block levels when performing data backup, there are some application scenarios that cannot be covered, and when different PostgreSQL database versions and different database types (such as MySQL) are crossed, backup and recovery cannot be realized, and there are many limitations. Therefore, in the embodiment of the application, the method of logical replication is used for data collection, which can conveniently restore the backup data to different versions of PostgreSQL, and can also conveniently restore to other types of relational databases (such as MySQL).

[0063] In the embodiment of the application, the PostgreSQL database to be processed is first determined, and database backup or data recovery is performed on the PostgreSQL database to be processed. Since the method of logical replication is used for data collection, in order to overcome the problem that traditional data backup tools cannot directly support DDL incremental data synchronization of the PostgreSQL database due to the fact that only data manipulation language (DML) corresponding records are distributed to the replication slot in logical replication, and the data definition language (DDL) corresponding operations are not distributed, a preset data processing object is created in the PostgreSQL database to be processed.

[0064] Specifically, an operation record table for recording the data definition language is created in the PostgreSQL database to be processed, a statement capture function for capturing the data definition language and an event trigger for capturing execution completion of the statement capture function are created, a logical slot for replicating data of the specified library is created, and user information for accessing the logical slot is created, to obtain the target database. ​

[0065] It can be understood that first, the user creates a DDL operation record table in the to-be-processed PostgreSQL database, which is used to record the DDL statement generated in the current database, mainly including fields such as execution time, SQL statement, search path, etc. Second, the user creates a DDL statement capture function in the to-be-processed PostgreSQL database, and creates an event trigger, which is used to realize the function of capturing the DDL statement executed in the current database, mainly including obtaining the DDL search path, inserting the DDL statement information into the DDL operation record table, etc. The trigger is used to capture the message of the completion of the DDL statement execution, and to call the DDL statement capture function after capturing. Finally, the user creates a logical slot for copying data in the to-be-processed PostgreSQL database, which can specify a specific database to be backed up, and at the same time, user information for accessing the logical slot is created. In this way, the flexible customization requirement of the user to back up only the specified key database can be met, the influence on other irrelevant databases is reduced, and the occupation of backup storage space is greatly saved. The logical decoding plug-in loaded by the logical slot parses the WAL log (Write Ahead Log, pre-write log), and filters the insert, update, delete data and other modification statements in the WAL log.

[0066] Step S12: When it is monitored that there is a data manipulation language operating on the target database, the incremental change data generated in the target database is published, and based on the pre- backup strategy, the backup tool deployed on the target device is used to backup according to the incremental change data, to obtain a backup database.

[0067] In the embodiment of the application, when the DML statement operating on the target database is executed, the target database is taken as a publishing end of the incremental data, and the logical decoding plug-in mounted by the logical slot of the target database is used to distribute the parsed DML statement, and the backup tool is taken as a subscribing end to receive the incremental change data. Therefore, remote access to the database system to be backed up is supported, and the storage space occupation of the database system running machine is avoided. Moreover, the incremental backup mode is used to store the backup data, which greatly reduces the storage space occupation of the backup data.

[0068] In the embodiments of the present application, since the DDL operation record table is created in the target database, when the incremental change data generated in the target database is published, the corresponding operation of DDL is also published. Specifically, when the DDL statement for operating the database is executed, the event trigger is triggered after the execution of the DDL statement is completed, the DDL statement capture function is called through the event trigger, and the DDL statement is inserted into the DDL operation record table. Further, when it is monitored that there is a DML modification to the DDL operation record table, the DML modification of the table is distributed to the subscription end of the backup tool. Therefore, the limitation that the backup tool cannot support the DDL incremental data synchronization of the PostgreSQL database in the prior art is solved.

[0069] Further, the user configures the connection information of the database to be backed up on the remote machine where the backup tool is deployed, including the IP address, the port number, the login user information, and the like, and starts the backup tool. The backup tool runs as a resident process and actively connects to the logical replication slot of the target database as a subscription end of the incremental backup data, and obtains the backup database after the backup is completed. It should be noted that the incremental backup strategy of different databases can be flexibly configured, including the backup period, compression, encryption, expired data cleaning, which will be specifically described in subsequent embodiments.

[0070] Step S13: restoring the target database based on the backup database through the recovery tool deployed on the target device to obtain a recovery database.

[0071] In the embodiments of the present application, when the user needs to perform a recovery operation according to the incremental backup data, the obtained recovery database can be a cross-version compatible PostgreSQL database, or a cross-type compatible SQL syntax other relational database. In this way, the backup data is used to support multiple recovery scenarios, which supports recovery into a cross-version PostgreSQL database, and supports other types of relational databases compatible with the SQL syntax, and has wide versatility.

[0072] Exemplary, in the execution of the recovery command, the user login information, database type and data coverage options of the target database need to be transmitted, and the backup database name and the recovery time point need to be transmitted, and if the data is encrypted, the optional data decryption private key can also be transmitted. When data recovery is performed, the recovery tool of the target device first performs connectivity checking on the database to be recovered according to the login information, collects version information and SQL compatibility information of the database to be recovered, compares the collected information with the information carried by the target database, and starts the recovery process when the pre-checking of the recovery tool is passed. After the recovery is completed, the recovered database is obtained. Compared with the prior art, a more flexible and reliable, more economical and safe, and more efficient method is provided for the incremental backup and recovery method of the specified database.

[0073] In the present application, a to-be-processed PostgreSQL database is first determined, and a preset data processing object is created in the to-be-processed PostgreSQL database to obtain a target database; wherein the preset data processing object is used to collect, analyze and record the data definition language in the to-be-processed PostgreSQL database, and the preset data processing object is used to specify the specified library in the to-be-processed PostgreSQL database that needs data processing; when it is monitored that there is a data manipulation language operating on the target database, the incremental change data generated in the target database is published, and based on a pre-backup strategy, a backup tool deployed on a target device performs backup according to the incremental change data to obtain a backup database; the target database is recovered by a recovery tool deployed on the target device based on the backup database to obtain a recovery database. It can be seen that the preset data processing object is created in the to-be-processed PostgreSQL database, which realizes the collection, analysis and record support of the data definition language, supports the backup of the data definition language such as database table structure change, and has no limitation of only supporting data manipulation language backup, has strong practicability and innovation. At the same time, the method can flexibly specify a specific database for backup according to business needs, and different databases can set different backup strategies. Compared with the existing database backup method, the backup data storage space occupation is small, the backup data collection and storage security is high, and the problem of large traditional backup storage space occupation and many irrelevant data is effectively solved. Secondly, the incremental backup method is used to store backup data, which can safely and efficiently collect incremental change data of the data and store it in the storage device, greatly reducing the storage space occupation of the backup data. The backup and recovery of the target database are realized by the backup tool and the recovery tool of the target device, which realizes the support of remote access, and the method can be compatible with different versions of PostgreSQL, and can be conveniently recovered to other types of relational databases.

[0074] The embodiment of the application discloses a specific data processing method based on a PostgreSQL database, referring to Figure 2 The method comprises the following steps:

[0075] Step S21: determining a PostgreSQL database to be processed, and creating a preset data processing object in the PostgreSQL database to be processed to obtain a target database; wherein the preset data processing object is used for collecting, analyzing and recording a data definition language in the PostgreSQL database to be processed, and the preset data processing object is used for specifying a specified library in the PostgreSQL database to be processed which needs to be processed.

[0076] The more specific processing process of the above step S21 can refer to the corresponding content disclosed in the foregoing embodiments, and will not be described here.

[0077] Step S22: continuously calling the statement capture function based on the event trigger to capture the data definition language.

[0078] In the embodiment of the application, when a DDL statement for operating a target database is executed, an event trigger created in the target database will be triggered after the execution of the DDL statement is completed, the event trigger calls its associated DDL capture function, the DDL capture function collects necessary information of the DDL statement, such as a search path, and inserts the DDL statement into a DDL operation record table.

[0079] Step S23: inserting the data information of the captured data definition language into the operation record table, and counting the number of records in the operation record table, so as to update the operation record table according to the number of records by using a preset update rule to obtain a current operation record table.

[0080] In the embodiment of the application, since the operation record table is used for recording the DDL statement generated in the current database, after the DDL statement is captured, the necessary information of the DDL statement is inserted into the operation record table. It should be noted that, in order to prevent the DDL operation table from increasing indefinitely, the DDL capture function simultaneously counts the number of records in the DDL operation table, and performs regular cleaning. In addition, a threshold value can be set after counting the number of records in the DDL operation table, when the number of records reaches the set threshold value, the operation record table is updated, and the earlier records are deleted according to the default rule defined by the function to prevent the backup storage space from being occupied. Furthermore, when the DDL operation record table is modified, the target database is taken as a publishing end of incremental data, and the DML modification of the table is distributed to a subscribing end of a backup tool.

[0081] Step S24: When it is detected that a data manipulation language is operating on the current operation record table, the logic decoding plugin mounted on the logic slot is called to parse the write-ahead log.

[0082] In this embodiment of the application, when a DML statement that operates on the target database is executed, the target database, as the publisher of incremental data, first parses the WAL (Write-Ahead Log) by calling the mounted logical decoding plugin in its logical replication slot. For example... Figure 3 The diagram shows the WAL log commit. The WAL log is parsed by the logic decoding plugin, and insert, update, and delete data modification statements are filtered out.

[0083] Step S25: Filter the data manipulation language in the parsed write-ahead log, then publish the data manipulation language, and back up the database based on the incremental change data using the backup tool deployed on the target device according to the preset backup strategy, so as to obtain the backup database.

[0084] In this embodiment, the logical replication slot calls the mounted logical decoding plugin to distribute the parsed DML statement, waiting for the backup tool subscription end to receive the incremental change data.

[0085] like Figure 4 The diagram illustrates the incremental backup data publishing process. Users create pre-defined data processing objects in the target database, including DDL operation tables, DDL capture functions, event triggers, logical replication slots, and access users. Their specific functions are detailed in previous embodiments and will not be repeated here. By creating these pre-defined data processing objects, the method achieves support for the collection, parsing, and recording of database DDL statements, ensuring support for DDL statements related to table structure changes, thereby guaranteeing the continuous availability of incremental backup data and minimizing the performance impact on the backup database. When operating on the target database, statements are executed in a loop. If the execution is a DML statement directly operating on the backup database, the DML statement is parsed through logical slots and published to the subscriber. If the execution is a DDL statement operating on the database, the pre-defined data processing objects created in the target database are first used to process the DDL statement. Then, when the DDL operation record table is modified, the DML modifications to that table are distributed to the backup tool's subscriber.

[0086] Step S26: Based on the backup database, the target database is restored using the recovery tool deployed on the target device to obtain the restored database.

[0087] For a more detailed explanation of the process of step S26, please refer to the relevant content disclosed in the foregoing embodiments, which will not be repeated here.

[0088] It can be seen that the preset data processing object is created in the to-be-processed PostgreSQL database, the collection, analysis and record support of the data definition language are realized, the backup of the data definition language such as the database table structure change is supported, the limitation of only supporting the backup of the data manipulation language is not existed, and the method has strong practicability and innovation. Meanwhile, the method can flexibly specify a specific database for backup according to business needs, different databases can be set with different backup strategies, the backup data storage space occupation is small compared with the existing database backup method, the backup data collection and storage safety are high, and the problems of large backup storage space occupation and much irrelevant data in the traditional backup are effectively solved. Secondly, the incremental backup mode is used to store the backup data, the incremental change data of the data can be safely and efficiently collected, and the incremental change data is stored in the storage device, so that the storage space occupation of the backup data is greatly reduced. The backup and recovery of the target database are realized through the backup tool and the recovery tool of the target device, the remote access is supported, the method can be compatible with different versions of PostgreSQL, and the method can be conveniently recovered to other types of relational databases.

[0089] The embodiment of the application discloses a specific data processing method based on a PostgreSQL database, referring to Figure 5 The method comprises the following steps.

[0090] Step S31: determining a to-be-processed PostgreSQL database, and creating a preset data processing object in the to-be-processed PostgreSQL database to obtain a target database; wherein the preset data processing object is used for collecting, analyzing and recording the data definition language in the to-be-processed PostgreSQL database, and the preset data processing object is used for specifying a specified library in the to-be-processed PostgreSQL database that needs to be processed.

[0091] The more specific processing process of step S31 can be referred to the corresponding content disclosed in the foregoing embodiment, and will not be described here.

[0092] Step S32: when it is monitored that the data manipulation language operates on the target database, the incremental change data generated in the target database is published, and the backup period, the data compression option and the data encryption option of the target database are set based on a pre-device backup strategy; wherein the backup period comprises a full backup period and an incremental backup period.

[0093] In the embodiment of the present application, in the process of remotely subscribing to incremental backup data to obtain a backup database, a user respectively configures different backup strategies of a target database according to needs. It can be understood that when different backup strategies are configured, a pre-backup strategy can be set for a specified database. The pre-backup strategy includes setting a full data backup period, setting an incremental backup period, setting whether to enable a compression option for backup data, setting whether to enable an encryption option for backup data, and setting a public key of the encryption option. The full data backup period can be set to backup once every few days to reduce the storage space occupied by backup data. The incremental backup period can be set to backup once every half an hour to one hour to ensure the time accuracy of data recovery. The compression option is enabled by default. The encryption option is not enabled by default.

[0094] Step S33: calling a data backup instruction to remotely perform data backup on the target database, and cyclically obtaining incremental change data through a backup tool deployed on a target device according to the full data backup period.

[0095] In the embodiment of the present application, the backup tool actively detects, and when the full data backup period of a specified database is reached, the backup tool first obtains the latest LSN (log sequence number) to locate an incremental point. Then, the backup tool remotely performs full data backup on the database by calling a backup instruction pg_dump of PostgreSQL data.

[0096] Further, according to the full data backup period, the backup tool cyclically obtains incremental change data using the latest LSN, and updates the latest LSN for next time of obtaining incremental change data. It should be noted that the process of cyclically obtaining incremental change data through a backup tool deployed on a target device according to the full data backup period further includes: establishing a first incremental data backup file according to the incremental change data, and monitoring a backup time length of the first incremental data backup file; when the backup time length reaches the incremental backup period, closing the first incremental data backup file through the backup tool, and creating a second incremental data backup file based on the incremental change data; compressing the first incremental data backup file, and cleaning up the first incremental data backup file after a storage time limit of the first incremental data backup file reaches a preset threshold.

[0097] In the embodiment of the present application, an incremental data backup file is generated in the backup process. When the backup time length of the incremental data backup file reaches the backup period length, the backup tool closes the backup file, and re-creates a new backup file for storing subsequent backup data. The closed backup file is compressed by the backup tool to reduce the storage space occupation. According to the business security needs, the compressed data can be further configured to be encrypted and stored.

[0098] In addition, the user can set a time threshold or a space threshold of the backup file. When the maximum backup time threshold of the backup file exceeds, the backup tool cleans up the expired backup data, and the default time threshold is 30 days. When the maximum backup space exceeds the threshold, the backup tool cleans up the earliest backup data, and the default space threshold is 0, i.e., the threshold is not effective.

[0099] Step S34: judging whether an operation table for operating the target database by using the data manipulation language is an operation record table in the target database. If not, the data manipulation language is directly appended to the incremental change data. If yes, judging whether the data manipulation language is a record for deleting the operation record table.

[0100] In the embodiment of the application, the backup tool analyzes the incremental change data. First, the table operated by the DML statement is filtered to judge whether the table operated by the DML operation language is a DDL operation record table created in the target database. If not, the DML statement is directly appended to the incremental change data to obtain a new incremental data backup file. If yes, it is further judged whether the data manipulation language is a record for deleting the operation record table.

[0101] Step S35: if the data manipulation language is a record for deleting the operation record table, the data manipulation language is ignored. If the data manipulation language is not a record for deleting the operation record table, a data definition language contained in the data manipulation language is extracted, and then the data definition language is appended to the incremental change data.

[0102] In the embodiment of the application, if the DML operation is a record for deleting the DDL operation table, it is directly ignored. Otherwise, for inserting the DLL operation record table, the backup tool extracts the DDL statement contained in the statement and appends the DDL statement to the incremental data backup file.

[0103] Step S36: the incremental change data is stream-compressed according to the data compression option, and then the backup compressed file obtained after compression is stored by encryption according to the data encryption option, so as to obtain a backup database by using the backup compressed file stored by encryption.

[0104] In the embodiment of the application, the backup tool receives the backup data and stores the data by stream compression. This method can reduce the storage space occupation and the performance overhead of multiple disks during compression. According to the business security requirement, the compressed data can be further configured to be stored by encryption, which further reduces the data space occupation and ensures the data security.

[0105] AsFigure 6 The process of remotely receiving incremental backup data is shown to obtain a backup database. The user performs data backup on the target device based on different pre-device backup strategies, including configuring the connection information of the backup database, configuring the full backup strategy of different databases, configuring the incremental backup strategy of different databases, and configuring the cleaning strategy of different databases, etc. After enabling the backup process, the backup tool actively detects, and when the full backup period of a specified database arrives, the backup tool first obtains the latest LSN number, locates the incremental point, and realizes the sparse full backup of the specified database. Then, the backup instruction pg_dump of PostgreSQL data is called to remotely perform full data backup on the database. At the same time, when the backup tool receives the backup data, it uses stream compression to compress and store the data, which can reduce the storage space occupation and the performance overhead of multiple disks during compression. According to the business security needs, the compressed data can be further configured to be stored in encrypted form. Then, the corresponding expired data cleaning threshold is set to clean the expired data. The backup tool uses the latest LSN number to cyclically obtain incremental change data and update the latest LSN number for the next incremental data acquisition. The backup tool parses the obtained incremental data, first filters the table operated by the DML statement, and if the operated table is not the created DDL operation record table, the DML statement is directly appended to the incremental data backup file. If the operated table is the created DDL operation record table, if the DML operation is a delete DDL operation record, it is directly ignored. Otherwise, for the insert DLL operation record table record, the backup tool extracts the DDL statement contained in the statement and appends the DDL statement to the incremental data backup file. When the backup time of the incremental backup data file reaches the backup period, the backup tool will close the backup file and create a new backup file for storing subsequent backup data. The backup tool performs compression operation on the closed backup file to reduce storage space occupation. According to the business security needs, the compressed data can be further configured to be stored in encrypted form.

[0106] Step S37: restoring the target database based on the backup database through the recovery tool deployed on the target device to obtain a recovery database.

[0107] Among them, the more specific processing process of the above step S37 can refer to the corresponding content disclosed in the foregoing embodiments, which will not be repeated here.

[0108] It can be seen that the preset data processing object is created in the to-be-processed PostgreSQL database, the collection, analysis and record support of the data definition language are realized, the backup of the data definition language such as the database table structure change is supported, the limitation of only supporting the backup of the data manipulation language is not existed, and the method has strong practicability and innovation. Meanwhile, the method can flexibly specify a specific database for backup according to business needs, different databases can be set with different backup strategies, the backup data storage space occupation is small compared with the existing database backup method, the backup data collection and storage safety are high, and the problems of large backup storage space occupation and much irrelevant data in the traditional backup are effectively solved. Secondly, the incremental backup mode is used to store the backup data, the incremental change data of the data can be safely and efficiently collected, and the incremental change data is stored in the storage device, so that the storage space occupation of the backup data is greatly reduced. The backup and recovery of the target database are realized through the backup tool and the recovery tool of the target device, the remote access is supported, the method can be compatible with different versions of PostgreSQL, and the method can be conveniently recovered to other types of relational databases.

[0109] The embodiment of the application discloses a specific data processing method based on a PostgreSQL database, as shown in Figure 7 The method comprises the following steps.

[0110] Step S41: determining a to-be-processed PostgreSQL database, and creating a preset data processing object in the to-be-processed PostgreSQL database to obtain a target database; wherein the preset data processing object is used for collecting, analyzing and recording a data definition language in the to-be-processed PostgreSQL database, and the preset data processing object is used for specifying a specified library in the to-be-processed PostgreSQL database that needs to be processed.

[0111] Step S42: when it is monitored that a data manipulation language exists and operates on the target database, incremental change data generated in the target database is published, and a backup tool deployed on a target device is used to backup the incremental change data according to a backup strategy to obtain a backup database.

[0112] The more specific processing process of the above steps S41 and S42 can be referred to the corresponding content disclosed in the foregoing embodiments, and will not be described here.

[0113] Step S43: checking the connectivity of the target database through a recovery tool deployed on the target device, and collecting first basic information of the backup database.

[0114] In the embodiment of the present application, when the user needs to perform a recovery operation according to the incremental backup data, the user login information of the database to be recovered, the database type, and the data overwrite option are transmitted when the recovery command is executed, and the backup database name and the recovery time point, and the optional data decryption private key are also transmitted. The recovery tool first performs connectivity check on the target database according to the login information, and collects the version information and the SQL compatibility information of the target database. It can be understood that the database to be recovered is the database on the target device. If the preset database exists, the data recovery is performed based on the existing preset database to obtain the recovered database. If the preset database does not exist on the target device, a new database is constructed.

[0115] Step S44: comparing the first basic information with the second basic information of the preset database pre-constructed on the target device, and judging whether the comparison result meets the preset recovery condition.

[0116] In the embodiment of the present application, the second basic information of the preset database collected is compared with the first basic information carried by the backup file of the target database. If the recovery addition is met, the recovery process is started. Otherwise, the corresponding incompatible error information is returned to the user.

[0117] Step S45: if the comparison result meets the preset recovery condition, the target database is recovered to obtain the recovered database; if the comparison result does not meet the preset recovery condition, the incompatible error information of the backup database and the pre-constructed database is returned.

[0118] In a specific embodiment, the recovery tool pre-checks the database to be recovered. When the recovery tool pre-checks is passed, the recovery process is started. In another specific embodiment, when the recovery tool pre-checks is not passed, the corresponding incompatible error information is returned to the user.

[0119] In the embodiment of the present application, in the recovery process, the recovery tool first finds the backup directory of the database according to the transmitted backup database name, and finds a full backup compressed file and a set of incremental backup compressed files closest to the time according to the time stamp. That is, the backup directory of the backup database is obtained by the recovery tool deployed on the target device according to the backup database. The backup file closest to the time of the time stamp is filtered out according to the time stamp in the backup directory.

[0120] Further, the recovery tool is connected to the preset database to be recovered, and if the preset database does not exist, a new database is created; if the preset database exists, corresponding operations are performed according to the data cover option transmitted by the user. That is, it is judged whether the preset database already exists on the target device; if the preset database already exists on the target device, data in the backup file is directly covered on data in the preset database in time sequence to obtain a recovery database; if the preset database does not exist on the target device, the preset database is created, and data in the backup file is covered on data in the preset database in time sequence to obtain a recovery database.

[0121] As shown in FIG. 6, the recovery tool judges whether the database cover option is data renaming, data cover or data merging. Figure 8 As shown in FIG. 6, the recovery tool judges whether the database cover option is data renaming, data cover or data merging.

[0122] In the process of data recovery, the recovery tool judges that the full backup file is an encrypted file, and first performs decryption; judges that the full backup file is a compressed file, and decompresses the full backup data file, and calls corresponding database client instructions to import the full backup data into the recovery database. The recovery tool judges that the incremental file is an encrypted file, and first performs decryption; judges that the incremental file is a compressed file, and decompresses a group of incremental backup data files, and executes the incremental backup files in time sequence. When the last backup file is executed, it is necessary to compare a time stamp carried by each record in the backup file with a time stamp transmitted by the recovery instruction, and accurately restore to a time point required by the user to be restored.

[0123] It can be seen that the preset data processing object is created in the to-be-processed PostgreSQL database, the collection, analysis and record support of the data definition language are realized, the backup of the data definition language such as the database table structure change is supported, the limitation of only supporting the backup of the data manipulation language is not existed, and the method has strong practicability and innovation. Meanwhile, the method can flexibly specify a specific database for backup according to business needs, different databases can set different backup strategies, the backup data storage space occupation is small compared with the existing database backup method, the backup data collection and storage safety are high, and the problems of large backup storage space occupation and much irrelevant data in the traditional backup are effectively solved. Secondly, the incremental backup mode is used to store the backup data, the incremental change data of the data can be safely and efficiently collected, and the incremental change data is stored in the storage device, so that the storage space occupation of the backup data is greatly reduced. The backup and recovery of the target database are realized through the backup tool and the recovery tool of the target device, the remote access is supported, the method can be compatible with different versions of PostgreSQL, and the method can be conveniently recovered to other types of relational databases.

[0124] Correspondingly, the embodiment of the application further discloses a data processing device based on a PostgreSQL database, referring to Figure 9 The device comprises:

[0125] A preset data processing object creation module 11 is configured to determine a to-be-processed PostgreSQL database, and create a preset data processing object in the to-be-processed PostgreSQL database to obtain a target database; wherein the preset data processing object is used to collect, analyze and record the data definition language in the to-be-processed PostgreSQL database, and the preset data processing object is used to specify a specified library in the to-be-processed PostgreSQL database that needs to be processed;

[0126] A database backup module 12 is configured to publish incremental change data generated in the target database when it is monitored that the data manipulation language operates on the target database, and backup the incremental change data based on a preset backup strategy through a backup tool deployed on a target device to obtain a backup database.

[0127] A database recovery module 13 is configured to recover the target database based on the backup database through a recovery tool deployed on the target device to obtain a recovery database.

[0128] The more specific working processes of the above modules can be referred to the corresponding contents disclosed in the foregoing embodiments, and will not be described here.

[0129] It can be seen that, by the above scheme of the embodiment, first, a to-be-processed PostgreSQL database is determined, and a preset data processing object is created in the to-be-processed PostgreSQL database to obtain a target database; the preset data processing object is used to collect, analyze and record the data definition language in the to-be-processed PostgreSQL database, and the preset data processing object is used to specify a specified library in the to-be-processed PostgreSQL database that needs data processing; when it is monitored that there is a data manipulation language to operate the target database, the incremental change data generated in the target database is published, and the backup tool deployed on the target device is used to backup according to the incremental change data based on a preset backup strategy to obtain a backup database; the target database is recovered based on the backup database by using the recovery tool deployed on the target device to obtain a recovery database. It can be seen that, by creating the preset data processing object in the to-be-processed PostgreSQL database, the collection, analysis and recording of the data definition language are supported, the backup of the data definition language such as the database table structure change is supported, there is no limitation of only supporting the backup of the data manipulation language, and the method has strong practicability and innovation. At the same time, the method can flexibly specify a specific database for backup according to business needs, different databases can set different backup strategies, the backup data storage space is small compared with the existing database backup method, the backup data collection and storage safety are high, and the problems of large backup storage space occupation and many irrelevant data in the traditional backup are effectively solved. Secondly, the incremental backup method is used to store the backup data, the incremental change data of the data can be safely and efficiently collected, and the incremental change data is stored in the storage device, which greatly reduces the storage space occupation of the backup data. The backup and recovery of the target database are realized by using the backup tool and the recovery tool of the target device, the remote access is supported, the method can be compatible with different versions of PostgreSQL, and the method can be conveniently recovered to other types of relational databases.

[0130] Further, the embodiment of the application further discloses an electronic device, Figure 10 FIG. 1 is a structural diagram of an electronic device 20 according to an example embodiment, and the contents in the figure cannot be considered as any limitation on the use range of the application.

[0131] Figure 10A structural schematic diagram of an electronic device 20 is provided in the embodiments of the present application. The electronic device 20 can specifically include at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input / output interface 25 and a communication bus 26. The memory 22 is configured to store a computer program, and the processor 21 is configured to load and execute the computer program to implement the related steps in the data processing method based on the PostgreSQL database disclosed in any of the foregoing embodiments. In addition, the electronic device 20 in the embodiments of the present application can be specifically a computer.

[0132] In the embodiments of the present application, the power supply 23 is configured to provide working voltage for each hardware device on the electronic device 20; the communication interface 24 is capable of creating a data transmission channel between the electronic device 20 and external devices, and the communication protocol followed by the communication interface 24 can be any communication protocol applicable to the technical solutions of the present application, which is not specifically limited herein; the input / output interface 25 is configured to obtain external input data or output data to the outside, and the specific interface type can be selected according to the specific application needs, which is not specifically limited herein.

[0133] In addition, the memory 22 as a carrier for resource storage can be a read-only memory, a random access memory, a magnetic disk or an optical disk, and the resources stored thereon can include an operating system 221, a computer program 222 and data 223, etc., and the data 223 can include various data. The storage mode can be temporary storage or permanent storage.

[0134] The operating system 221 is configured to manage and control each hardware device on the electronic device 20 and the computer program 222, and can be Windows Server, Netware, Unix, Linux, etc. In addition to the computer program capable of completing the data processing method based on the PostgreSQL database executed by the electronic device 20 disclosed in any of the foregoing embodiments, the computer program 222 can further include a computer program capable of completing other specific work.

[0135] Further, the embodiments of the present application further disclose a computer readable storage medium, which includes a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a magnetic disk or an optical disk or any other form of storage medium known in the technical field. The computer program is executed by the processor to implement the foregoing data processing method based on the PostgreSQL database. The specific steps of the method can refer to the corresponding content disclosed in the foregoing embodiments, which will not be described herein again.

[0136] The various embodiments described in this specification are presented by way of example, and each embodiment is not necessarily composed of all features described with respect to other embodiments. Each embodiment described in this specification can be implemented in software, firmware, hardware, or a combination thereof. The various embodiments described in this specification can be implemented in any combination of the following examples.

[0137] The steps of the data processing or algorithm based on the PostgreSQL database described in conjunction with the embodiments disclosed herein can be implemented directly in hardware, software modules executed by a processor, or a combination of both. The software modules can be placed in random access memory (RAM), memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disks, removable disks, CD-ROMs, or any other form of storage medium known in the art.

[0138] Finally, it should be noted that the relational terms herein, such as first and second, are used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any such actual relationship or order between such entities or actions. Moreover, the terms "comprises", "comprising", or any other variations thereof are intended to cover non-exclusive inclusions, so that a process, method, article, or apparatus that includes a list of elements does not only include those elements, but also includes other elements not expressly listed, or other elements inherent in such process, method, article, or apparatus. Without more limitations, an element defined by the phrase "comprising a" does not exclude the presence of additional identical elements in the process, method, article, or apparatus that includes the element.

[0139] The above provides a detailed introduction to the data processing method, device, equipment and medium based on the PostgreSQL database provided by the present application. The principles and implementation modes of the present application are described in this specification by applying specific examples. The above description of the embodiments is only to help understand the method of the present application and its core idea; at the same time, for those skilled in the art, according to the idea of the present application, the specific implementation mode and application range will be changed; in view of the above, the content of the specification should not be understood as a limitation of the present application.

Claims

1. A data processing method based on a PostgreSQL database, characterized in that, The application relates to a method for processing a PostgreSQL database, and the method comprises the following steps: determining a to-be-processed PostgreSQL database, and creating preset data processing objects in the to-be-processed PostgreSQL database to obtain a target database; wherein the preset data processing objects are used for collecting, analyzing and recording data definition languages in the to-be-processed PostgreSQL database, and the preset data processing objects are used for specifying specified libraries in the to-be-processed PostgreSQL database which need to be processed; when it is monitored that a data manipulation language exists and is used for operating the target database, publishing incremental change data generated in the target database, and based on a preset backup strategy, backing up the target database through a backup tool deployed on a target device according to the incremental change data to obtain a backup database; based on the backup database, restoring the target database through a recovery tool deployed on the target device to obtain a recovery database; the step of creating the preset data processing objects in the to-be-processed PostgreSQL database to obtain the target database comprises the following steps: creating an operation record table used for recording the data definition languages in the to-be-processed PostgreSQL database, creating a statement capturing function used for capturing the data definition languages and an event trigger used for capturing execution completion of the statement capturing function, creating a logical slot used for copying data of the specified library, and creating user information used for accessing the logical slot to obtain the target database; the step of backing up the target database through the backup tool deployed on the target device according to the incremental change data to obtain the backup database comprises the following steps: based on the preset backup strategy, setting a backup period, a data compression option and a data encryption option of the target database; wherein the backup period comprises a full backup period and an incremental backup period; calling a data backup instruction to remotely back up data of the target database, and according to the full backup period, cyclically acquiring incremental change data through the backup tool deployed on the target device; judging whether an operation table used for operating the target database by using the data manipulation language is the operation record table in the target database; if not, the data manipulation language is directly added to the incremental change data; if yes, judging whether the data manipulation language is used for deleting records of the operation record table; if the data manipulation language is used for deleting the records of the operation record table, the data manipulation language is ignored; if the data manipulation language is not used for deleting the records of the operation record table, data definition languages contained in the data manipulation language are extracted, and then the data definition languages are added to the incremental change data; according to the data compression option, the incremental change data is stream-compressed, then according to the data encryption option, a backup compressed file obtained after compression is stored in a manner of encryption, and the backup database is obtained by using the backup compressed file stored in the manner of encryption.

2. The PostgreSQL database-based data processing method according to claim 1, characterized in that, The incremental change data generated in the target database is published when it is monitored that the data manipulation language operates on the target database, comprising: continuously invoking the statement capture function based on the event trigger to capture the data definition language; inserting the captured data information of the data definition language into the operation record table, and counting the number of records in the operation record table, so as to update the operation record table according to the number of records by using a preset update rule to obtain a current operation record table; when it is monitored that the data manipulation language operates on the current operation record table, the logical decoding plug-in mounted through the logical slot is called to analyze the prewrite log; filtering the data manipulation language in the analyzed prewrite log, and then publishing the data manipulation language.

3. The PostgreSQL database-based data processing method according to claim 1, characterized in that, In the process of cyclically obtaining incremental change data by the backup tool deployed on the target device according to the full backup period, the method further comprises: establishing a first incremental data backup file according to the incremental change data, and monitoring the backup duration of the first incremental data backup file; when the backup duration reaches the incremental backup period, closing the first incremental data backup file through the backup tool, and creating a second incremental data backup file based on the incremental change data; compressing the first incremental data backup file, and cleaning up the first incremental data backup file when the storage time limit of the first incremental data backup file reaches a preset threshold.

4. The PostgreSQL database-based data processing method according to any one of claims 1 to 3, characterized in that, The method of recovering the target database based on the backup database by the recovery tool deployed on the target device to obtain a recovery database, comprising: checking the connectivity with the target database by the recovery tool deployed on the target device, and collecting first basic information of the backup database; comparing the first basic information with second basic information of a preset database pre-constructed on the target device, and determining whether the comparison result meets a preset recovery condition; if the comparison result meets the preset recovery condition, recovering the target database to obtain a recovery database; if the comparison result does not meet the preset recovery condition, returning incompatible error information of the backup database and the pre-constructed database.

5. The PostgreSQL database-based data processing method according to claim 4, characterized in that, The method of recovering the target database based on the backup database by the recovery tool deployed on the target device to obtain a recovery database, comprising: acquiring a backup directory of the backup database by the recovery tool deployed on the target device according to the backup database; filtering out a backup file closest to the time of the timestamp according to the timestamp in the backup directory; determining whether the preset database already exists on the target device; if the preset database already exists on the target device, sequentially covering the data in the preset database with the data in the backup file in time sequence to obtain a recovery database; If the preset database does not exist on the target device, the preset database is created, and data in the backup file is sequentially overwritten on data in the preset database in chronological order to obtain a recovery database.

6. A data processing apparatus based on a PostgreSQL database, characterized in that, Comprise: A preset data processing object creation module is configured to determine a to-be-processed PostgreSQL database, and create a preset data processing object in the to-be-processed PostgreSQL database to obtain a target database; wherein the preset data processing object is configured to collect, analyze and record data definition languages in the to-be-processed PostgreSQL database, and the preset data processing object is configured to specify a specified library in the to-be-processed PostgreSQL database that needs to be processed; A database backup module is configured to publish incremental change data generated in the target database when it is monitored that there is a data manipulation language operating on the target database, and backup the incremental change data based on a preset backup strategy through a backup tool deployed on the target device to obtain a backup database; A database recovery module is configured to recover the target database based on the backup database through a recovery tool deployed on the target device to obtain a recovery database; The preset data processing object creation module is specifically configured to: create an operation record table for recording the data definition languages, create a statement capture function for capturing the data definition languages and an event trigger for capturing execution completion of the statement capture function, create a logical slot for copying data of the specified library, and create user information for accessing the logical slot in the to-be-processed PostgreSQL database to obtain a target database; The database backup module is specifically configured to: set a backup period, a data compression option and a data encryption option of the target database based on a preset backup strategy; wherein the backup period comprises a full backup period and an incremental backup period; call a data backup instruction to remotely backup data of the target database, and cyclically acquire incremental change data through a backup tool deployed on the target device according to the full backup period; determine whether an operation table operating on the target database by the data manipulation language is an operation record table in the target database; if not, directly append the data manipulation language to the incremental change data; if yes, determine whether the data manipulation language is a record deleting the operation record table; if the data manipulation language is a record deleting the operation record table, ignore the data manipulation language; if the data manipulation language is not a record deleting the operation record table, extract a data definition language contained in the data manipulation language, and then append the data definition language to the incremental change data; stream-compress the incremental change data according to the data compression option, and then encrypt a backup compressed file obtained after compression according to the data encryption option to obtain a backup database by using the backup compressed file stored after encryption.

7. An electronic device, comprising: The electronic device comprises a processor and a memory; wherein the memory is used to store a computer program, the computer program is loaded and executed by the processor to realize the PostgreSQL database-based data processing method as claimed in any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that, A computer program product for storing a computer program; wherein the computer program is executed by a processor to realize the PostgreSQL database-based data processing method as claimed in any one of claims 1 to 5.

Images