Data batch backup method, device, equipment and storage medium

Abstract

The present invention relates to the field of artificial intelligence technology, and discloses a method, device, equipment and storage medium for batch data backup. The method comprises: obtaining backup configuration information corresponding to multiple source databases, and using the backup configuration information to configure a structure configuration table and a rule configuration table; based on the structure configuration table, screening out the data source table to be backed up, and constructing a data backup table sample according to the table structure of the data source table; based on the rule configuration table, selecting source data corresponding to the data backup table sample from the data source table, and using each source data to configure multiple backup data shards; storing each backup data shard to the corresponding data backup table sample and deploying it on a preset target database node to obtain a data backup table corresponding to the data source table. The present invention realizes fast and accurate backup of large quantities.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a data batch backup method, device, equipment and storage medium. Background Art

[0002] The current society is an era of artificial intelligence and big data. The term big data is no longer unfamiliar to each of us. All walks of life have their own big data. Data records valuable information from various industries and plays an irreplaceable role in analyzing user consumption habits and exploring the market. However, in the process of operating data, it is inevitable that problems such as accidental deletion will occur. At this time, flexible backup of data is very important.

[0003] In the process of work, there are many operations that need to be processed on data. For example, in Oracle or MySQL, data will be added, deleted, checked, and modified. For general information, it can be modified or deleted, but for important information, it is necessary to be cautious. At this time, for stability or security, data backup is particularly important. In many cases, it is necessary to back up one table at a time, and it is impossible to accurately back up the same type of batch data at one time. This backup is time-consuming and inefficient. So how to quickly and accurately back up large quantities of data in a targeted manner has become a problem that needs to be answered urgently. Summary of the invention

[0004] The main purpose of the present invention is to solve the technical problem of how to perform targeted, rapid and accurate backup of large quantities of data.

[0005] A first aspect of the present invention provides a method for batch data backup, comprising: obtaining backup configuration information corresponding to multiple source databases, and using the backup configuration information to configure a structure configuration table and a rule configuration table; based on the structure configuration table, screening out a data source table to be backed up, and building a data backup table sample according to the table structure of the data source table; based on the rule configuration table, selecting source data corresponding to the data backup table sample from the data source table, and using each of the source data to configure multiple backup data shards; storing each of the backup data shards in a corresponding data backup table sample and deploying them on a preset target database node to obtain a data backup table corresponding to the data source table.

[0006] Optionally, in a first implementation method of the first aspect of the present invention, the use of the backup configuration information to configure the structure configuration table and the rule configuration table includes: extracting source table identification information in the backup configuration information, and using the source table identification information to create a structure configuration table; extracting multiple rule configuration fields in the backup configuration information, and using each of the rule configurations to create a rule configuration table.

[0007] Optionally, in a second implementation method of the first aspect of the present invention, the selecting the source data corresponding to the data backup table sample from the data source table based on the rule configuration table includes: based on the rule configuration table, selecting a query instruction that matches each of the rule configuration fields; according to the query instruction, calling a preset interface to obtain multiple class attribute information corresponding to the data backup table sample; according to each of the class attribute information, selecting the source data corresponding to the data backup table sample.

[0008] Optionally, in a third implementation of the first aspect of the present invention, the selection of query instructions matching each of the rule configuration fields based on the rule configuration table includes: extracting backup rule fields corresponding to each rule configuration field in the rule configuration table, and identifying the rule status of each of the backup rule fields; respectively determining whether the rule status of each of the backup rule fields is valid; if valid, selecting the query instructions corresponding to the backup rule fields whose rule status is valid.

[0009] Optionally, in a fourth implementation method of the first aspect of the present invention, after storing each of the backup data shards in a corresponding data backup table sample and deploying them on a preset target database node to obtain a data backup table corresponding to the data source table, it also includes: obtaining data modification rules and data modification parameters, and performing a security check on the data modification parameters; if the check is successful, using the successfully verified data modification parameters as variables and combining them with the data modification rules to generate an SQL statement; according to the SQL statement, using a preset data modification tool to perform data modification operations on the data backup table to obtain a new data backup table.

[0010] Optionally, in a fifth implementation manner of the first aspect of the present invention, the data modification operation is performed on the data backup table using a preset data modification tool according to the SQL statement to obtain a new data backup table, including: determining the statement type of the SQL, and extracting the table name, field name and modification condition to be processed in the SQL statement according to the statement type of the SQL statement; generating a query affecting statement, a new and old value change statement and a backup statement corresponding to the SQL statement according to the table name, the field name and the modification condition; executing the SQL statement based on the query affecting statement, the new and old value change statement and the backup statement; and modifying the target data in the data backup table according to the execution result of the SQL statement to obtain a new data backup table.

[0011] The second aspect of the present invention provides a data batch backup device, including: a configuration module, used to obtain backup configuration information corresponding to multiple source databases, and use the backup configuration information to configure a structure configuration table and a rule configuration table; a construction module, used to screen out the data source table to be backed up based on the structure configuration table, and build a data backup table sample according to the table structure of the data source table; a selection module, used to select source data corresponding to the data backup table sample from the data source table based on the rule configuration table, and use each of the source data to configure multiple backup data shards; a backup module, used to store each of the backup data shards to the corresponding data backup table sample and deploy it on a preset target database node to obtain a data backup table corresponding to the data source table.

[0012] Optionally, in a first implementation method of the second aspect of the present invention, the configuration module includes: a structure configuration unit, used to extract source table identification information in the backup configuration information, and use the source table identification information to create a structure configuration table; a rule configuration unit, used to extract multiple rule configuration fields in the backup configuration information, and use each of the rule configurations to create a rule configuration table.

[0013] Optionally, in a second implementation of the second aspect of the present invention, the selection module includes: an instruction selection unit, used to select query instructions matching each of the rule configuration fields based on the rule configuration table; an acquisition unit, used to call a preset interface according to the query instruction to obtain multiple class attribute information corresponding to the data backup table sample; and a data selection unit, used to select source data corresponding to the data backup table sample according to each of the class attribute information.

[0014] Optionally, in a third implementation method of the second aspect of the present invention, the instruction selection unit is also used to: extract the backup rule fields corresponding to each rule configuration field in the rule configuration table, and identify the rule status of each backup rule field; respectively determine whether the rule status of each backup rule field is valid; if valid, select the query instruction corresponding to the backup rule field whose rule status is valid.

[0015] Optionally, in a fourth implementation method of the second aspect of the present invention, the data batch backup device also includes a modification module, which is used to: obtain data modification rules and data modification parameters, and perform a security check on the data modification parameters; if the check is successful, use the successfully checked data modification parameters as variables and combine them with the data modification rules to generate an SQL statement; according to the SQL statement, use a preset data modification tool to perform data modification operations on the data backup table to obtain a new data backup table.

[0016] Optionally, in a fifth implementation of the second aspect of the present invention, the modification module is also used to: determine the statement type of the SQL, and extract the table name, field name and modification conditions to be processed in the SQL statement according to the statement type of the SQL statement; generate a query impact statement, a new and old value change statement and a backup statement corresponding to the SQL statement according to the table name, the field name and the modification condition; execute the SQL statement based on the query impact statement, the new and old value change statement and the backup statement; and modify the target data in the data backup table according to the execution result of the SQL statement to obtain a new data backup table.

[0017] The third aspect of the present invention provides a data batch backup device, comprising: a memory and at least one processor, wherein the memory stores instructions; the at least one processor calls the instructions in the memory so that the data batch backup device executes the above-mentioned data batch backup method.

[0018] A fourth aspect of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores instructions, which, when executed on a computer, enable the computer to execute the above-mentioned batch data backup method.

[0019] In the technical solution provided by the present invention, by acquiring backup configuration information corresponding to multiple source databases, a structure configuration table for screening data source tables to be backed up and a rule configuration table for specifying how to back up the data source table are configured; then, through the rule configuration table, source data corresponding to the data backup table sample is selected from the data source table to be backed up and stored in the corresponding data backup table sample to obtain a data backup table corresponding to the data source table, wherein one data source table corresponds to one data backup table, so as to realize configured batch automatic backup of data and improve the accuracy of large-scale rapid backup. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 This is a schematic diagram of a first embodiment of a method for batch data backup in an embodiment of the present invention;

[0021] Figure 2 Schematic diagram of a second embodiment of the method for batch data backup in an embodiment of the present invention;

[0022] Figure 3 Schematic diagram of a third embodiment of the method for batch data backup in an embodiment of the present invention;

[0023] Figure 4 A schematic diagram of an embodiment of a data batch backup device in an embodiment of the present invention;

[0024] Figure 5A schematic diagram of another embodiment of the data batch backup device in the embodiment of the present invention;

[0025] Figure 6 The figure is a schematic diagram of an embodiment of a data batch backup device in an embodiment of the present invention. DETAILED DESCRIPTION

[0026] The embodiment of the present invention provides a method, device, equipment and storage medium for batch data backup, which obtains backup configuration information corresponding to multiple source databases, and uses the backup configuration information to configure a structure configuration table and a rule configuration table; based on the structure configuration table, the data source table to be backed up is screened out, and a data backup table sample is constructed according to the table structure of the data source table; based on the rule configuration table, source data corresponding to the data backup table sample is selected from the data source table, and multiple backup data shards are configured using each source data; each backup data shard is stored in the corresponding data backup table sample and deployed on a preset target database node to obtain a data backup table corresponding to the data source table. The present invention realizes fast and accurate backup of large quantities.

[0027] The terms "first", "second", "third", "fourth", etc. (if any) in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "including" or "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0028] For ease of understanding, the specific process of the embodiment of the present invention is described below. Figure 1 The first embodiment of the method for batch data backup in the embodiment of the present invention includes:

[0029] 101. Obtain backup configuration information corresponding to multiple source databases, and configure a structure configuration table and a rule configuration table using the backup configuration information;

[0030] It is understandable that the execution subject of the present invention may be a data batch backup device, or a terminal or a server, which is not limited here. The embodiment of the present invention is described by taking a server as the execution subject as an example.

[0031] In this embodiment, developers can write a front-end page on the JAVA side to control the input backup configuration information and upload it to this server, and upload the backup configuration information by triggering a button to perform data backup. Among them, the backup configuration information obtained by this server includes structure configuration information and rule configuration information, which are used to configure the structure configuration table and the rule configuration table, wherein the structure configuration table is used to screen the data source table that needs to be backed up during the data backup process, and the rule configuration table is used to screen the source data that needs to be backed up and the backup rules of each source data during the data backup process. That is, through the configurability of data backup, the source data that needs to be backed up in the data source table can be backed up in large quantities in a targeted manner.

[0032] Among them, the backup configuration information includes a structure configuration table and a rule configuration table corresponding to a database cluster composed of multiple source databases. The rule configuration table here is used to determine how each source database in the database cluster stores source data in other databases, which can be other source databases or other databases in the database cluster, and is not specifically limited here. In addition, the structure configuration table here is also used to determine the source data and corresponding data structure that need to be backed up in each source database.

[0033] 102. Based on the structure configuration table, filter out the data source table to be backed up, and build a data backup table sample according to the table structure of the data source table;

[0034] In this embodiment, the structure configuration table contains the data source table that needs to be backed up, that is, the source table identification information of the data source table to be backed up. The data source table to be backed up can be found through the source table identification information. Then, dynamic SQL is used to automatically create a data backup table sample of the data source table to be backed up, so that its table structure is consistent with the table structure of the data source table. Specifically, it can be implemented with the INSERT INTO FROM statement, such as: INSERT INTO data backup table sample SELECT * FROM data source table. Among them, one data source table corresponds to one data backup table sample.

[0035] 103. Based on the rule configuration table, source data corresponding to the data backup table sample is selected from the data source table, and multiple backup data shards are configured using each source data;

[0036] In this embodiment, the source data that needs to be backed up in the corresponding data source table is determined according to the rule configuration field set in the structure configuration table, and then the backup is performed according to the rule corresponding to the rule configuration field. For example, for the backup of data source table M, the data source table M is found according to the source table in the rule configuration table. If the data of a certain time period needs to be backed up, the creation time in the rule configuration table can be configured. For example, if proc_date=2021-4-30 is configured, the source data updated on the date "2021-4-30" will be automatically backed up. For another example, if the source data in region A needs to be backed up, region_sid=A can be configured, and the source data in region A will be automatically backed up at this time.

[0037] Specifically, assume that the target database node has n nodes and each node has k primary shards. First, you can not group each target database node, and put the k backup data shards of the first database node L1 on the database nodes L2-L k+1 Or, place all the backup shards of the k data shards of the first database node L1 on the next database node L2. If n is an integer multiple of k+1 and n>k+1, then group the database nodes into n groups, each with k+1 primary shards, and place the k backup data shards in the other k database nodes of the group in turn. If n is an integer multiple of k and n>k. Divide the database nodes into n groups, each with k backup data shards, and place the k backup data shards in the k database nodes of the next group in turn. If n is an integer multiple of 2, divide the database nodes into n / 2 groups, each with 2 database nodes, storing k backup data shards in each other.

[0038] 104. Store each backup data shard into a corresponding data backup table sample and deploy it on a preset target database node to obtain a data backup table corresponding to the data source table.

[0039] In this embodiment, the source data is stored according to the configured backup data shards. Here, the source data in the data source table can be stored in an Oracle database. In the Oracle database, stored procedures and dynamic SQL are used to read the information of the two configuration tables, and a large amount of source data is backed up from the data source table of the Oracle database to the data backup table sample to obtain a data backup table corresponding to the data source table, wherein one data source table corresponds to one data backup table.

[0040] In an embodiment of the present invention, by acquiring backup configuration information corresponding to multiple source databases, a structure configuration table for screening data source tables to be backed up and a rule configuration table for specifying how to back up the data source table are configured; then, through the rule configuration table, source data corresponding to the data backup table sample is selected from the data source table to be backed up and stored in the corresponding data backup table sample to obtain a data backup table corresponding to the data source table, wherein one data source table corresponds to one data backup table, so as to realize configured batch automatic backup of data and improve the accuracy of large-scale rapid backup.

[0041] See also Figure 2 The second embodiment of the method for batch data backup in the embodiment of the present invention includes:

[0042] 201. Obtain backup configuration information corresponding to multiple source databases, extract source table identification information in the backup configuration information, and create a structure configuration table using the source table identification information;

[0043] 202. Extract multiple rule configuration fields in the backup configuration information, and create a rule configuration table using each rule configuration;

[0044] In this embodiment, the backup configuration information includes source table identification information of the data source table that needs to be backed up, which is written into the corresponding table of the structure configuration table in sequence to create a structure configuration table for constructing a data backup table sample corresponding to the data source table; the backup configuration information also includes various rule configuration fields of the source data that needs to be backed up in the data source table, which are used to indicate where to obtain source data for backup, how to back up the source data, the name of the backup data table after the backup, etc.

[0045] Specifically, the rule configuration fields may include: backup rule, source table name, backup table name, whether it is effective, deletion rule, backup rule, creation time, update time, etc. Among them, when backing up the source data rules, the source data can be filtered in the corresponding data source table according to the settings of each rule configuration field in the rule configuration table.

[0046] 203. Based on the structure configuration table, filter out the data source table to be backed up, and build a data backup table sample according to the table structure of the data source table;

[0047] 204. Based on the rule configuration table, select a query instruction that matches each rule configuration field;

[0048] 205. According to the query instruction, call the preset interface to obtain multiple class attribute information corresponding to the data backup table sample;

[0049] 206. Select source data corresponding to the data backup table sample according to various types of attribute information;

[0050] In this embodiment, based on the rule configuration table, a query instruction for querying the attribute information of the source data that needs to be backed up can be selected. Among them, the query instruction can indicate a preset interface, such as a batch query interface to obtain all class attribute information of the source data that needs to be backed up in one or more data source tables at one time. For example, when a POSIX (Portable Operating System Interface) is used for querying, the class attribute information of the source data is queried from the data backup table through the mount point, VFS and PIM corresponding to the interface. After all the class attribute information is obtained, the source data that needs to be backed up in the data backup source table is indicated for subsequent backup.

[0051] 207. Store each backup data slice in a corresponding data backup table sample and deploy it on a preset target database node to obtain a data backup table corresponding to the data source table.

[0052] In an embodiment of the present invention, a structure configuration table is created through source table identification information in the backup configuration information, which is used for subsequent screening of data source tables to be backed up, and then a rule configuration table is created through rule configuration fields, which is used for subsequent determination of how to back up each data source table according to the rule configuration fields, so as to achieve configured and standardized batch backup of data.

[0053] See also Figure 3 A third embodiment of the method for batch data backup in the embodiment of the present invention includes:

[0054] 301. Obtain backup configuration information corresponding to multiple source databases, and configure a structure configuration table and a rule configuration table using the backup configuration information;

[0055] 302. Based on the structure configuration table, filter out the data source table to be backed up, and construct a data backup table sample according to the table structure of the data source table;

[0056] 303. Extract the backup rule fields corresponding to the rule configuration fields in the rule configuration table, and identify the rule status of each backup rule field;

[0057] 304. Determine whether the rule status of each backup rule field is valid;

[0058] 305. If valid, select the query instruction corresponding to the backup rule field whose rule status is valid;

[0059] In this embodiment, the rule configuration table needs to be manually configured and uploaded in the front-end page written on the JAVA side, and can also be modified. At the same time, the effectiveness of the corresponding backup rule can be controlled by the rule configuration field. Specifically, the field that controls whether the backup rule is effective can be the rule status field. For example, when IS_VALID=Y, the corresponding backup rule is effective, otherwise it is not effective, so as to determine the rule status of the filing configuration field.

[0060] 306. According to the query instruction, call the preset interface to obtain multiple class attribute information corresponding to the data backup table sample;

[0061] 307. Select source data corresponding to the data backup table sample according to various types of attribute information;

[0062] 308. Store each backup data shard into a corresponding data backup table sample and deploy it on a preset target database node to obtain a data backup table corresponding to the data source table;

[0063] 309. Obtain data modification rules and data modification parameters, and perform security verification on the data modification parameters;

[0064] 310. If the verification is successful, the successfully verified data modification parameters are used as variables and combined with the data modification rules to generate SQL statements;

[0065] 311. According to the SQL statement, use the preset data modification tool to perform data modification operations on the data backup table to obtain a new data backup table.

[0066] In this embodiment, the developer develops multiple data modification rules in advance and stores them in the data modification rule library. When applied, the corresponding data modification rules are selected in the front-end page written on the JAVA side, and the corresponding data modification parameters are input and uploaded. After obtaining the data modification rules and data modification parameters uploaded on the front-end page, the data modification parameters are first security checked to determine whether the format of the data modification parameters is correct. After the check is passed, the data modification parameters and the corresponding data modification rules are combined to generate an SQL statement for backup data modification, which is executed in the preset data modification tool to modify the backup data in the data backup table.

[0067] The data modification rules include SQL statements containing variables, and record the dependency between each SQL statement and the data modification parameters, including name, type, length, default value, and whether it is optional. The data modification tool can be a data ETL (Extract-Transform-Load) tool that automatically obtains data modification execution content and performs data modification on the data backup table.

[0068] Specifically, according to the SQL statement, a preset data modification tool is used to perform a data modification operation on the data backup table, and the modified data backup table includes:

[0069] (1) Determine the SQL statement type, and extract the table name, field name, and modification condition to be processed in the SQL statement according to the SQL statement type;

[0070] (2) Generate the query impact statement, new and old value change statement, and backup statement corresponding to the SQL statement based on the table name, field name, and modification condition;

[0071] (3) Execute SQL statements based on query impact statements, new and old value change statements, and backup statements;

[0072] (4) According to the execution result of the SQL statement, the target data in the data backup table is modified to obtain a new data backup table.

[0073] In an embodiment of the present invention, the user can modify the rule status of each backup rule field in the rule configuration table at the front end to determine which backup rule field is currently valid, so as to execute the corresponding backup rule and realize the configuration of the backup rule. By modifying the backed-up data backup table through SQL statements, flexible operation of the backup data can be realized.

[0074] The above describes the batch data backup method in the embodiment of the present invention. The following describes the batch data backup device in the embodiment of the present invention. Figure 4 In one embodiment of the present invention, a data batch backup device includes:

[0075] Configuration module 401, used to obtain backup configuration information corresponding to multiple source databases, and configure a structure configuration table and a rule configuration table using the backup configuration information;

[0076] A construction module 402 is used to filter out the data source table to be backed up based on the structure configuration table, and to construct a data backup table sample according to the table structure of the data source table;

[0077] A selection module 403 is used to select source data corresponding to the data backup table sample from the data source table based on the rule configuration table, and configure multiple backup data slices using each source data;

[0078] The backup module 404 is used to store each of the backup data slices into a corresponding data backup table sample and deploy it on a preset target database node to obtain a data backup table corresponding to the data source table.

[0079] In an embodiment of the present invention, by acquiring backup configuration information corresponding to multiple source databases, a structure configuration table for screening data source tables to be backed up and a rule configuration table for specifying how to back up the data source table are configured; then, through the rule configuration table, source data corresponding to the data backup table sample is selected from the data source table to be backed up and stored in the corresponding data backup table sample to obtain a data backup table corresponding to the data source table, wherein one data source table corresponds to one data backup table, so as to realize configured batch automatic backup of data and improve the accuracy of large-scale rapid backup.

[0080] See also Figure 5 Another embodiment of the data batch backup device in the embodiment of the present invention includes:

[0081] Configuration module 401, used to obtain backup configuration information corresponding to multiple source databases, and configure a structure configuration table and a rule configuration table using the backup configuration information;

[0082] A construction module 402 is used to filter out the data source table to be backed up based on the structure configuration table, and to construct a data backup table sample according to the table structure of the data source table;

[0083] A selection module 403 is used to select source data corresponding to the data backup table sample from the data source table based on the rule configuration table, and configure multiple backup data slices using each source data;

[0084] The backup module 404 is used to store each of the backup data slices into a corresponding data backup table sample and deploy it on a preset target database node to obtain a data backup table corresponding to the data source table.

[0085] Specifically, the configuration module 401 includes:

[0086] The structure configuration unit 4011 is used to extract the source table identification information in the backup configuration information and create a structure configuration table using the source table identification information;

[0087] The rule configuration unit 4012 is used to extract multiple rule configuration fields in the backup configuration information and create a rule configuration table using each of the rule configurations.

[0088] Specifically, the selection module 403 includes:

[0089] An instruction selection unit 4031 is used to select a query instruction matching each of the rule configuration fields based on the rule configuration table;

[0090] The acquisition unit 4032 is used to call a preset interface according to the query instruction to acquire multiple class attribute information corresponding to the data backup table sample;

[0091] The data selection unit 4033 is used to select source data corresponding to the data backup table sample according to each of the class attribute information.

[0092] Specifically, the instruction selection unit 4031 is further used for:

[0093] Extracting the backup rule fields corresponding to the respective rule configuration fields in the rule configuration table, and identifying the rule status of each of the backup rule fields;

[0094] respectively determining whether the rule status of each backup rule field is valid;

[0095] If valid, the query instruction corresponding to the backup rule field whose rule status is valid is selected.

[0096] Specifically, the data batch backup device further includes a modification module 405, which is used to:

[0097] Obtaining data modification rules and data modification parameters, and performing security verification on the data modification parameters;

[0098] If the verification is successful, the successfully verified data modification parameters are used as variables and combined with the data modification rules to generate SQL statements;

[0099] According to the SQL statement, a preset data modification tool is used to perform a data modification operation on the data backup table to obtain a new data backup table.

[0100] Specifically, the modification module 405 is further used for:

[0101] Determine the statement type of the SQL statement, and extract the table name, field name and modification condition to be processed in the SQL statement according to the statement type of the SQL statement;

[0102] Generate a query impact statement, a new and old value change statement, and a backup statement corresponding to the SQL statement according to the table name, the field name, and the modification condition;

[0103] Execute the SQL statement based on the query impact statement, the new and old value change statement, and the backup statement;

[0104] According to the execution result of the SQL statement, the target data in the data backup table is modified to obtain a new data backup table.

[0105] In an embodiment of the present invention, a structure configuration table is created through source table identification information in the backup configuration information, so as to be used for subsequent screening of data source tables to be backed up, and then a rule configuration table is created through rule configuration fields, so as to be used for subsequent determination of how each data source table is to be backed up according to the rule configuration fields, so as to achieve configured and standardized batch backup of data; in addition, the user can modify the rule status of each backup rule field in the rule configuration table at the front end to determine which backup rule field is currently valid, so as to execute the corresponding backup rule, thereby achieving configuration of the backup rule, and modifying the backed-up data backup table through SQL statements to achieve flexible operation of the backup data.

[0106] above Figure 4 and Figure 5 The data batch backup device in the embodiment of the present invention is described in detail from the perspective of modular functional entities. The data batch backup device in the embodiment of the present invention is described in detail from the perspective of hardware processing.

[0107] Figure 6 1 is a schematic diagram of the structure of a data batch backup device provided by an embodiment of the present invention. The data batch backup device 600 may have relatively large differences due to different configurations or performances, and may include one or more processors (central processing units, CPU) 610 (for example, one or more processors) and a memory 620, and one or more storage media 630 (for example, one or more mass storage devices) storing application programs 633 or data 632. Among them, the memory 620 and the storage medium 630 may be short-term storage or permanent storage. The program stored in the storage medium 630 may include one or more modules (not shown in the figure), and each module may include a series of instruction operations in the data batch backup device 600. Furthermore, the processor 610 may be configured to communicate with the storage medium 630 to execute a series of instruction operations in the storage medium 630 on the data batch backup device 600.

[0108] The data batch backup device 600 may also include one or more power supplies 640, one or more wired or wireless network interfaces 650, one or more input and output interfaces 660, and / or one or more operating systems 631, such as Windows Server, Mac OS X, Unix, Linux, FreeBSD, etc. Those skilled in the art will appreciate that Figure 6 The structure of the data batch backup device shown does not constitute a limitation on the data batch backup device, and may include more or fewer components than shown in the figure, or a combination of certain components, or a different arrangement of components.

[0109] The present invention also provides a data batch backup device, the computer device includes a memory and a processor, the memory stores computer-readable instructions, and when the computer-readable instructions are executed by the processor, the processor executes the steps of the data batch backup method in the above embodiments.

[0110] The present invention also provides a computer-readable storage medium, which may be a non-volatile computer-readable storage medium or a volatile computer-readable storage medium. Instructions are stored in the computer-readable storage medium. When the instructions are executed on a computer, the computer executes the steps of the data batch backup method.

[0111] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0112] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk and other media that can store program codes.

[0113] As described above, the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features thereof may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for batch data backup, characterized in that: The data batch backup method comprises: Obtain backup configuration information corresponding to multiple source databases, and configure a structure configuration table and a rule configuration table using the backup configuration information; Based on the structure configuration table, a data source table to be backed up is screened out, and a data backup table sample is constructed according to the table structure of the data source table; Based on the rule configuration table, source data corresponding to the data backup table sample is selected from the data source table, and multiple backup data slices are configured using each source data; Each of the backup data slices is stored in a corresponding data backup table sample and deployed on a preset target database node to obtain a data backup table corresponding to the data source table; The step of configuring the structure configuration table and the rule configuration table using the backup configuration information includes: extracting source table identification information in the backup configuration information, and creating a structure configuration table using the source table identification information; extracting multiple rule configuration fields in the backup configuration information, and creating a rule configuration table using each of the rule configurations; The selecting, based on the rule configuration table, source data corresponding to the data backup table sample from the data source table comprises: selecting a query instruction matching each of the rule configuration fields based on the rule configuration table; calling a preset interface according to the query instruction to obtain a plurality of class attribute information corresponding to the data backup table sample; selecting source data corresponding to the data backup table sample according to each of the class attribute information; The backup configuration information includes source table identification information of the data source table that needs to be backed up, which is sequentially written into the corresponding table of the structure configuration table to create a structure configuration table for building a data backup table sample corresponding to the data source table; the backup configuration information also includes various rule configuration fields of the source data that needs to be backed up in the data source table, so as to indicate where to obtain the source data for backup; The rule configuration fields include: backup rules, source table name, backup table name, whether effective, deletion rules, backup rules, creation time, and update time; wherein, when backing up the source data rules, the source data is screened in the corresponding data source table according to the settings of each rule configuration field in the rule configuration table.

2. The method for batch data backup according to claim 1, characterized in that: The selecting, based on the rule configuration table, a query instruction matching each of the rule configuration fields comprises: Extracting the backup rule fields corresponding to the respective rule configuration fields in the rule configuration table, and identifying the rule status of each of the backup rule fields; respectively determining whether the rule status of each backup rule field is valid; If valid, the query instruction corresponding to the backup rule field whose rule status is valid is selected.

3. The method for batch data backup according to any one of claims 1 to 2, characterized in that: After storing each of the backup data shards in a corresponding data backup table sample and deploying them on a preset target database node to obtain a data backup table corresponding to the data source table, the method further includes: Obtaining data modification rules and data modification parameters, and performing security verification on the data modification parameters; If the verification is successful, the successfully verified data modification parameters are used as variables and combined with the data modification rules to generate SQL statements; According to the SQL statement, a data modification operation is performed on the data backup table using a preset data modification tool to obtain a new data backup table.

4. The method for batch data backup according to claim 3, characterized in that: The step of performing a data modification operation on the data backup table using a preset data modification tool according to the SQL statement to obtain a new data backup table includes: Determine the statement type of the SQL statement, and extract the table name, field name and modification condition to be processed in the SQL statement according to the statement type of the SQL statement; Generate a query impact statement, a new and old value change statement, and a backup statement corresponding to the SQL statement according to the table name, the field name, and the modification condition; Execute the SQL statement based on the query impact statement, the new and old value change statement, and the backup statement; According to the execution result of the SQL statement, the target data in the data backup table is modified to obtain a new data backup table.

5. A data batch backup device, characterized in that: The data batch backup device comprises: A configuration module, used to obtain backup configuration information corresponding to multiple source databases, and configure a structure configuration table and a rule configuration table using the backup configuration information; A construction module, used to screen out the data source table to be backed up based on the structure configuration table, and construct a data backup table sample according to the table structure of the data source table; A selection module, configured to select source data corresponding to the data backup table sample from the data source table based on the rule configuration table, and configure multiple backup data slices using each source data; A backup module, used to store each of the backup data slices into a corresponding data backup table sample and deploy it on a preset target database node to obtain a data backup table corresponding to the data source table; The configuration module includes: a structure configuration unit, used to extract source table identification information in the backup configuration information, and use the source table identification information to create a structure configuration table; a rule configuration unit, used to extract multiple rule configuration fields in the backup configuration information, and use each of the rule configurations to create a rule configuration table; The selection module includes: an instruction selection unit, which is used to select a query instruction matching each of the rule configuration fields based on the rule configuration table; an acquisition unit, which is used to call a preset interface according to the query instruction to obtain a plurality of class attribute information corresponding to the data backup table sample; a data selection unit, which is used to select source data corresponding to the data backup table sample according to each of the class attribute information; The backup configuration information includes source table identification information of the data source table that needs to be backed up, which is sequentially written into the corresponding table of the structure configuration table to create a structure configuration table for building a data backup table sample corresponding to the data source table; the backup configuration information also includes various rule configuration fields of the source data that needs to be backed up in the data source table, so as to indicate where to obtain the source data for backup; The rule configuration fields include: backup rules, source table name, backup table name, whether effective, deletion rules, backup rules, creation time, and update time; wherein, when backing up the source data rules, the source data is screened in the corresponding data source table according to the settings of each rule configuration field in the rule configuration table.

6. The data batch backup device according to claim 5, characterized in that: The data batch backup device also includes a modification module, which is used to: Obtaining data modification rules and data modification parameters, and performing security verification on the data modification parameters; If the verification is successful, the successfully verified data modification parameters are used as variables and combined with the data modification rules to generate SQL statements; According to the SQL statement, a data modification operation is performed on the data backup table using a preset data modification tool to obtain a new data backup table.

7. A data batch backup device, characterized in that: The data batch backup device comprises: a memory and at least one processor, wherein instructions are stored in the memory; The at least one processor calls the instruction in the memory to enable the data batch backup device to perform the steps of the data batch backup method according to any one of claims 1 to 4.

8. A computer-readable storage medium having instructions stored thereon, characterized in that: When the instructions are executed by the processor, the steps of the data batch backup method as described in any one of claims 1 to 4 are implemented.

Images