List acquisition method and device based on database backup recovery BR

Abstract

The invention provides a list acquisition method and device based on database backup recovery BR, and relates to the technical field of data processing, in particular to the field of cloud computing and big data. According to the specific implementation scheme, BR operation is executed on a database so as to export a backup file containing metadata from the database; in the execution process of the BR operation, synchronously analyzing the exported backup file to obtain target attribute information required by the metadata list; and generating a metadata list according to the target attribute information, and storing the metadata list.

Description

Technical Field

[0001] The present disclosure relates to the field of data processing technology, in particular to the field of cloud computing and big data, and specifically to a method and device for obtaining a list based on database backup and recovery BR. Background Art

[0002] Inventory export can be understood as the enumeration and persistence of metadata of storage system data (objects, directories or files), which can be implemented based on the service internally or based on offline tasks. When exporting inventories of large-scale data, offline tasks are usually used to complete it. However, the current offline inventory export relies on many external components, consumes too much computing and storage resources, and has poor maintainability. Summary of the invention

[0003] The present disclosure provides a method and device for obtaining a list based on database backup and recovery BR.

[0004] According to one aspect of the present disclosure, a method for obtaining a list of BRs based on database backup recovery is provided, comprising:

[0005] Performing a BR operation on a database to export a backup file containing metadata from the database;

[0006] During the execution of the BR operation, the exported backup file is synchronously parsed to obtain target attribute information required by the metadata list;

[0007] The metadata list is generated according to the target attribute information, and the metadata list is stored.

[0008] According to another aspect of the present disclosure, there is provided a list acquisition device for restoring BR based on database backup, comprising:

[0009] An export module, used for performing a BR operation on a database to export a backup file containing metadata from the database;

[0010] A parsing module, used for synchronously parsing the exported backup file during the execution of the BR operation to obtain target attribute information required by the metadata list;

[0011] The acquisition module is used to generate the metadata list according to the target attribute information and store the metadata list.

[0012] According to a third aspect of the present disclosure, there is provided an electronic device, at least one processor; and

[0013] a memory communicatively connected to the at least one processor; wherein,

[0014] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can perform the method described in the embodiment of the first aspect.

[0015] According to a fourth aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable the computer to execute the method described in the embodiment of the first aspect.

[0016] According to a fifth aspect of the present disclosure, a computer program product is provided, including a computer program, wherein the computer program implements the method described in the embodiment of the first aspect when executed by a processor.

[0017] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The accompanying drawings are used to better understand the present solution and do not constitute a limitation of the present disclosure.

[0019] Figure 1 It is a schematic diagram of a method for obtaining a list based on database backup and recovery BR provided in an embodiment of the present disclosure;

[0020] Figure 2 It is a schematic diagram of another method for obtaining a list based on database backup and recovery BR provided by an embodiment of the present disclosure;

[0021] Figure 3 It is a schematic diagram of another method for obtaining a list based on database backup and recovery BR provided by an embodiment of the present disclosure;

[0022] Figure 4 It is a schematic diagram of another method for obtaining a list based on database backup and recovery BR provided by an embodiment of the present disclosure;

[0023] Figure 5 It is a logic diagram of a method for obtaining a list based on database backup and recovery BR provided in an embodiment of the present disclosure;

[0024] Figure 6 It is a block diagram of a list acquisition device based on database backup and recovery BR provided by an embodiment of the present disclosure;

[0025] Figure 7 A schematic block diagram of an example electronic device that can be used to implement an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0026] The following is a description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0027] Data processing is the collection, storage, retrieval, processing, transformation and transmission of data. It runs through all areas of social production and social life. Its basic purpose is to extract and derive valuable and meaningful data for certain specific people from large amounts of disorganized and difficult to understand data.

[0028] Cloud computing is a type of distributed computing. It refers to breaking down huge data computing programs into countless small programs through the network "cloud". These small programs are then processed and analyzed by a system composed of multiple servers to obtain results and return them to users. It can complete the processing of tens of thousands of data in a very short time (a few seconds), thereby achieving powerful network services.

[0029] Big Data, or massive data, refers to large or complex data sets that are not adequately processed by traditional data processing applications. It can also be defined as large amounts of unstructured or structured data from various sources.

[0030] The present disclosure can be applied to the distributed storage scenario of massive user data. The existing list export based on internal service is suitable for list export when the data volume is small. It is achieved by regularly initiating List operation to obtain part or all of the meta information of the corresponding data entry, and after serializing it in a specified format, the final result is persisted to a specific location, such as local or online storage; the list export based on offline task is to externally implement the list export scheduled task, directly interact with the metadata service, periodically pull the data meta information, parse and filter the meta information, and then serialize it to the local or online storage in a specific format. It does not interact or couple with the internal services of the system, and is suitable for the list export of massive data. However, the current offline export relies on external components such as data warehouses, which is complex to implement and has poor maintainability. In addition, it consumes a lot of computing and storage resources. Insufficient resources may cause task failure or delay, and the overall execution time is uncontrollable.

[0031] At present, the implementation process of offline inventory export is usually to transfer data metadata from the database and regularly merge it into the table specified by the Hive data warehouse. The offline export service regularly queries the Hive table, filters out the data metadata required for inventory export, serializes the inventory information and delivers it to the specified location, such as local or online storage. It can be seen that the current offline export implementation depends on the Hive data warehouse, and the metadata needs to be imported into Hive for filtering to obtain the final result. It needs to rely on additional components, and the implementation is relatively complex and there is a waste of additional storage space. There are also multiple writes and exports, and the process is too long, with obvious time loss.

[0032] Figure 1 is a schematic diagram of a method for obtaining a list based on database backup and recovery BR provided by an embodiment of the present disclosure; Figure 1 As shown, the method includes:

[0033] S101: Perform a BR operation on a database to export a backup file containing metadata from the database.

[0034] In some implementations, the backup-restore (BR) operation is a process of copying the complete data in the database to another location so that it can be safely restored in the event of hardware, network, disk, etc. failures. Optionally, the master node can initiate a backup request, which includes a BR backup operation, so that the BR operation is performed on the database to obtain a backup file.

[0035] It can be understood that a database is a warehouse that organizes, stores and manages data according to a data structure, and is a collection of related data stored together in a certain organizational manner; performing BR operations on a database, that is, backing up and restoring data from the database, thereby exporting a backup file containing metadata from the database.

[0036] Metadata is data about data. It provides information about the data's attributes, organization, domains, and relationships. It is mainly used to describe data and its environment. Metadata is not just information such as the data's type, name, value, etc., but a group of information used to describe the data. This information reflects certain characteristics of the data.

[0037] S102: During the execution of the BR operation, the exported backup file is synchronously parsed to obtain target attribute information required by the metadata list.

[0038] In some implementations, the exported backup files can be obtained regularly and analyzed synchronously. That is, during the data backup process performed by the BR operation, the exported backup data can be analyzed synchronously to reduce the time consumption of analysis after the backup is completed.

[0039] Optionally, the backup file to be exported can be divided into multiple sub-files, each sub-file is stored in the cloud storage when exported, and the sub-files in the cloud storage are checked regularly. When the sub-file backup is completed, the exported backup sub-file is obtained and parsed to obtain parsed data, thereby reducing time consumption.

[0040] In some implementations, the parsed data includes attribute information of the metadata, and the target attributes required by the pre-configured metadata list can be obtained, and the target attribute information required for the metadata list can be determined from the parsed data, that is, the parsed data is further screened and filtered according to the required target attributes to obtain the target attribute information required for the metadata list.

[0041] S103: Generate a metadata list according to the target attribute information, and store the metadata list.

[0042] The corresponding metadata is determined according to the target attribute information, the metadata corresponding to the target attribute information is spliced ​​to obtain a metadata list, and then the metadata list is stored. In some implementations, the metadata list can also be formatted according to the storage format requirements, and the format-converted metadata list is stored to improve the efficiency of list export.

[0043] In this embodiment, a BR operation is performed from the database to export the backup file. During the backup file export process, the exported backup file is synchronously parsed to reduce the time consumption of backup and parsing, obtain the target attribute information required for the metadata list, and generate the metadata list according to the target attribute information for storage. The whole list export process does not need to rely on additional components and excessive resource consumption, and the problem of list exporting of massive data is solved with higher efficiency and lower resource consumption. The list export process is simpler and more efficient.

[0044] Figure 2 is a schematic diagram of another method for obtaining a list based on database backup and recovery BR provided by an embodiment of the present disclosure; Figure 2 As shown, the method includes:

[0045] S201: Perform a BR operation on a database to export a backup file containing metadata from the database.

[0046] In the embodiments of the present disclosure, the implementation method of step S201 can be implemented by any of the methods in the embodiments of the present disclosure, which is not limited here and will not be described in detail.

[0047] S202, during the execution of the BR operation, the task execution status of the candidate nodes in the distributed system is obtained, and based on the task execution status, a plurality of target nodes are scheduled.

[0048] In some implementations, the task execution status of the candidate node refers to the file backup status, and multiple target nodes are scheduled according to the task execution status. For example, when the backup of some files is completed, multiple target nodes are scheduled to parse the exported backup files to improve the parsing efficiency.

[0049] S203, sending a backup parsing request to the target node, and the target node pulls the backup file for parsing.

[0050] Optionally, the backup parsing request can be a request sent by the Master node to the target node to instruct the target node to pull the backup file for parsing; the backup and parsing are performed simultaneously without waiting for the overall backup operation to be completed. In the case of large data, the backup and parsing can be completed almost at the same time. Compared with the existing list export based on the Hive data warehouse, it is necessary to filter the information output results from the Hive data warehouse, which involves multiple reading and writing of data, while the present embodiment only involves one data reading and writing, that is, reading from the backup file and writing the parsing results to the specified location, which greatly shortens the overall time of the list export.

[0051] In some implementations, the storage information of the backup file can be determined and the backup file can be stored based on the storage information; optionally, the storage information can be carried in the backup request sent by the Master node, and the exported backup file can be stored according to the storage information in the backup request, that is, the exported backup file is stored in the storage address in the storage information.

[0052] After the target node receives the backup resolution request, the target node can obtain the storage information of the backup file from the backup resolution request, that is, the storage information of the backup file storage location included in the backup resolution request, obtain the storage information of the backup file and pull the backup file based on the storage information of the backup file, thereby realizing fast pulling of the backup file.

[0053] In some implementations, it is also possible to obtain a data storage standard file in the storage space where the storage information of the backup file is located, the data storage standard file includes the file identifier of the stored backup file; in response to the backup file being pulled, the pulling status of the backup file is marked; that is, the file identifier of the stored backup file is recorded in the data storage standard file, and after the backup file is pulled, the pulling status of the backup file is marked in the data storage standard file to determine that the current backup file has been pulled.

[0054] Furthermore, according to the pull status marks of the data storage standard files and the backup files, the unpulled backup files are determined, and the unpulled backup files stored in the storage space are pulled, thereby avoiding repeated parsing of the backup files and improving the parsing efficiency.

[0055] In some implementations, it is also possible to obtain a file list of parsed backup files maintained by the target node itself, and deduplicate the pulled backup files based on the file list. That is, the target node records the parsed backup files to obtain a file list, and when pulling and parsing the backup files, the backup files in the file list are removed to avoid repeated parsing of the backup files.

[0056] S204: Acquire target attribute information required by the metadata list.

[0057] In some implementations, the metadata in the backup file can be parsed to obtain a set of attribute information; in response to the end of the BR operation, the attribute information set is filtered according to the list configuration information to obtain the target attribute information; that is, the attribute information set in all metadata in the backup file is determined by parsing the metadata in the backup file, and the attribute information can be data name, data size, data type, data structure (such as length, field or data column) and data association information (location or owner) and other information; the attribute information set is filtered according to the pre-set list configuration information, and the list configuration information is the attribute information required for the client configuration, so the required target attribute information is filtered out from the attribute information set according to the list configuration information to reduce resource waste.

[0058] In some other implementations, in response to the BR operation ending, a subset of attribute information parsed by the target node may be obtained; the attribute information subsets are combined to obtain an attribute information set, and the attribute information set is filtered according to the manifest configuration information to obtain the target attribute information.

[0059] That is, obtain the attribute information subset of the backup file parsed by each target node, combine the attribute information subsets parsed by all target nodes, obtain the attribute information set in all metadata in the backup file, and further filter the attribute information set according to the pre-set inventory configuration information to screen out the required target attribute information, improve the accuracy of inventory export, and avoid resource consumption on the critical path.

[0060] S205: Generate a metadata list according to the target attribute information, and store the metadata list.

[0061] In the embodiments of the present disclosure, the implementation method of step S205 can be implemented by any of the methods in the embodiments of the present disclosure, which is not limited here and will not be described in detail.

[0062] In this embodiment, during the process of performing BR operation on the database, the file backup status is determined, and when the file backup is completed, multiple target nodes are scheduled to parse the exported backup files to improve the parsing efficiency. By sending a backup parsing request carrying storage information, the target node is instructed to pull the exported backup files in the storage location. During the pulling process, the backup files that have been parsed and processed can be screened by file identifiers or the file list of the target node itself, avoiding repeated pulling and repeated parsing of the exported backup files, reducing resource consumption in the processing process, and obtaining target attribute information through efficient parsing of multiple target nodes. Irrelevant attribute information is filtered and screened through pre-configured list configuration information, ensuring the accuracy of the metadata list exported by the list, storing the metadata list finally obtained, reducing multi-disk reading and writing and frequent reading of intermediate results, avoiding repeated operations and repeated processing, and improving the list export processing efficiency.

[0063] Figure 3 is a schematic diagram of another method for obtaining a list based on database backup and recovery BR provided by an embodiment of the present disclosure; Figure 3 As shown, the method includes:

[0064] S301: Perform a BR operation on a database to export a backup file containing metadata from the database.

[0065] In the embodiments of the present disclosure, the implementation method of step S301 may be implemented in any manner in the embodiments of the present disclosure, which is not limited here and will not be described in detail.

[0066] S302: During the execution of the BR operation, the exported backup file is synchronously parsed to obtain target attribute information required by the metadata list.

[0067] In the embodiments of the present disclosure, the implementation method of step S302 may be implemented in any manner in the embodiments of the present disclosure, which is not limited here and will not be described in detail.

[0068] S303: Generate the full path of each directory according to the target attribute information.

[0069] Optionally, the directory attribute information of each directory can be obtained from the target attribute information; the path is traced back according to the directory attribute information to obtain the full path of each directory. In some implementations, the directory attribute information may include information such as inodes, file types and sizes of each directory index node; inode is a unique identifier of a file, which is used to quickly locate the location of a file or directory.

[0070] In some implementations, when tracing the path, the parent directory of the directory can be obtained, and the directories belonging to the same parent directory can be determined; based on the full path of the parent directory, the full path of the directories belonging to the same parent directory can be obtained; and the full path of the parent directory and the directories belonging to the parent directory can be added to the metadata list.

[0071] It can be understood that the parent directory is the parent directory of a directory, that is, the path is traced back according to the directory attribute information to determine the parent directory of the directory and the directories belonging to the same parent directory. The full path of the parent directory is determined according to the directory attribute information, and then the full path of the directory belonging to the same parent directory is determined according to the full path of the parent directory, so as to reduce the number of full path splicing and effectively reduce the time spent on path backtracing. After determining the full path of each directory, the full path of the parent directory and the directory belonging to the parent directory is added to the metadata list, that is, the metadata list includes the full path information of the directory.

[0072] S304: Generate a metadata list according to the target attribute information and the full path of the directory.

[0073] It can be understood that the required metadata in the backup parsing file is filtered out according to the target attribute information, and the filtered metadata information and the full path information of the directory are used as the metadata list, that is, the metadata list includes the exported list data and the full path information of the corresponding directory.

[0074] S305: Store the metadata list.

[0075] Optionally, the metadata list may be formatted to obtain a target metadata list; the target metadata list is processed in a unified data format to improve data readability and facilitate data processing; further, storage information of the metadata list is obtained, the storage information includes location information where the metadata list is to be stored, and the target metadata list is stored according to the storage information.

[0076] In the embodiments of the present disclosure, the implementation method of step S305 can be implemented by any of the methods in the embodiments of the present disclosure, which is not limited here and will not be described in detail.

[0077] In this embodiment, after the backup file is parsed and the target attribute information is obtained, backtracking is performed according to the directory attribute information in the target attribute information to obtain the full path information corresponding to each directory, and in the directory backtracking process, the full path of each parent directory is recorded, and based on the full path of the parent directory, the full path information of the directory belonging to the same parent directory is obtained, thereby reducing the number of full path splicing and the time consumption of path backtracking, and generating a metadata list according to the full path of the spliced ​​directory and the filtered target attribute information obtained by parsing, and formatting the metadata list to obtain a target metadata list that is easy to operate and easy to store, and storing the target metadata list, thereby reducing the dependence on external components and the time consumption during the list export process, and having obvious performance advantages compared to the existing list export method.

[0078] Figure 4 is a schematic diagram of another method for obtaining a list based on database backup and recovery BR provided by an embodiment of the present disclosure; Figure 4 As shown, the method includes:

[0079] S401: Perform a BR operation on a database to export a backup file containing metadata from the database.

[0080] In the embodiments of the present disclosure, the implementation method of step S401 may be implemented in any manner in the embodiments of the present disclosure, which is not limited here and will not be described in detail.

[0081] S402, during the execution of the BR operation, the task execution status of the candidate nodes in the distributed system is obtained, and based on the task execution status, a plurality of target nodes are scheduled.

[0082] In the embodiments of the present disclosure, the implementation method of step S402 may be implemented in any manner in the embodiments of the present disclosure, which is not limited here and will not be described in detail.

[0083] S403, sending a backup parsing request to the target node, and the target node pulls the backup file for parsing.

[0084] In the embodiments of the present disclosure, the implementation method of step S403 may be implemented in any manner in the embodiments of the present disclosure, which is not limited here and will not be described in detail.

[0085] S404, obtaining target attribute information required by the metadata list.

[0086] In the embodiments of the present disclosure, the implementation method of step S404 may be implemented in any manner in the embodiments of the present disclosure, which is not limited here and will not be described in detail.

[0087] S405: Generate the full path of each directory according to the target attribute information.

[0088] In the embodiments of the present disclosure, the implementation method of step S405 can be implemented by any of the methods in the embodiments of the present disclosure, which is not limited here and will not be described in detail.

[0089] S406: Generate a metadata list according to the target attribute information and the full path of the directory.

[0090] In the embodiments of the present disclosure, the implementation method of step S406 may be implemented in any manner in the embodiments of the present disclosure, which is not limited here and will not be described in detail.

[0091] S407: Store the metadata list.

[0092] In the embodiments of the present disclosure, the implementation method of step S407 may be implemented in any manner in the embodiments of the present disclosure, which is not limited here and will not be described in detail.

[0093] In this embodiment, during the process of performing BR operation on the database, multiple target nodes are scheduled to parse the exported backup files to improve the parsing efficiency. By sending a backup parsing request carrying storage information, the target node is instructed to pull the exported backup files in the storage location. During the pulling process, the backup files that have been parsed can be filtered by file identifiers or the file list of the target node itself, so as to avoid repeated pulling and parsing of the exported backup files and reduce resource consumption in the processing process. After the backup file is parsed and the target attribute information is obtained, the directory attribute information in the target attribute information is backtracked to obtain the corresponding backup files of each directory. The full path information of the corresponding directory is obtained, and the full path of each parent directory is recorded during the directory backtracking process. Based on the full path of the parent directory, the full path information of the directory belonging to the same parent directory is obtained, so as to reduce the number of full path splicing and the time consumption of path backtracking. According to the full path of the spliced ​​directory and the filtered target attribute information obtained by parsing, a metadata list is generated and formatted to obtain a target metadata list that is easy to operate and easy to store. This embodiment can reduce the dependence on external components and the time consumption in the list export process, solve the list export problem of massive data with higher efficiency and lower resource consumption, and the list export process is simpler and more efficient.

[0094] Figure 5 is a logic diagram of a method for obtaining a list based on database backup and recovery BR provided by an embodiment of the present disclosure; Figure 5As shown, when the task is initiated, the Master node initiates a round of inventory export tasks, that is, the BR operation is performed on the database (DataBase, DB) to export a backup file, and the backup file includes metadata information. During the execution of the BR operation, the Master node sends a backup parsing request to multiple target nodes to instruct the target nodes to pull and parse the backup file to obtain all attribute information, filter all attribute information to obtain the attribute information required for the metadata list, and further splice the entire directory based on the attribute information to obtain the full path of each directory. The full path of the directory and the attribute information are serialized to the specified format to obtain the metadata list, and the list information is pushed, that is, the exported list information is stored so that users can obtain and view the list information, thereby realizing efficient export of list data.

[0095] Figure 6 is a block diagram of a list acquisition device based on database backup and recovery BR provided by an embodiment of the present disclosure; Figure 6 As shown, the list acquisition device 600 based on database backup recovery BR includes:

[0096] The export module 601 is used to perform a BR operation on the database to export a backup file containing metadata from the database;

[0097] The parsing module 602 is used to synchronously parse the exported backup file during the execution of the BR operation to obtain the target attribute information required by the metadata list;

[0098] The acquisition module 603 is used to generate a metadata list according to the target attribute information and store the metadata list.

[0099] In some implementations, the parsing module 602 includes:

[0100] Parse the metadata in the backup file to obtain a set of attribute information;

[0101] In response to the BR operation ending, the attribute information set is filtered according to the inventory configuration information to obtain target attribute information.

[0102] In some implementations, the acquisition module 603 includes:

[0103] Generate the full path of each directory based on the target attribute information;

[0104] Generates a metadata list based on the target attribute information and the full path of the directory.

[0105] In some implementations, the acquisition module 603 includes:

[0106] Obtain directory attribute information of each directory from the target attribute information;

[0107] Perform path backtracking based on directory attribute information to obtain the full path of each directory.

[0108] In some implementations, the apparatus 600 further includes:

[0109] Get the parent directory of a directory and determine the directories that belong to the same parent directory;

[0110] Based on the full path of the parent directory, get the full path of the directory belonging to the same parent directory;

[0111] Add the full path of the parent directory and the directories under it to the metadata manifest.

[0112] In some implementations, the acquisition module 603 includes:

[0113] Formatting the metadata list to obtain a target metadata list;

[0114] The storage information of the metadata list is obtained, and the target metadata list is stored according to the storage information.

[0115] In some implementations, the parsing module 602 includes:

[0116] Obtain the task execution status of candidate nodes in the distributed system, and schedule multiple target nodes based on the task execution status;

[0117] A backup parsing request is sent to the target node, and the target node pulls the backup file for parsing.

[0118] In some implementations, the parsing module 602 includes:

[0119] In response to the BR operation ending, obtaining a subset of attribute information parsed by the target node;

[0120] The attribute information subsets are combined to obtain an attribute information set, and the attribute information set is filtered according to the inventory configuration information to obtain target attribute information.

[0121] In some implementations, the apparatus 600 further includes:

[0122] Determine storage information of the backup file, and store the backup file based on the storage information;

[0123] The target node obtains the storage information of the backup file from the backup resolution request, and pulls the backup file based on the storage information of the backup file.

[0124] In some implementations, the parsing module 602 includes:

[0125] Obtaining a data storage standard file in the storage space where the storage information of the backup file is located, the data storage standard file including a file identifier of the stored backup file;

[0126] In response to the backup file being pulled, marking the pulling status of the backup file;

[0127] According to the pull status marks of the data storage standard files and the backup files, the unpulled backup files are determined, and the unpulled backup files stored in the storage space are pulled.

[0128] In some implementations, the apparatus 600 further includes:

[0129] Get the file list of the parsed backup files maintained by the target node itself, and deduplicate the pulled backup files based on the file list.

[0130] In this embodiment, during the process of performing BR operation on the database, multiple target nodes are scheduled to parse the exported backup files to improve the parsing efficiency. By sending a backup parsing request carrying storage information, the target node is instructed to pull the exported backup files in the storage location. During the pulling process, the backup files that have been parsed can be filtered by file identifiers or the file list of the target node itself, so as to avoid repeated pulling and parsing of the exported backup files and reduce resource consumption in the processing process. After the backup file is parsed and the target attribute information is obtained, the directory attribute information in the target attribute information is backtracked to obtain the corresponding backup files of each directory. The full path information of the corresponding directory is obtained, and the full path of each parent directory is recorded during the directory backtracking process. Based on the full path of the parent directory, the full path information of the directory belonging to the same parent directory is obtained, so as to reduce the number of full path splicing and the time consumption of path backtracking. According to the full path of the spliced ​​directory and the filtered target attribute information obtained by parsing, a metadata list is generated and formatted to obtain a target metadata list that is easy to operate and easy to store. This embodiment can reduce the dependence on external components and the time consumption in the list export process, solve the list export problem of massive data with higher efficiency and lower resource consumption, and the list export process is simpler and more efficient.

[0131] In the technical solution disclosed herein, the acquisition, storage and application of user personal information involved are in compliance with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0132] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium and a computer program product.

[0133] Figure 7A schematic block diagram of an example electronic device that can be used to implement an embodiment of the present disclosure is shown. Electronic device 700 is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.

[0134] like Figure 7 As shown, the device 700 includes a computing unit 701, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 702 or a computer program loaded from a storage unit 708 into a random access memory (RAM) 703. In the RAM 703, various programs and data required for the operation of the device 700 can also be stored. The computing unit 701, the ROM 702, and the RAM 703 are connected to each other via a bus 704. An input / output (I / O) interface 705 is also connected to the bus 704.

[0135] A number of components in the device 700 are connected to the I / O interface 705, including: an input unit 706, such as a keyboard, a mouse, etc.; an output unit 707, such as various types of displays, speakers, etc.; a storage unit 708, such as a disk, an optical disk, etc.; and a communication unit 709, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 709 allows the device 700 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0136] The computing unit 701 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 701 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 701 performs the various methods and processes described above, such as a method for obtaining a list based on database backup recovery BR. For example, in some embodiments, the method for obtaining a list based on database backup recovery BR may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 708. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 700 via the ROM 702 and / or the communication unit 709. When the computer program is loaded into the RAM 703 and executed by the computing unit 701, one or more steps of the method for obtaining a list based on database backup recovery BR described above may be performed. Alternatively, in other embodiments, the computing unit 701 may be configured to execute the inventory acquisition method based on database backup recovery BR in any other appropriate manner (for example, by means of firmware).

[0137] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0138] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0139] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0140] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0141] The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0142] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship of client and server is generated by computer programs running on respective computers and having a client-server relationship with each other. The server may be a cloud server, a server of a distributed system, or a server combined with a blockchain.

[0143] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.

[0144] The above specific implementations do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. A method for obtaining a list of BRs based on database backup and recovery, comprising: Performing a BR operation on a database to export a backup file containing metadata from the database; During the execution of the BR operation, the exported backup file is synchronously parsed to obtain target attribute information required by the metadata list; The metadata list is generated according to the target attribute information, and the metadata list is stored.

2. The method according to claim 1, wherein: The synchronous parsing of the exported backup file to obtain target attribute information required by the metadata list includes: Parsing metadata in the backup file to obtain a set of attribute information; In response to the BR operation being completed, the attribute information set is filtered according to the inventory configuration information to obtain the target attribute information.

3. The method according to claim 1, wherein: The step of generating the metadata list according to the target attribute information includes: Generate the full path of each directory according to the target attribute information; The metadata list is generated according to the target attribute information and the full path of the directory.

4. The method according to claim 3, wherein: The generating of the full path of each directory according to the target attribute information includes: Acquire directory attribute information of each directory from the target attribute information; Path backtracking is performed according to the directory attribute information to obtain the full path of each directory.

5. The method according to claim 3 or 4, wherein: The method further comprises: Obtain the parent directory of the directory, and determine the directories that belong to the same parent directory; Based on the full path of the parent directory, obtain the full path of the directory belonging to the same parent directory; The full paths of the parent directory and the directories subordinate to the parent directory are added to the metadata list.

6. The method according to claim 1, wherein: The storing of the metadata list includes: Formatting the metadata list to obtain a target metadata list; The storage information of the metadata list is acquired, and the target metadata list is stored according to the storage information.

7. The method according to claim 1, wherein: The synchronously parsing the exported backup file includes: Obtaining task execution status of candidate nodes in the distributed system, and scheduling the multiple target nodes based on the task execution status; A backup parsing request is sent to the target node, and the target node pulls the backup file for parsing.

8. The method according to claim 7, wherein: The target attribute information required for obtaining the metadata list includes: In response to the BR operation ending, obtaining a subset of attribute information parsed by the target node; The attribute information subsets are combined to obtain an attribute information set, and the attribute information set is filtered according to the inventory configuration information to obtain the target attribute information.

9. The method according to claim 7, wherein: The method further comprises: Determining storage information of the backup file, and storing the backup file based on the storage information; The target node obtains the storage information of the backup file from the backup parsing request, and pulls the backup file based on the storage information of the backup file.

10. The method according to claim 7, wherein: The process of the target node pulling the backup file includes: Acquire a data storage standard file in the storage space where the storage information of the backup file is located, wherein the data storage standard file includes a file identifier of the stored backup file; In response to the backup file being pulled, marking a pulling status of the backup file; According to the data storage standard file and the pull status mark of the backup file, the unpulled backup file is determined, and the unpulled backup file stored in the storage space is pulled.

11. The method according to claim 7, wherein: The method further comprises: A file list of parsed backup files maintained by the target node itself is obtained, and the pulled backup files are deduplicated based on the file list.

12. A list acquisition device based on database backup and recovery BR, comprising: An export module, used for performing a BR operation on a database to export a backup file containing metadata from the database; A parsing module, used for synchronously parsing the exported backup file during the execution of the BR operation to obtain target attribute information required by the metadata list; The acquisition module is used to generate the metadata list according to the target attribute information and store the metadata list.

13. The device according to claim 12, wherein: The parsing module comprises: Parsing metadata in the backup file to obtain a set of attribute information; In response to the BR operation being completed, the attribute information set is filtered according to the inventory configuration information to obtain the target attribute information.

14. The device according to claim 12, wherein: The acquisition module comprises: Generate the full path of each directory according to the target attribute information; The metadata list is generated according to the target attribute information and the full path of the directory.

15. The device according to claim 14, wherein: The acquisition module comprises: Acquire directory attribute information of each directory from the target attribute information; Path backtracking is performed according to the directory attribute information to obtain the full path of each directory.

16. The device according to claim 14 or 15, wherein: The device also includes: Obtain the parent directory of the directory, and determine the directories that belong to the same parent directory; Based on the full path of the parent directory, obtain the full path of the directory belonging to the same parent directory; The full paths of the parent directory and the directories subordinate to the parent directory are added to the metadata list.

17. The device according to claim 12, wherein: The acquisition module comprises: Formatting the metadata list to obtain a target metadata list; The storage information of the metadata list is acquired, and the target metadata list is stored according to the storage information.

18. The device according to claim 12, wherein: The parsing module comprises: Obtaining task execution status of candidate nodes in the distributed system, and scheduling the multiple target nodes based on the task execution status; A backup parsing request is sent to the target node, and the target node pulls the backup file for parsing.

19. The device according to claim 18, wherein: The parsing module comprises: In response to the BR operation ending, obtaining a subset of attribute information parsed by the target node; The attribute information subsets are combined to obtain an attribute information set, and the attribute information set is filtered according to the inventory configuration information to obtain the target attribute information.

20. The device according to claim 18, wherein The device also includes: Determining storage information of the backup file, and storing the backup file based on the storage information; The target node obtains the storage information of the backup file from the backup parsing request, and pulls the backup file based on the storage information of the backup file.

21. The device according to claim 18, wherein The parsing module comprises: Acquire a data storage standard file in the storage space where the storage information of the backup file is located, wherein the data storage standard file includes a file identifier of the stored backup file; In response to the backup file being pulled, marking a pulling status of the backup file; According to the data storage standard file and the pull status mark of the backup file, the unpulled backup file is determined, and the unpulled backup file stored in the storage space is pulled.

22. The device according to claim 18, wherein The device also includes: A file list of parsed backup files maintained by the target node itself is obtained, and the pulled backup files are deduplicated based on the file list.

23. An electronic device, comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 11.

24. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause the computer to execute the method according to any one of claims 1-11.

25. A computer program product comprising a computer program, which, when executed by a processor, implements the steps of the method according to any one of claims 1 to 11.

Images