Method and apparatus for obtaining directory quota information based on database backup recovery (BR)

Abstract

The present disclosure provides a directory quota information acquisition method and device based on database backup recovery (BR), relates to the technical field of data processing, and particularly relates to the fields of cloud computing and big data. The specific implementation scheme is as follows: performing a BR operation on a database to export a backup file containing metadata from the database; in the execution process of the BR operation, synchronously analyzing the exported backup file to acquire directory attribute information; generating a directory full path according to the directory attribute information, and acquiring quota information of each directory according to the directory full path.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of data processing, in particular to the field of cloud computing and big data, and more particularly to a directory quota information obtaining method and device based on database backup recovery BR. BACKGROUND

[0002] The concept of quota originates from a local file system. Directory quota is directory-granularity, which identifies the total amount of available space and index node of the corresponding directory. When the user's usage of the directory space / inode exceeds the set quota total amount, the system usually refuses the user's subsequent write or creation, so as to avoid possible resource over-limit or depletion.

[0003] There are two ways to count directory quota, namely incremental real-time statistics and timing analysis based on full amount. However, the incremental real-time statistics method needs specific restrictions and has poor usability, while the timing analysis based on full amount will lose some timeliness, so a directory quota obtaining method that takes into account timeliness and ease of use is needed. SUMMARY

[0004] The present disclosure provides a directory quota information obtaining method and device based on database backup recovery BR.

[0005] According to an aspect of the present disclosure, a directory quota information obtaining method based on database backup recovery BR is provided, comprising:

[0006] performing a BR operation on a database to export a backup file containing metadata from the database;

[0007] synchronously parsing the exported backup file to obtain directory attribute information during the execution of the BR operation;

[0008] generating a directory full path according to the directory attribute information, and obtaining quota information of each directory according to the directory full path.

[0009] According to another aspect of the present disclosure, a directory quota information obtaining device based on database backup recovery BR is provided, comprising:

[0010] an exporting module configured to perform a BR operation on a database to export a backup file containing metadata from the database;

[0011] a parsing module configured to synchronously parse the exported backup file to obtain directory attribute information during the execution of the BR operation;

[0012] The acquisition module is configured to generate a full path of the directory according to the directory attribute information, and acquire the quota information of each directory according to the full path of the directory.

[0013] According to a third aspect of the present disclosure, an electronic device is provided, comprising:

[0014] at least one processor; and

[0015] a memory in communication with the at least one processor; wherein

[0016] The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method of the first aspect.

[0017] According to a fourth aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable the computer to perform the method of the first aspect.

[0018] According to a fifth aspect of the present disclosure, a computer program product is provided, comprising a computer program which, when executed by a processor, implements the steps of the method of the first aspect.

[0019] It should be understood that the contents described in this part are not intended to identify key or important features of the embodiments of the present disclosure, nor to limit the scope of the present disclosure. Other features of the present disclosure will become apparent from the following description. BRIEF DESCRIPTION OF DRAWINGS

[0020] The accompanying drawings are used to better understand the present scheme, and do not constitute a limitation on the present disclosure. Among them:

[0021] Figure 1 is a schematic diagram of a directory quota information acquisition method based on database backup recovery BR provided by an embodiment of the present disclosure;

[0022] Figure 2 is a schematic diagram of another directory quota information acquisition method based on database backup recovery BR provided by an embodiment of the present disclosure;

[0023] Figure 3 is a schematic diagram of another directory quota information acquisition method based on database backup recovery BR provided by an embodiment of the present disclosure;

[0024] Figure 4 is a schematic diagram of another directory quota information acquisition method based on database backup recovery BR provided by an embodiment of the present disclosure;

[0025] Figure 5is a logic diagram of another method for obtaining directory quota information based on database backup recovery (BR) provided by an embodiment of the present disclosure;

[0026] Figure 6 is a block diagram of a device for obtaining directory quota information based on database backup recovery (BR) provided by an embodiment of the present disclosure;

[0027] Figure 7 A schematic block diagram of an example electronic device that can be used to implement embodiments of the present disclosure is shown. DETAILED DESCRIPTION

[0028] Exemplary embodiments of the present disclosure are described herein with reference to the accompanying drawings, which are provided to assist in the understanding of the present disclosure, and should be considered in their entirety when interpreting the disclosure. Accordingly, those of ordinary skill in the art will recognize that the embodiments described herein can be practiced with various modifications and alterations, and that the disclosure is not limited to the embodiments described herein. Similarly, although claims can be dependent on other claims in this or other embodiments, their scope does not necessarily involve the combination of all limitations of the dependent claim. In addition, any single claim can include one, some or all limitations of that claim. The disclosure is not limited to the embodiments described herein, but can be practiced with variations that will occur to those of ordinary skill in the art upon reading the description. Therefore, the disclosure should not be considered limited to the embodiments described herein, but should be understood to include all variations that will occur to those of ordinary skill in the art upon reading the description. The words "comprise," "comprising," "include," "including," and "has" are used synonymously to mean "including but not limited to."

[0029] Data processing is the collection, storage, retrieval, processing, transformation and transmission of data, which is used in various fields of social production and social life. The basic purpose is to extract and deduce data that is valuable and meaningful to certain people from a large amount of disorganized and difficult-to-understand data.

[0030] Cloud computing is a kind of distributed computing, which means that a huge data processing program is divided into countless small programs through the network "cloud", and then the system composed of multiple servers processes and analyzes these small programs to get the results and return them to the user, which can complete the processing of hundreds of thousands of data in a very short time (a few seconds), so as to achieve powerful network service.

[0031] Big data, or massive data, refers to a large or complex data set that cannot be processed by traditional data processing application software, or can be defined as a large amount of unstructured or structured data from various sources.

[0032] The present disclosure can be applied to the scenario of obtaining directory Quota in a distributed file storage system with massive data. The current directory Quota function is incomplete or limited in use, which is manifested in limiting directory renaming operation, and the number of directory Quota layers and quantity, etc. The incremental real-time statistical method increases the complexity of file system design and implementation, and at the same time causes the decline of file system performance; the offline method for timing full analysis prolongs the directory Quota statistical time length, and sacrifices timeliness, which is not friendly to delay-sensitive businesses.

[0033] The current directory Quota acquisition method is based on the increment of input output (Input Output, IO) real-time statistics, which is divided into binary log (binary log, binlog) export or IO-based companion statistics; based on metadata snapshot, periodic snapshot file and parsing, output the text that can be recognized by Map Reduce (MR) job, and perform directory Quota statistics in the form of MR job; the parsing of snapshot file and MR job in the acquisition method is difficult to estimate the time cost, and the timeliness performance is poor, and a large amount of system resources will be consumed in the snapshot or parsing process, and the disk space may be exhausted, affecting the overall service stability.

[0034] Figure 1 is a schematic diagram of a directory quota information acquisition method based on database backup and recovery (BR) provided by the embodiment of the disclosure; as shown in Figure 1 , the method comprises:

[0035] S101, performing a BR operation on the database to export a backup file containing metadata from the database.

[0036] In some implementations, the backup and restore (Backup-Restore, BR) operation is a process of copying and storing complete data in the database to another location, so as to safely recover in the event of hardware, network, disk, etc. Failure. Optionally, a backup request can be initiated by the master node Master node, and the backup request includes the BR backup operation, so as to perform the BR operation on the database to obtain the backup file.

[0037] It can be understood that the database is a warehouse that organizes, stores and manages data according to data structure, and is a collection of related data stored together in a certain organization way; performing a BR operation on the database is to backup and restore data from the database, thereby exporting a backup file containing metadata from the database; in this embodiment, the BR operation obtains full backup of data in the database, compared with the real-time incremental statistics method, each round of quota statistics can be full correction, ensuring the accuracy of the directory Quota acquisition.

[0038] Metadata is data about data, which provides information about data attributes, organization, domain and their relationships, and is mainly used to describe data and its environment. Metadata is not only the type, name, value and other information of data, but also a set of information used to describe data, which reflects some aspects of data characteristics.

[0039] S102, in the execution process of the BR operation, synchronously parsing the exported backup file to obtain directory attribute information.

[0040] In some implementations, the exported backup file can be acquired in a timed manner, and the exported backup file can be synchronously parsed, that is, the exported backup data is synchronously parsed in the data backup process of the BR operation execution, so as to reduce the time consumption of parsing after the backup is completed.

[0041] Optionally, the backup file to be exported can be divided into a plurality of sub-files, each sub-file is stored to the cloud storage when being exported, the sub-files of the cloud storage are detected periodically, when the backup of the sub-file is completed, the exported backup sub-file is acquired and parsed to obtain parsed data, so as to reduce the time consumption.

[0042] In some implementations, the parsed data includes attribute information of the metadata, the directory-related attribute information is acquired from the attribute information, so as to obtain the directory attribute information; in some implementations, the required directory attribute can also be acquired, the required directory attribute information is determined from the parsed data, that is, the parsed data is filtered according to the required directory attribute, so as to obtain the directory attribute information.

[0043] S103, according to the directory attribute information, a directory full path is generated, and according to the directory full path, the quota information of each directory is acquired.

[0044] In some implementations, the directory full path of each directory can be generated according to the path backtracking of the directory attribute information; in some implementations, the directory attribute information can include inode, file type and size of each directory index node, and the like; the inode is the unique identifier of the file, which is used to quickly locate the position of the file or the directory.

[0045] In some implementations, when the path is backtracked, the parent directory of the directory can be acquired, and the directories belonging to the same parent directory can be determined; based on the full path of the parent directory, the full paths of the directories belonging to the same parent directory are acquired; the full paths of the parent directory and the directories belonging to the parent directory are added in the metadata list.

[0046] It can be understood that the parent directory is the upper directory of a directory, that is, according to the path backtracking of the directory attribute information, the parent directory of the directory is determined, the directories belonging to the same parent directory are determined, the full path of the parent directory is determined according to the directory attribute information, so as to determine the full paths of the directories belonging to the same parent directory according to the full path of the parent directory, so as to reduce the number of full path splicing and effectively reduce the time consumption of path backtracking.

[0047] Further, the quota information corresponding to each directory can be counted, for example, the file amount corresponding to the directory and its subdirectories in different full paths is obtained, the file amount of all subdirectories and the file amount of the directory are accumulated to obtain the file amount of the current directory, which is taken as the quota information corresponding to the current directory, that is, the quota information of the directory is obtained.

[0048] In the embodiment, the full backup file is obtained by performing the BR operation on the database, compared with the existing real-time incremental statistical method, the directory quota statistics based on the backup file in each round can be full rectification to ensure the accuracy of the directory quota; during the execution of the BR operation, the exported backup file is parsed synchronously, without waiting for the completion of the backup operation for parsing, which greatly shortens the time consumption of the directory quota statistics, the target attribute information obtained by parsing is used to backtrack the directory full path, and the quota information is counted according to the target full path, which reduces the resource consumption and improves the accuracy of the directory quota information acquisition.

[0049] Figure 2 is another schematic diagram of a directory quota information acquisition method based on database backup and recovery (BR) provided by the embodiment of the disclosure; as shown in Figure 2 the method comprises:

[0050] S201, performing a BR operation on the database to export a backup file containing metadata from the database.

[0051] In the embodiment of the disclosure, the implementation method of step S201 can be implemented by any one of the embodiments of the disclosure, which is not limited here and will not be repeated.

[0052] S202, during the execution of the BR operation, obtaining the task execution situation of the candidate node, and scheduling a plurality of target nodes based on the task execution situation.

[0053] In some implementations, the task execution situation of the candidate node refers to the file backup situation, and a plurality of target nodes are scheduled according to the task execution situation, for example, when part of the file backup is completed, a plurality of target nodes are scheduled to parse the exported backup file to improve the parsing efficiency.

[0054] S203, sending a backup parsing request to the target node, and pulling the backup file by the target node for parsing.

[0055] Optionally, the backup analysis request can be a request sent by the Master node to the target node to instruct the target node to pull the backup file for analysis; the backup and analysis are synchronized, without waiting for the completion of the overall backup operation, compared with the traditional serial directory quota statistics based on the fast dump and analysis of the snapshot, the overall time of the directory quota statistics can be greatly shortened.

[0056] In some implementations, storage information of the backup file can be determined, and the backup file can be stored based on the storage information; optionally, the storage information can be carried in the backup request sent by the Master node, and the exported backup file can be stored according to the storage information in the backup request, that is, the exported backup file is stored in the storage address in the storage information.

[0057] After the target node receives the backup analysis request, the target node can obtain the storage information of the backup file from the backup analysis request, that is, the backup analysis request includes the storage information of the storage location of the backup file, the storage information of the backup file is obtained and the backup file is pulled based on the storage information of the backup file, to realize fast positioning and pulling of the backup file.

[0058] In some implementations, the data storage standard file in the storage space where the storage information of the backup file is located can also be obtained, the data storage standard file includes the file identifier of the stored backup file; in response to the backup file being pulled, the pulling state of the backup file is marked; that is, the file identifier of the stored backup file is recorded in the data storage standard file, and after the backup file is pulled, the pulling state of the backup file is marked in the data storage standard file to determine that the current backup file has been pulled.

[0059] Further, according to the data storage standard file and the pulling state marking of the backup file, the backup file that has not been pulled is determined, and the backup file that has not been pulled stored in the storage space is pulled, thereby avoiding repeated analysis of the backup file and improving the analysis efficiency.

[0060] In some implementations, the file list of the analyzed backup file maintained by the target node itself can also be obtained, and the pulled backup file is de-duplicated based on the file list, that is, the target node records the analyzed backup file to obtain the file list, and the backup file in the file list is removed when the backup file is pulled and analyzed, to avoid repeated analysis of the backup file.

[0061] S204, obtain directory attribute information.

[0062] In some implementations, the metadata in the backup file can be parsed to obtain a set of attribute information; in response to the end of the BR operation, the set of attribute information is filtered according to directory configuration information to obtain target attribute information; that is, the set of attribute information in all metadata in the backup file is determined by parsing the metadata in the backup file, and the attribute information can be information such as data name, data size, data type, data structure (such as length, field, or data column), and data association information (position or owner). The set of attribute information is filtered according to the pre-set directory configuration information, and the directory configuration information is the attribute information required by the client configuration, so the required directory attribute information is screened from the set of attribute information according to the directory configuration information, and unnecessary resource consumption is reduced.

[0063] In some implementations, in response to the end of the BR operation, a subset of attribute information parsed by the target node can also be obtained; the subset of attribute information is combined to obtain the set of attribute information, and the set of attribute information is filtered according to the directory configuration information to obtain the directory attribute information.

[0064] That is, a subset of attribute information of the backup file parsed by each target node is obtained, the subsets of attribute information parsed by all target nodes are combined to obtain the set of attribute information in all metadata in the backup file, and the set of attribute information is further filtered according to the pre-set directory configuration information to screen the required directory attribute information, thereby avoiding resource consumption on the critical path.

[0065] S205, according to the directory attribute information, a directory full path is generated, and according to the directory full path, the quota information of each directory is obtained.

[0066] In the embodiments of the present disclosure, the implementation method of step S205 can be implemented in any of the embodiments of the present disclosure, and this is not limited herein, and will not be repeated here.

[0067] In the embodiment, during the BR operation on the database, the file backup situation is determined, and when the file backup is completed, a plurality of target nodes are scheduled to parse the exported backup file, so as to improve the parsing efficiency. By sending a backup parsing request carrying storage information, the target node is instructed to pull the exported backup file in the storage location. In the pulling process, the backup file that has been parsed and processed can be filtered through the file identifier or the file list of the target node itself, so as to avoid repeated pulling and parsing of the exported backup file, reduce resource consumption in the processing process, avoid repeated operation and processing, obtain attribute information through efficient parsing of a plurality of target nodes, filter and screen irrelevant attribute information through the preconfigured directory configuration information, generate a directory full path based on the screened directory attribute information, and obtain the quota information of the directory, so as to improve the timeliness of the directory quota information acquisition.

[0068] Figure 3 FIG. 2 is a schematic diagram of another method for obtaining directory quota information based on database backup and recovery (BR) provided by the present disclosure; as shown in the figure, the method comprises the following steps: Figure 3

[0069] S301, performing a BR operation on a database to export a backup file containing metadata from the database.

[0070] In the embodiment of the present disclosure, the implementation method of step S301 can be realized by any one of the embodiments of the present disclosure, which will not be limited here and will not be repeated.

[0071] S302, synchronously parsing the exported backup file during the execution of the BR operation to obtain directory attribute information.

[0072] In the embodiment of the present disclosure, the implementation method of step S302 can be realized by any one of the embodiments of the present disclosure, which will not be limited here and will not be repeated.

[0073] S303, generating a directory full path according to the directory attribute information.

[0074] In the embodiment of the present disclosure, the implementation method of step S303 can be realized by any one of the embodiments of the present disclosure, which will not be limited here and will not be repeated.

[0075] S304, splitting the directory full path to obtain the directory hierarchical relationship corresponding to the directory full path.

[0076] ​In some implementations, a separator can be identified from the directory full path, the directory full path can be split according to the separator, and a hierarchical directory can be extracted from information between adjacent separators; a position sequence of the separator in the directory full path can be obtained, and a level of the hierarchical directory can be determined according to the position sequence of the separator; and a statistic of the hierarchical directory can be performed on each level to generate a directory level relationship.

[0077] Optionally, the separator of the directory can be a separator symbol such as “\” or “ / ”, the separator can be identified from the directory full path and the directory full path can be split according to the separator, and a hierarchical directory can be extracted from information between adjacent separators, for example, the directory full path is C:\Users\username, the directory full path is split into “C:”, “Users”, and “username” according to the separator, and the hierarchical directory extracted from the information between adjacent separators is “C:”, “Users”, and “username”.

[0078] Further, a level of a hierarchical directory can be determined according to a position sequence of the separator in the target full path, for example, a hierarchical directory between the first separator is level 1, a hierarchical directory between the first separator and the second separator is level 2, and so on, that is, the level of “C:” is level 1, the level of “Users” is level 2, and the level of “username” is level 3; and a statistic of the hierarchical directory of each level can be performed to generate a directory level relationship corresponding to the directory full path, and each directory level can be more intuitively presented and analyzed.

[0079] S305, according to the directory level relationship, obtaining quota information of each directory in the directory full path.

[0080] In some implementations, first directory attribute information of a hierarchical directory can be obtained from directory attribute information; a file quantity and a file total size included in the hierarchical directory can be determined according to the first directory attribute information; and quota information of the hierarchical directory can be determined according to the file quantity and the file total size. That is, first directory attribute information corresponding to any hierarchical directory can be obtained from the directory attribute information, the first directory attribute information can include a file quantity and a file total size included in the hierarchical directory, and therefore the file quantity and the file total size included in the hierarchical directory can be obtained according to the first directory attribute information, the file quantity and the file total size can be taken as the quota information of the hierarchical directory, and the quota information of the hierarchical directory can be taken as a statistical basis to improve the accuracy of the quota information.

[0081] Further, after determining the quota information of the hierarchical directories under each level, the quota information of the hierarchical directories is recursively accumulated according to the directory hierarchical relationship, to obtain the quota information of the directory. For example, the quota information of the hierarchical directory "C:" split from the directory full path C:\Users\username is recursively accumulated according to the directory hierarchical relationship, that is, "username", "Users", and "C:" are accumulated according to the directory hierarchical relationship, to obtain the quota information of the directory "C:". Correspondingly, the quota information of the directory "Users" is the accumulation of the quota information of "username" and the quota information of "Users", and the quota information of the directory "username" is the quota information of the hierarchical directory itself, so that the accuracy of the quota information of the directory is higher.

[0082] In some implementations, for any hierarchical directory "Users", it can appear in other directory full paths, and when the quota information of the directory "Users" is accumulated, the quota information of all hierarchical directories under the "Users" level in all directory full paths is accumulated, to obtain the accurate quota information of the directory "Users".

[0083] Further, the storage information of the quota information of the directory can also be obtained, and the quota information of the directory is stored according to the storage information. Optionally, the storage information can include the storage location of the quota information, and the quota information of the directory is stored according to the storage location in the storage information, to facilitate subsequent user acquisition and viewing.

[0084] In this embodiment, after obtaining the directory full path, the directory full path is split to obtain the hierarchical directories corresponding to the target full path and the level of the hierarchical directories, to more accurately analyze the directory full path, obtain the quota information of each hierarchical directory, recursively accumulate according to the directory hierarchical relationship, obtain the quota information of each directory in the target full path, and obtain the storage information of the quota information of the directory, and store based on the storage information, to facilitate user acquisition and viewing. In the process of obtaining the quota information of the directory, no disk read and write are involved, compared with the traditional real-time incremental statistics and full-amount timing statistics, the timeliness and ease of use can be considered, and through the split of the directory full path, more detailed quota information statistics are ensured, and the absolute accuracy of the directory quota statistics is ensured.

[0085] Figure 4 is another schematic diagram of a directory quota information acquisition method based on database backup and recovery (BR) provided by the embodiment of the disclosure; as shown in Figure 4 the method comprises:

[0086] S401, performing a BR operation on the database to export a backup file containing metadata from the database.

[0087] In the embodiments of the present disclosure, the implementation method of step S401 can be implemented by any one of the embodiments of the present disclosure, and here it is not limited, and will not be repeated.

[0088] S402, during the execution of the BR operation, obtaining the task execution condition of the candidate node, and scheduling multiple target nodes based on the task execution condition.

[0089] In the embodiments of the present disclosure, the implementation method of step S402 can be implemented by any one of the embodiments of the present disclosure, and here it is not limited, and will not be repeated.

[0090] S403, sending a backup analysis request to the target node, and pulling the backup file by the target node for analysis.

[0091] In the embodiments of the present disclosure, the implementation method of step S403 can be implemented by any one of the embodiments of the present disclosure, and here it is not limited, and will not be repeated.

[0092] S404, obtaining directory attribute information.

[0093] In the embodiments of the present disclosure, the implementation method of step S404 can be implemented by any one of the embodiments of the present disclosure, and here it is not limited, and will not be repeated.

[0094] S405, generating a directory full path according to the directory attribute information.

[0095] In the embodiments of the present disclosure, the implementation method of step S405 can be implemented by any one of the embodiments of the present disclosure, and here it is not limited, and will not be repeated.

[0096] S406, splitting the directory full path to obtain the directory hierarchy relationship corresponding to the directory full path.

[0097] In the embodiments of the present disclosure, the implementation method of step S406 can be implemented by any one of the embodiments of the present disclosure, and here it is not limited, and will not be repeated.

[0098] S407, obtaining the quota information of each directory in the directory full path according to the directory hierarchy relationship.

[0099] In the embodiments of the present disclosure, the implementation method of step S407 can be implemented by any one of the embodiments of the present disclosure, and here it is not limited, and will not be repeated.

[0100] In this embodiment, the full backup file is obtained by performing the BR operation on the database. Compared with the existing real-time incremental statistics method, the directory Quota statistics based on the backup file in each round can be fully corrected to ensure the accuracy of the directory Quota. In the process of performing the BR operation on the database, the file backup situation is determined, and when the file backup is completed, a plurality of target nodes are scheduled to parse the exported backup file to improve the parsing efficiency. By sending a backup parsing request carrying storage information, the target nodes are instructed to pull the exported backup file in the storage location. In the pulling process, the backup file that has been parsed and processed can be filtered through the file identifier or the file list of the target node itself, so as to avoid repeated pulling and parsing of the exported backup file, reduce resource consumption in the processing process, avoid repeated operation and processing, filter and screen the irrelevant attribute information based on the preconfigured directory configuration information, generate the directory full path based on the screened directory attribute information, split the directory full path to obtain the hierarchical directory corresponding to the target full path and the level of the hierarchical directory, so as to more accurately analyze the directory full path, obtain the quota information of each hierarchical directory, recursively accumulate the quota information according to the directory hierarchical relationship, obtain the quota information of each directory in the target full path, and obtain the storage information of the quota information of the directory. Based on the storage information, the storage is performed to facilitate the user to obtain and view. In the process of obtaining the directory quota information, the disk reading and writing are not involved, the timeliness and ease of use are considered, and the absolute accuracy of the directory quota statistics is ensured.

[0101] Figure 5 is another logic diagram of a directory quota information acquisition method based on database backup recovery BR provided by the embodiment of the present disclosure; as shown in Figure 5 shown, the Master node initiates the directory quota statistics task at the task initiation, that is, performs the BR operation on the database (Data Base, DB) to export the backup file, and the backup file includes the metadata information. In the process of performing the BR operation, the Master node sends a backup parsing request to a plurality of target nodes to instruct the target nodes to pull and parse the backup file to obtain all attribute information, filter the directory attribute information from the all attribute information, further splice the full directory based on the directory attribute information to obtain the full path of each directory, split the full path of the directory after splicing the full path of the directory to obtain a plurality of hierarchical directories, obtain and recursively accumulate the quota information of the hierarchical directories to obtain the Quota information of each directory, and further push and store the full directory Quota information to facilitate the user to obtain and view.

[0102] Figure 6 is a block diagram of a directory quota information acquisition device based on database backup recovery BR provided by the embodiment of the present disclosure; as shown in Figure 6As shown, the directory quota information acquisition device 600 based on database backup and recovery BR includes:

[0103] The export module 601 is used to perform a BR operation on the database to export a backup file containing metadata from the database;

[0104] Parsing module 602, used to synchronously parse the exported backup file during the BR operation to obtain directory attribute information;

[0105] The acquisition module 603 is used to generate a full directory path according to the directory attribute information, and acquire the quota information of each directory according to the full directory path.

[0106] In some implementations, obtaining a module includes:

[0107] Split the full directory path and obtain the directory hierarchy relationship corresponding to the full directory path;

[0108] Get the quota information for each directory in the full directory path based on the directory hierarchy.

[0109] In some implementations, the acquisition module 603 includes:

[0110] Get the quota information of the hierarchical directory under each level;

[0111] For the quota information of the hierarchical directory, the quota is recursively applied according to the directory hierarchy to obtain the quota information of the directory.

[0112] In some implementations, the acquisition module 603 includes:

[0113] Identify separators from the full directory path, split the full directory path according to the separators, and extract the hierarchical directory from the information between adjacent separators;

[0114] Get the position order of the separators in the full directory path, and determine the level of the hierarchical directory based on the position order of the separators;

[0115] Statistics of hierarchical directories are performed on each level to generate directory hierarchical relationships.

[0116] In some implementations, the acquisition module 603 includes:

[0117] Obtaining first directory attribute information of the hierarchical directory from the directory attribute information;

[0118] Determine the number of files and the total size of the files included in the hierarchical directory according to the first directory attribute information;

[0119] Determine the quota information for the hierarchical directory based on the number of files and the total size of the files.

[0120] In some implementations, the apparatus 600 further includes:

[0121] Obtaining storage information of quota information of the directory, and storing the quota information of the directory according to the storage information.

[0122] In some implementations, the parsing module 602 includes:

[0123] Parsing the metadata in the backup file to obtain a set of attribute information;

[0124] In response to the end of the BR operation, filtering the set of attribute information according to the directory configuration information to obtain directory attribute information.

[0125] In some implementations, the parsing module 602 includes:

[0126] Obtaining a task execution status of the candidate node, and scheduling a plurality of target nodes based on the task execution status;

[0127] Sending a backup parsing request to the target node, and pulling the backup file by the target node for parsing.

[0128] In some implementations, the parsing module 602 includes:

[0129] In response to the end of the BR operation, obtaining a subset of attribute information parsed by the target node;

[0130] Combining the subset of attribute information to obtain the set of attribute information.

[0131] In some implementations, the apparatus 600 further includes:

[0132] Determining storage information of the backup file, and storing the backup file based on the storage information;

[0133] The target node obtains the storage information of the backup file from the backup parsing request, and pulls the backup file based on the storage information of the backup file.

[0134] In some implementations, the parsing module 602 includes:

[0135] Obtaining a data storage standard file in a storage space where the storage information of the backup file is located, the data storage standard file including a file identifier of a stored backup file;

[0136] In response to the backup file being pulled, marking a pulling state of the backup file;

[0137] Determining a not-pulled backup file according to the data storage standard file and the pulling state mark of the backup file, and pulling the not-pulled backup file stored in the storage space.

[0138] In some implementations, the apparatus 600 further includes:

[0139] The file list of the resolved files maintained by the target node itself is acquired, and the pulled backup file is deduplicated based on the file list.

[0140] In the embodiment, the full backup file is obtained by performing the BR operation on the database, compared with the existing real-time incremental statistical method, the directory Quota statistics based on the backup file in each round can be full rectification, and the accuracy of the directory Quota is ensured; in the process of performing the BR operation on the database, the file backup condition is determined, and when the file backup is completed, the multiple target nodes are scheduled to analyze the exported backup file, so as to improve the analysis efficiency; by sending the backup analysis request carrying the storage information, the target node is instructed to pull the exported backup file in the storage location, in the pulling process, the backup file that has been analyzed and processed can be screened through the file identifier or the file list of the target node itself, so as to avoid repeated pulling and repeated analysis of the exported backup file, reduce the resource consumption in the processing process, avoid repeated operation and repeated processing, filter and screen the irrelevant attribute information through the preconfigured directory configuration information, generate the directory full path based on the screened directory attribute information, split the directory full path to obtain the hierarchical directory corresponding to the target full path and the level where the hierarchical directory is located, so as to more accurately analyze the directory full path, obtain the quota information of each hierarchical directory, recursively accumulate according to the directory hierarchical relationship, obtain the quota information of each directory in the target full path, and further obtain the storage information of the quota information of the directory, and store the storage information based on the storage information, so as to facilitate the user to obtain and view, the disk reading and writing are not involved in the process of obtaining the directory quota information, the timeliness and the ease of use can be considered, and the absolute accuracy of the directory quota statistics is ensured.

[0141] In the technical scheme of the present disclosure, the acquisition, storage and application of the user personal information involved comply with the relevant laws and regulations and do not violate public order and good customs.

[0142] According to the embodiments of the present disclosure, the present disclosure further provides an electronic device, a readable storage medium and a computer program product.

[0143] Figure 7A schematic block diagram of an example electronic device that can be used to implement embodiments of the present disclosure is shown. Electronic device 700 is intended to represent various forms of digital computers, such as laptops, desktops, tablets, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. Electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular telephones, smartphones, wearable devices, and other similar computing devices. The components shown here, their connections and relationships, and their functions, are meant to be examples only, and are not intended to limit the implementations of the present disclosure described and / or claimed in this document.

[0144] As shown, Figure 7 Device 700 includes a computing unit 701 that can perform various suitable actions and processes in accordance with computer programs stored in read-only memory (ROM) 702 or loaded into random access memory (RAM) 703 from storage unit 708. Various programs and data used by device 700 can also be stored in RAM 703 as needed. Computing unit 701, ROM 702, and RAM 703 are connected to each other by bus 704. Input / output (I / O) interface 705 is also connected to bus 704.

[0145] Various components in device 700 are connected to I / O interface 705, including: input unit 706, such as a keyboard, mouse, etc.; output unit 707, such as various types of displays, speakers, etc.; storage unit 708, such as magnetic disks, optical disks, etc.; and communication unit 709, such as a network card, modem, wireless communication transceiver, etc. Communication unit 709 allows device 700 to exchange information / data with other devices over computer networks, such as the Internet, and / or various telecommunication networks.

[0146] The computing unit 701 can be various general and / or special purpose processing components with processing and computing capabilities. Some examples of the computing unit 701 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various specialized artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The computing unit 701 performs various methods and processes described above, such as the method for obtaining directory quota information based on database backup recovery BR. For example, in some embodiments, the method for obtaining directory quota information based on database backup recovery BR can be implemented as a computer software program tangibly embodied in a machine-readable medium, such as the storage unit 708. In some embodiments, part or all of the computer program can be loaded and / or installed onto the device 700 via the ROM 702 and / or the communication unit 709. When the computer program is loaded onto the RAM 703 and executed by the computing unit 701, one or more steps of the method for obtaining directory quota information based on database backup recovery BR described above can be performed. Alternatively, in other embodiments, the computing unit 701 can be configured to perform the method for obtaining directory quota information based on database backup recovery BR by other any appropriate means, such as by means of firmware.

[0147] Various implementations of the systems and techniques described above herein can be realized in digital electronic circuitry, integrated circuitry, a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), a system on a chip system (SOC), a complex programmable logic device (CPLD), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include implementation in one or more computer programs that are executable and / or interpretable on a programmable system including at least one programmable processor, which can be special or general purpose, coupled to receive data and instructions from, and to transmit data and instructions to, a storage system, at least one input device, and at least one output device.

[0148] Program code for carrying out methods of the present disclosure can be written in any combination of one or more programming languages. The program code can be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the program code, when executed by the processor or controller, produces the functions / operations specified in the flowcharts and / or block diagrams. The program code can be executed entirely on a machine, partially on a machine, partially on a machine as a standalone software package, or entirely on a remote machine or server.

[0149] In the context of the present disclosure, a machine-readable medium can be a tangible medium that contains or stores a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include but is not limited to an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of the machine-readable storage medium will include one or more lines of electrical connections, portable computer disks, hard disk drives, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or Flash memory), optical fibers, portable compact disc read-only memories (CD-ROMs), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0150] To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the computer. Other kinds of devices can be used to provide for interaction with a user as well; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form, including acoustic, speech, or tactile input.

[0151] The systems and techniques described here can be implemented in a computing system that includes a back end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front end component (e.g., a user computer having a graphical user interface or a Web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back end, middleware, or front end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), and the Internet.

[0152] The computer system can include clients and servers. This relationship can be. The servers are typically remote from the clients with the interactions between them occurring over a communication network. The relationship between a client and a server is one of client-server. The server can be a cloud server, a server of a distributed system, or a server incorporating a blockchain.

[0153] It should be understood that the various forms of flow shown above can be re-ordered, added to, or deleted from without departing from the scope of the present disclosure. For example, the steps recited in the present disclosure can be performed in parallel, in series, or in a different order, as long as the desired results of the technology disclosed in the present disclosure are achieved, and the present disclosure is not limited herein.

[0154] The specific embodiments described above are not intended to be limiting. One of skill in the art will understand that various modifications, combinations, sub-combinations, and alternatives can be made to the specific embodiments described above without departing from the scope of the disclosure. Any modifications, combinations, sub-combinations, and alternatives falling within the spirit and principles of the disclosure should be considered.

Claims

1. A method for obtaining directory quota information based on database backup and recovery BR, wherein: The method comprises: Performing a BR operation on a database to export a backup file containing metadata from the database; During the BR operation, the task execution status of the candidate node is obtained. Based on the task execution status, multiple target nodes are scheduled and a backup resolution request is sent to the target node. The target node pulls the backup file and synchronously resolves the metadata in the exported backup file to obtain a set of attribute information. In response to the BR operation being completed, filtering the attribute information set according to the directory configuration information to obtain directory attribute information; A full directory path is generated according to the directory attribute information, and quota information of each directory is obtained according to the full directory path.

2. The method according to claim 1, wherein The step of obtaining the quota information of each directory according to the full path of the directory includes: Split the full directory path to obtain the directory hierarchy relationship corresponding to the full directory path; According to the directory hierarchy, the quota information of each directory in the directory full path is obtained.

3. The method according to claim 2, wherein: The step of obtaining quota information for each directory in the full directory path according to the directory hierarchy relationship includes: Get the quota information of the hierarchical directory under each level; The quota information of the hierarchical directory is recursively calculated according to the directory hierarchy to obtain the quota information of the directory.

4. The method according to claim 2, wherein: The step of segmenting the full directory path and obtaining the directory hierarchy relationship corresponding to the full directory path includes: Identifying separators from the full directory path, segmenting the full directory path according to the separators, and extracting hierarchical directories from information between adjacent separators; Obtaining the position sequence of the separators in the full path of the directory, and determining the level of the hierarchical directory according to the position sequence of the separators; Statistics of the hierarchical directory are performed on each level to generate the directory hierarchical relationship.

5. The method according to claim 3, wherein The process of obtaining quota information for each level directory includes: Acquire first directory attribute information of the hierarchical directory from the directory attribute information; Determining the number of files and the total size of the files included in the hierarchical directory according to the first directory attribute information; The quota information of the hierarchical directory is determined according to the number of files and the total size of the files.

6. The method according to claim 1, wherein After obtaining the quota information of the directory according to the full path of the directory, the method further includes: Acquire storage information of the quota information of the directory, and store the quota information of the directory according to the storage information.

7. The method according to claim 1, wherein The step of parsing the metadata in the backup file to obtain a set of attribute information includes: In response to the BR operation ending, obtaining a subset of attribute information parsed by the target node; The attribute information subsets are combined to obtain an attribute information set.

8. The method according to claim 1, wherein The method further comprises: Determining storage information of the backup file, and storing the backup file based on the storage information; The target node obtains the storage information of the backup file from the backup parsing request, and pulls the backup file based on the storage information of the backup file.

9. The method according to claim 1, wherein The process of the target node pulling the backup file includes: Acquire a data storage standard file in the storage space where the storage information of the backup file is located, wherein the data storage standard file includes a file identifier of the stored backup file; In response to the backup file being pulled, marking a pulling status of the backup file; The unpulled backup files are determined according to the data storage standard file and the pull status mark of the backup file, and the unpulled backup files stored in the storage space are pulled.

10. The method according to claim 1, wherein The method further comprises: A file list of parsed files maintained by the target node itself is obtained, and duplicate removal is performed on the pulled backup file based on the file list.

11. A directory quota information acquisition device based on database backup and recovery BR, comprising: an export module, configured to perform a BR operation on a database to export a backup file containing metadata from the database; A parsing module is configured to obtain the task execution status of candidate nodes during the execution of a backup resolution operation, schedule multiple target nodes based on the task execution status, and send a backup resolution request to the target nodes. The target nodes retrieve the backup files containing the storage information of the backup files in the backup resolution request and synchronously parse the metadata in the exported backup files to obtain a set of attribute information. In response to the completion of the backup resolution operation, the attribute information set is filtered based on directory configuration information to obtain directory attribute information. The acquisition module is used to generate a full directory path according to the directory attribute information, and obtain quota information of each directory according to the full directory path.

12. The device according to claim 11, wherein The acquisition module includes: Split the full directory path to obtain the directory hierarchy relationship corresponding to the full directory path; According to the directory hierarchy, the quota information of each directory in the directory full path is obtained.

13. The device according to claim 12, wherein The acquisition module includes: Get the quota information of the hierarchical directory under each level; The quota information of the hierarchical directory is recursively calculated according to the directory hierarchy to obtain the quota information of the directory.

14. The device according to claim 12, wherein The acquisition module includes: Identifying separators from the full directory path, segmenting the full directory path according to the separators, and extracting hierarchical directories from information between adjacent separators; Obtaining the position sequence of the separators in the full path of the directory, and determining the level of the hierarchical directory according to the position sequence of the separators; Statistics of the hierarchical directory are performed on each level to generate the directory hierarchical relationship.

15. The device according to claim 13, wherein The acquisition module includes: Acquire first directory attribute information of the hierarchical directory from the directory attribute information; Determining the number of files and the total size of the files included in the hierarchical directory according to the first directory attribute information; The quota information of the hierarchical directory is determined according to the number of files and the total size of the files.

16. The device according to claim 11, wherein The device further comprises: Acquire storage information of the quota information of the directory, and store the quota information of the directory according to the storage information.

17. The device according to claim 12, wherein The parsing module includes: In response to the BR operation ending, obtaining a subset of attribute information parsed by the target node; The attribute information subsets are combined to obtain an attribute information set.

18. The device according to claim 12, wherein The device further comprises: Determining storage information of the backup file, and storing the backup file based on the storage information; The target node obtains the storage information of the backup file from the backup parsing request, and pulls the backup file based on the storage information of the backup file.

19. The device according to claim 12, wherein The parsing module includes: Acquire a data storage standard file in the storage space where the storage information of the backup file is located, wherein the data storage standard file includes a file identifier of the stored backup file; In response to the backup file being pulled, marking a pulling status of the backup file; The unpulled backup files are determined according to the data storage standard file and the pull status mark of the backup file, and the unpulled backup files stored in the storage space are pulled.

20. The device according to claim 12, wherein The device further comprises: A file list of parsed files maintained by the target node itself is obtained, and duplicate removal is performed on the pulled backup file based on the file list.

21. An electronic device comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 10.

22. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause the computer to execute the method according to any one of claims 1-10.

23. A computer program product comprising a computer program, which, when executed by a processor, implements the steps of the method according to any one of claims 1 to 10.

Images