Data backup and recovery method, equipment, medium and product

Abstract

The invention provides a data backup and recovery method and device, a medium and a product. The method relates to the technical field of data processing, and comprises the following steps: in response to a received target backup request triggered for a target cloud pool, obtaining target backup information in the target backup request; the target backup information comprises at least one database address and an external storage service address; according to the target backup information, obtaining corresponding to-be-backed-up data from a database corresponding to each database address, and uploading each to-be-backed-up data to a corresponding distributed file system through a management scheduling device; and backing up the to-be-backed-up data in each distributed file system to an external storage service corresponding to the external storage service address in parallel. According to the method, the data backup and recovery efficiency is improved, the user experience is improved, the AK and the SK are prevented from being recorded by the management scheduling equipment, the risk of database equipment information leakage is reduced, and the safety of the backup and recovery process is improved.

Description

Technical Field

[0001] This application relates to the field of data processing technology, and in particular to a data backup and recovery method, device, medium and product. Background Technology

[0002] With the development of the Internet and data processing technologies, more and more data will be stored in databases. This requires the backup and recovery of data in the database to ensure that the stored data is not lost.

[0003] Currently, when backing up and restoring data, a communication connection is typically established between the database to which the data belongs and the external storage service. Then, the backup and restore tool corresponding to the database is used to perform data backup and restore operations between the database and the external storage service according to the Linux commands entered by the user and the command-line interface of the database, through the scheduling instructions of the management and scheduling device.

[0004] However, the aforementioned backup and recovery process requires a large number of commands to execute, reducing the efficiency of data backup and recovery, impacting user experience. Furthermore, when users have backup and recovery needs for multiple databases, they need to perform backup and recovery sequentially based on commands, further reducing the efficiency of data backup and recovery. Moreover, the scheduling instructions based on external storage services directly transmit AK (Access Key) and SK (Secret Key), making it easy for the management and scheduling device to record AK and SK, increasing the risk of database device information leakage and reducing the security of the backup and recovery process. Summary of the Invention

[0005] This application provides a data backup and recovery method, device, medium, and product to solve the technical problems in the prior art where data backup and recovery is inefficient, affects user experience, and AK and SK are easily recorded by management and scheduling devices, increasing the risk of database device information leakage and reducing the security of the backup and recovery process.

[0006] Firstly, this application provides a data backup method applied to a backup and recovery service. The backup and recovery service includes at least one node, each node deploying a shell data backup and recovery script corresponding to a cloud pool. Each cloud pool includes at least one database and an external storage service. Each database is configured with a distributed file system. When the shell data backup and recovery script is run by the corresponding node, it is used to execute the data backup method. The method includes: in response to receiving a target backup request triggered for a target cloud pool, obtaining target backup information from the target backup request; the target backup information includes: at least one database address and an external storage service address.

[0007] Based on the target backup information, the corresponding backup data is obtained from the database corresponding to each database address, and the backup data is uploaded to the corresponding distributed file system through the management and scheduling device.

[0008] The data to be backed up in each distributed file system is backed up in parallel to the external storage service corresponding to the external storage service address.

[0009] Optionally, in the method described above, the target backup information further includes an incremental backup identifier or a full backup identifier; the step of obtaining the corresponding backup data in the database corresponding to each database address according to the target backup information includes: obtaining the corresponding backup data in the database corresponding to all database addresses according to the full backup identifier; or, obtaining the corresponding backup data in the database corresponding to some database addresses according to the incremental backup identifier.

[0010] Optionally, the method described above, in which the corresponding data to be backed up is obtained from the database corresponding to the database address, specifically includes: performing version verification on the database and column family verification on the data table in the database; the data table is used to store the data to be backed up; in response to both the version verification result and the column family verification result being passed, the corresponding data to be backed up is obtained from the database corresponding to the database address.

[0011] Optionally, in the method described above, the target backup information further includes an encrypted login password; the step of uploading each piece of data to be backed up to the corresponding distributed file system through the management and scheduling device includes: decrypting the encrypted login password to obtain a login password, and using the login password to log in to each distributed file system; determining the memory value occupied by each piece of data to be backed up, and determining the current capacity value of each distributed file system; determining whether each memory value is less than the corresponding capacity value, and if so, uploading each piece of data to be backed up to the corresponding distributed file system through the management and scheduling device; if not, generating a corresponding prompt message based on the result of being less than the capacity value.

[0012] Optionally, in the method described above, after obtaining the target backup information in the target backup request, the method further includes: performing a legality check on the content included in the target backup information, and continuing to execute the data backup process when the legality check result is passed.

[0013] Optionally, the method described above further includes: in response to an abnormality in the data backup process, suspending the execution of the data backup process, and automatically obtaining target backup information when the data backup process resumes normal operation; and continuing to execute the data backup process based on the target backup information.

[0014] Secondly, this application provides a data recovery method applied to a backup and recovery service. The backup and recovery service includes at least one node, each node deploying a shell data backup and recovery script corresponding to a cloud pool. Each cloud pool includes at least one database and an external storage service. Each database is configured with a distributed file system. When the shell data backup and recovery script is run by the corresponding node, it is used to execute the data recovery method. The method includes: in response to receiving a target recovery request triggered for a target cloud pool, obtaining target recovery information from the target recovery request; the target recovery information includes: at least one database address and an external storage service address.

[0015] Based on the target recovery information, the data to be recovered corresponding to each database address is obtained from the external storage service corresponding to the storage service address, and the data to be recovered is restored to the corresponding distributed file system in parallel.

[0016] The management and scheduling equipment uploads the data to be recovered from each distributed file system to the corresponding database.

[0017] Thirdly, this application provides a data backup device, the device comprising: an acquisition module, configured to, in response to receiving a target backup request triggered for a target cloud pool, acquire target backup information in the target backup request; the target backup information includes: at least one database address and an external storage service address; and, based on the target backup information, acquire corresponding data to be backed up from the databases corresponding to each database address;

[0018] The processing module is used to upload each piece of data to be backed up to the corresponding distributed file system through the management and scheduling device; and to back up the data to be backed up in each distributed file system in parallel to the external storage service corresponding to the external storage service address.

[0019] Fourthly, this application provides a data recovery device, the device comprising: an acquisition module, configured to, in response to receiving a target recovery request triggered for a target cloud pool, acquire target recovery information in the target recovery request; the target recovery information includes: at least one database address and an external storage service address; and, based on the target recovery information, acquire the data to be recovered corresponding to each database address from the external storage service corresponding to the storage service address;

[0020] The processing module is used to restore each piece of data to be recovered to its corresponding distributed file system in parallel; and to upload the data to be recovered from each distributed file system to its corresponding database through the management and scheduling device.

[0021] Fifthly, this application provides an electronic device, including: a processor, and a memory communicatively connected to the processor;

[0022] The memory stores computer-executed instructions;

[0023] The processor executes computer execution instructions stored in the memory to implement the data backup and recovery methods as described in the first and second aspects.

[0024] In a sixth aspect, this application provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement the data backup and recovery methods described in the first and second aspects.

[0025] In a seventh aspect, this application provides a computer program product, including a computer program that, when executed by a processor, implements the data backup and recovery methods described in the first and second aspects.

[0026] This application provides a data backup and recovery method, device, medium, and product. The method is applied to a backup and recovery service, which includes at least one node. Each node deploys a shell data backup and recovery script corresponding to a cloud pool. Each cloud pool includes at least one database and an external storage service. Each database is configured with a distributed file system. When the shell data backup and recovery script is run by the corresponding node, it is used to execute the data backup method. The method includes: in response to receiving a target backup request triggered for a target cloud pool, obtaining target backup information from the target backup request; the target backup information includes at least one database address and an external storage service address; according to the target backup information, obtaining corresponding data to be backed up from the databases corresponding to each database address, and uploading each data to be backed up to the corresponding distributed file system through a management scheduling device; and backing up the data to be backed up in each distributed file system in parallel to the external storage service corresponding to the external storage service address. Since each node in the backup and recovery service deploys shell data backup and recovery scripts corresponding to the cloud pool, and each cloud pool has at least one database and an external storage system, and each database is configured with a distributed file system, the node corresponding to the target cloud pool, after receiving a target backup request triggered by the user for the target cloud pool, can obtain the target backup information in the request by running the shell data backup and recovery script corresponding to the target cloud pool. That is, the database address corresponding to at least one database included in the target cloud pool and the external storage service address corresponding to the external storage service. Based on the target backup information, the node can obtain the corresponding data to be backed up in the database corresponding to each database address. Then, the management and scheduling device can upload the data to be backed up to the corresponding distributed file system, and then back up the data to be backed up in each distributed file system in parallel to the external storage service address corresponding to the external storage service. This allows for the backup and recovery of one or more databases without the need to execute numerous commands, improving the efficiency of data backup and recovery, enhancing the user experience, and eliminating the need for AK and SK in the scheduling instructions based on the distributed file system. Furthermore, data transfer between the distributed file systems and external storage services does not require management and scheduling devices, preventing the transmitted AK and SK from being recorded by the management and scheduling devices, reducing the risk of database device information leakage, and improving the security of the backup and recovery process. Attached Figure Description

[0027] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0028] Figure 1 This is an application scenario diagram of the data backup and recovery method provided in one embodiment of this application;

[0029] Figure 2 A flowchart illustrating a data backup method provided in an embodiment of this application;

[0030] Figure 3 A flowchart illustrating a data recovery method provided in an embodiment of this application;

[0031] Figure 4 A complete flowchart of a data backup and recovery method provided in an embodiment of this application;

[0032] Figure 5 A schematic diagram of the data backup and recovery architecture provided for this application;

[0033] Figure 6 This is a schematic diagram of the structure of a data backup device provided in an embodiment of this application;

[0034] Figure 7 This is a schematic diagram of the structure of a data recovery device provided in an embodiment of this application;

[0035] Figure 8 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application.

[0036] The accompanying drawings illustrate specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concept of this application to those skilled in the art through reference to particular embodiments. Detailed Implementation

[0037] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0038] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use and processing of the relevant data must comply with relevant laws, regulations and standards, and corresponding operation entry points are provided for users to choose to authorize or refuse.

[0039] Currently, data backup and recovery typically involves establishing a communication connection between the database containing the data and the external storage service. Then, using the backup and recovery tool corresponding to the database, when backup is needed, the data is backed up to the external storage service according to user-input Linux commands and the database's command-line interface, via scheduling instructions from the management and scheduling device. Similarly, when recovery is needed, the data is restored to the database from the external storage service according to user-input Linux commands and the database's command-line interface, via scheduling instructions from the management and scheduling device.

[0040] For example, for backing up and restoring HBase database data, a communication connection can be established between the HBase database and an external storage service, such as OSS (Object Storage Service). Then, using the backup and restore tool corresponding to the HBase database, based on backup requirements and following user-inputted Linux commands and the command-line interface of the HBase database, the data can be backed up to OSS via scheduling instructions from a management scheduling device, such as yarn (Yet Another Resource Negotiator). Similarly, based on recovery requirements and following user-inputted Linux commands and the command-line interface of the HBase database, the HBase data in OSS can be restored to the HBase database via yarn's scheduling instructions.

[0041] This results in the entire backup and recovery process requiring a large number of commands to be executed, reducing the efficiency of data backup and recovery, impacting user experience, and when users have backup and recovery needs for multiple databases, they need to perform backup and recovery sequentially based on commands, further reducing the efficiency of data backup and recovery. In addition, the scheduling instructions based on external storage services will directly transmit AK and SK, making AK and SK easy to be recorded by management and scheduling devices, increasing the risk of database device information leakage and reducing the security of the backup and recovery process.

[0042] Therefore, to address the issues in existing technologies and improve the efficiency of data backup and recovery while enhancing user experience, instead of sequentially backing up and restoring at least one database using a large number of commands, at least one database is deployed in a cloud pool. A shell script for data backup and recovery is then deployed within the cloud pool, and backup and recovery of at least one database is performed based on this shell script. To prevent AK and SK from being recorded by the management and scheduling device, reduce the risk of database device information leakage, and improve the security of the backup and recovery process, data backup and recovery is not performed directly between the database and the external storage service via the management and scheduling device. Instead, the built-in distributed file system within the database is used as a data transfer point. First, the data in the database is uploaded to the built-in distributed file system via the management and scheduling device. Then, based on the communication connection between this distributed file system and the external storage service, the data in the distributed file system is backed up to the storage service.

[0043] Based on this, after determining that a user has triggered a backup request for a specific cloud pool, the shell script described above retrieves the corresponding data from at least one database within the cloud pool. The management and scheduling device then uploads the data from each database to its corresponding distributed file system (DFS). Finally, through the communication connection between each DFS and the external storage service, the data in each DFS is backed up to the external storage service. Similarly, after determining that a user has triggered a recovery request for a specific cloud pool, the shell script described above retrieves the corresponding data from the external storage service within the cloud pool. The DFS then restores the data from the external storage service to its respective DFS. Finally, the management and scheduling device uploads the data from each DFS to its corresponding database. This allows for the backup and recovery of one or more databases without executing numerous commands, improving the efficiency of data backup and recovery, enhancing the user experience, and because the scheduling instructions based on the distributed file system do not involve AK and SK, and data transfer between the distributed file systems and the external storage service does not require the management and scheduling device, it avoids the AK and SK being recorded by the management and scheduling device, reducing the risk of database device information leakage and improving the security of the backup and recovery process.

[0044] Figure 1 This is an application scenario diagram of the data backup and recovery method provided in an embodiment of this application, such as... Figure 1As shown, the data backup and recovery system provided in this application includes: a user terminal 1, an electronic device 2, and a target cloud pool 3. User terminal 1 is a terminal with data backup and recovery needs, such as the backend where data administrators reside. Electronic device 2 is a device that executes the data backup and recovery process, specifically located in a node included in the backup and recovery service. This node deploys the shell data backup and recovery scripts corresponding to the target cloud pool 3. Target cloud pool 3 is a cloud pool located in the management domain corresponding to the backup and recovery service, specifically including at least one database and an external storage service. Each database is the database containing the data to be backed up and recovered; specifically, it can be an HBase database. Each database is configured with a corresponding distributed file system, specifically, it can be the Hadoop Distributed File System (HDFS). The external storage service is an external storage service used for backing up data; specifically, it can be OSS. Electronic device 2 is communicatively connected to both user terminal 1 and target cloud pool 3.

[0045] First, user terminal 1 generates target backup information based on at least one database address in target cloud pool 3 and an external storage service address, and triggers a target backup request. Then, electronic device 2 responds to the above request by running the shell data backup and recovery script corresponding to the target cloud pool. Based on the running result, it obtains the target backup information in the request and retrieves the corresponding backup data from the databases corresponding to each database address. Then, through the management and scheduling device, it uploads each backup data to the corresponding distributed file system, and then backs up the backup data in each distributed file system in parallel to the external storage service address corresponding to the external storage service.

[0046] The data backup and recovery method provided in this application is intended to solve the above-mentioned technical problems of the prior art.

[0047] The technical solution of this application and how the technical solution of this application solves the above-mentioned technical problems are described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will now be described with reference to the accompanying drawings.

[0048] Figure 2 A flowchart of a data backup method provided in an embodiment of this application is shown below. Figure 2As shown, the execution subject of the data backup method provided in this embodiment is a data backup device, which is specifically located in an electronic device. This electronic device is specifically located in each node included in the backup and recovery service. Each node deploys a shell data backup and recovery script corresponding to the cloud pool. Each cloud pool includes at least one database and an external storage service. Each database is configured with a distributed file system. When the shell data backup and recovery script is run by the corresponding node, it is used to execute the data backup method provided in this embodiment. The specific execution steps of this embodiment include:

[0049] S201. In response to receiving a target backup request triggered for a target cloud pool, obtain the target backup information in the target backup request; the target backup information includes: at least one database address and an external storage service address.

[0050] The backup and recovery service is a service for backing up and restoring database data. Under its corresponding management domain, it can contain at least one node. This node can be a computer, server, or similar device used to back up and restore data from each database within the corresponding cloud pool. The cloud pool can be an isolated cloud environment, with each cloud pool having one corresponding node.

[0051] Specifically, the shell data backup and recovery scripts are pre-written scripts based on the corresponding cloud pool to perform data backup and recovery processes. The database is the big data cluster in the corresponding cloud pool, specifically an HBase database. The external storage service is a storage service located on the external network, specifically an OSS (Object Storage Service). The distributed file system is a file system configured in the database, specifically an HDFS (High-Definition File System).

[0052] It is understood that the above illustrative examples are merely examples and should not constitute any limitation on this application.

[0053] In this embodiment, the target cloud pool can be any cloud pool under the management domain corresponding to the backup and recovery service. The target backup request is a request instructing the node corresponding to the target cloud pool to perform data backup, specifically including target backup information. This target backup information reflects relevant information reflecting the user's specific backup needs, specifically including at least one database address and an external storage service address. The database address is the IP address where the database resides, and the external storage service address is the IP address where the external storage service resides.

[0054] Specifically, users can input at least one database address and storage service address in the target cloud pool through the corresponding terminal according to specific backup needs, form target backup information, and trigger the component where the target backup request is located. Then, the corresponding terminal responds to receiving the above component being triggered and sends the target backup request to the node corresponding to the target cloud pool.

[0055] Optionally, users can also pre-set the backup cycle, as well as at least one database address and storage service address in the target cloud pool to be backed up, i.e., the target backup information, on the corresponding terminal. Then, in response to the current time meeting the user's pre-set backup cycle, the corresponding terminal will automatically obtain the user's pre-set target backup information and automatically generate a target backup request, and send the target backup request to the corresponding node of the target cloud pool to achieve automatic backup, further improve backup efficiency, and enhance user experience.

[0056] Based on this, after receiving a target backup request triggered for the target cloud pool, the node corresponding to the target cloud pool runs the shell data backup and recovery script corresponding to the target cloud pool, and obtains the target backup information in the target backup request based on the script, namely at least one database address and external storage service address.

[0057] It is understood that the above-mentioned method of triggering the target backup request can also be other methods, and this embodiment does not limit it.

[0058] S202. Based on the target backup information, retrieve the corresponding backup data from the databases corresponding to each database address, and upload each backup data to the corresponding distributed file system through the management and scheduling device.

[0059] It is understandable that each database address contains corresponding data to be backed up. This data is the data that the user needs to back up.

[0060] Based on this, the user's specific backup requirements are determined according to the target backup information. Then, the database addresses in the target backup information are accessed according to these specific backup requirements to retrieve the data to be backed up from the databases corresponding to each database address. Further, a backup task is generated based on the data to be backed up and submitted to the management and scheduling device. Following the scheduling instructions of the management and scheduling device, the data to be backed up in the backup task is uploaded to the corresponding distributed file system.

[0061] S203. Back up the data to be backed up in each distributed file system in parallel to the external storage service corresponding to the external storage service address.

[0062] In this embodiment, after uploading each piece of data to be backed up to the corresponding distributed file system, the data to be backed up in each distributed file system is simultaneously backed up to the external storage service based on the communication connection between each distributed file system and the external storage service, according to the address of the external storage service where the external storage service is located.

[0063] It is understandable that after backing up the data to be backed up in each distributed file system to an external storage service address, the data to be backed up in each distributed file system can be deleted to avoid occupying the memory of each distributed file system, ensure that each distributed file system has enough space for data transfer, improve transfer performance, and thus improve the success rate of data backup and recovery.

[0064] The data backup method provided in this embodiment utilizes a cloud pool-specific shell data backup and recovery script deployed on each node of the backup and recovery service. Each cloud pool contains at least one database and an external storage system, and each database is configured with a distributed file system. Therefore, when the node corresponding to the target cloud pool receives a target backup request triggered by a user for the target cloud pool, it can obtain the target backup information in the request by running the shell data backup and recovery script corresponding to the target cloud pool. This information includes the database address of at least one database included in the target cloud pool and the external storage service address. Based on the target backup information, the node can retrieve the corresponding backup data from the databases corresponding to each database address. Then, the backup data can be uploaded to the corresponding distributed file system via a management scheduling device, and the backup data in each distributed file system can be backed up in parallel to the external storage service address corresponding to the external storage service address. This allows for the backup of one or more databases without the need to execute numerous commands, improving data backup efficiency and enhancing user experience. Furthermore, the scheduling instructions based on the distributed file system do not involve AK and SK, and data transmission between the distributed file systems and external storage services does not require management and scheduling devices. This avoids the AK and SK being recorded by the management and scheduling devices, reducing the risk of database device information leakage and improving the security of the backup process.

[0065] As an optional embodiment, this embodiment refines the content of the target backup information and the process of retrieving the corresponding backup data from the databases corresponding to each database address based on the target backup information. In this embodiment, the target backup information further includes an incremental backup identifier or a full backup identifier. The specific steps for retrieving the corresponding backup data from the databases corresponding to each database address based on the target backup information are as follows:

[0066] The corresponding backup data is retrieved from the databases corresponding to all database addresses based on the full backup identifier; or, the corresponding backup data is retrieved from the databases corresponding to some database addresses based on the incremental backup identifier.

[0067] The full backup flag indicates that a backup of all databases at at least one database address is being performed. The incremental backup flag indicates that a backup of a portion of the databases at at least one database address is being performed; the specific settings can be selected by the user according to their needs.

[0068] Specifically, users can select either an incremental backup identifier or a full backup identifier based on their needs. Based on the selection result, at least one database address, and external storage services, target backup information is generated, and a target backup request is triggered. If the target backup information also includes a full backup identifier, the backup data for all databases is retrieved from at least one database address. If the target backup information also includes an incremental backup identifier, the time range indicated by the incremental backup identifier is determined, and database addresses whose creation time falls within the aforementioned time range are retrieved from at least one database address. The backup data is then retrieved from the databases corresponding to these database addresses. For example, if the time range indicated by the incremental backup identifier is from January 2022 to March 2022, then database addresses whose creation time falls within the period from January 2022 to March 2022 are retrieved from at least one database address, and the backup data is then retrieved from the databases corresponding to these database addresses.

[0069] The data backup method provided in this embodiment includes incremental backup identifiers or full backup identifiers in the target backup information. Therefore, by determining the relevant backup identifiers, the corresponding data to be backed up can be obtained from the databases corresponding to all database addresses based on the full backup identifier. Alternatively, the corresponding data to be backed up can be obtained from the databases corresponding to some database addresses based on the incremental backup identifier. This allows for the user to choose which data to back up, improving backup flexibility and further enhancing the user experience.

[0070] As an optional embodiment, this embodiment further refines the process of retrieving the corresponding data to be backed up from the database corresponding to the database address, based on the above embodiment. Specifically, retrieving the corresponding data to be backed up from the database corresponding to the database address includes the following steps:

[0071] Step a1: Perform version verification on the database and column family verification on the data tables in the database; the data tables are used to store the data to be backed up.

[0072] The data table is a table that stores the data to be backed up. If the database is an HBase database, the data table can be an HBase table or a Phoenix table.

[0073] Specifically, it checks if the database version is the latest version. If so, the version verification result is considered passed; otherwise, it is considered failed. Further, it queries the structure of the data tables contained in the database to obtain the column family information of the data tables, and compares it with the expected structure to check if the expected column family exists. If it does, the column family verification result is considered passed; otherwise, it is considered failed.

[0074] When querying the structure of a table in a database, if the database is an HBase database, you can use the `describe 'table_name'` command in the HBase Shell to query the structure of the HBase table contained in the HBase database.

[0075] Step a2: In response to the version verification result and column family verification result both being passed, retrieve the corresponding data to be backed up from the database corresponding to the database address.

[0076] In this embodiment, when both the version verification result and the column family verification result are passed, the backup data corresponding to the verified database is obtained based on the verification result.

[0077] It should be noted that if at least one of the version verification results and column family verification results fails, the data backup process will be stopped, and corresponding prompts will be generated based on the failed verification results, such as failed version verification results and / or failed column family verification results, so that users can be informed of the problems in the version and / or column families through the prompts, thereby further improving the user experience.

[0078] It is understood that the generated prompt message is used to inform the user that a certain database version is incorrect, and / or that a certain column family of a certain data table in a certain database is incorrect. The specific form of the prompt message is not limited in this embodiment. For example, it may be a prompt message generated using a preset error code encoding rule, or it may be a string of characters representing the result generated based on the failed verification result. This embodiment does not limit the specific method of generating the prompt message.

[0079] The data backup method provided in this embodiment, since the data table stores the data to be backed up, can obtain the corresponding data to be backed up from the database corresponding to the database address by performing version verification on the database and column family verification on the data table in the database when it is determined that both the version verification result and the column family verification result are passed. This ensures that the backed-up data is the latest and accurate, thereby improving the accuracy of data backup.

[0080] As an optional embodiment, this embodiment is in Figure 2Based on the corresponding embodiment, the content of the target backup information and the uploading of each piece of data to be backed up to the corresponding distributed file system through the management and scheduling device have been further refined. In this embodiment, the target backup information also includes an encrypted login password. When uploading each piece of data to be backed up to the corresponding distributed file system through the management and scheduling device, the specific steps include:

[0081] Step b1: Decrypt the encrypted login password to obtain the login password, and use the login password to log in to each distributed file system.

[0082] The encrypted login password is the password that has been encrypted before logging in. Specifically, this password is the password required to log in to each distributed file system.

[0083] In this embodiment, the user can generate target backup information and trigger a target backup request based on the login password, at least one database address and external storage service, or based on the login password, incremental backup identifier / full backup identifier, at least one database address and external storage service. The corresponding terminal encrypts the login password and sends a target backup request containing the target backup information after encryption.

[0084] Based on this, when the target backup request also includes an encrypted login password, the encrypted login password is decrypted to obtain the login password, and then the login password is used to authenticate each distributed file system, so that the login is successful when the authentication result is passed.

[0085] Step b2: Determine the memory value occupied by each piece of data to be backed up, and determine the current capacity value of each distributed file system.

[0086] The memory value reflects the amount of memory needed to back up the data. The capacity value reflects the amount of memory the distributed file system can hold.

[0087] Specifically, the memory size occupied by each piece of data to be backed up is detected to obtain the memory value occupied by each piece of data to be backed up, and the current capacity of each distributed file system is detected to obtain the current capacity value of each distributed file system.

[0088] Step b3: Determine whether each memory value is less than the corresponding capacity value. If yes, upload each data to be backed up to the corresponding distributed file system through the management scheduling device. If no, generate the corresponding prompt information based on the result that it is less than the capacity value.

[0089] In this embodiment, based on determining the memory value corresponding to the data to be backed up for each database and the capacity value corresponding to the distributed file system for each database, the memory value and capacity value for each database are compared. The comparison result determines whether the memory value is less than the capacity value. If so, it indicates that each distributed file system currently has sufficient capacity to store the corresponding data to be backed up, and the data to be backed up is uploaded to the corresponding distributed file system through the management scheduling device. The specific execution method is similar to S202 and will not be repeated here. If not, it indicates that each distributed file system currently does not have sufficient capacity to store the corresponding data to be backed up, and a corresponding prompt message is generated based on the result of the data being less than the capacity value.

[0090] It is understood that the generated prompt message is used to inform the user that the distributed file system corresponding to a certain database cannot hold the corresponding data to be backed up, and its specific form is not limited in this embodiment. For example, it can be a prompt message generated using a preset error code encoding rule, or it can be a string of characters representing the result generated based on the failed verification result. This embodiment does not limit the specific method of generating the prompt message.

[0091] The data backup method provided in this embodiment improves security by decrypting the encrypted login password to obtain the actual login password, which can then be used to log in to each distributed file system. By determining the memory occupied by each piece of data to be backed up and the current capacity of each distributed file system, the management scheduling device can upload the data to the corresponding distributed file system when the memory value is less than the corresponding capacity value, thus successfully uploading the data and increasing the success rate of subsequent data backups. When the memory value is greater than or equal to the corresponding capacity value, a corresponding prompt message is generated based on the result of being less than the capacity value, allowing the user to understand the upload failure based on the prompt message, further improving the user experience.

[0092] As an optional embodiment, this embodiment, based on any of the above embodiments, further includes the following steps after obtaining the target backup information in the target backup request:

[0093] The system performs a validity check on the content included in the target backup information, and continues the data backup process if the validity check result is satisfactory.

[0094] Among them, the validity check is to verify the validity of the user's input content.

[0095] It is understandable that at least one database address and storage service address in the target backup information, or encrypted login password, at least one database address and external storage service, or encrypted login password, incremental backup identifier / full backup identifier, at least one database address and external storage service, are all manually entered by the user, so there may be input errors. Therefore, after obtaining the target backup information, it is necessary to verify the legality of its contents.

[0096] Specifically, the system identifies at least one database address and storage service address, or an encrypted login password, at least one database address and external storage service, or an encrypted login password, incremental backup identifier / full backup identifier, at least one database address and external storage service field, and determines whether these fields are complete. If they are, the validity check result is determined to be passed; otherwise, the validity check result is determined to be failed. If the validity check result is passed, the subsequent data backup process continues.

[0097] It should be noted that if the validity check fails, the data backup process will be stopped, and a corresponding prompt message will be generated based on the failure result, so that the user is aware that the input content is incorrect, thereby further improving the user experience.

[0098] It is understood that the prompt message generated here is used to inform the user that a certain piece of information in the target prompt message has been entered incorrectly, and its specific form is not limited in this embodiment. For example, it can be a prompt message generated using a preset error code encoding rule, or it can be a string of characters representing the result generated based on a failed verification result. This embodiment does not limit the specific method of generating the prompt message.

[0099] The data backup method provided in this embodiment can continue the data backup process if the legality verification result is passed by performing a legality verification on the contents of the target backup information, thereby improving the success rate of subsequent data backup.

[0100] As an optional embodiment, this embodiment further includes the following steps based on any of the above embodiments:

[0101] Step c1: In response to an abnormality in the data backup process, the data backup process is terminated, and the target backup information is automatically obtained when the data backup process resumes normally.

[0102] Understandably, the data backup process will fail to execute properly when hardware failure or power failure occurs, resulting in an abnormal data backup process. Therefore, the data backup process is aborted, and the target backup information is automatically retrieved when the data backup process proceeds normally, such as after hardware repair and power restoration.

[0103] Step c2: Continue the data backup process based on the target backup information.

[0104] In this embodiment, after automatically obtaining the target backup information, the data backup process continues according to the content in the target backup information, such as at least one database address and storage service address, or encrypted login password, at least one database address and external storage service, or encrypted login password, incremental backup identifier / full backup identifier, at least one database address and external storage service.

[0105] The data backup method provided in this embodiment can automatically obtain the target backup information after recovery, i.e., when the data backup process is normal, because the data backup process may be abnormal. Therefore, when the data backup process is determined to be abnormal, the execution of the data backup process can be stopped. The data backup process can be resumed according to the target backup information, realizing end-to-end resuming. The user does not need to enter the target backup information again, which improves the user experience and increases the data backup efficiency.

[0106] It should be noted that, during the data backup process described above, key information generated during the backup process can also be obtained in real time, such as data backup time, backup operation, backup progress, and desensitization information, forming a backup log. This backup log is then stored in the storage medium, allowing users to understand the specific execution process of the data backup through the backup log in the storage medium, further improving the user experience.

[0107] Figure 3 A flowchart of a data recovery method provided in an embodiment of this application is shown below. Figure 3 As shown, the execution entity of the data recovery method provided in this embodiment is a data recovery device, which is specifically located in an electronic device. This electronic device is specifically located in each node included in the backup and recovery service. Each node deploys a shell data backup and recovery script corresponding to the cloud pool. Each cloud pool includes at least one database and an external storage service. Each database is configured with a distributed file system. When the shell data backup and recovery script is run by the corresponding node, it is used to execute the data recovery method provided in this embodiment. The specific execution steps of this embodiment include:

[0108] S301. In response to receiving a target recovery request triggered for a target cloud pool, obtain the target recovery information in the target recovery request; the target recovery information includes: at least one database address and an external storage service address.

[0109] The target recovery request is a request to instruct the node corresponding to the target cloud pool to perform data recovery. Specifically, it includes target recovery information, which is relevant information reflecting the user's specific recovery needs, including at least one database address and an external storage service address.

[0110] The specific execution process of this embodiment is similar to that of S201, and will not be described again here.

[0111] S302. Based on the target recovery information, obtain the data to be recovered corresponding to each database address from the external storage service corresponding to the storage service address, and restore each data to be recovered to the corresponding distributed file system in parallel.

[0112] It is understandable that the prerequisite for performing data recovery is that the data in the database corresponding to at least one database address mentioned in S301 has been backed up in the storage service corresponding to the external storage service address, where the data to be recovered is the data that the user needs to recover.

[0113] Based on this, the user's specific recovery needs are determined according to the target recovery information, and the databases corresponding to each database address are identified based on these specific recovery needs. The external storage service addresses in the target recovery information are then accessed to retrieve the data to be recovered for each database from the corresponding external storage service. Furthermore, based on the communication connection between the distributed file system corresponding to each database and the external storage service, the data to be recovered for each database is restored to the corresponding distributed file system.

[0114] S303. The data to be recovered in each distributed file system is uploaded to the corresponding database through the management and scheduling device.

[0115] In this embodiment, after restoring each piece of data to be recovered to the corresponding distributed file system, a recovery task is further generated based on the data to be recovered in each distributed file system, and the recovery task is submitted to the management and scheduling device. Then, according to the scheduling instructions of the management and scheduling device, each piece of data to be recovered in the recovery task is uploaded to the corresponding database.

[0116] Specifically, when uploading the data to be recovered in the recovery task to the corresponding database, it can be done by: determining whether the user-specified data table exists in each database; if so, uploading the data to be recovered to the specified data table in the corresponding database; if not, automatically creating a new data table in each database and uploading the data to be recovered to the new data table in the corresponding database.

[0117] Understandably, after uploading the data to be recovered from each distributed file system to the database address, the data to be recovered from each distributed file system can be deleted to avoid occupying the memory of each distributed file system, ensuring that each distributed file system has enough space for data transfer, improving transfer performance, and thus improving the success rate of data backup and recovery.

[0118] The data recovery method provided in this embodiment utilizes a cloud pool-specific shell data backup and recovery script deployed on each node of the backup and recovery service. Each cloud pool contains at least one database and an external storage system, and each database is configured with a distributed file system. Therefore, upon receiving a target recovery request triggered by a user for the target cloud pool, the node corresponding to the target cloud pool can obtain the target recovery information in the request by running the shell data backup and recovery script for the target cloud pool. This information includes the database address of at least one database within the target cloud pool and the external storage service address. Based on the target recovery information, the node can retrieve the data to be recovered corresponding to each database address from the external storage service corresponding to the storage service address. This allows the data to be recovered in parallel to the corresponding distributed file system. Furthermore, the management and scheduling device uploads the data to be recovered from each distributed file system to its corresponding database. This eliminates the need to execute numerous commands to recover data from one or more databases, improving data recovery efficiency and user experience. Moreover, the scheduling instructions based on the distributed file system do not involve AK and SK, and data transmission between the distributed file systems and the external storage service does not require management and scheduling, preventing the transmitted AK and SK from being recorded by the management and scheduling device, reducing the risk of database device information leakage, and improving the security of the recovery process.

[0119] Optionally, the target recovery information may also include an incremental recovery identifier or a full recovery identifier. When retrieving the data to be recovered for each database address from the external storage service corresponding to the storage service address based on the target recovery information, the full recovery identifier is used to retrieve the data to be recovered for all database addresses from the external storage service corresponding to the storage service address; or, the incremental recovery identifier is used to retrieve the data to be recovered for a portion of the database addresses from the external storage service corresponding to the storage service address. The specific execution process is similar to retrieving the corresponding backup data from the databases corresponding to each database address based on the target backup information during data backup, and will not be elaborated here. This allows users to choose the data to be recovered, improving recovery flexibility and further enhancing the user experience.

[0120] Optionally, when retrieving the data to be recovered corresponding to the database address from the external storage service corresponding to the external storage service address, version verification of the database and column family verification of the data table to which the data to be recovered belongs are performed; the data table is used to store the data to be recovered; in response to both the version verification result and the column family verification result being passed, the data to be recovered corresponding to the database address is retrieved from the external storage service corresponding to the external storage service address. The specific execution process is similar to steps a1-a2, and will not be repeated here. This ensures that the recovered data is up-to-date and accurate, improving the accuracy of data recovery.

[0121] Optionally, the target recovery information also includes an encrypted login password. When restoring each piece of data to be recovered to its corresponding distributed file system in parallel, the encrypted login password is decrypted to obtain the login password, which is then used to log in to each distributed file system. The memory value occupied by each piece of data to be recovered is determined, as is the current capacity of each distributed file system. It is then determined whether each memory value is less than its corresponding capacity value. If so, the data to be recovered is restored to its corresponding distributed file system in parallel; otherwise, a corresponding prompt message is generated based on the result of the memory value being less than the capacity value. The specific execution process is similar to steps b1-b3, and will not be elaborated here. This allows for successful data recovery to the file system, improving the success rate of subsequent data recovery and enabling users to understand the upload failure result based on the prompt message, further improving the user experience.

[0122] Optionally, after obtaining the target recovery information from the target recovery request, the process further includes: verifying the validity of the content included in the target recovery information, and continuing the data recovery process if the validity verification result is successful. The specific execution process is similar to the validity verification based on the target backup information during data backup, and will not be elaborated here. This improves the success rate of subsequent data recovery.

[0123] Optionally, in response to an anomaly during the data recovery process, the data recovery process is aborted, and the target recovery information is automatically obtained when the data recovery process resumes normally; the data recovery process then continues based on the target recovery information. The specific execution process is similar to steps c1-c2, and will not be described in detail here. This enables end-point resuming without requiring the user to re-enter the target recovery information, improving user experience and increasing data recovery efficiency.

[0124] It should be noted that, during the data recovery process described above, key information generated during the data recovery process can also be obtained in real time, such as data recovery time, recovery operations, recovery progress, and de-identification information, forming a recovery log. This recovery log is then stored in a storage medium, allowing users to understand the specific execution process of data recovery through the recovery log in the aforementioned storage medium, further improving the user experience.

[0125] It should be noted that the data backup and recovery process based on the above embodiments will not affect the user's normal use of HBase services, further improving the user experience.

[0126] Figure 4 A complete flowchart of a data backup and recovery method provided in an embodiment of this application is shown below. Figure 4 As shown in the figure, this embodiment schematically illustrates the specific execution process of the data backup and recovery method.

[0127] S401. In response to receiving a target backup request triggered for a target cloud pool, obtain the target backup information in the target backup request; the target backup information includes: at least one database address, external storage service address, incremental backup identifier or full backup identifier, and encrypted login password.

[0128] S402. Verify the legality of the content included in the target backup information.

[0129] S403. When the legality verification result is passed, perform version verification on all databases / part of the databases based on the full backup identifier / incremental backup identifier, and perform column family verification on the data tables in the database.

[0130] S404: In response to both version verification and column family verification results being passed, retrieve the corresponding backup data from the databases corresponding to all database addresses / partial database addresses.

[0131] S405. Decrypt the encrypted login password to obtain the login password, and use the login password to log in to each distributed file system.

[0132] S406. Determine the memory value occupied by each piece of data to be backed up, and determine the current capacity value of each distributed file system.

[0133] S407. Determine if each memory value is less than its corresponding capacity value. If yes, execute S409-S410; otherwise, execute S408.

[0134] S408. Generate corresponding prompt information based on the result that is less than the capacity value.

[0135] S409. Upload each piece of data to be backed up to the corresponding distributed file system through the management and scheduling device.

[0136] S410. Back up the data to be backed up in each distributed file system in parallel to the external storage service corresponding to the external storage service address.

[0137] S411. In response to receiving a target recovery request triggered for a target cloud pool, obtain the target recovery information in the target recovery request; the target recovery information includes: at least one database address, external storage service address, incremental recovery identifier or full recovery identifier, and encrypted login password.

[0138] S412. Verify the legality of the content included in the target recovery information.

[0139] S413. When the legality verification result is passed, perform version verification on all databases / part of the databases according to the full recovery identifier / incremental recovery identifier, and perform column family verification on the data table to which the data to be recovered belongs corresponding to all database addresses / part of the database addresses.

[0140] S414. In response to both the version verification result and the column family verification result being passed, retrieve the data to be recovered corresponding to all database addresses / partial database addresses from the external storage service corresponding to the external storage service address.

[0141] S415. Decrypt the encrypted login password to obtain the login password, and use the login password to log in to each distributed file system.

[0142] S416. Determine the memory value occupied by each piece of data to be recovered, and determine the current capacity value of each distributed file system.

[0143] S417. Determine if each memory value is less than its corresponding capacity value. If yes, execute S419-S420; otherwise, execute S418.

[0144] S418. Generate a corresponding prompt message based on the result that is less than the capacity value.

[0145] S419. Restore each piece of data to be restored to the corresponding distributed file system in parallel.

[0146] S420: The data to be recovered in each distributed file system is uploaded to the corresponding database through the management and scheduling device.

[0147] S421. In response to an abnormality in the data backup / restore process, the execution of the data backup / restore process is terminated, and the target backup / restore information is automatically obtained when the data backup / restore process resumes normal operation.

[0148] S422. Continue the data backup / restore process based on the target backup / restore information.

[0149] Figure 5 The data backup and recovery architecture diagram provided for this application is as follows: Figure 5As shown, the architecture provided in this application includes a storage layer and a business layer. The storage layer includes at least one database to be backed up and a storage service for backup. The business layer is responsible for completing the specific execution processes such as data backup, data recovery, breakpoint resumption, secure transmission, and Phoenix compatibility. This process also involves the parsing and decryption of relevant parameters, such as the parsing and verification of target backup / recovery information and the decryption of encrypted login passwords. Throughout the entire method execution process, logs are also recorded to obtain information about the specific execution process of data backup and recovery.

[0150] Figure 6 This is a schematic diagram of the structure of a data backup device provided in an embodiment of this application, as shown below. Figure 6 As shown, the data backup device is specifically located in an electronic device, which is specifically located in each node included in the backup and recovery service. Each node deploys a shell data backup and recovery script corresponding to the cloud pool. Each cloud pool includes at least one database and an external storage service. Each database is configured with a distributed file system. When the shell data backup and recovery script is run by the corresponding node, it is used to execute the data backup method provided in this embodiment. Therefore, the data backup device 60 provided in this embodiment includes: an acquisition module 61 and a processing module 62.

[0151] The acquisition module 61 is used to respond to a target backup request triggered for a target cloud pool and acquire the target backup information in the target backup request. The target backup information includes at least one database address and an external storage service address. Based on the target backup information, the module acquires the corresponding data to be backed up in the database corresponding to each database address. The processing module 62 is used to upload each data to be backed up to the corresponding distributed file system through the management and scheduling device. The module backs up the data to be backed up in each distributed file system in parallel to the external storage service address corresponding to the external storage service address.

[0152] The data backup device provided in this application embodiment can be used to execute the technical solution of the data backup method in the above embodiment. Its implementation principle and technical effect are similar, and will not be described again here.

[0153] Optionally, the target backup information may also include an incremental backup identifier or a full backup identifier;

[0154] Accordingly, module 61, when retrieving the corresponding backup data from the databases corresponding to each database address based on the target backup information, is specifically used for:

[0155] The corresponding backup data is retrieved from the databases corresponding to all database addresses based on the full backup identifier; or, the corresponding backup data is retrieved from the databases corresponding to some database addresses based on the incremental backup identifier.

[0156] Optionally, when retrieving the corresponding backup data from the database corresponding to the database address, module 61 is specifically used for:

[0157] Perform version verification on the database and column family verification on the data tables in the database; the data tables are used to store the data to be backed up; in response to the version verification result and the column family verification result both being passed, retrieve the corresponding data to be backed up from the database corresponding to the database address.

[0158] Optionally, the target backup information may also include an encrypted login password;

[0159] Accordingly, when processing module 62 uploads each piece of data to be backed up to the corresponding distributed file system through the management and scheduling device, it is specifically used for:

[0160] The encrypted login password is decrypted to obtain the login password, and the login password is used to log in to each distributed file system; the memory value occupied by each piece of data to be backed up is determined, and the current capacity value of each distributed file system is determined; it is determined whether each memory value is less than the corresponding capacity value. If so, the data to be backed up is uploaded to the corresponding distributed file system through the management scheduling device. If not, the corresponding prompt message is generated based on the result that it is less than the capacity value.

[0161] Optionally, the processing module 62 is further configured to perform a validity check on the content included in the target backup information after obtaining the target backup information in the target backup request, and continue to execute the data backup process when the validity check result is passed.

[0162] Optionally, the processing module 62 is also configured to, in response to an abnormality in the data backup process, suspend the execution of the data backup process, and automatically obtain the target backup information when the data backup process resumes normal operation; and continue the data backup process based on the target backup information.

[0163] The data backup device provided in this application embodiment can be used to execute the technical solution of any of the above embodiments. Its implementation principle and technical effect are similar, and will not be described again here.

[0164] Figure 7 This is a schematic diagram of the structure of a data recovery device provided in an embodiment of this application, as shown below. Figure 7As shown, the data recovery device is specifically located in an electronic device, which is specifically located in each node included in the backup and recovery service. Each node deploys a shell data backup and recovery script corresponding to the cloud pool. Each cloud pool includes at least one database and an external storage service. Each database is configured with a distributed file system. When the shell data backup and recovery script is run by the corresponding node, it is used to execute the data recovery method provided in this embodiment. Therefore, the data recovery device 70 provided in this embodiment includes: an acquisition module 71 and a processing module 72.

[0165] The acquisition module 71 is used to respond to a target recovery request triggered for a target cloud pool and acquire the target recovery information in the target recovery request. The target recovery information includes at least one database address and an external storage service address. Based on the target recovery information, the module acquires the data to be recovered corresponding to each database address from the external storage service corresponding to the external storage service address. The processing module 72 is used to restore each data to be recovered to the corresponding distributed file system in parallel. The module uploads the data to be recovered in each distributed file system to the corresponding database through the management and scheduling device.

[0166] The data recovery device provided in this application embodiment can be used to execute the technical solution of the data recovery method in the above embodiment. Its implementation principle and technical effect are similar, and will not be described again here.

[0167] Figure 8 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Figure 8 As shown, the electronic device 80 may include: a processor 81 and a memory 82.

[0168] Processor 81 executes computer execution instructions stored in memory, causing processor 81 to perform the scheme in the above embodiments. Processor 81 can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0169] The memory 82 is connected to the processor 81 via the system bus and completes communication between them. The memory 82 is used to store computer program instructions.

[0170] The electronic device 80 may also include a transceiver that can be used to send and receive data.

[0171] The system bus can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The system bus can be divided into address bus, data bus, control bus, etc. For ease of representation, only one thick line is used in the diagram, but this does not indicate that there is only one bus or one type of bus. Transceivers are used to enable communication between database access devices and other computers (e.g., clients, read-write libraries, and read-only libraries). Memory may include random access memory (RAM) and may also include non-volatile memory.

[0172] The electronic device provided in this application embodiment can also be the terminal device described in the above embodiments.

[0173] This application also provides a chip for executing instructions, which is used to execute the data backup and recovery methods described in the above embodiments.

[0174] This application also provides a computer-readable storage medium storing computer instructions that, when executed on a computer, cause the computer to perform the data backup and recovery method described in the above embodiments.

[0175] This application also provides a computer program product, which includes a computer program stored in a computer-readable storage medium. At least one processor can read the computer program from the computer-readable storage medium. When the at least one processor executes the computer program, it can implement the technical solution of the data backup and recovery method in the above embodiments.

[0176] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are all optional embodiments, and the actions and modules involved are not necessarily essential to this application.

[0177] It should be further noted that although the steps in the flowchart are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowchart may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the sub-steps or stages of other steps.

[0178] It should be understood that the above-described device embodiments are merely illustrative, and the device of this application can also be implemented in other ways. For example, the division of units / modules in the above embodiments is only a logical functional division, and there may be other division methods in actual implementation. For example, multiple units, modules, or components may be combined, or integrated into another system, or some features may be ignored or not executed.

[0179] Furthermore, unless otherwise specified, the functional units / modules in the various embodiments of this application can be integrated into one unit / module, or each unit / module can exist physically separately, or two or more units / modules can be integrated together. The integrated units / modules described above can be implemented in hardware or as software program modules.

[0180] When integrated units / modules are implemented in hardware, the hardware can be digital circuits, analog circuits, etc. The physical implementation of the hardware structure includes, but is not limited to, transistors, memristors, etc. Unless otherwise specified, the processor can be any suitable hardware processor, such as a CPU, GPU, FPGA, DSP, and ASIC, etc. Unless otherwise specified, the storage unit can be any suitable magnetic or magneto-optical storage medium, such as Resistive Random Access Memory (RRAM), Dynamic Random Access Memory (DRAM), Static Random Access Memory (SRAM), Enhanced Dynamic Random Access Memory (EDRAM), High-Bandwidth Memory (HBM), Hybrid Memory Cube (HMC), etc.

[0181] If the integrated unit / module is implemented as a software program module and sold or used as an independent product, it can be stored in a computer-readable storage device (CMD). Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a memory and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned memory includes various media capable of storing program code, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard drive, magnetic disk, or optical disk.

[0182] In the above embodiments, the descriptions of each embodiment have their own emphasis. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments. The technical features of the above embodiments can be combined arbitrarily. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as these combinations of technical features do not contradict each other, they should be considered within the scope of this specification.

[0183] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this application are indicated by the following claims.

[0184] It should be understood that this application is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this application is limited only by the appended claims.

Claims

1. A data backup method, characterized in that, The method is applied to a backup and recovery service, which includes at least one node. Each node deploys a shell data backup and recovery script corresponding to a cloud pool. Each cloud pool includes at least one database and an external storage service. Each database is configured with a distributed file system. When the shell data backup and recovery script is run by the corresponding node, it is used to execute the data backup method. The method includes: In response to receiving a target backup request triggered for a target cloud pool, the system obtains the target backup information from the target backup request; the target backup information includes: at least one database address and an external storage service address. Based on the target backup information, the corresponding backup data is obtained from the database corresponding to each database address, and the backup data is uploaded to the corresponding distributed file system through the management and scheduling device. The data to be backed up in each distributed file system is backed up in parallel to the external storage service corresponding to the external storage service address.

2. The method according to claim 1, characterized in that, The target backup information also includes an incremental backup identifier or a full backup identifier; The step of retrieving the corresponding backup data from the databases corresponding to each database address based on the target backup information includes: The corresponding backup data is retrieved from the databases corresponding to all database addresses based on the full backup identifier; or, the corresponding backup data is retrieved from the databases corresponding to some database addresses based on the incremental backup identifier.

3. The method according to claim 2, characterized in that, Retrieve the corresponding data to be backed up from the database corresponding to the database address, specifically including: The database undergoes version verification, and the data tables within the database undergo column family verification; these data tables are used to store data to be backed up. If both the version verification result and the column family verification result are passed, the corresponding backup data is retrieved from the database corresponding to the database address.

4. The method according to claim 1, characterized in that, The target backup information also includes an encrypted login password; The process of uploading each piece of data to be backed up to the corresponding distributed file system through the management and scheduling device includes: The encrypted login password is decrypted to obtain the login password, and the login password is used to log in to each distributed file system; Determine the memory usage of each piece of data to be backed up, and determine the current capacity of each distributed file system. Determine if each memory value is less than its corresponding capacity value. If so, upload each piece of data to be backed up to the corresponding distributed file system through the management scheduling device. If not, generate the corresponding prompt message based on the result that the data is less than the capacity value.

5. The method according to any one of claims 1-4, characterized in that, After obtaining the target backup information from the target backup request, the process further includes: The legality of the content included in the target backup information is verified, and the data backup process continues if the legality verification result is successful.

6. The method according to any one of claims 1-4, characterized in that, Also includes: In response to an anomaly in the data backup process, the data backup process is aborted, and the target backup information is automatically retrieved when the data backup process resumes normally. Continue the data backup process based on the target backup information.

7. A data recovery method, characterized in that, The method is applied to a backup and recovery service, which includes at least one node. Each node deploys a shell data backup and recovery script corresponding to a cloud pool. Each cloud pool includes at least one database and an external storage service. Each database is configured with a distributed file system. When the shell data backup and recovery script is run by the corresponding node, it is used to execute the data recovery method. The method includes: In response to receiving a target recovery request triggered for a target cloud pool, the system obtains the target recovery information from the target recovery request; the target recovery information includes: at least one database address and an external storage service address. Based on the target recovery information, the data to be recovered corresponding to each database address is obtained from the external storage service corresponding to the external storage service address, and the data to be recovered is restored to the corresponding distributed file system in parallel; The management and scheduling equipment uploads the data to be recovered from each distributed file system to the corresponding database.

8. An electronic device, characterized in that, include: A processor, and a memory communicatively connected to the processor; The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory to implement the data backup and recovery method as described in any one of claims 1 to 7.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the data backup and recovery method as described in any one of claims 1 to 7.

10. A computer program product, characterized in that, Includes a computer program that, when executed by a processor, implements the data backup and recovery method according to any one of claims 1 to 7.

Images