Data backup method and equipment

Abstract

According to the data backup method and device provided by the specification, description information corresponding to target data is obtained, and the description information comprises at least one of data change frequency, health state information of a storage medium stored in the target data or table association information; the table association information comprises a plurality of tables used for storing the target data and a dependency relationship among the plurality of tables, and a target backup strategy is intelligently generated through a pre-trained backup strategy model, so that backup of the target data according to the target backup strategy is more intelligent. The target backup strategy is intelligently obtained according to the data change frequency of the target data, the health state information of the storage medium and / or the table association information instead of a fixed backup strategy, so that the data can be effectively protected by performing data backup through the target backup strategy generated by the method in the specification.

Description

Technical Field

[0001] This specification relates to the field of data processing, and in particular, to a data backup method and device. Background Art

[0002] Data backup refers to the process of copying data in a computer system to a location independent of the original storage medium. This process aims to prevent data loss caused by software errors, virus attacks, human errors, or natural disasters. An effective backup strategy can not only recover lost data but also ensure transaction continuity and reduce downtime caused by unavailable data.

[0003] The method of backing up data according to a fixed backup strategy is too mechanical and difficult to meet the high requirements of modern database systems. Therefore, a data backup method is needed that can back up data more intelligently and flexibly to effectively protect the data.

[0004] The content in the background art section is only the information known to the inventor personally, and does not represent that the above information has entered the public domain before the filing date of this disclosure, nor does it represent that it can become the prior art of this disclosure. Summary of the Invention

[0005] The data backup method and device provided in this specification can back up data intelligently and flexibly.

[0006] In a first aspect, this specification provides a data backup method applied to a data backup device, including: obtaining description information corresponding to target data, where the description information includes at least one of a data change frequency, health status information of a storage medium storing the target data, or table association information, and the table association information includes a plurality of tables for storing the target data and a dependency relationship between the plurality of tables; generating a target backup strategy based on the description information and a pre-trained backup strategy model; and backing up the target data according to the target backup strategy.

[0007] In some embodiments, the data change frequency includes a heat matrix, and the heat matrix represents the change frequency of each partition data of the target data corresponding to each tenant.

[0008] In some embodiments, the generating a target backup strategy based on the description information and a pre-trained backup strategy model includes: inputting the health status information of the storage medium into a pre-trained failure probability prediction model, and obtaining the failure probability of the storage medium predicted by the failure probability prediction model within a preset period; and inputting the failure probability into the backup strategy model, and obtaining the target backup strategy output by the backup strategy model.

[0009] In some embodiments, the health status information of the storage medium includes at least one of temperature, reallocated sector count, error rate, or access frequency.

[0010] In some embodiments, generating the target backup policy based on the description information and a pre-trained backup policy model includes: inputting the table association information into a pre-trained graph neural network model, and obtaining a cross-table consistency vector output by the graph neural network model; and inputting the cross-table consistency vector into the backup policy model, and obtaining the target backup policy output by the backup policy model.

[0011] In some embodiments, the description information further includes the load information of the data backup device, and the load information includes at least one of CPU usage rate, memory usage rate, number of I / O operations, or number of transaction conflicts.

[0012] In some embodiments, the backup policy model includes a multi-agent reinforcement model, the multi-agent reinforcement model includes multiple agents, the target backup policy includes multiple sub-policies, and each agent is configured to output at least one sub-policy of the multiple sub-policies based on the description information.

[0013] In some embodiments, the multiple sub-policies include at least two of backup granularity, backup method, backup period, backup concurrency, backup destination, or execution quality when performing the backup task.

[0014] In some embodiments, the multi-agent reinforcement model is obtained through federated training based on the training data of the subject to which the target data belongs and the training data of other subjects, and the training data of different subjects is data with added noise.

[0015] In some embodiments, the multi-agent reinforcement model is trained with the minimization of a target loss function as the training objective, and the target loss function includes at least one of the maximum data loss amount, data recovery duration, cost of performing the backup task, data consistency risk, or carbon emission amount of performing the backup task.

[0016] In some embodiments, after backing up the target data according to the target backup policy, it further includes: extracting test data from the backed-up backup data, and creating a temporary tenant; recovering the test data in the temporary tenant, and determining the evaluation result corresponding to the recovery; and updating the backup policy model based on the evaluation result.

[0017] In a second aspect, this specification also provides a data backup device, including: at least one storage medium storing at least one instruction set for implementing data backup; and at least one processor communicatively connected to the at least one storage medium, wherein when the data backup device runs, the at least one processor reads the at least one instruction set and implements the data backup method according to any one of the first aspect.

[0018] As can be seen from the above technical solutions, the data backup method and device provided in this specification obtain description information corresponding to target data, where the description information includes at least one of data change frequency, health status information of the storage medium storing the target data, or table association information, and the table association information includes multiple tables for storing the target data and the dependency relationship between the multiple tables, and intelligently generate a target backup strategy through a pre-trained backup strategy model, so that backing up the target data according to the target backup strategy is more intelligent. The target backup strategy is intelligently obtained based on the data change frequency, health status information of the storage medium, and / or table association information of the target data, rather than a fixed backup strategy. Therefore, when data is backed up using the target backup strategy generated by the method in this specification, the data can be effectively protected.

[0019] Other functions of the data backup method and device provided in this specification will be partially listed in the following description. According to the description, the content introduced by the following numbers and examples will be obvious to those of ordinary skill in the art. The creative aspects of the data backup method and device provided in this specification can be fully explained through practice or the use of the methods, devices, and combinations described in the detailed examples below. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] To more clearly illustrate the technical solutions in the embodiments of this specification, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the drawings in the following description are only some embodiments of this specification. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings.

[0021] Figure 1 Shows a schematic diagram of a data backup scenario provided according to some embodiments of this specification; Figure 2 Shows a hardware structure diagram of a computing device provided according to some embodiments of this specification; Figure 3 Shows a flowchart of a data backup method provided according to some embodiments of this specification; and Figure 4 Shows a schematic diagram of a data backup method provided according to some embodiments of this specification. Detailed Implementation Modes

[0022] The following description provides specific application scenarios and requirements of this specification, aiming to enable those skilled in the art to manufacture and use the content in this specification. For those skilled in the art, various local modifications to the disclosed embodiments are obvious, and without departing from the spirit and scope of this specification, the general principles defined here can be applied to other embodiments and applications. Therefore, this specification is not limited to the disclosed embodiments, but has the broadest scope consistent with the claims.

[0023] The terms used here are only for the purpose of describing specific example embodiments and are not restrictive. For example, unless the context clearly indicates otherwise, the singular forms "a", "an", and "the" used here may also include the plural forms. When used in this specification, the terms "comprise", "include", and / or "contain" mean that the associated integers, steps, operations, elements, and / or components exist, but do not exclude the existence of one or more other features, integers, steps, operations, elements, components, and / or groups, or the addition of other features, integers, steps, operations, elements, components, and / or groups in the device / method.

[0024] In view of the following description, these features of this specification and other features, as well as the operations and functions of the related elements of the structure, and the combination and manufacturing economy of the components can be significantly improved. Referring to the accompanying drawings, all of these form a part of this specification. However, it should be clearly understood that the drawings are only for the purpose of illustration and description and are not intended to limit the scope of this specification. It should also be understood that the drawings are not drawn to scale.

[0025] The flowcharts used in this specification illustrate the operations implemented by a device according to some embodiments in this specification. It should be clearly understood that the operations in the flowchart may not be implemented in sequence. On the contrary, the operations may be implemented in reverse order or simultaneously. In addition, one or more other operations may be added to the flowchart. One or more operations may be removed from the flowchart.

[0026] In this specification, "X includes at least one of A, B, or C" means that X includes at least A, or X includes at least B, or X includes at least C. That is, X may only include any one of A, B, C, or may simultaneously include any combination of A, B, C and other possible contents / elements. Any combination of A, B, C may be A, B, C, AB, AC, BC, or ABC.

[0027] In this specification, unless otherwise clearly stated, the association relationships generated between structures can be direct association relationships or indirect association relationships. For example, when describing "A is connected to B", unless it is clearly stated that A is directly connected to B, it should be understood that A can be directly connected to B or indirectly connected to B; for another example, when describing "A is above B", unless it is clearly stated that A is directly above B (A and B are adjacent and A is above B), it should be understood that A can be directly above B or A can be indirectly above B (there are other elements between A and B and A is above B). And so on.

[0028] Before describing the specific embodiments of this specification, an overall introduction to the solution of this specification will be given first: Traditional data backup all adopts a fixed backup strategy. The fixed backup strategy usually adopts a fixed backup frequency and backup method. For example, a full backup is performed once a week on Monday, and incremental backups are performed on Wednesday, Friday, and Sunday. That is, regardless of how the data-related information changes, the backup strategy always remains fixed, resulting in relatively rigid data backup, lacking flexibility and intelligence. Moreover, the fixed backup strategy is also difficult to meet the high availability and high performance requirements of modern database systems. However, the data backup method provided in this specification can intelligently generate a backup strategy through a backup strategy model, and the backup strategy model can generate a more optimal backup strategy that fits the actual situation according to the actual description information of the data, so that when backing up data according to this backup strategy, data can be more effectively protected and data loss can be prevented.

[0029] The data backup method provided in this specification can be applied to various types of databases, such as distributed databases, distributed relational databases, relational databases, non-relational databases, etc. Any enterprise or organization can adopt the data backup method provided in this specification for data backup when using a database. For example, an e-commerce enterprise can adopt a backup strategy model and use the relevant description information of e-commerce data to generate a backup strategy to back up e-commerce data. A financial institution can adopt a backup strategy model and use the relevant description information of financial data to generate a backup strategy to back up financial data.

[0030] It should be noted that the above example scenario is only one of the multiple usage scenarios provided in this specification. The data backup method provided in this specification can not only be applied to the above scenario, but also to other scenarios. Those skilled in the art should understand that the application of the data backup method described in this specification to other usage scenarios is also within the protection scope of this specification.

[0031] Figure 1 Shows a schematic diagram of a data backup scenario 001 provided according to some embodiments of this specification. As Figure 1As shown, the scenario 001 may include a target user 100, a client 200, a server 300, and a network 400.

[0032] The target user 100 may be a manager who manually manages the data backup method. The target user 100 may configure the data backup process. For example, import a pre-trained backup policy into the computing device, set the execution time for executing the data backup method, etc.

[0033] In some embodiments, the client 200 may display various interfaces for the target user 100 to view and trigger, so as to achieve human-computer interaction. The triggering may be click triggering, voice triggering, image triggering, etc. It should be noted that the user data obtained in this specification has been authorized by the user and does not involve user privacy.

[0034] The data backup method may be executed on a computing device (i.e., a data backup device). In some embodiments, the computing device may be the client 200. At this time, the client 200 may store the data or instructions for executing the data backup method described in this specification and may execute or be used to execute the data or instructions. In some embodiments, the client 200 may include a hardware device with data information processing functions and the necessary programs for driving the hardware device to work. Such as Figure 1As shown, the client 200 can communicate with the server 300. In some embodiments, the server 300 can communicate with multiple clients 200. In some embodiments, the client 200 can interact with the server 300 through the network 400 to receive or send messages, such as receiving or sending biological images or various feature information, such as two-dimensional images / features and / or three-dimensional images / features. In some embodiments, the client 200 can include a mobile device, a tablet computer, a laptop computer, an in-vehicle device of a motor vehicle or the like, a vending machine, a vending cabinet, or any combination thereof. In some embodiments, the mobile device can include a smart home device, a smart mobile device, a virtual reality device, an augmented reality device or the like, or any combination thereof. In some embodiments, the smart home device can include a smart TV, a desktop computer, etc., or any combination. In some embodiments, the smart mobile device can include a smart phone, a personal digital assistant, a gaming device, a navigation device, etc., or any combination thereof. In some embodiments, the virtual reality device or the augmented reality device may include a virtual reality helmet, virtual reality glasses, a virtual reality patch, an augmented reality helmet, augmented reality glasses, an augmented reality patch or the like, or any combination thereof. For example, the virtual reality device or the augmented reality device may include smart glasses, a head-mounted display, VR, etc. In some embodiments, the in-vehicle device in the motor vehicle can include an in-vehicle computer, an in-vehicle TV, etc. In some embodiments, the client 200 can include an image acquisition device for acquiring biological images, such as the face image of the target user 100. In some embodiments, the image acquisition device can be a two-dimensional image acquisition device (such as an RGB camera), or a two-dimensional image acquisition device (such as an RGB camera) and a depth image acquisition device (such as a 3D structured light camera, a laser detector, etc.). In some embodiments, the client 200 can be a device with positioning technology for positioning the location of the client 200. In some embodiments, the client 200 can have one or more of the following functions: NFC (Near Field Communication), WIFI (Wireless Fidelity), 3G / 4G / 5G, POS (Point Of Sale) machine card swiping function, two-dimensional code scanning function, bar code scanning function, Bluetooth, infrared, SMS (Short Message Service), MMS (Multimedia Message Service).

[0035] In some embodiments, the client 200 may be installed with one or more applications (APPs). The APPs can provide the target user 100 with the ability to interact with the outside world through the network 400 and an interface. The APPs include, but are not limited to: web browser APP programs, search APP programs, chat APP programs, shopping APP programs, video APP programs, financial management APP programs, instant messaging tools, email clients, social platform software, and so on. In some embodiments, a target APP may be installed on the client 200. In some embodiments, the target APP can execute the data backup method. The target user 100 can trigger a request to execute data backup through the target APP, and the target APP can respond to the request and execute the data backup method.

[0036] In some embodiments, the computing device may be the server 300. At this time, the server 300 may store data or instructions for executing the data backup method described in this specification, and may execute or be used to execute the data or instructions. In some embodiments, the server 300 may include a hardware device with data information processing capabilities and the necessary programs for driving the hardware device to work. The server 300 may be a server that provides various services, such as a backend server that supports the pages displayed by the target APP on the client 200.

[0037] The network 400 is a medium for providing a communication connection between the client 200 and the server 300. The network 400 can facilitate the exchange of information or data. As Figure 1 shown, the client 200 and the server 300 can be connected to the network 400 and transmit information or data to each other through the network 400. In some embodiments, the network 400 can be any type of wired or wireless network, or a combination thereof. For example, the network 400 can include a cable network, a wired network, an optical fiber network, a telecommunication network, an intranet, the Internet, a local area network (LAN), a wide area network (WAN), a wireless local area network (WLAN), a metropolitan area network (MAN), a public switched telephone network (PSTN), a Bluetooth network, a ZigBee network, a near field communication (NFC) network, or a similar network. In some embodiments, the network 400 can include one or more network access points. For example, the network 400 can include wired or wireless network access points, such as base stations or Internet exchange points, through which one or more components of the client 200 and the server 300 can be connected to the network 400 to exchange data or information.

[0038] It should be understood, Figure 1The numbers of the client 200, the server 300, and the network 400 in [description] are merely illustrative. According to the implementation requirements, there can be any number of clients 200, servers 300, and networks 400.

[0039] It should be noted that the data backup method can be executed entirely on the client 200, entirely on the server 300, or partially on the client 200 and partially on the server 300.

[0040] Figure 2 The hardware structure diagram of a computing device 600 provided according to some embodiments of the present specification is shown. The computing device 600 can be a data backup device for executing the data backup method described in the present specification. The data backup method is introduced in other parts of the present specification. The computing device 600 can be a device of the client 200, a device of the server 300, or other computing devices, or even any combination of the above devices.

[0041] As Figure 2 shown, the computing device 600 can include at least one storage medium 630 and at least one processor 620. In some embodiments, the computing device 600 can also include a communication port 650 and an internal communication bus 610. At the same time, the computing device 600 can also include I / O components 660.

[0042] The internal communication bus 610 can connect different components, including the storage medium 630, the processor 620, and the communication port 650.

[0043] The I / O components 660 support input / output between the computing device 600 and other components.

[0044] The communication port 650 is used for data communication between the computing device 600 and the outside world. For example, the communication port 650 can be used for data communication between the computing device 600 and the network 400. The communication port 650 can be a wired communication port or a wireless communication port.

[0045] The storage medium 630 may include a data storage device. The data storage device may be a non-transitory storage medium or a transitory storage medium. For example, the data storage device may include one or more of a magnetic disk 632, a read-only storage medium (ROM) 634, or a random access storage medium (RAM) 636. The storage medium 630 may store at least one instruction set for implementing data backup. The instructions are computer program code, and the computer program code may include programs, routines, objects, components, data structures, procedures, modules, etc. for executing the data backup method provided in this specification. The storage medium 630 may also store a backup policy model. At this time, the model may be one or more instruction sets stored in the storage medium 630 for executing corresponding instructions and executed by the processor 620 in the computing device 600. Of course, the model may also be a part of the circuit, a hardware device, or a module in the computing device 600. At this time, at least one set of instruction sets for controlling one or the instruction sets of the model may be stored in the processor 620.

[0046] At least one processor 620 can be communicatively connected to at least one storage medium 630 and a communication port 650 via an internal communication bus 610. The at least one processor 620 is configured to execute the at least one instruction set described above. When the computing device 600 is running, the at least one processor 620 can read the at least one instruction set and, according to the instructions of the at least one instruction set, execute the data backup method provided in this specification. The processor 620 can execute all the steps included in the data backup method. The processor 620 can be in the form of one or more processors. In some embodiments, the processor 620 can include one or more hardware processors, such as a microcontroller, a microprocessor, a reduced instruction set computer (RISC), an application specific integrated circuit (ASIC), an application specific instruction set processor (ASIP), a central processing unit (CPU), a graphics processing unit (GPU), a physics processing unit (PPU), a microcontroller unit, a digital signal processor (DSP), a field programmable gate array (FPGA), an advanced RISC machine (ARM), a programmable logic device (PLD), any circuit or processor capable of executing one or more functions, etc., or any combination thereof. For illustrative purposes only, only one processor 620 is described in the computing device 600 in this specification. However, it should be noted that the computing device 600 in this specification can also include multiple processors. Therefore, the operations and / or method steps disclosed in this specification can be executed by one processor as described in this specification, or jointly executed by multiple processors. For example, if the processor 620 of the computing device 600 executes step A and step B in this specification, it should be understood that step A and step B can also be jointly or separately executed by two different processors 620 (e.g., the first processor executes step A, the second processor executes step B, or the first and second processors jointly execute steps A and B).

[0047] Figure 3 FIG. shows a flowchart of a data backup method P100 provided according to some embodiments of this specification. As described above, the computing device 600 can execute the data backup method P100 described in this specification. As Figure 3 shown, the method P100 can include: S120: Obtain description information corresponding to the target data, where the description information includes at least one of a data change frequency, health status information of the storage medium in which the target data is stored, or table association information. The table association information includes a plurality of tables for storing the target data and the dependency relationship between the plurality of tables.

[0048] The target data can be data stored in a database, such as the Ocean Base database. The target data can be data stored in a database by a specific entity (such as data from a company, a platform, an organization, or a department within a company). For example, e-commerce data stored in a database by an e-commerce platform or financial data stored in a database by a financial institution. The target data can be data that the entity to which it belongs needs to back up. The descriptive information corresponding to the target data can characterize the characteristics of the target data in various dimensions (such as data changes, data storage hardware, and data organization).

[0049] In some embodiments, the descriptive information may include data change frequency. Data change frequency may refer to the number of modifications (including insert, update, delete, and other operations) performed on the target data within a certain period of time. In some embodiments, the data change frequency may include a heat matrix, which represents the frequency of changes to each partition of the target data by each tenant corresponding to the target data.

[0050] Among them, the tenant is a logical unit of resource isolation in the database system, and each tenant is a logically independent database instance. Each tenant has its own independent database environment / resource pool, including CPU, memory, storage, etc., to avoid mutual interference between different tenants, but these tenants can share a set of physical infrastructure at the same time. The entity to which the target data belongs can create multiple tenants in the database, for example, creating separate tenants for different departments. The tenants corresponding to the target data can be tenants that can access at least part of the target data. The partition is the basic unit of data storage and management in the database system. The data in the database can be stored in the form of a table, and the table can be horizontally divided into multiple partitions, each partition containing a portion of row data. The data of each partition of the target data can contain part of the row data of the target data.

[0051] Each tenant corresponding to the target data can change the partition data of the target data. It should be noted that different tenants can change the data of the same partition, but they can only change the data within their permissions in that partition. For example, a certain partition in the order details table includes four rows of order detail data. Both tenants can change the data in this partition, but one tenant can change the first two rows of data in this partition, and the other tenant can change the last two rows of data in this partition. The computing device 600 can capture the changes made by each tenant corresponding to the target data to the partition data of the target data and determine the corresponding change frequency. For example, the CDC (Change Data Capture) tool of the OceanBase database can identify which partition data is being modified by which tenant and can count the change frequency.

[0052] In some embodiments, the computing device 600 can construct a heat matrix from the change frequencies of each tenant corresponding to the target data for the partition data of the target data. The heat matrix can be a two-dimensional matrix, where the rows represent tenants, the columns represent partitions, and the values represent the change frequency of the partition data under a certain tenant. The unit of the change frequency can be, for example, times / second, times / minute, etc. Partitions with a high change frequency in the heat matrix can be called hot partitions, and those with a low or even unchanged change frequency can be called cold partitions. The following is an example of a heat matrix:

[0053] Among them, partition A and partition C of tenant T1 are hot partitions, partition B of tenant T2 is a hot partition, and the change frequency of partition D is low and can be regarded as a cold partition.

[0054] In the embodiments of this specification, based on the data change frequencies of different partitions in the heat matrix, the backup policy model can generate different backup policies for different partitions. For example, since the data in hot partitions changes frequently, the backup period is short and the backup method can be full backup to avoid data loss. While the data in cold partitions changes less, the backup period can be extended and the backup method can be incremental backup to save backup costs.

[0055] In some embodiments, the data change frequency can also be a total change frequency corresponding to the target data. For example, the total change frequency obtained by summing up the change frequencies of each tenant corresponding to the target data for the partition data of the target data. Furthermore, the backup policy model can determine the target backup policy (such as the backup period) based on the high or low total change frequency. The embodiments of this specification do not make specific limitations on the data change frequency.

[0056] In some embodiments, the described information may include the health status information of the storage medium where the target data is stored. The storage medium where the target data is stored may be any type of storage medium, such as a hard disk (mechanical hard disk, solid-state drive), ROM, RAM, and so on. The health status information may be used to characterize the failure probability of the storage medium, that is, the probability of a failure occurring within a certain period of time. In some embodiments, the health status information of the storage medium includes at least one of temperature, reallocated sector count, error rate, or access frequency.

[0057] Among them, the temperature may be the internal or surface temperature of the storage medium during operation. The temperature has a direct impact on the lifespan of the storage medium. For example, if a hard disk operates in a high-temperature environment for a long time, it will cause the internal electronic components to age faster, increasing the probability of the hard disk failing. The reallocated sector count refers to the number of bad sectors that have been detected and attempted to be repaired on the hard disk, which can indicate the possible physical damage on the hard disk surface. If the reallocated sector count is 0, it means that no sectors need to be reallocated and the hard disk has no physical damage. If the value of the reallocated sector count starts to increase, it means that the hard disk has started to show physical damage. The error rate may be the ratio of the number of errors that occur during the read and write operations of the storage medium to the total number of operations. The error rate includes, for example, read errors, write errors, parity errors, etc. For example, as the usage time of a hard disk increases, some sectors may start to develop bad sectors, or the read and write heads of the hard disk may deviate, resulting in an increase in the read and write error rate. This trend is usually a warning signal that the hard disk is about to experience a serious failure. The access frequency may be the read and write (I / O) frequency, such as IOPS. As the storage medium ages or is damaged, the access frequency may decrease.

[0058] In the embodiments of this specification, the backup strategy model may evaluate the health status of the storage medium based on the health status information of the storage medium, give an early warning of possible problems with the storage medium, and generate corresponding backup strategies. For example, when the health status of the storage medium is poor, the backup strategy model may generate a high backup concurrency, that is, execute multiple backup tasks simultaneously to quickly back up the data and prevent the data from being lost due to storage medium problems. The backup strategy model may also generate a shorter backup cycle to perform backups frequently, so as to back up the data before the storage medium has health problems. Another example is that if the storage medium where a certain table is stored is about to have health problems, the backup strategy model may generate a backup granularity at the table level to quickly back up the data of that table.

[0059] In some embodiments, the description information may include table association information, and the table association information includes multiple tables for storing the target data and the dependency relationships between the multiple tables. In some embodiments, the target data may be stored in the form of tables, where a table may refer to the most basic data storage unit in a database or a view (virtual table). There may be dependency relationships between different tables. The dependency relationships are, for example, the relationship between a primary key and a foreign key, that is, a foreign key field in one table references the primary key field in another table. For example, each order detail in the order detail table can be associated with an order in the order table through the foreign key and primary key relationship. The dependency relationships are, for example, a one-to-many relationship, that is, a record in one table can be associated with multiple records in another table. For example, one product in the product table is associated with multiple orders in the order table, that is, one product can generate multiple orders. The dependency relationships may also be others, which are not limited in the embodiments of this specification. When there is no such dependency relationship between two tables, it can be considered that these two tables are independent tables and there is no dependency relationship between them.

[0060] In the embodiments of this specification, the backup policy model can determine which tables are associated with each other through the table association information, so as to generate a backup policy that can ensure cross-table data consistency. For example, when the data backup method P100 is being executed and a transaction involving multiple tables is in progress, at this time, the data of some tables has been changed and the data of some tables has not been changed. Then, based on the table association information, the backup policy model can know that there are dependency relationships between these tables. Therefore, when generating a backup policy, the execution time of the backup task (that is, the task of backing up the target data) can be determined to be after the completion of the transaction involving multiple tables, and the backup object of the backup task can be determined to be the multiple tables involved in the transaction, so as to ensure that the backed-up data meets cross-table consistency and improve the accuracy of data backup.

[0061] In some embodiments, the description information further includes the load information of the data backup device, and the load information includes at least one of CPU usage rate, memory usage rate, number of I / O operations, or number of transaction conflicts. The load information of the data backup device can reflect the current running state and resource consumption of the data backup device (computing device 600). A high CPU usage rate may indicate that the computing device 600 is processing a large number of computing tasks. At this time, performing a backup may further increase the burden on the computing device 600, resulting in performance degradation or a slow backup speed. When the memory usage rate is too high, there is insufficient memory, and insufficient memory may cause errors or data loss during the backup process. The number of I / O operations may affect the backup speed and efficiency. When the number of transaction conflicts is large, it can be shown that the computing device 600 is currently unstable, which may affect the consistency and integrity of the backup.

[0062] In the embodiments of this specification, by providing the load information of the computing device 600 to the backup policy model, the backup policy model can determine the occupied resources for executing the backup task. For example, if the current state is a high-load state, the occupied resources for executing the backup task can be relatively reduced to avoid affecting online transactions. If the current state is a low-load state, the occupied resources for executing the backup task can be increased to speed up the backup. The backup policy model can also determine the time for executing the backup task based on the load information. For example, the backup task can be selected to be executed during a period when the CPU usage rate is low, the memory is sufficient, the number of I / O operations is small, and the number of transaction conflicts is small. In this way, the efficient completion of the backup task can be ensured, and the impact of the backup on the performance of the production environment can be reduced.

[0063] In some embodiments, the description information may further include others, such as the free backup space owned by the subject to which the target data belongs, the cost, security, and access convenience of the free backup space, etc. The embodiments of this specification do not make specific limitations on the description information.

[0064] S140: Generate a target backup policy based on the description information and a pre-trained backup policy model.

[0065] In some embodiments, the computing device 600 can input the description information into the backup policy model. For example, at least one of the heat matrix, health status information, and table association information is input into the backup policy model, so as to quickly obtain the target backup policy output by the backup policy model.

[0066] In some embodiments, the computing device 600 can process the description information, input the processing result into the backup policy model, and obtain the target backup policy output by the backup policy model.

[0067] In some embodiments, the computing device 600 can input the health status information of the storage medium into a pre-trained failure probability prediction model, and obtain the failure probability of the storage medium predicted by the failure probability prediction model within a preset period. Furthermore, the computing device 600 can input the failure probability into the backup policy model, and obtain the target backup policy output by the backup policy model.

[0068] Among them, the failure probability may be the probability of a failure of the storage medium. The failure probability prediction model may be, for example, an isolation forest model, a logistic regression model, etc. For example, it is predicted by the isolation forest model that the probability p of the hard disk failing within the next 24 hours is 0.5. In some embodiments, the backup policy model may determine the backup time and backup method in the target backup policy according to the failure probability. For example, when the failure probability p > 0.3, a strategy for an emergency full backup is generated, that is, a full backup starts immediately (e.g., after 1 minute), and a strategy for cloud mirror storage may also be generated, that is, the data is further mirrored to cloud storage to improve the redundancy and recoverability of the data.

[0069] In the embodiments of this specification, by providing the backup policy model with the failure probability of the storage medium, the backup policy model can quickly generate the target backup policy based on the magnitude of the failure probability, thereby improving the efficiency of data backup. Moreover, when the failure probability of the storage medium is relatively high, the embodiments of this specification can perform preemptive backup, that is, perform backup before the storage medium fails, avoiding data loss.

[0070] In some embodiments, the computing device 600 may input the table association information into a pre-trained graph neural network model and obtain the cross-table consistency vector output by the graph neural network model. Furthermore, the computing device 600 may input the cross-table consistency vector into the backup policy model and obtain the target backup policy output by the backup policy model.

[0071] Among them, the graph neural network (GNN, Graph Neural Networks) may be, for example, GraphSAGE (GraphSAmple and aggreGatE), and GraphSAGE generates the embedding representation of each node (table) by sampling neighbor nodes (tables) and aggregating their features. The cross-table consistency vector represents, in the form of a vector, multiple tables for storing target data and their dependencies, quantifying the tables and their dependencies so that the backup policy model can better capture and process the dependencies between different tables, enabling the computing device 600 to effectively solve the cross-table consistency problem during the backup process.

[0072] In some embodiments, the computing device 600 may perform feature encoding on the load information of the data backup device to obtain a load vector and input the load vector into the backup policy model to generate a target backup policy.

[0073] The computing device 600 can provide multi-dimensional features for the backup policy model. For example, inputting the heat matrix, the failure probability of the storage medium, the cross-table consistency vector, and the load vector into the backup policy model, so that the backup policy model can generate a more accurate backup policy. When the computing device 600 performs data backup according to this backup policy, the backup efficiency and success rate can be significantly improved. In some embodiments, the backup policy model may include a feature encoder trained by self-supervised learning to perform feature encoding on the description information. Alternatively, the computing device 600 can train a feature encoder by self-supervised learning to encode the heat matrix, failure probability, and / or load information, and then provide the encoded result to the backup policy model.

[0074] In some embodiments, the target backup policy may include multiple sub-policies, and the multiple sub-policies may include at least two of backup granularity, backup method, backup period, backup concurrency, backup destination, or execution quality when performing the backup task. The multiple sub-policies may also include others, which are not limited in the embodiments of this specification. The embodiments of this specification can provide a detailed backup policy for the target data, thereby improving the security, success rate, and efficiency of the backup.

[0075] Among them, the backup time refers to the time when the backup task is executed. The backup granularity refers to the data range or refinement level covered by the backup operation, such as full database backup (backing up all data in the database), table-level backup (backing up data of a certain table), row-level backup (backing up data that meets certain rows), partition backup (backing up data of certain partitions), etc. The backup method refers to the specific method used when performing the backup task, such as full backup (performing a complete backup of all data), incremental backup (backing up data that has changed since the last backup), differential backup (backing up data that has changed since the last full backup), etc. The backup period refers to the time interval at which the backup task is executed, such as real-time backup (responding immediately to data changes, applicable to scenarios with extremely high requirements for data consistency), daily backup (performing a backup once at a fixed time every day), weekly backup, monthly backup, etc. The backup concurrency refers to the ability to execute multiple backup tasks simultaneously. High concurrency means processing more backup requests within the same time period, which can improve the backup efficiency. The backup destination refers to the location where the backup data is stored, such as a local disk, an external storage device, cloud storage, a tape library, etc. The execution quality when performing the backup task, such as QoS (Quality of Service), may refer to some performance manifestations when performing the backup task, such as backup time, backup speed, the priority of the backup task (the priority order between different backup tasks), and the resources occupied when performing the backup task (computing resources, storage resources, network resources, etc.).

[0076] In some embodiments, the backup policy model includes a multi-agent reinforcement model, and the multi-agent reinforcement model includes multiple agents, and each agent is configured to output at least one of the multiple sub-policies based on the description information. The multi-agent reinforcement model can flexibly cope with complex environmental changes and provide a comprehensive, efficient and highly adaptable backup policy.

[0077] The multi-agent reinforcement model can be a MARL (Multi-Agent Reinforcement Learning) model. Multiple agents in the multi-agent reinforcement model can cooperate or compete with each other to jointly output the target backup policy. Each agent can adopt the Actor-Critic method in reinforcement learning, where Actor is used to generate policies and Critic is used to evaluate the quality of policies. Each agent can output one or more sub-policies based on one or more pieces of description information. For example, MARL includes Scheduler-Agent (scheduling agent), Type-Agent (type agent), and QoS-Agent (quality of service agent). The Scheduler-Agent can generate a backup period according to the data change frequency (such as a heat matrix). For example, for data in a hot partition, a shorter backup period is generated, such as real-time backup, while for data in a cold partition, a longer backup period is generated, such as monthly backup. The Type-Agent can generate a backup method according to the data change frequency. For example, if the data changes rapidly, incremental backup is preferred. The Scheduler-Agent and the Type-Agent can also output jointly, such as full backup daily plus incremental backup hourly. The QoS-Agent can generate the quality of execution when performing backup tasks according to the load information of the data backup device, such as resource occupancy. For example, during peak periods, the computing resources and network resources used by backup tasks are restricted.

[0078] For another example, the multi-intelligent reinforcement model may further include a backup granularity agent, a backup concurrency agent, and a backup destination agent. The backup granularity agent can evaluate the data change frequency and importance of different tables. For critical transaction data that changes frequently (such as the order table), it recommends table-level backup. For data that does not change frequently (such as the product catalog), it uses full-database backup. The backup concurrency agent can calculate the maximum number of backup tasks that can be executed in parallel based on the health status information of the storage medium and the load information of the data backup device, so as to maximize the utilization of system resources while avoiding overloading. For example, three backup tasks are allowed to run simultaneously during off-peak hours and when the hardware condition is good. The backup destination agent can weigh the cost, security, and access convenience of the idle backup space and select the most suitable backup storage location. For example, for critical transaction data, it selects the cloud storage service of an off-site disaster recovery center, and for general data, it uses local NAS (Network Attached Storage) storage.

[0079] In some embodiments, the multi-agent reinforcement model is obtained through federated training based on the training data of the subject to which the target data belongs and the training data of other subjects. The data used by each subject for the federated training is noise-added. For example, each subject can use its own training data to train the multi-agent reinforcement model. For example, the computing device 600 aggregates the training data of each subject and trains the model. To avoid data leakage of each subject, the training data of each subject can be noise-added data. For another example, each subject uses its own training data to train the multi-agent reinforcement model to obtain its respective model parameters. The computing device 600 can aggregate the respective model parameters and return the aggregated parameters to each subject to continue training the multi-agent reinforcement model. To avoid leakage of the model parameters of each subject, the model parameters of each subject can be noise-added data. Among them, the aggregation is, for example, taking the average of the respective training data or model parameters. Through federated training, the model can be trained using rich training data, improving the model performance, thereby improving the accuracy of generating backup strategies. At the same time, data privacy can be protected through data noise-adding.

[0080] In some embodiments, the multi-agent reinforcement model is trained with the goal of minimizing the target loss function, and the target loss function includes at least one of the maximum data loss amount, data recovery duration, cost of executing the backup task, data consistency risk, or carbon emissions of executing the backup task. In some embodiments, different metrics in the target loss function can be assigned different weights according to their importance. Through the multi-objective optimization method, it can be ensured that the generated backup strategy comprehensively considers various factors, and the backup strategy is more balanced, comprehensive, and adaptable.

[0081] Among them, the maximum data loss amount, such as RPO (Recovery Time Objective), refers to the maximum data loss amount that can be tolerated, usually expressed in time. For example, if the set RPO is 15 minutes, then at the time of a disaster, at most only the data in the past 15 minutes can be lost. The lower the RPO, the smaller the risk of data loss. The data recovery duration, such as RTO (Recovery Point Objective), refers to the time required to recover the data to normal after a disaster occurs. For example, if the set RTO is 2 hours, then all necessary recovery operations must be completed within 2 hours. The shorter the RTO, the less time the transaction is interrupted. The cost of performing the backup task can represent the economic cost required to perform the backup task, including related expenses such as hardware, software, network bandwidth, storage space, and manpower. The data consistency risk can refer to the possible inconsistency problems that may exist during the data recovery process. For example, in a distributed database, the data of tables with dependencies may not reach a completely consistent state at the same time, resulting in partial data loss or damage. A high consistency risk may lead to data integrity problems and affect the normal operation of transactions. Therefore, reducing the consistency risk is the key to ensuring data reliability. The carbon emissions generated by performing the backup task can refer to the carbon emissions generated during the execution of the backup task. Using carbon emissions as an evaluation indicator to train the multi-agent reinforcement model takes into account the impact of data backup on the environment and helps environmental protection. The trained multi-agent reinforcement model can adjust the backup time so that the backup task falls within the peak of green electricity (the time period with the highest renewable energy generation), or adjust the backup destination so that the backup task falls within a computer room with a low PUE (Power Usage Effectiveness), thereby reducing carbon emissions.

[0082] Figure 4 FIG. shows a schematic diagram of a data backup method provided according to some embodiments of the present specification. As Figure 4 shown, the computing device 600 encodes the load information to obtain a load vector, s_(t) encodes the table association information to obtain a cross-table consistency vector g_(t), encodes the health status information of the hardware to obtain a failure probability h_(t), and uses CDC to capture the data change frequency and create a heat matrix c_(t). The computing device 600 concatenates s_(t), g_(t), h_(t), and c_(t) into a vector s_(t)` and inputs the vector s_(t)` to the multi-agent reinforcement model, and the multi-agent reinforcement model can output a target backup strategy.

[0083] S160: Back up the target data according to the target backup strategy.

[0084] As shown Figure 4 in FIG. 1, the computing device 600 may perform a backup task according to the target backup policy, that is, back up the target data. After the computing device 600 backs up the target data, it may immediately perform a micro-restore verification on the backup data to ensure the effectiveness of the backup, and expose in advance potential risks that are available but cannot be restored, that is, identify backup data that seems normal but cannot be effectively restored. The computing device 600 may also optimize the model based on the evaluation results of the micro-restore verification to improve the model performance. As shown Figure 4 in FIG. 2, the computing device 600 performs a micro-restore verification after performing the backup task, and updates the multi-agent reinforcement model accordingly.

[0085] In some embodiments, the computing device 600 may extract test data (such as 0.5% of the data) from the backup data that has been backed up, and create a temporary tenant. The computing device 600 may quickly create a temporary tenant, and the temporary tenant may be a tenant different from each tenant corresponding to the target data. The temporary tenant may be an independent environment for performing restore verification, and may be destroyed after the verification is completed, thereby saving resources. The computing device 600 may perform a restore on the test data in the temporary tenant, and determine the evaluation result corresponding to the restore. The evaluation results may be, for example, RPO, RTO, cost, consistency, carbon emissions, etc. Thus, the computing device 600 may update the backup policy model based on the evaluation results. For example, the parameters of the multi-agent reinforcement model may be adjusted according to the evaluation results to optimize the model.

[0086] In summary, the data backup method and device provided in this specification obtain description information corresponding to the target data, where the description information includes at least one of the data change frequency, the health status information of the storage medium storing the target data, or the table association information. The table association information includes multiple tables for storing the target data and the dependency relationship between the multiple tables, and intelligently generates a target backup policy through a pre-trained backup policy model, so that backing up the target data according to the target backup policy is more intelligent. The target backup policy is intelligently obtained based on the data change frequency, the health status information of the storage medium, and / or the table association information of the target data, rather than a fixed backup policy. Therefore, backing up data through the target backup policy generated by the method in this specification can effectively protect the data.

[0087] On the other hand, this specification provides a non-transitory storage medium storing at least one set of executable instructions for data backup. When the executable instructions are executed by a processor, the executable instructions direct the processor to perform the steps of the data backup method P100 described in this specification. In some possible implementation manners, various aspects of this specification may also be implemented in the form of a program product, which includes program code. When the program product runs on a data backup device, the program code is used to cause the data backup device to perform the steps of the data backup method P100 described in this specification. The program product for implementing the above method may be a portable compact disc read-only memory (CD-ROM) including program code and may run on a data backup device. However, the program product of this specification is not limited thereto. In this specification, a readable storage medium may be any tangible medium that contains or stores a program, and the program may be used by or in combination with an instruction execution device. The program product may adopt any combination of one or more readable media. The readable media may be a readable signal medium or a readable storage medium. The readable storage medium may, for example, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above. More specific examples of the readable storage medium include: an electrical connection having one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. The computer-readable storage medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries the readable program code. Such a propagated data signal may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The readable storage medium may also be any readable medium other than the readable storage medium, and the readable medium may send, propagate, or transmit a program for use by or in combination with an instruction execution device, apparatus, or device. The program code included on the readable storage medium may be transmitted using any appropriate medium, including but not limited to wireless, wired, optical fiber cable, RF, etc., or any suitable combination of the above. The program code for performing the operations of this specification may be written in any combination of one or more programming languages, including object-oriented programming languages such as Java, C++, etc., and also including conventional procedural programming languages such as the "C" language or similar programming languages. The program code may be executed entirely on the data backup device, partially on the data backup device, executed as an independent software package, partially on the data backup device and partially on a remote computing device, or entirely on a remote computing device.

[0088] The above description has been made of specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the acts or steps recited in the claims may be performed in a different order than in the embodiments and still achieve the desired result. Additionally, the processes depicted in the drawings do not necessarily require a particular order or a sequential order to achieve the desired result. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0089] In summary, after reading this detailed disclosure, those skilled in the art will appreciate that the foregoing detailed disclosure may be presented by way of example only and is not necessarily limiting. Although not explicitly stated herein, those skilled in the art will understand that this specification is intended to embrace various reasonable changes, improvements, and modifications to the embodiments. These changes, improvements, and modifications are intended to be proposed by this specification and are within the spirit and scope of the exemplary embodiments of this specification.

[0090] Furthermore, certain terms in this specification have been used to describe embodiments of this specification. For example, "one embodiment", "an embodiment", and / or "some embodiments" mean that the specific features, structures, or characteristics described in connection with that embodiment may be included in at least one embodiment of this specification. Thus, it should be emphasized and understood that two or more references to "an embodiment" or "one embodiment" or "alternative embodiments" in various parts of this specification do not necessarily all refer to the same embodiment. Additionally, the specific features, structures, or characteristics may be appropriately combined in one or more embodiments of this specification.

[0091] It should be understood that in the foregoing description of the embodiments of this specification, for the purpose of helping to understand a feature, and for the purpose of simplifying this specification, this specification combines various features in a single embodiment, drawing, or its description. However, this does not mean that the combination of these features is necessary, and it is entirely possible for those skilled in the art to identify some of these devices as separate embodiments when reading this specification. That is to say, the embodiments in this specification can also be understood as an integration of multiple sub - embodiments. And it also holds when the content of each sub - embodiment contains fewer features than all the features of a single foregoing disclosed embodiment.

[0092] Each patent, patent application, published patent application, and other materials cited herein, such as articles, books, specifications, publications, documents, items, etc., except for any historical prosecution documents associated therewith, any identical ones that may be inconsistent or in conflict with this document, or any identical historical prosecution documents that may have a limiting effect on the broadest scope of the claims, may be incorporated herein by reference and used for all purposes now or hereafter associated with this document. In addition, if there is any inconsistency or conflict between the description, definition, and / or use of terms associated with any of the materials incorporated herein and the terms, descriptions, definitions, and / or uses associated with this document, the terms of this document shall control.

[0093] Finally, it should be understood that the embodiments of the application disclosed herein are illustrative of the principles of the embodiments of this specification. Other modified embodiments are also within the scope of this specification. Therefore, the embodiments disclosed in this specification are merely examples and not limitations. Those skilled in the art can adopt alternative configurations based on the embodiments in this specification to implement the application in this specification. Therefore, the embodiments of this specification are not limited to the embodiments precisely described in the application.

Claims

1. A data backup method, applied to a data backup device, includes: Obtaining description information corresponding to target data, where the description information includes at least one of data change frequency, health status information of the storage medium where the target data is stored, or table association information, and the table association information includes multiple tables for storing the target data and the dependency relationship between the multiple tables; Generating a target backup policy based on the description information and a pre-trained backup policy model; And Backing up the target data according to the target backup policy.

2. The method according to claim 1, wherein The data change frequency includes a heat matrix, and the heat matrix characterizes the change frequency of each partition data of the target data by each tenant corresponding to the target data.

3. The method according to claim 1, wherein, The generating a target backup policy based on the description information and a pre-trained backup policy model includes: Inputting the health status information of the storage medium into a pre-trained failure probability prediction model, and obtaining the failure probability of the storage medium predicted by the failure probability prediction model within a preset time period; and Inputting the failure probability into the backup policy model, and obtaining the target backup policy output by the backup policy model.

4. The method according to claim 3, wherein, The health status information of the storage medium includes at least one of temperature, reallocated sector count, error rate, or access frequency.

5. The method according to claim 1, wherein The generating a target backup policy based on the description information and a pre-trained backup policy model includes: Inputting the table association information into a pre-trained graph neural network model, and obtaining a cross-table consistency vector output by the graph neural network model; and Inputting the cross-table consistency vector into the backup policy model, and obtaining the target backup policy output by the backup policy model.

6. The method according to claim 1, wherein, The description information further includes load information of the data backup device, and the load information includes at least one of CPU usage rate, memory usage rate, number of I / O operations, or number of transaction conflicts.

7. The method according to claim 1, wherein The backup policy model includes a multi-agent reinforcement model, the multi-agent reinforcement model includes multiple agents, the target backup policy includes multiple sub-policies, and each agent is configured to output at least one sub-policy among the multiple sub-policies based on the description information.

8. The method according to claim 7, wherein, The multiple sub-policies include at least two of backup granularity, backup method, backup period, backup concurrency, backup destination, or execution quality when performing a backup task.

9. The method according to claim 7, wherein, The multi-agent reinforcement model is obtained through federated training based on the training data of the subject to which the target data belongs and the training data of other subjects, and the training data of different subjects is data with added noise.

10. The method according to claim 7, wherein, The multi-agent reinforcement model is trained with the minimization of a target loss function as the training objective, and the target loss function includes at least one of the maximum data loss amount, data recovery duration, cost of performing a backup task, data consistency risk, or carbon emission amount of performing the backup task.

11. The method according to claim 1, wherein, After backing up the target data according to the target backup policy, it further includes: Extracting test data from the backed-up backup data and creating a temporary tenant; Restore the test data in the temporary tenant and determine the evaluation result corresponding to the restoration; and Update the backup policy model based on the evaluation result.

12. A data backup device, comprising: At least one storage medium storing at least one instruction set for implementing data backup; And At least one processor communicatively connected to the at least one storage medium, Wherein when the data backup device runs, the at least one processor reads the at least one instruction set and implements the method according to any one of claims 1-11.

Images