Backup method and device for unstructured object data, equipment and medium

Abstract

The invention provides a backup method and device for unstructured object data, equipment and a medium, and effectively solves the problem of poor user experience caused by the phenomena that the backup efficiency is low and faults are easy to occur when the existing unstructured object data is backed up. The method comprises the following steps: according to a target backup stage of unstructured object data, a backup server obtains target backup object data in a target object storage container in the target backup stage; downloading target backup object data, and identifying a data type of the target backup object data to select a target compression strategy corresponding to the data type; compressing the target backup object data to obtain backup data packets, and generating index information of each backup data packet; and writing index information into the backup data packet to obtain a target backup data packet, and uploading the target backup data packet to the tape library through a pre-configured data interface to complete backup of the unstructured object data.

Description

Technical Field

[0001] This application relates to the field of data processing technology, and more specifically, to a method, apparatus, device, and medium for backing up unstructured object data. Background Technology

[0002] As required by regulatory agencies such as the State Financial Regulatory Commission, insurance companies and their agents are required to conduct dual recording (audio and video recording) when selling certain types of insurance products, resulting in a large amount of audio and video files. Furthermore, with the rapid development of information technology, insurance companies generate a large number of image files during the application, claims, and policy maintenance processes, such as electronic application forms, ID cards, and bank cards. This unstructured data, including audio, video, and images, is not only large in volume but also infrequently used and contains sensitive information. Due to the needs of insurance business and regulatory requirements, there is a need for data backup.

[0003] Due to its low cost and high scalability, object storage is currently widely used for storing unstructured data in production environments. Therefore, backing up unstructured data stored as objects is particularly important.

[0004] Currently, the existing technologies can be roughly categorized as follows: 1. Synchronous replication: Real-time data replication is achieved through underlying storage replication, but it cannot handle situations such as manual deletion or logical errors; 2. Multi-version protection: Using the version control function of object storage, multiple versions of objects are retained, but it cannot cope with storage-level failures; 3. Backup software backup: Backups based on buckets or directories require a full scan for each backup, and a full scan and comparison are also required for incremental backups, resulting in low backup efficiency; while the tape library backup solution not only relies on the SAN network, leading to high construction costs, but also depends on the proprietary interface of the tape library, which is not conducive to widespread use. Summary of the Invention

[0005] In view of this, the purpose of this application is to provide a method, apparatus, device and medium for backing up unstructured object data, which effectively solves the problems of low backup efficiency and easy failure in the backup of existing unstructured object data, resulting in poor user experience.

[0006] In a first aspect, embodiments of this application provide a method for backing up unstructured object data, applicable to a backup system, the backup system including a backup server and a tape library, the method comprising: Based on the target backup stage of the unstructured object data, the backup server obtains the target backup object data in the target object storage container under the target backup stage; the backup stage includes full backup and incremental backup; the target backup object data includes object data and operation logs. Download the target backup object data and identify the data type of the target backup object data to select the target compression strategy corresponding to the data type; different data types correspond to different compression strategies; The target backup object data is compressed using the target compression strategy to obtain a backup data package. Index information for each backup data package is generated according to the index generation method under the target backup stage. Different backup stages correspond to different index generation methods. The backup data package includes at least one index. The target backup data packet is obtained by writing the index information to the backup data packet, and then uploaded to the tape library through a pre-configured data interface to complete the backup of the unstructured object data.

[0007] In conjunction with the first aspect, this application provides a first possible implementation of the first aspect, wherein the backup server acquires target backup object data from the target object storage container during the target backup stage, including: Based on the target backup stage, the target object storage container and data range of the target backup object data are determined; Call the interface of the target object storage container to read the target backup object data within the data range of the target object storage container.

[0008] In conjunction with the first aspect, this application provides a second possible implementation of the first aspect, wherein the target backup stage includes a full backup stage; The step of generating index information for each backup data packet according to the index generation method under the target backup stage includes: Based on the pre-defined index standard fields in the full backup phase, the backup server obtains the corresponding index data. By integrating the index data corresponding to the index standard fields, the full index information of the full backup stage is obtained.

[0009] In conjunction with the first aspect, this application provides a third possible implementation of the first aspect, wherein the target backup stage includes an incremental backup stage; After reading the target backup object data within the data range in the target object storage container, the process includes: Analyze the operation logs of the incremental backup phase to obtain the object-level operations after the previous incremental backup time point; The operation type of the object-level operation is determined, and the operation log is optimized based on the optimization strategy corresponding to the operation type.

[0010] In conjunction with the first aspect, this application provides a fourth possible implementation of the first aspect, wherein generating index information for each backup data packet according to the index generation method under the target backup stage includes: Obtain and associate the task IDs of the incremental backup phase and the corresponding full backup phase to obtain the ID combination; The IDs are combined and incorporated into the index data of the incremental backup phase to obtain the incremental index information of the incremental backup phase.

[0011] In conjunction with the first aspect, this application provides a fifth possible implementation of the first aspect, wherein, after uploading the target backup data package to the tape library, the following steps are included: Select the target query method from a variety of pre-set query methods, and determine the attribute data of the corresponding object data to be queried based on the target query method; Based on the attribute data, information on full backup tasks and incremental backup tasks is searched in the target database to determine the backup status of the data to be queried.

[0012] In conjunction with the first aspect, this application provides a sixth possible implementation of the first aspect, wherein, after determining the backup status of the data to be queried, the following steps are included: Select the target recovery method from the recovery methods corresponding to the various query methods, and download the target backup data package of the data to be queried based on the backup status; Download the target backup data package and decompress the target backup data package in reverse order of compression to obtain the unstructured object data corresponding to the object data to be queried.

[0013] Secondly, embodiments of this application provide a backup device for unstructured object data, suitable for backup systems, wherein the backup system includes a backup server and a tape library, and the device includes: The acquisition module is used to acquire target backup object data in the target object storage container under the target backup stage, based on the target backup stage of the unstructured object data; the backup stage includes full backup and incremental backup; the target backup object data includes object data and operation logs. The identification module is used to download the target backup object data and identify the data type of the target backup object data in order to select the target compression strategy corresponding to the data type; different data types correspond to different compression strategies. The compression module is used to compress the target backup object data according to the target compression strategy to obtain a backup data package, and to generate index information for each backup data package according to the index generation method under the target backup stage; different backup stages correspond to different index generation methods; the backup data package includes at least one; The backup module is used to write the index information to the backup data packet to obtain the target backup data packet, and upload the target backup data packet to the tape library through a pre-configured data interface to complete the backup of the unstructured object data.

[0014] Thirdly, embodiments of this application provide an electronic device, including: a processor, a memory, and a bus. The memory stores machine-readable instructions executable by the processor. When the electronic device is running, the processor communicates with the memory via the bus. When the machine-readable instructions are executed by the processor, the steps of any one of the methods for backing up unstructured object data are performed.

[0015] Fourthly, embodiments of this application provide a computer-readable storage medium storing a computer program, which, when executed by a processor, performs the steps of any one of the methods for backing up unstructured object data.

[0016] This application provides a method for backing up unstructured object data, applicable to a backup system. The backup system includes a backup server and a tape library. The method first involves the backup server acquiring target backup object data from a target object storage container at the target backup stage, based on the target backup stage of the unstructured object data. The backup stage includes full and incremental backups. The target backup object data includes object data and operation logs. Next, the method downloads the target backup object data and identifies its data type to select a target compression strategy corresponding to that data type. Different data types correspond to different compression strategies. Then, the method compresses the target backup object data using the target compression strategy to obtain a backup data package. Index information is generated for each backup data package according to the index generation method at the target backup stage. Different backup stages correspond to different index generation methods. The backup data package includes at least one index. Finally, the method writes the index information to the backup data package to obtain a target backup data package. The target backup data package is then uploaded to the tape library through a pre-configured data interface to complete the backup of the unstructured object data. Based on the above methods, not only is the backup efficiency for unstructured object data improved in both full and incremental backup phases, but the use cases for backing up unstructured object data are also expanded. This not only solves the problem that tape library backup solutions rely on SAN networks, leading to high construction costs, but also addresses the issue that relying on tape library proprietary interfaces is not conducive to widespread use, and avoids the need for full scans during incremental backups. Attached Figure Description To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 A flowchart illustrating a method for backing up unstructured object data according to an embodiment of this application is shown. Figure 2 This application provides a schematic diagram of the process for obtaining target backup object data according to an embodiment of the present application. Figure 3 This document illustrates a flowchart of the process for obtaining full index information according to an embodiment of this application. Figure 4 This paper shows a structural block diagram of a backup device for unstructured object data provided in an embodiment of this application; Figure 5 A structural block diagram of an electronic device provided in an embodiment of this application is shown. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. It should be understood that the accompanying drawings in this application are for illustrative and descriptive purposes only and are not intended to limit the scope of protection of this application. Furthermore, it should be understood that the schematic drawings are not drawn to scale. The flowcharts used in this application illustrate operations implemented according to some embodiments of this application. It should be understood that the operations in the flowcharts may not be implemented in sequence, and steps without logical contextual relationships may be reversed or implemented simultaneously. In addition, those skilled in the art, guided by the content of this application, may add one or more other operations to the flowcharts, or remove one or more operations from the flowcharts.

[0019] Furthermore, the described embodiments are merely some, not all, of the embodiments of this application. The components of the embodiments of this application described and illustrated herein can typically be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely to illustrate selected embodiments of the application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

[0020] It should be noted that the term "comprising" will be used in the embodiments of this application to indicate the presence of the features declared thereafter, but does not exclude the addition of other features.

[0021] Current backup methods for unstructured object data suffer from problems such as inability to handle human deletion, logical errors, storage-level failures, and low backup efficiency. Furthermore, most industry solutions use disk-type storage devices as backup targets. On the other hand, tape library backup solutions not only rely on SAN networks, resulting in high construction costs, but also depend on tape library proprietary interfaces, which are not conducive to widespread use.

[0022] Based on this, embodiments of this application provide a method, apparatus, device, and medium for backing up unstructured object data, which are described below through embodiments.

[0023] Example 1 To facilitate understanding of this embodiment, a method for backing up unstructured object data disclosed in this application will first be described in detail. For example... Figure 1The flowchart shown illustrates a method for backing up unstructured object data. This application provides a method for backing up unstructured object data, applicable to backup systems, which include a backup server and a tape library. The method includes: S101. Based on the target backup stage of the unstructured object data, the backup server obtains the target backup object data in the target object storage container under the target backup stage; the backup stage includes full backup and incremental backup; the target backup object data includes object data and operation logs. S102. Download the target backup object data and identify the data type of the target backup object data to select the target compression strategy corresponding to the data type; different data types correspond to different compression strategies; S103. Compress the target backup object data using the target compression strategy to obtain a backup data package, and generate index information for each backup data package according to the index generation method under the target backup stage; different backup stages correspond to different index generation methods; the backup data package includes at least one; S104. Write the index information to the backup data packet to obtain the target backup data packet, and upload the target backup data packet to the tape library through the pre-configured data interface to complete the backup of the unstructured object data.

[0024] In this application, the backup system includes a backup server and a tape library. In addition, the backup system is connected to an external Elasticsearch database, referred to as ES database, to provide data support for the backup system.

[0025] In step S101, this application determines the user's backup stage based on user needs. The backup stage includes full backup and incremental backup, i.e., full backup stage and incremental backup stage. Different backup stages correspond to different target backup object data. The target backup object data includes object data and operation logs. The object data is unstructured. That is, the target backup object data to be obtained in the full backup stage is object data, while the target backup object data corresponding to the incremental backup stage is operation logs. Different data types correspond to different data acquisition methods. In the incremental backup stage, the unstructured object data has already been fully backed up. Based on this method, the inefficiency caused by the need for a full scan in the incremental backup stage is avoided, and the backup efficiency is improved. According to the target backup stage of the unstructured object data, the backup server obtains the target backup object data in the target object storage container of the target backup stage by calling the corresponding data acquisition method, so as to perform further processing on the target backup object data. In the specific implementation of step S101, one embodiment is as follows: Figure 2 As shown, the backup server obtains the target backup object data from the target object storage container during the target backup stage, including: S1011. Based on the target backup stage, determine the target object storage container and data range of the target backup object data; S1012. Call the interface of the target object storage container to read the target backup object data within the data range in the target object storage container.

[0026] In steps S1011-S1012, based on the determined target backup stage, this application determines the target object storage container and data range of the target backup object data. For the full backup stage, the target object storage container is set as one or more target buckets in the specified source object storage, and the corresponding data range is all unstructured object data in one or more target buckets in the specified source object storage. For the incremental backup stage, the target object storage container is set as one or more source buckets configured for monitoring by the backup server, and the corresponding data range is the operation logs in one or more source buckets, such as Server AccessLogging, Change Notification, or Bucket Versioning event streams. For the full backup stage, the backup server reads all unstructured object data in one or more target buckets in the specified source object storage by calling the target object storage container interface, including the ListObjects object storage API. For the full backup stage, the backup server periodically, such as hourly, pulls the operation logs of one or more monitored source buckets.

[0027] In the specific implementation of step S1012, one embodiment is as follows: the target backup stage includes an incremental backup stage; After reading the target backup object data within the data range in the target object storage container, the process includes: A1. Analyze the operation log of the incremental backup phase to obtain the object-level operations after the previous incremental backup time point; A2. Determine the operation type of the object-level operation, and optimize the operation log based on the optimization strategy corresponding to the operation type.

[0028] In steps A1-A2, during the incremental backup phase, this application parses the operation logs of the incremental backup phase through the backup server to obtain object-level operations since the previous incremental backup time point, including: PUT (upload / overwrite), POST, COPY, DELETE, etc., and counts the number of these object-level operations to determine the operation type based on the number of operations. The operation type includes single and multiple operations. If it is a multiple operation type, the operation logs are optimized based on the optimization strategy corresponding to the operation type. The optimization strategy includes deduplication and merging; that is, for multiple operation types, the operation type is further classified. If it is a duplicate operation type... The corresponding optimization strategy is deduplication. Specifically, when parsing the operation logs of the source bucket, duplicate log entries are removed, and all operations on the same object key within the same incremental backup period are merged into a single operation record. For complex operation types, the corresponding optimization strategy is merging. Specifically, for complex change sequences, such as multiple PUTs followed by DELETEs, index information reflecting the final state is directly generated. For example, if an object undergoes two PUT updates within a period and is finally deleted by DELETE, only one DELETE operation and its related metadata, such as the operation timestamp, need to be recorded, thus facilitating the backup of the operation logs.

[0029] In step S102, after obtaining the target backup object data, the backup server downloads all target backup object data in parallel from the source bucket. For incremental backup, the download of unstructured object data corresponding to the operation logs included in the target backup object data depends on the type of the operation logs. For PUT / COPY operations, the unstructured object data corresponding to the operation logs is downloaded from the source bucket. For DELETE operations, only the object key and deletion time are recorded, and the unstructured object data corresponding to the operation logs is not downloaded. This achieves the effect of handling various scenarios such as human deletion, logical errors, and storage failures, and identifies the data type of the target backup object data. Specifically, the identification method is based on its MIME type or file extension. For the unstructured object data corresponding to the target backup object data, its data type includes audio, video, and images. The target compression strategy corresponding to the data type can be selected. Different data types correspond to different compression strategies; that is, for audio, a corresponding audio compression strategy is set; for video, a corresponding video compression strategy is set; for images, a corresponding image compression strategy is set. Alternatively, the data can be packaged according to the bucket to form one or more backup data packages (Archive). Package, thereby enabling targeted compression of the data of the target backup object, achieving a significant compression effect.

[0030] In step S103, after determining the target compression strategy, the backup server compresses the target backup object data according to the target compression strategy to obtain a backup data packet. The backup data packet includes at least one packet; that is, the number of backup data packets can be multiple or one, depending on the size of the target backup object data and the performance of the backup server. Index information for each backup data packet is generated according to the index generation method under the target backup stage. Different backup stages correspond to different index generation methods; that is, corresponding index generation methods are set for the full backup stage and the incremental backup stage, thereby generating corresponding indexes to obtain corresponding index information. For the index information in the full backup stage, "sampling verification" is used to ensure that the index information is consistent with the original unstructured object data. Specifically, a portion of the object data is randomly sampled, and the checksum of the index information is compared with the checksum of the actual object data. After successful verification, the index information is "fixed," i.e., marked as unmodifiable, and used as the "baseline index" for the subsequent incremental backup stage. For the incremental backup stage, only the checksum of the "new / modified data" recorded in the index information is verified to be consistent with the original unstructured object data; there is no need to verify the entire unstructured object data, reducing verification time.

[0031] In the specific implementation of step S103, one embodiment is as follows: Figure 3 As shown, the target backup phase includes a full backup phase; The step of generating index information for each backup data packet according to the index generation method under the target backup stage includes: S10311. Based on the index standard fields preset in the full backup stage, the backup server obtains the corresponding index data; S10312. Integrate the index data corresponding to the index standard fields to obtain the full index information of the full backup stage.

[0032] In steps S10311-S10312, during the full backup phase, the backup server pre-sets various index standard fields, including but not limited to: source bucket name (source_bucket), source object key (full path) (object_key), object size, last modified time (last_modified), compression algorithm name used, compressed size, packaging location information (such as package name, offset within the package), checksum (such as ETag, MD5), packaging timestamp (timestamp), and full backup task ID (full_backup_task_id). Based on the pre-set index standard fields in the full backup phase, the backup server obtains the index data corresponding to the index standard fields, integrates the index data corresponding to the index standard fields, and obtains the full index information of the full backup phase. All collected index data and all unstructured object data are organized into a structured index library according to the "container-object hierarchy" or "unique identifier hash" method. This index can be stored in a relational database, NoSQL database, or a dedicated index file, such as LevelDB, forming a "basic directory" covering all data.

[0033] In the specific implementation of step S103, another embodiment exists: generating index information for each backup data packet according to the index generation method under the target backup stage includes: S10321. Obtain and associate the task IDs of the incremental backup stage and the corresponding full backup stage to obtain the ID combination; S10322. Integrate the ID combination into the index data of the incremental backup stage to obtain the incremental index information of the incremental backup stage.

[0034] In steps S10321-S10322, during the incremental backup phase, the backup server obtains and associates the task IDs of the incremental backup phase and the corresponding full backup phase to obtain an ID combination. This ID combination is then integrated into the index data of the incremental backup phase. The index data includes the source bucket name (source_bucket), source object key (object_key), operation type (operation_type: CREATE / UPDATE / DELETE), data type, application compression method, (for CREATE / UPDATE) compressed size, checksum, packaging location, operation timestamp, incremental task ID (incremental_task_id), and associated full task ID (full_backup_task_id). The full task ID points to the most recent full backup on which this incremental backup is based. This is the key to logically associating the incremental backup with a specific full backup baseline, thereby obtaining the incremental index information of the incremental backup stage and forming an index chain of "full + incremental". If an index record needs to be added, the information of "new / modified data" needs to be added to the historical index library in the form of "incremental fragments". At the same time, the incremental index is marked with association information such as "associated baseline index ID", "incremental backup timestamp", and "backup task ID", forming a new "index chain". This allows for the identification of different data by "baseline index comparison" without having to re-traverse the full amount of unstructured object data, reducing the occupation of storage I / O and network bandwidth. The efficiency difference is significant, especially when the target container data volume reaches TB / PB level. At the same time, the structure of the index chain makes it easy to trace the "data change trajectory".

[0035] In step S104, the backup server writes the corresponding index information obtained in the target backup stage to the backup data packet to obtain the target backup data packet. The number of target backup data packets is the same as the number of backup data packets. For the full backup stage, the result is a full data packet; for the incremental backup stage, the result is an incremental data packet. The incremental index information for the incremental backup stage is written to an external Elasticsearch (ES) database. Utilizing the existing index structure, the ES database index mapping should be pre-designed to ensure that key fields such as bucket_name, object_key, full_backup_task_id, and timestamp are correctly mapped, enabling efficient execution of subsequent queries. For example, when querying all backup versions of a certain object: First, query using bucket_name and object_key to find the full_backup_task_id of the most recent full backup. Then, using bucket_name, object_key, and the full_backup_task_id as query conditions, retrieve all relevant incremental backup records. Deletion operation records can be packaged separately or attached to the metadata of the incremental data packet. The target backup data package is uploaded to the tape library via a pre-configured data interface, which is an S3 API. The target backup data package is then uploaded to a designated archive bucket in the tape library that supports the S3 protocol via the S3 API, thereby completing a full or incremental backup of the unstructured object data. This application uses a tape library as an S3 storage device, significantly reducing costs and making it suitable for long-term and offline storage. If off-site or same-city backup is required, the data can be uploaded to two S3 tape libraries, and one copy of the data can be selected for tape export for offline backup protection in the same city or off-site.

[0036] In a specific implementation of step S104, one embodiment is as follows: after uploading the target backup data package to the tape library, the process includes: S1041. Select a target query method from a variety of pre-set query methods, and determine the attribute data of the corresponding object data to be queried based on the target query method; S1042. Based on the attribute data, search the target database for information on full backup tasks and incremental backup tasks to determine the backup status of the data to be queried.

[0037] In steps S1041-S1042, when a user queries data for an object in a specific bucket on the front end, they select a target query method from several pre-set query methods. These methods include regular queries and time-based queries. For regular queries, the user needs to access the attribute data of the object being queried, namely the bucket name and object key. For time-based queries, the attribute data includes the bucket name, object key, and a target recovery time. Based on the attribute data obtained from both query methods, the user queries the ES database to find the information from the most recent full backup. If subsequent incremental backups exist, the user continues to search for and apply these incremental backups. This process obtains the backup status of an object in a specific bucket, indicating which full / incremental data package it resides in.

[0038] In a specific implementation of step S1041, one embodiment is as follows: after determining the backup status of the data to be queried, the process includes: S10421. Select the target recovery method from the recovery methods corresponding to the multiple query methods, and download the target backup data package of the data to be queried based on the backup status; S10422. Download the target backup data package and decompress the target backup data package in reverse order of compression to obtain the unstructured object data corresponding to the object data to be queried.

[0039] In steps S10421-S10422, when a user restores an object in a specific Bucket on the front end, after the query operation, the user needs to select the target recovery method from the recovery methods corresponding to the various query methods. The regular query method corresponds to the regular recovery method, and the time-based query method corresponds to the time-based recovery method. The target backup data package of the object to be queried is downloaded based on the backup status. The target backup data package is downloaded and decompressed in reverse order of compression to obtain the unstructured object data corresponding to the object to be queried. In conventional recovery methods, the backup system searches for relevant information and locates the required data packets in the Elasticsearch database using the Bucket name and object key input by the user, and then downloads and unpacks them from the tape library. In point-in-time recovery methods, the backup system needs to find the closest full backup that occurred before the specified recovery time using the Bucket name, object key, and recovery timestamp input by the user. This will become the recovery baseline. After finding the baseline full backup, the backup system needs to find all incremental operations that occurred after the baseline full backup but before the specified recovery time.

[0040] This application also sets up a direct recovery method, in which users directly download the target recovery object, and the entire recovery process is carried out automatically on the backup server. This is done without the user's awareness and does not rely on the backup administrator to perform the recovery operation, thus achieving source format backup and recovery.

[0041] Example 2 This application also provides a backup device for unstructured object data, such as... Figure 4 The diagram shows a block diagram of a backup device for unstructured object data. This backup device performs functions corresponding to the steps of the aforementioned method for backing up unstructured object data on a terminal device. The device can be understood as a component of a server including a processor. The backup device for unstructured object data described in this application is suitable for backup systems, which include backup servers and tape libraries. The device includes: The acquisition module 401 is used to acquire target backup object data in the target object storage container under the target backup stage according to the target backup stage of the unstructured object data; the backup stage includes full backup and incremental backup; the target backup object data includes object data and operation logs. The identification module 402 is used to download the target backup object data and identify the data type of the target backup object data in order to select the target compression strategy corresponding to the data type; different data types correspond to different compression strategies; Compression module 403 is used to compress the target backup object data to obtain a backup data packet using the target compression strategy, and to generate index information for each backup data packet according to the index generation method under the target backup stage; different backup stages correspond to different index generation methods; the backup data packet includes at least one; The backup module 404 is used to write the index information to the backup data packet to obtain the target backup data packet, and upload the target backup data packet to the tape library through a pre-configured data interface to complete the backup of the unstructured object data.

[0042] In one feasible implementation, the acquisition module includes: The determination module is used to determine the target object storage container and data range of the target backup object data based on the target backup stage. The read module is used to call the interface of the target object storage container to read the target backup object data within the data range of the target object storage container.

[0043] In one feasible implementation, the compression module includes: The first acquisition module is used to obtain the corresponding index data from the backup server based on the index standard fields preset in the full backup stage. The first integration module is used to integrate the index data corresponding to the index standard fields to obtain the full index information in the full backup stage.

[0044] In one feasible implementation, the acquisition module further includes: The parsing module is used to parse the operation log of the incremental backup phase to obtain the object-level operations after the previous incremental backup time point; An optimization module is used to determine the operation type of the object-level operation and optimize the operation log based on the optimization strategy corresponding to the operation type.

[0045] In one feasible implementation, the compression module further includes: The association module is used to obtain and associate the task IDs of the incremental backup stage and the corresponding full backup stage to obtain the ID combination. The second integration module is used to integrate the ID combination into the index data of the incremental backup stage to obtain the incremental index information of the incremental backup stage.

[0046] In one feasible implementation, the backup module includes: The query module is used to select a target query method from a variety of pre-set query methods, and to determine the attribute data of the corresponding object data to be queried based on the target query method; The search module is used to search for information on full backup tasks and incremental backup tasks in the target database based on the attribute data, so as to determine the backup status of the data to be queried.

[0047] In one feasible implementation, the backup module further includes: The selection module is used to select the target recovery method from the recovery methods corresponding to the multiple query methods, and download the target backup data package of the data to be queried based on the backup status; The decompression module is used to download the target backup data package and decompress the target backup data package in reverse order of compression to obtain the unstructured object data corresponding to the object data to be queried.

[0048] Example 3 This application also provides an electronic device, such as Figure 5As shown, it includes: a processor 501, a memory 502, and a bus 503. The memory 502 stores machine-readable instructions that can be executed by the processor 501. When the electronic device is running, the processor 501 and the memory 502 communicate through the bus 503. When the machine-readable instructions are executed by the processor 501, the steps of any one of the unstructured object data backup methods described above are performed.

[0049] Example 4 This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, performs the steps of any one of the methods for backing up unstructured object data.

[0050] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems and devices described above can be referred to the corresponding processes in the method embodiments, and will not be repeated here. In the several embodiments provided in this application, it should be understood that the disclosed systems, devices, and methods can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods. Furthermore, multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed mutual coupling or direct coupling or communication connection can be through some communication interfaces; the indirect coupling or communication connection of devices or modules can be electrical, mechanical, or other forms.

[0051] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0052] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0053] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a processor-executable, non-volatile, computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, a platform server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, ROM, RAM, magnetic disks, or optical disks.

[0054] The above are merely specific embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A backup method of unstructured object data, characterized by, The method is suitable for a backup system including a backup server and a tape library, and the method comprises the following steps: According to the target backup stage of the unstructured object data, the backup server acquires target backup object data in a target object storage container in the target backup stage; the backup stage includes full backup and incremental backup; the target backup object data includes object data and operation logs; The target backup object data is downloaded, and the data type of the target backup object data is identified to select a target compression strategy corresponding to the data type; different data types correspond to different compression strategies; The target backup object data is compressed by the target compression strategy to obtain backup data packets, and index information of each backup data packet is generated according to an index generation mode in the target backup stage; different backup stages correspond to different index generation modes; the backup data packet includes at least one; The index information is written into the backup data packet to obtain target backup data packets, and the target backup data packets are uploaded to the tape library through a preconfigured data interface to complete the backup of the unstructured object data.

2. The method of claim 1, wherein, The backup server acquires target backup object data in a target object storage container in the target backup stage, which comprises the following steps: According to the target backup stage, the target object storage container and the data range of the target backup object data are determined; The interface of the target object storage container is called to read the target backup object data in the data range in the target object storage container.

3. The method of claim 2, wherein, The target backup stage includes a full backup stage; According to the index standard field preset in the full backup stage, the backup server acquires corresponding index data; The index data corresponding to the index standard field is integrated to obtain full index information of the full backup stage. The target backup stage includes an incremental backup stage; 4. The method of claim 2, wherein, After reading the target backup object data in the data range in the target object storage container, the following steps are included: The operation logs of the incremental backup stage are parsed to obtain object-level operations after a previous incremental backup time point; The operation type of the object-level operations is determined to optimize the operation logs based on an optimization strategy corresponding to the operation type. According to the index generation mode in the target backup stage, the index information of each backup data packet is generated, which comprises the following steps:

5. The method of claim 4, wherein, The task ID of the incremental backup stage and the corresponding full backup stage is acquired and associated to obtain an ID combination; The ID combination is integrated into the index data of the incremental backup stage to obtain incremental index information of the incremental backup stage. After uploading the target backup data packets to the tape library, the following steps are included:

6. The method of claim 1, wherein, A target query mode is selected from a plurality of pre-set query modes to determine attribute data of corresponding to-be-queried object data based on the target query mode; Based on the attribute data, information of full backup tasks and incremental backup tasks is searched in a target database to determine the backup state of the to-be-queried object data. ​ 7. The method of claim 6, wherein, The determining the backup state of the to-be-queried object data comprises: selecting a target recovery mode from the recovery modes corresponding to the plurality of query modes, and downloading a target backup data package of the to-be-queried object data based on the backup state; downloading the target backup data package, and decompressing the target backup data package in a reverse order of compression to obtain unstructured object data corresponding to the to-be-queried object data.

8. A backup apparatus of unstructured object data, characterized by, The device is suitable for a backup system comprising a backup server and a tape library, and comprises: an acquisition module configured to acquire, by the backup server, target backup object data in a target object storage container according to a target backup phase of unstructured object data; the backup phase comprises full backup and incremental backup; and the target backup object data comprises object data and operation logs; an identification module configured to download the target backup object data, identify a data type of the target backup object data, and select a target compression strategy corresponding to the data type; different data types correspond to different compression strategies; a compression module configured to compress the target backup object data by the target compression strategy to obtain a backup data package, and generate index information of each backup data package according to an index generation mode of the target backup phase; different backup phases correspond to different index generation modes; and the backup data package comprises at least one; a backup module configured to write the index information into the backup data package to obtain a target backup data package, and upload the target backup data package to the tape library through a preconfigured data interface, so as to complete backup of the unstructured object data.

9. An electronic device, comprising: The device comprises: a processor, a memory, and a bus; the memory stores machine-readable instructions executable by the processor; when the electronic device is running, the processor and the memory communicate through the bus; and the machine-readable instructions are executed by the processor to perform the steps of the backup method of the unstructured object data according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, which is executed by the processor to perform the steps of the backup method of the unstructured object data according to any one of claims 1 to 7.

Images