Compression method, decompression method, compression device, and decompression device for object storage

Abstract

The application provides a compression method, a decompression method, a compression device and a decompression device of object storage, comprising: obtaining a compression rule, wherein the compression rule comprises a compression condition, object metadata of a source file and a compressed storage location, the compressed storage location comprises a target path corresponding to a target bucket in the object storage, locating a storage path corresponding to a storage bucket in the object storage according to the object metadata of the source file, reading the source file according to the storage path, wherein the source file is encapsulated into an input stream, the input stream represents a byte sequence, creating a compression task according to the compression rule, executing the compression task to compress and package the source file and generate a packaged file, and uploading the packaged file to the target path corresponding to the target bucket in the object storage if the data amount of the packaged file meets the compression condition. The source file is read in a streaming mode and the source file is compressed and packaged, so that the uploading time is shortened and the resource integrity is ensured.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of object storage, and in particular to an object storage compression method, an object storage decompression method, an object storage compression device and an object storage decompression device. BACKGROUND

[0002] Object storage is a technology for storing data, which stores data as objects in a unified storage pool and accesses the data through a unique identifier. Unlike traditional file systems, object storage does not organize data in a hierarchical structure, but stores data in a flat structure. Object storage has the characteristics of high scalability, large capacity, high reliability, and good data redundancy, and is suitable for large-scale data storage and distributed storage scenarios. However, when a user uploads a file to object storage or downloads a file from object storage, a large file will consume a lot of time and occupy a large amount of network bandwidth resources. Due to the large size of the file, the resource integrity rate is easily reduced due to network interruption during the uploading or downloading process. SUMMARY

[0003] The present application provides an object storage compression method, an object storage decompression method, an object storage compression device and an object storage decompression device, which solves the problem that in the prior art, when a user uploads a file to object storage or downloads a file from object storage, a large file will consume a lot of time and occupy a large amount of network bandwidth resources, and due to the large size of the file, the resource integrity rate is easily reduced due to network interruption during the uploading or downloading process.

[0004] The present application provides an object storage compression method, which comprises:

[0005] Obtaining a compression rule, wherein the compression rule comprises a compression condition, object metadata of a source file and a compressed storage location, and the compressed storage location comprises a target path corresponding to a target bucket in the object storage;

[0006] Locating a storage path corresponding to a storage bucket in the object storage according to the object metadata of the source file;

[0007] Reading the source file according to the storage path, wherein the source file is encapsulated into an input stream, and the input stream represents a byte sequence;

[0008] Creating a compression task according to the compression rule, wherein the compression task is a task for compressing the source file;

[0009] Executing the compression task to compress and package the source file and generate a packaged file;

[0010] If the data volume of the packaged file upload meets the compression condition, the packaged file is uploaded to a target path corresponding to a target bucket in the object storage.

[0011] In an embodiment, the obtaining the compression rule comprises:

[0012] Based on the user's compression instruction, a blank table is generated; wherein the blank table includes the compression condition, source file name, source file number, target file name, target file number, compression type, the compressed storage location, and the object metadata of the source file;

[0013] According to the content filled in the blank table and the preset encoding format, a compression table is generated;

[0014] Based on the content of the compression table, a compression rule is created.

[0015] In an embodiment, the creating a compression task according to the compression rule comprises:

[0016] According to the compression rule, a plurality of compression task entries are generated; wherein each compression task entry includes a task identifier, a task content, a task type, and a task progress;

[0017] The priority of each compression task entry is determined;

[0018] According to the priority of each compression task entry, the plurality of compression task entries are sorted;

[0019] Each sorted compression task entry is generated into a corresponding compression task.

[0020] In an embodiment, after the creating a compression task according to the compression rule, the compression method of the object storage further comprises:

[0021] The compression task entry is persisted into a relational database.

[0022] In an embodiment, if the data volume of the packaged file upload meets the compression condition, the packaged file is uploaded to a target path corresponding to a target bucket in the object storage, which comprises:

[0023] If the data volume of the packaged file upload is less than a preset quantity threshold, the packaged file is uploaded to a target path corresponding to a target bucket in the object storage.

[0024] In an embodiment, after the if the data volume of the packaged file upload meets the compression condition, the packaged file is uploaded to a target path corresponding to a target bucket in the object storage, the compression method of the object storage further comprises:

[0025] generating an execution result of performing the compression task;

[0026] if the execution result indicates a compressed file error, obtaining a backup file corresponding to the source file; wherein the storage path of the backup file is different from the storage path of the source file;

[0027] if the source file and the backup file do not match, determining that the source file is damaged;

[0028] dividing a plurality of backup files to generate a plurality of backup file groups;

[0029] compressing and packaging each backup file group;

[0030] obtaining a word from a preset dictionary; wherein the number of English letters in the word is the same as the number of processed backup file groups;

[0031] assigning each English letter of the word to the processed backup file group and using the assigned English letter as the name of the processed backup file group;

[0032] uploading the processed backup file group to a target path corresponding to a target bucket in the object storage based on the pronunciation order of the English letters in the word.

[0033] Another embodiment of the present application provides a decompression method of an object storage, comprising:

[0034] obtaining a decompression rule; wherein the decompression rule includes the object metadata of a target file;

[0035] locating a target path corresponding to a target bucket in the object storage according to the object metadata of the target file;

[0036] reading the target file according to the target path; wherein the source file is encapsulated as an input stream, and the input stream represents a byte sequence;

[0037] creating a decompression task according to the decompression rule; wherein the decompression task is a task of decompressing the target file;

[0038] executing the decompression task to perform decompression processing on the decompression task and obtaining a decompressed source file.

[0039] In an embodiment, after executing the decompression task to perform decompression processing on the decompression task to obtain a source file, the decompression method of the object storage further comprises:

[0040] if the task is interrupted during the execution of the decompression task, obtaining the decompression task that has been executed when the task is interrupted from a relational database;

[0041] obtaining a task progress corresponding to each executed decompression task;

[0042] determining a decompression task with a task progress less than a preset progress as an abnormal task;

[0043] obtaining a plurality of preset reasons causing the abnormal task to be abnormal;

[0044] according to each preset reason, modifying the abnormal task and re-executing the modified executed decompression task;

[0045] if the task progress of the modified executed decompression task is greater than the preset progress, determining that a preset reason corresponding to the modified executed decompression task is an abnormal reason and persisting the abnormal reason in the relational database.

[0046] Another embodiment of the present application provides a compression device of object storage, comprising:

[0047] a first task module configured to obtain a compression rule; wherein the compression rule comprises a compression condition, object metadata of a source file, and a storage location after compression, the storage location after compression comprising a target path corresponding to a target bucket in the object storage; and locate a storage path corresponding to a storage bucket in the object storage according to the object metadata of the source file;

[0048] a first storage module configured to read the source file according to the storage path; wherein the source file is encapsulated into an input stream, and the input stream represents a byte sequence;

[0049] a compression module configured to create a compression task according to the compression rule; wherein the compression task is a task of compressing the source file; execute the compression task to compress and package the source file and generate a packaged file; and if a data volume uploaded by the packaged file meets the compression condition, upload the packaged file to the target path corresponding to the target bucket in the object storage.

[0050] Another embodiment of the present application provides a decompression device of object storage, comprising:

[0051] a second task module configured to obtain a decompression rule; wherein the decompression rule comprises object metadata of a target file; and locate a target path corresponding to a target bucket in the object storage according to the object metadata of the target file;

[0052] a second storage module configured to read the target file according to the target path; wherein the source file is encapsulated into an input stream, and the input stream represents a byte sequence;

[0053] a decompression module configured to create a decompression task according to the decompression rule, wherein the decompression task is a task of decompressing the target file, and execute the decompression task to perform decompression processing on the decompression task and obtain a decompressed source file.

[0054] The compression method for object storage provided by the application, the decompression method, the compression device and the decompression device, comprising: obtaining a compression rule, wherein the compression rule comprises a compression condition, object metadata of a source file and a storage location after compression, the storage location after compression comprises a target path corresponding to a target bucket in the object storage, locating a storage path corresponding to a storage bucket in the object storage according to the object metadata of the source file, reading the source file according to the storage path, wherein the source file is encapsulated into an input stream, the input stream represents a byte sequence, creating a compression task according to the compression rule, wherein the compression task is a task of compressing the source file, executing the compression task to perform compression and packaging processing on the source file and generate a packaged file, and uploading the packaged file to the target path corresponding to the target bucket in the object storage if the data amount of the packaged file uploaded meets the compression condition. Through the object metadata of the source file in the compression rule, the source file is obtained, and the source file is encapsulated into an input stream in a streaming mode, so as to reduce the memory occupation amount in the compression process. Then, a compression task is created according to the compression rule, and the compression task is executed to perform compression and packaging processing on the source file, so as to guarantee the resource integrity of the source file. Then, it is determined whether the data amount of the packaged file is small according to the compression condition, the packaged file is small, which can not only guarantee the time of uploading to be shortened, but also can improve the efficiency of uploading to the object storage. Finally, the packaged file is uploaded to the target path corresponding to the target bucket in the object storage, so as to further guarantee the resource integrity of the source file. BRIEF DESCRIPTION OF DRAWINGS

[0055] Figure 1 FIG. 1 is a flowchart of a compression method for object storage provided by an exemplary embodiment of the application.

[0056] Figure 2 FIG. 2 is a flowchart of a compression method for object storage provided by another exemplary embodiment of the application.

[0057] Figure 3 FIG. 3 is a flowchart of a method for obtaining a compression rule provided by an exemplary embodiment of the application.

[0058] Figure 4 FIG. 4 is a flowchart of a decompression method for object storage provided by an exemplary embodiment of the application.

[0059] Figure 5 FIG. 5 is a structural diagram of a compression device for object storage provided by an exemplary embodiment of the application.

[0060] Figure 6 is a structural schematic diagram of a compression device for object storage provided by another exemplary embodiment of the present application.

[0061] Figure 7 is a structural schematic diagram of a decompression device for object storage provided by an exemplary embodiment of the present application.

[0062] Figure 8 is a structural schematic diagram of a decompression device for object storage provided by an exemplary embodiment of the present application.

[0063] Figure 9 is a structural diagram of an electronic device provided by an exemplary embodiment of the present application. DETAILED DESCRIPTION

[0064] Figure 1 is a flowchart of a compression method for object storage provided by an exemplary embodiment of the present application. Figure 2 is a flowchart of a compression method for object storage provided by another exemplary embodiment of the present application.

[0065] As shown in Figures 1-2 , the compression method for object storage comprises:

[0066] Step 110: obtaining a compression rule, wherein the compression rule comprises a compression condition, object metadata of a source file, and a storage location after compression, and the storage location after compression comprises a target path corresponding to a target bucket in the object storage.

[0067] In the embodiments of the present application, in order to shorten the file uploading time, the present application first sets a compression rule, through which a more effective compression scheme can be designed for a specific type of file or data to improve the compression rate, and a more suitable compression algorithm can be designed according to the characteristics and needs of the file or data to be compressed, thereby improving the compression efficiency, shortening the file uploading time, etc. The compression rule can be a custom compression rule created by a user according to actual needs, wherein the compression rule can include a compression condition, object metadata of a source file, a storage location after compression, a number of source files, a target file name after compression, and the storage location after compression includes a target path corresponding to a target bucket in the object storage.

[0068] Step 120: locating a storage path corresponding to a storage bucket in the object storage according to the object metadata of the source file.

[0069] In this embodiment of the invention, to expedite the retrieval of files that a user needs to compress, the storage path corresponding to the storage bucket in the object storage can be located using the object metadata of the source file, thereby obtaining the corresponding source file through the storage path. Therefore, by using only the object metadata and avoiding traversing the entire object storage to find the file's location, time and resources can be saved. Furthermore, determining the storage path through object metadata reduces the possibility of incorrect location, thus ensuring that the correct file storage location is found.

[0070] Object metadata is a collection of data describing the attributes and characteristics of objects (such as files, database records, messages, etc.). This metadata typically includes information about the object, such as creation time, modification time, size, owner, permissions, tags, relationships, and location. Object storage is a storage model used to store and manage large amounts of unstructured data. In object storage, data is stored in the form of objects, each containing the data itself and its associated metadata. A bucket is the basic container used to store objects in an object storage system. In an object storage system, a bucket is similar to a folder in a file system, used to organize and manage the objects stored within it. Each bucket has a unique name, can contain a large number of objects, and can be configured with different access permissions and configuration options.

[0071] Step 130: Read the source file according to the storage path. The source file is encapsulated as an input stream, which represents a sequence of bytes.

[0072] In this embodiment of the invention, to reduce memory usage and improve system performance during compression, the source file can be encapsulated as an input stream, where the input stream represents a sequence of bytes or an object containing a sequence of bytes. When the source file needs to be read, it can be read from the input stream, thereby reducing memory usage and improving system performance during compression. Specifically, reading the target file uses a GET request, calling the Ceph RGW (Object Storage Gateway) download interface.

[0073] Step 140: Create a compression task according to the compression rules, where the compression task is a task that compresses the source file.

[0074] In the embodiments of the present application, in order to actually execute the compression rule to compress the source file, it is necessary to create a compression task according to the compression rule, that is, even if the compression rule has been designed, it is necessary to create a compression task to trigger the system to execute the compression rule. It should be understood that the designed compression rule only defines when and how to compress the source file, and does not actually execute the compression operation, so the system needs to execute the compression operation according to the rule by creating a compression task to compress the source file into a smaller storage space. Moreover, the execution time of the compression can be controlled by creating a compression task, so as to be executed during the time when the system load is low or the non-core business time, thereby avoiding affecting the system performance.

[0075] In addition, by creating a compression task, the execution of the compression operation can be monitored and tracked, including execution time, success or failure, compression rate and other indicators.

[0076] Step 150: executing the compression task to compress and package the source file to generate a packaged file.

[0077] In the embodiments of the present application, by compressing and packaging the source file, the size of the file can be significantly reduced, thereby saving storage space. The compressed file is smaller in size and requires less bandwidth for file transmission, which can save transmission cost and time. Since the compressed file is smaller, the transmission speed is faster, especially when the network condition is poor or large files are transmitted, which can reduce the transmission time. Packaging the source file into a packaged file can better organize and manage the file, facilitating unified backup, transmission and storage.

[0078] Step 160: If the data amount of the packaged file uploaded meets the compression condition, the packaged file is uploaded to the target path corresponding to the target bucket in the object storage.

[0079] In the embodiments of the present application, in order to improve the uploading efficiency of the packaged file and shorten the uploading time, the compression condition can be set to determine whether the data amount of the packaged file uploaded meets the regulation, and if the data amount of the packaged file uploaded meets the compression condition, the packaged file is uploaded to the target path corresponding to the target bucket in the object storage. The packaged file is uploaded using a put request, which calls the upload interface of the ceph rgw (object storage gateway). The put request is a HTTP request method for uploading new data to a specified resource location. The object storage gateway is a technology designed to provide an interface that enables applications to access and store data to an object storage-based backend storage system. The existence of this intermediate layer helps to simplify the interaction between the application and the underlying object storage system, and provides additional functionality and flexibility.

[0080] Specifically, step 160 can specifically include the following steps:

[0081] Step 161: If the data amount of the packaged file upload is less than the preset number threshold, the packaged file is uploaded to the target path corresponding to the target bucket in the object storage.

[0082] In the embodiment of the application, in order to improve the compression efficiency and considering that the object storage allows uploading of a single file with a limited size, a compression condition is set in the compression rule, that is, whether the data amount of the packaged file upload is less than the preset number threshold is judged, and if the data amount of the packaged file upload is less than the preset number threshold, the packaged file is uploaded to the target path corresponding to the target bucket in the object storage, wherein the preset number threshold can be 5T.

[0083] The compression method of the object storage provided by the application comprises: obtaining a compression rule, wherein the compression rule comprises a compression condition, object metadata of a source file, and a storage location after compression, the storage location after compression comprises a target path corresponding to a target bucket in the object storage, locating a storage path corresponding to a storage bucket in the object storage according to the object metadata of the source file, reading the source file according to the storage path, wherein the source file is encapsulated into an input stream, the input stream represents a byte sequence, creating a compression task according to the compression rule, wherein the compression task is a task of compressing the source file, executing the compression task to compress and package the source file and generate a packaged file, and if the data amount of the packaged file upload meets the compression condition, uploading the packaged file to the target path corresponding to the target bucket in the object storage. The source file is obtained through the object metadata of the source file in the compression rule, and the source file is encapsulated into an input stream in a streaming mode, so as to reduce the memory occupation amount in the compression process. Then, the compression task is created according to the compression rule, and the compression task is executed to compress and package the source file, so as to ensure the resource integrity of the source file. Then, whether the data amount of the packaged file is small is determined according to the compression condition, the packaged file with a small data amount can not only shorten the uploading time but also improve the uploading efficiency to the object storage. Finally, the packaged file is uploaded to the target path corresponding to the target bucket in the object storage, so as to further ensure the resource integrity of the source file.

[0084] Figure 3 FIG. 1 is a flowchart of an example embodiment of the application, which shows a method for obtaining a compression rule. As shown in FIG. 1, step 110 can comprise: Figure 3

[0085] Step 111: generating a blank table based on the compression instruction of the user, wherein the blank table comprises a compression condition, a source file name, a source file number, a compression type, a storage location after compression, object metadata of a source file, a target file name, and a target file number.

[0086] ​In the embodiment of the present application, when the user needs to compress the source file, the compression instruction can be sent, the blank form is generated through the compression instruction, and the user fills in the blank form according to the indication of the blank form, that is, fills in the compression requirement in the blank form, wherein the blank form includes compression condition, source file name or directory name, source file number, compression type, and storage location after compression. The compression type can include zip, tar.gz and other compression formats. For the zip format, even if the user's computer does not install decompression software, the zip format compressed file can be opened and made. The source file name refers to the name of the compressed file to be compressed. The target file name refers to the name of the file or folder generated after compression. Usually, the source file name is the same as the target file name, and the source file number is the same as the target file number.

[0087] Step 112: generating a compression form according to the content filled in the blank form and the preset encoding format.

[0088] In the embodiment of the present application, since problems such as file or folder name garbled code and task process interruption may occur during compression, after the blank form is filled in, the text in the blank form is encoded by setting the preset encoding format, for example, the source file name is encoded, so as to prevent the problem of name garbled code.

[0089] Specifically, the preset encoding format can be UTF-8 or GB2312. UTF-8 is a variable-length Unicode encoding scheme that supports global character sets, GB2312 is a Chinese national standard simplified Chinese character set, which uses double-byte encoding, each byte is 8 bits, and represents 6763 Chinese characters and 682 non-Chinese characters.

[0090] Step 113: creating a compression rule based on the content of the compression form.

[0091] In an embodiment, step 140 can be specifically implemented as: generating a plurality of compression task entries according to the compression rule; wherein each compression task entry includes task identification, task content, task type, and task progress; determining the priority of each compression task entry; sorting the plurality of compression task entries according to the priority of each compression task entry; and generating a corresponding compression task for each sorted compression task entry.

[0092] In the embodiments of the present application, in order to make the compression task can be accurately executed and ensure the accuracy of the compression task execution result, therefore can be generated according to the compression rule, a plurality of compression task entries, through the compression task entry can determine the task execution content and can make a plurality of compression tasks orderly executed, prevent the task loss situation when executing the task. Among them, each compression task entry includes task identification (ID), task content, task type, task progress. Through the task identification, the detailed information of the specific task can be quickly tracked, and the dependency relationship between different tasks can also be associated, helping to determine the sequence or logical relationship between tasks. The task content is mainly generated by the content of the compression rule, and the compression condition in the compression rule, the storage location after compression and the like are converted into operation steps, which are task contents. When executing the task content, the operation steps corresponding to the task content are executed. By setting the task type, the task can be classified into different categories or types, which helps to better organize and manage the task. The task progress can be real-time understood the completion of the task, including the part of the task that has been completed and the part that has not been completed. If the overall compression task is taken as a reference object to understand the task progress, when all the tasks are executed, the overall task progress should be 100%. When half of the tasks are executed, the corresponding overall task progress should be 50%. If a single compression task is taken as a reference object to understand the task progress of a single compression task, when the single compression task is completed, the corresponding task progress is 100%. If the single compression task includes a plurality of sub-compression tasks, then the task progress of the single compression task is 100% when half of the sub-compression tasks are completed. If the single compression task has only one compression task, then the task progress is usually only the completion condition, because a compression task can be completed in milliseconds, and the task progress has been completed in the process of milliseconds.

[0093] In order to shorten the time, the priority of each compression task entry can be set, and through the setting of the priority, the system can help to determine the importance of the compression task entry with priority in advance, and can execute the compression task corresponding to the compression task entry with priority. That is, the plurality of compression task entries are sorted according to the priority from large to small, and the sorted compression task entries are created into a compression task package or a compression task, so that the system can execute the compression task as a whole. In the compression process, the progress of the task entry is updated at a certain period. After the compression is completed, the state of the compression task entry is updated to compression completion.

[0094] In an embodiment, after step 140, the compression method of the object storage can be specifically implemented as: persisting the compression task entry into a relational database.

[0095] In the embodiment of the present application, in order to realize the sharing of data or files, the persistence of data, the compressed task entry is therefore persisted into a relational database, the data in the database can be accessed and shared by multiple users or application programs, realizing the sharing and cooperation of data, improving the work efficiency and the accuracy of information, and even if the system or application program is closed or restarted, the data can still be retained and will not be lost. The relational database includes mysql, oracle and other relational databases.

[0096] In addition, the result of each sorted compressed task entry corresponding to the execution of the compressed task can be persisted into the relational database. The task progress corresponding to the execution of each compressed task can also be persisted into the relational database, thereby facilitating the user to view and use. In addition, the overall task progress of multiple compressed tasks completed in the relational database can also be persisted, for example, the total number of compressed tasks is 10, the overall task progress corresponding to the completion of 5 compressed tasks is 50%, and the overall task progress corresponding to the completion of 10 compressed tasks is 100%. If a single compressed task is referred to, the compression progress after the completion of a compressed task is 100%.

[0097] In an embodiment, before step 160, the compression method of the object storage can be specifically implemented as follows: generating an execution result of executing the compression task; if the execution result indicates that the compressed file is incorrect, obtaining a backup file corresponding to the source file; wherein the storage path of the backup file is different from the storage path of the source file; if the source file and the backup file do not match, determining that the source file is damaged; dividing the multiple backup files to generate multiple backup file groups; compressing and packaging each backup file group; obtaining a word from a preset dictionary; wherein the number of English letters of the word is the same as the number of the processed backup file groups; assigning each English letter of the word to the processed backup file group and taking the assigned English letter as the name of the processed backup file group; based on the arrangement order of the English letters in the word, uploading the processed backup file group to a target path corresponding to a target bucket in the object storage.

[0098] In the embodiment of the present application, since the source file is obtained from the object storage, the phenomenon of damage of the source file may occur in the process of obtaining, and therefore the execution result needs to be output after executing the compression task, and it is determined whether the compression of the source file is successful through the execution result. If the execution result indicates that the compressed file is incorrect, the backup file corresponding to the source file is obtained, and it is determined whether the source file is damaged through the backup file.

[0099] Specifically, the backup file needs to be obtained from a storage path different from the storage path of the source file, because the source file obtained from the object storage may itself be damaged, so the backup file of the source file is obtained from the other storage path, and the backup file is taken as a reference object to determine whether the source file itself is damaged, if the source file itself is damaged, that is, the source file does not match the backup file, then it is determined that the source file is damaged, then the backup file can be compressed and packaged and uploaded to the target path of the target bucket in the object storage.

[0100] When there are multiple backup files, the multiple backup files can be divided to generate multiple backup file groups, and then each backup file group is compressed and packaged, and a word is obtained from a preset dictionary, wherein the number of letters of the word is the same as the number of the processed backup file groups, for example, the processed backup file groups are 3, and the word that can be found from the preset dictionary is "egg". Then each letter of the word is assigned to the processed backup file group, and the assigned English letters are taken as the name of the processed backup file group, for example, the letters "e", "g" and "g" in "egg" are assigned to the processed backup file group, and "e", "g" and "g" are taken as the name of the processed backup file group. Based on the pronunciation order of the English letters in the word, the processed backup file group is uploaded to the target path corresponding to the target bucket in the object storage, so as to prevent the network from being congested during the uploading process caused by simultaneous uploading of the processed backup file groups.

[0101] In an embodiment, the compression method of the object storage can be specifically implemented as: if the execution result indicates compression interruption, the next unexecuted compression task at the time of compression interruption is obtained from the relational database; and the next unexecuted compression task is executed.

[0102] In the embodiment of the application, because the execution of the compression task is all persisted to the relational database, the task progress, task type, etc. of each compression task execution can be obtained through the relational database.

[0103] If the execution result indicates compression interruption, that is, the currently executed compression task is interrupted after completion, and the next unexecuted compression task has not been executed. Therefore, the next unexecuted compression task at the time of compression interruption can be obtained from the relational database and executed.

[0104] In an embodiment, the compression method of the object storage can be specifically implemented as: if the execution result indicates encoding error, the preset encoding format is changed.

[0105] In the embodiment of the present application, if the preset encoding format of the compression is UTF-8, but the use of UTF-8 results in encoding error, the preset encoding format can be changed, that is, UTF-8 is changed to GB2312. In other words, if the preset encoding format of the compression is GB2312, but the use of GB2312 results in encoding error, the preset encoding format can be changed, that is, GB2312 is changed to UTF-8. After the preset encoding format is changed, the source file is compressed and packaged again.

[0106] Figure 4 is a flowchart of an object storage decompression method provided by an exemplary embodiment of the present application. As shown in Figure 4 the object storage decompression method includes:

[0107] Step 210: Obtain a decompression rule, wherein the decompression rule includes object metadata of the target file.

[0108] In the embodiment of the present application, in order to quickly obtain the compressed target file and thus perform application, a decompression rule needs to be formulated before decompression, and the decompression rule can be used to find the file or data that needs to be decompressed.

[0109] Specifically, the decompression rule can include object metadata of the target file and a decompression condition. The object metadata is a collection of data describing the properties and characteristics of an object (such as a file, a database record, a message, etc.). These metadata usually contain information about the object, such as creation time, modification time, size, owner, permission, tag, association relationship, location, etc.

[0110] If the system memory space is less, the decompression condition can be set to prevent the data volume of the decompressed file from being too large and causing decompression failure. If the data volume of the decompressed file is greater than a preset decompression data volume threshold, a prompt information is generated, which indicates that the source files are compressed in batches, that is, the decompressed source files are divided into multiple file groups, each file group retains only one or a few source files, and then each file group is repeatedly compressed to ensure that part of the source files can be obtained in the case of less system memory space.

[0111] Step 220: According to the object metadata of the target file, locate the target path corresponding to the target bucket in the object storage.

[0112] In the embodiment of the present application, in order to speed up the search for the file that the user needs to decompress, the object metadata of the target file is used to locate the target path corresponding to the target bucket in the object storage, so that the corresponding target file is obtained through the target path. As can be seen, only the object metadata is needed, and the entire object storage does not need to be traversed to find the file location, which can save time and resources. Moreover, the storage path determined by the object metadata can reduce the possibility of positioning errors, thereby ensuring that the correct file storage location is found.

[0113] Step 230: reading the target file according to the target path, wherein the source file is encapsulated into an input stream, and the input stream represents a byte sequence.

[0114] In the embodiment of the present application, in order to reduce the memory occupation during the decompression process and improve the system performance, the target file can be encapsulated into an input stream, and the input stream represents a byte sequence or an object of a byte sequence. When the target file needs to be read, the target file can be read from the input stream, thereby reducing the memory occupation during the decompression process and improving the system performance. The target file is read using a get request to call a ceph rgw (object storage gateway) download interface. The get request is a request method defined in the HTTP protocol, which is used to obtain resources from a specified server.

[0115] Step 240: creating a decompression task according to the decompression rule, wherein the decompression task is a task for decompressing the target file.

[0116] In the embodiment of the present application, in order to actually execute the decompression rule to decompress the target file, a decompression task needs to be created according to the decompression rule, that is, even if the decompression rule has been designed, a decompression task needs to be created to trigger the system to execute these decompression rules. It should be understood that the designed decompression rule only defines when and how to decompress the target file, and does not actually perform the decompression operation, so the system needs to perform the decompression operation according to the rule by creating a decompression task

[0117] Step 250: executing the decompression task to perform decompression processing on the decompression task and obtain the decompressed source file.

[0118] The decompression method of the object storage provided by the application comprises the following steps: obtaining a decompression rule, wherein the decompression rule comprises object metadata of a target file; locating a target path corresponding to a target bucket in the object storage according to the object metadata of the target file; and reading the target file according to the target path; wherein the source file is encapsulated into an input stream, the input stream represents a byte sequence, a decompression task is created according to the decompression rule, the decompression task is a task of decompressing the target file, the decompression task is executed to perform decompression processing on the decompression task and obtain a decompressed source file. The object metadata of the target file in the decompression rule is used to obtain the target file, and the target file is encapsulated into the input stream in a streaming mode, so that the memory occupation in the decompression process is reduced. Then, the decompression task is created according to the decompression rule, and the decompression task is executed to perform decompression processing on the target file, so that the resource integrity of the target file is ensured, and the user can quickly obtain the source file.

[0119] In an embodiment, step 240 can be specifically implemented as: generating a plurality of decompression task entries according to the decompression rule; wherein each decompression task entry comprises a task identifier, task content, a task type, and a task progress; determining a priority of each decompression task entry; sorting the plurality of decompression task entries according to the priority of each decompression task entry; and generating a corresponding decompression task for each sorted decompression task entry.

[0120] In the embodiments of the present application, in order to make the decompression task can be accurately executed and ensure the accuracy of the decompression task execution result, therefore can be generated according to the decompression rule, a plurality of decompression task entries, through the decompression task entry can determine the task execution content and can make a plurality of decompression task entries orderly execute, prevent the execution task when the task loss situation. Among them, each decompression task entry includes task identification (ID), task content, task type, task progress. Through the task identification can quickly track the detailed information of a specific task, and can also associate the dependency between different tasks, help to determine the order or logical relationship between tasks. Task content is mainly generated by the content of the decompression rule, and the decompression condition in the decompression rule, the storage location after decompression and the like are converted into operation steps, which are task content. When executing the task content, execute the operation steps corresponding to the task content. By setting the task type, the task can be classified into different categories or types, which helps to better organize and manage the task. The task progress can understand the completion of the task in real time, including the part of the task that has been completed and the part that has not been completed. If the overall decompression task is taken as the reference object to understand the task progress, when all the tasks are executed, the overall task progress should be 100%. When half of the tasks are executed, the corresponding overall task progress should be 50%. If a single decompression task is taken as a reference object to understand the task progress of a single decompression task, when a single compression task is completed, the corresponding task progress is 100%, if a single compression task includes a plurality of sub-decompression tasks, then the task progress of the single decompression task is 100% when half of the sub-decompression tasks are completed. If a single decompression task has only one decompression task, then its task progress is usually only the completion condition, because a decompression task can be completed in milliseconds, and its task progress has been completed in milliseconds.

[0121] In order to shorten the time, the priority of each decompression task entry can be set, through the setting of the priority can help the system to determine in advance the importance of the decompression task entry with priority, and can execute the decompression task entry with priority in priority. That is, the plurality of decompression task entries are sorted according to the priority from large to small, and the sorted decompression task entries are generated into executable decompression task package or decompression task, so that the system can execute the decompression task as a whole. In the decompression process, update the decompression progress of the decompression task entry at a certain period. After decompression, update the task state of the task entry to decompression completion.

[0122] In an embodiment, after step 250, the decompression method of the object storage can be implemented as follows: if the decompression task is interrupted during execution, obtaining the decompression task that has been executed at the time of interruption from the relational database; obtaining the task progress corresponding to each executed decompression task; determining the decompression task with a task progress less than the preset progress as an abnormal task; obtaining a plurality of preset reasons for the abnormal task; according to each preset reason, modifying the abnormal task and re-executing the modified executed decompression task; if the task progress of the modified executed decompression task is greater than the preset progress, determining the preset reason corresponding to the modified executed decompression task as an abnormal reason and persisting the abnormal reason in the relational database.

[0123] In an embodiment of the present application, since the decompression task may be interrupted due to file damage, unstable network, etc. during decompression, in order to find the reason for the interruption of the task, the present application obtains the decompression task that has been executed at the time of interruption from the relational database and obtains the task progress corresponding to each executed decompression task. The decompression task with a task progress less than the preset progress is determined as an abnormal task, because each decompression task may have a plurality of sub-decompression tasks, so if a sub-decompression task is not completed, the task progress will not be 100%. The present application needs to find the decompression task with a task progress less than 100% and take it as an abnormal task. A plurality of preset reasons for the abnormal task to appear abnormal may be stored in the system, for example, slow network speed. The abnormal task is modified according to the preset reason, for example, the abnormal task is to decompress file A when the network speed is 100M, so the abnormal task can be rewritten as decompressing file A when the network speed is 50M, and then the modified executed decompression task is re-executed. If the task progress of the modified executed decompression task is greater than the preset progress, the preset reason corresponding to the modified executed decompression task is determined as an abnormal reason and the abnormal reason is persisted in the relational database. That is, the abnormal reason of the abnormal task is that the current network speed is slow, which causes the decompression of the file to fail, so the abnormal reason is updated in the decompression task entry corresponding to the abnormal task and is persisted in the relational database.

[0124] In an embodiment, based on the decompression instruction of the user, a blank table is generated; wherein the blank table includes the source file name, the source file number, the target file name, the target file number, the decompression type, and the object metadata of the target file; according to the content filled in the blank table and the preset encoding format, a decompression table is generated; based on the content of the decompression table, a decompression rule is created.

[0125] In the embodiment of the present application, when a user needs to decompress a target file, a decompression instruction can be sent, a blank form is generated through the decompression instruction, and the user fills in the decompression requirement in the blank form according to the indication of the blank form. The blank form includes a source file name, a source file number, a decompression type, and object metadata of the target file. The decompression type can include zip, tar.gz, and other compression formats. For the zip format, even if the user's computer does not have decompression software installed, the zip format compressed file can be opened and created.

[0126] Since problems such as file or folder name garbled code and task interruption may occur during decompression, after the blank form is filled in, the text in the blank form is encoded by setting a preset encoding format, for example, the source file name is encoded, so as to prevent problems such as name garbled code.

[0127] Specifically, the preset encoding format can be UTF-8 or GB2312. UTF-8 is a variable-length Unicode encoding scheme that supports character sets worldwide. GB2312 is a Chinese national standard simplified Chinese character set, which uses double-byte encoding, each byte is 8 bits, and represents 6763 Chinese characters and 682 non-Chinese character symbols.

[0128] In an embodiment, after step 240, the compression method of the object storage can be specifically implemented as follows: persisting the decompression task item into a relational database.

[0129] In the embodiment of the present application, in order to realize data or file sharing and data persistence, the decompression task item is persisted into a relational database. The data in the database can be accessed and shared by multiple users or application programs, realizing data sharing and collaboration, improving work efficiency and information accuracy, and even if the system or application program is closed or restarted, the data can still be retained and will not be lost. The relational database includes mysql, oracle, and other relational databases.

[0130] In addition, the result of each decompression task after the corresponding decompression task execution is completed can be persisted into the relational database. The task progress corresponding to each decompression task execution completion can also be persisted into the relational database, thereby facilitating the user to view and use. In addition, the overall task progress of multiple decompression tasks completed in the relational database can also be persisted, for example, the total number of decompression tasks is 10, the overall task progress corresponding to 5 completed decompression tasks is 50%, and the overall task progress corresponding to 10 completed decompression tasks is 100%. If a single decompression task is referred to, the task progress after the completion of a decompression task is 100%.

[0131] In an embodiment, the decompression method of the object storage can be implemented as follows: if the execution result indicates that the decompression is interrupted, the next unexecuted decompression task at the time of the decompression interruption is obtained from the relational database; and the next unexecuted decompression task is executed.

[0132] In the embodiment of the present application, since the execution of the decompression task is persisted in the relational database, the task progress and the task type of each decompression task at the time of the execution can be obtained from the relational database.

[0133] If the execution result indicates that the decompression is interrupted, that is, the current executed decompression task is interrupted after being completed, and the next unexecuted decompression task is not executed. Therefore, the next unexecuted decompression task at the time of the decompression interruption can be obtained from the relational database and executed.

[0134] In an embodiment, the decompression method of the object storage can be implemented as follows: if the execution result indicates that the encoding is incorrect, the preset encoding format is changed.

[0135] In the embodiment of the present application, if the preset encoding format of the decompression is UTF-8, but the encoding error is caused by using UTF-8, the preset encoding format can be changed, that is, the UTF-8 is changed to GB2312. In other words, if the preset encoding format of the decompression is GB2312, but the encoding error is caused by using GB2312, the preset encoding format can be changed, that is, the GB2312 is changed to UTF-8. After the preset encoding format is changed, the target file is decompressed again.

[0136] Figure 5 FIG. 1 is a structural schematic diagram of a compression device of object storage provided by an exemplary embodiment of the present application. As shown in FIG. 1, the compression device of object storage 30 comprises: Figure 5

[0137] A first task module 301 is configured to obtain a compression rule; wherein the compression rule comprises a compression condition, object metadata of a source file, and a storage location after compression, and the storage location after compression comprises a target path corresponding to a target bucket in the object storage; and a storage path corresponding to a storage bucket in the object storage is located according to the object metadata of the source file.A first storage module 302 is configured to read the source file according to the storage path; wherein the source file is encapsulated into an input stream, and the input stream represents a byte sequence. A compression module 303 is configured to create a compression task according to the compression rule; wherein the compression task is a task of compressing the source file; and the compression task is executed to compress and package the source file and generate a packaged file; and if the data amount of the packaged file uploaded satisfies the compression condition, the packaged file is uploaded to the target path corresponding to the target bucket in the object storage.

[0138] Figure 6 is a structural schematic diagram of the object storage compression device provided by another exemplary embodiment of the present application. As shown in Figure 6 the first task module 301 can include: an instruction unit 3011 configured to generate a blank form based on a user's compression instruction; wherein the blank form includes compression conditions, source file names, source file numbers, target file names, target file numbers, compression types, compressed storage locations, and object metadata of source files; a generation unit 3012 configured to generate a compression form according to the content filled in the blank form and a preset encoding format; and a creation unit 2013 configured to create a compression rule based on the content of the compression form.

[0139] In an embodiment, the compression module 303 is specifically configured to: generate a plurality of compression task entries according to the compression rule; wherein each compression task entry includes a task identifier, a task content, a task type, and a task progress; determine the priority of each compression task entry; sort the plurality of compression task entries according to the priority of each compression task entry; and generate a corresponding compression task for each sorted compression task entry.

[0140] In an embodiment, after the first task module 301, the object storage compression device can be specifically configured to: persist the compression task entry in a relational database.

[0141] In an embodiment, the compression module 303 is specifically configured to: if the data amount of the packaged file upload is less than a preset number threshold, upload the packaged file to a target path corresponding to a target bucket in the object storage.

[0142] In an embodiment, after the compression module 303, the object storage compression device can be specifically configured to include a first monitoring module 304 configured to generate an execution result of executing a compression task; if the execution result indicates a compression file error, obtain a backup file corresponding to the source file; wherein the storage path of the backup file is different from the storage path of the source file; if the source file and the backup file do not match, determine that the source file is damaged; divide a plurality of backup files to generate a plurality of backup file groups; compress and package each backup file group; obtain a word from a preset dictionary; wherein the number of English letters in the word is the same as the number of processed backup file groups; assign each English letter of the word to a processed backup file group and use the assigned English letter as the name of the processed backup file group; and upload the processed backup file group to a target path corresponding to a target bucket in the object storage based on the pronunciation order of the English letters in the word.

[0143] Figure 7 is a structural schematic diagram of the object storage decompression device provided by an exemplary embodiment of the present application. As shown in Figure 7As shown, the decompression device 40 of the object storage comprises: a second task module 401, configured to obtain a decompression rule; wherein the decompression rule comprises object metadata of a target file; and the target path corresponding to the target bucket in the object storage is located according to the object metadata of the target file; a second storage module 402, configured to read the target file according to the target path; wherein the source file is encapsulated into an input stream, and the input stream represents a byte sequence; and a decompression module 403, configured to create a decompression task according to the decompression rule; wherein the decompression task is a task of decompressing the target file; and the decompression task is executed to perform decompression processing on the decompression task and obtain the decompressed source file.

[0144] Figure 8 FIG. 1 is a structural schematic diagram of a decompression device of an object storage according to an example embodiment of the present application. As shown in FIG. 1, the decompression device 40 of the object storage comprises: a second task module 401, configured to obtain a decompression rule; wherein the decompression rule comprises object metadata of a target file; and the target path corresponding to the target bucket in the object storage is located according to the object metadata of the target file; a second storage module 402, configured to read the target file according to the target path; wherein the source file is encapsulated into an input stream, and the input stream represents a byte sequence; and a decompression module 403, configured to create a decompression task according to the decompression rule; wherein the decompression task is a task of decompressing the target file; and the decompression task is executed to perform decompression processing on the decompression task and obtain the decompressed source file. Figure 8 As shown in FIG. 1, after the decompression module 403, the decompression device 40 of the object storage can further comprise: a second monitoring module 404, configured to: if the task is interrupted in the process of executing the decompression task, obtain the decompression task that has been executed at the time of the task interruption from the relational database; obtain the task progress corresponding to each executed decompression task; determine the decompression task with a task progress less than a preset progress as an abnormal task; obtain a plurality of preset reasons causing the abnormal task to be abnormal; according to each preset reason, change the abnormal task and re-execute the changed executed decompression task; and if the task progress of the changed executed decompression task is greater than the preset progress, determine the preset reason corresponding to the changed executed decompression task as an abnormal reason and persist the abnormal reason in the relational database.

[0145] Figure 9 FIG. 2 is a block diagram of an electronic device according to an example embodiment of the present application.

[0146] As shown in FIG. 2, the electronic device 10 comprises one or more processors 11 and a memory 12. Figure 9 The processor 11 can be a central processing unit (CPU) or other form of processing unit having data processing and / or instruction execution capabilities, and can control other components in the electronic device 10 to perform desired functions.

[0147] The processor 11 can be a central processing unit (CPU) or other form of processing unit having data processing and / or instruction execution capabilities, and can control other components in the electronic device 10 to perform desired functions.

[0148] The memory 12 can include one or more computer program products that can include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory, for example, can include random access memory (RAM), cache memory, and / or the like. The non-volatile memory, for example, can include read only memory (ROM), hard disk, flash memory, and / or the like. One or more computer program instructions can be stored on the computer-readable storage media, and the processor 11 can execute the program instructions to implement the compression method of object storage and the decompression method of object storage of various embodiments of the present application described above and / or other desired functions. Various contents such as input signals, signal components, noise components, and the like can also be stored in the computer-readable storage media.

[0149] In one example, the electronic device 10 can further include an input device 13 and an output device 14, which are interconnected through a bus system and / or other forms of connection mechanisms (not shown).

[0150] When the electronic device is a stand-alone device, the input device 13 can be a communication network connector for receiving the acquired input signals from the first device and the second device.

[0151] In addition, the input device 13 can further include, for example, a keyboard, a mouse, and the like.

[0152] The output device 14 can output various information including the determined distance information, direction information, and the like to the outside. The output device 14 can include, for example, a display, a speaker, a printer, a communication network and a remote output device connected thereto, and the like.

[0153] Of course, in order to simplify, Figure 9 In FIG. 1, only some of the components related to the present application among the components of the electronic device 10 are illustrated, and components such as a bus, an input / output interface, and the like are omitted. In addition to this, the electronic device 10 can further include any other appropriate components according to the specific application.

[0154] The computer program product can be written in any combination of one or more programming languages, including an object oriented programming language such as Java, C++, and the like, and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code can execute entirely on the user's computing device, partly on the user's device, as a stand-alone software package, partly on the user's computing device and partly on a remote computing device or entirely on the remote cloud device or server.

[0155] The computer readable storage medium can be any combination of one or more computer readable media. The computer readable medium can be a computer readable signal medium or a computer readable storage medium. A computer readable storage medium can be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of the computer readable storage medium include an electrical connection having one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0156] The above description is merely illustrative of the application, and not restrictive.

Claims

1. A compression method of object storage, characterized by, The method comprises the following steps: acquiring a compression rule; wherein the compression rule comprises a compression condition, object metadata of a source file, and a storage location after compression, and the storage location after compression comprises a target path corresponding to a target bucket in an object storage; locating a storage path corresponding to a storage bucket in the object storage according to the object metadata of the source file; reading the source file according to the storage path; wherein the source file is encapsulated into an input stream, and the input stream represents a byte sequence; creating a compression task according to the compression rule; wherein the compression task is a task of compressing the source file; performing the compression task to compress and package the source file and generate a packaged file; if a data amount uploaded by the packaged file meets the compression condition, uploading the packaged file to the target path corresponding to the target bucket in the object storage; after the step of uploading the packaged file to the target path corresponding to the target bucket in the object storage, the method further comprises the following steps: generating an execution result of performing the compression task; if the execution result indicates a compression file error, acquiring a backup file corresponding to the source file; wherein a storage path of the backup file is different from a storage path of the source file; if the source file does not match the backup file, determining that the source file is damaged; dividing a plurality of backup files to generate a plurality of backup file groups; compressing and packaging each backup file group.

2. The object storage compression method of claim 1, wherein, The step of acquiring the compression rule comprises the following steps: generating a blank table based on a compression instruction of a user; wherein the blank table comprises the compression condition, a source file name, a source file number, a target file name, a target file number, a compression type, the storage location after compression, and the object metadata of the source file; generating a compression table according to content filled in the blank table and a preset encoding format; creating the compression rule based on content of the compression table.

3. The compression method of object storage according to claim 1, wherein, The step of creating the compression task according to the compression rule comprises the following steps: generating a plurality of compression task entries according to the compression rule; wherein each compression task entry comprises a task identifier, task content, a task type, and a task progress; determining a priority of each compression task entry; sorting the plurality of compression task entries according to the priority of each compression task entry; generating a corresponding compression task for each sorted compression task entry.

4. The object storage compression method of claim 3, wherein, After the step of creating the compression task according to the compression rule, the method further comprises the following step: persisting the compression task entry into a relational database.

5. The compression method of object storage according to claim 1, wherein, The step of uploading the packaged file to the target path corresponding to the target bucket in the object storage if the data amount uploaded by the packaged file meets the compression condition comprises the following step: if the data amount uploaded by the packaged file is less than a preset quantity threshold, uploading the packaged file to the target path corresponding to the target bucket in the object storage.

6. The object storage compression method of claim 1, wherein, acquiring a word from a preset dictionary; wherein the number of English letters of the word is the same as the number of backup file groups after processing; Assign each English letter of the word to the processed backup file group and assign the assigned English letter as the name of the processed backup file group; Based on the pronunciation order of the English letters in the word, upload the processed backup file group to the target path corresponding to the target bucket in the object storage.

7. A decompression method of object storage, characterized by, Comprise: Obtain decompression rules; wherein the decompression rules include object metadata of a target file; According to the object metadata of the target file, locate the target path corresponding to the target bucket in the object storage; According to the target path, read the target file; wherein the target file is encapsulated into an input stream, and the input stream represents a byte sequence; According to the decompression rules, create a decompression task; wherein the decompression task is a task of decompressing the target file; Execute the decompression task to perform decompression processing on the decompression task and obtain the decompressed source file.

8. The object storage decompression method of claim 7, wherein, After executing the decompression task to perform decompression processing on the decompression task to obtain the source file, further comprising: If the task is interrupted during the execution of the decompression task, obtain the decompression task that has been executed when the task is interrupted from the relational database; Obtain the task progress corresponding to each executed decompression task; Determine the decompression task whose task progress is less than the preset progress as an abnormal task; Obtain a plurality of preset reasons for causing the abnormal task to appear abnormal; According to each preset reason, change the abnormal task and re-execute the changed executed decompression task; If the task progress of the changed executed decompression task is greater than the preset progress, determine that the preset reason corresponding to the changed executed decompression task is an abnormal reason and persist the abnormal reason in the relational database.

9. A compression apparatus for object storage, the apparatus comprising: Comprise: The first task module is used for obtaining compression rules; wherein the compression rules include compression conditions, object metadata of a source file, and a compressed storage location, and the compressed storage location includes a target path corresponding to a target bucket in an object storage; according to the object metadata of the source file, locate the storage path corresponding to the storage bucket in the object storage; The first storage module is used for reading the source file according to the storage path; wherein the source file is encapsulated into an input stream, and the input stream represents a byte sequence; The compression module is used for creating a compression task according to the compression rules; wherein the compression task is a task of compressing the source file; execute the compression task to perform compression and packaging processing on the source file and generate a packaged file; if the data amount uploaded by the packaged file meets the compression conditions, upload the packaged file to the target path corresponding to the target bucket in the object storage; After the if the data amount uploaded by the packaged file meets the compression conditions, upload the packaged file to the target path corresponding to the target bucket in the object storage, further comprising: Generate an execution result of executing the compression task; If the execution result indicates that the compressed file is incorrect, a backup file corresponding to the source file is obtained; a storage path of the backup file is different from a storage path of the source file; If the source file does not match the backup file, it is determined that the source file is damaged; The plurality of backup files are divided to generate a plurality of backup file groups; Each backup file group is compressed and packaged.

10. A decompression apparatus for object storage, the apparatus comprising: Comprise: The second task module is configured to obtain a decompression rule, wherein the decompression rule comprises object metadata of a target file; and locate a target path corresponding to a target bucket in object storage according to the object metadata of the target file; The second storage module is configured to read the target file according to the target path; wherein the target file is encapsulated into an input stream, and the input stream represents a byte sequence; The decompression module is configured to create a decompression task according to the decompression rule, wherein the decompression task is a task of decompressing the target file; and execute the decompression task to perform decompression processing on the decompression task and obtain a decompressed source file.

Images