Method, device, electronic device and storage medium for storing objects

Abstract

The application discloses a method and device for storing an object, electronic equipment and a storage medium. The method for storing an object comprises the following steps: generating an index file and a data file of the object, the file name of the index file is obtained based on an object name, the data of the index file comprises object metadata and the file name of the data file; the data of the data file comprises object data, and the file name of the data file is a unique identifier of the object data obtained based on the object data; and storing the generated index file and data file. According to the above technical scheme, the data file is uniquely identified based on the object data without modifying the original storage algorithm, repeated data is not stored repeatedly, the stored repeated data can be used, the efficiency of object writing is improved, and the cost of object storage is reduced.

Description

TECHNICAL FIELD

[0001] One or more embodiments of the present specification relate to the technical field of computer technology, and particularly relate to a method and device for storing an object, an electronic device, and a storage medium. BACKGROUND

[0002] The construction of the information age has given birth to a large number of related high-techs, and at the same time, various data has shown an explosive growth. The growing data in various industries has brought mankind into the era of big data. The huge data has given rise to the problem of data storage.

[0003] Objects of object storage are divided into PUT uploaded objects and segmented uploaded objects. The PUT uploaded object corresponds to only one file, the metadata and data of the object are stored in a file, and the file name is obtained based on the object name. Since the object name is unique, the file name of the PUT uploaded file is also unique. The segmented upload first generates a unique identifier UploadId of the segmented task, and each segmented upload is also identified by a segment number PartNumber. The data file of each segment is calculated based on the object name, UploadId and PartNumber to obtain the file name, so the file name of each segment data file is also unique.

[0004] In summary, for repeated data, the PUT uploaded object and the segmented uploaded object will be stored multiple times, which affects the efficiency of file uploading and wastes storage space. SUMMARY

[0005] To solve the above technical problems, the present application provides a method and device for storing an object, an electronic device, and a storage medium, which do not store the same data of different objects repeatedly, improve the efficiency of object writing, and reduce the cost of object storage. The technical solutions are as follows:

[0006] A method for storing an object, characterized in that it comprises:

[0007] generating an index file and a data file of the object, the file name of the index file being obtained based on the object name, the data of the index file including object metadata and the file name of the data file, the data of the data file including object data, and the file name of the data file being a unique identifier of the object data obtained based on the object data;

[0008] storing the generated index file and data file.

[0009] A device for storing an object, characterized in that it comprises:

[0010] The processing module generates an index file and a data file of the object, a file name of the index file is obtained based on an object name, data of the index file includes object metadata and a file name of the data file, data of the data file includes object data, and the file name of the data file is a unique identifier of the object data obtained based on the object data;

[0011] The storage module stores the generated index file and the data file.

[0012] An electronic device comprises:

[0013] A processor;

[0014] A memory for storing processor-executable instructions;

[0015] The processor implements a method for storing an object by running the executable instructions;

[0016] The method for storing an object comprises:

[0017] Generating an index file and a data file of the object, a file name of the index file is obtained based on an object name, data of the index file includes object metadata and a file name of the data file, data of the data file includes object data, and the file name of the data file is a unique identifier of the object data obtained based on the object data;

[0018] Storing the generated index file and the data file.

[0019] A computer-readable storage medium having computer instructions stored thereon, the instructions being executed by a processor to implement the method steps for storing an object;

[0020] The method for storing an object comprises:

[0021] Generating an index file and a data file of the object, a file name of the index file is obtained based on an object name, data of the index file includes object metadata and a file name of the data file, data of the data file includes object data, and the file name of the data file is a unique identifier of the object data obtained based on the object data;

[0022] Storing the generated index file and the data file.

[0023] In the embodiment of the application, the object is divided into an index file and a data file, and the file name of the data file is the unique identifier of the object data; the file name of the data file exists in the data of the index file. Without modifying the original storage algorithm, the data file is uniquely identified based on the object data, so that the data files of the same object data will not be repeatedly stored, that is, the feature of deleting duplicate data is supported, the stored duplicate data can be maximally utilized, the efficiency of object writing is improved, and the cost of object storage is reduced. BRIEF DESCRIPTION OF DRAWINGS

[0024] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed to be used in the embodiments or the prior art description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments described in the present application, and other drawings can also be obtained by those skilled in the art based on these drawings.

[0025] Figure 1 is a form diagram of a storage file of a PUT uploaded object according to the prior art of the present application;

[0026] Figure 2 is a form diagram of a storage file of a segmented uploaded object according to the prior art of the present application;

[0027] Figure 3 is a flow chart of a method for storing an object according to an embodiment of the present application;

[0028] Figure 4 is a form diagram of a storage file of a PUT uploaded object according to an embodiment of the present application;

[0029] Figure 5 is a form diagram of a storage file of a segmented uploaded object according to an embodiment of the present application;

[0030] Figure 6 is a storage process diagram of a method for storing an object according to an embodiment of the present application;

[0031] Figure 7 is a structure diagram of an apparatus for storing an object according to an embodiment of the present application;

[0032] Figure 8 is a hardware structure diagram of a computer device in which the apparatus for storing an object according to an embodiment of the present application is located. DETAILED DESCRIPTION

[0033] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with one or more embodiments of this specification. Rather, they are merely examples of apparatuses and methods consistent with some aspects of one or more embodiments of this specification as detailed in the appended claims. All other embodiments obtained by those skilled in the art based on the embodiments in this application should fall within the scope of protection of this application.

[0034] It should be noted that the steps of the corresponding methods are not necessarily performed in the order shown and described in this specification in other embodiments. In some other embodiments, the methods may include more or fewer steps than described in this specification. Furthermore, a single step described in this specification may be broken down into multiple steps in other embodiments; and multiple steps described in this specification may be combined into a single step in other embodiments.

[0035] First, let's introduce some relevant concepts involved in this application:

[0036] Object storage, also known as object-based storage, is a method for solving and processing discrete units, providing data storage services in the form of objects on a distributed system.

[0037] A bucket is a basic entity in object storage; it's a container used to hold objects. The bucket name is globally unique throughout the entire object storage system. Objects are stored as files, and their names are unique within each bucket, uniquely identifying the object.

[0038] An object is the basic entity stored in object storage. An object is a collection of data from a file and its related attribute information, including the object's key, object data, and object metadata.

[0039] A key, also called an object name, is a unique identifier for an object in a storage bucket.

[0040] Object metadata is a set of name-value pairs, including system metadata and user-defined metadata. Object metadata can be set when uploading an object, and cannot be modified afterward. The only way to modify object metadata is to create a copy of the object and set its metadata there.

[0041] Object data: The actual data of an object.

[0042] Hash value: A hash value maps a long piece of data to a shorter piece of data; this shorter piece of data is the hash value of the large piece of data. A key characteristic of hash values ​​is that they are unique; if the large piece of data changes, its hash value will change. Therefore, hash values ​​are often used to determine whether two files are identical.

[0043] Since entering the information age, humanity has witnessed the emergence of numerous cutting-edge technologies, leading to an explosive growth of data across various fields. This continuous increase in data has ushered in the era of big data. This massive amount of data has brought about challenges in data storage. People are no longer satisfied with simply preserving data intact; they also have demands regarding data retrieval speed, storage resource consumption, and other aspects.

[0044] In the era of big data, traditional data centers are transforming into cloud computing, big data, and artificial intelligence. Object storage, as a new type of storage, is undergoing continuous technological upgrades and its application scale is expanding daily. Objects in object storage are categorized into two types: objects uploaded via PUT and objects uploaded in segments.

[0045] Each object uploaded via PUT corresponds uniquely to a file. The object's metadata and data are stored in a single file. See [link / reference] Figure 1 As shown. The filename of the file uploaded by PUT is a hash value obtained based on the bucket name and object name; the file data includes object system metadata, object data, and object user metadata.

[0046] When multiple versions are enabled in a bucket, to prevent objects with the same name from being overwritten, the system will randomly generate a version number for each uploaded object. The version number is used in the calculation when storing the object, so the filename of the object uploaded by PUT is a hash value obtained based on the bucket name, object name, and version number. If multiple versions are not enabled in the bucket or multiple versions are paused, the filename of the object uploaded by PUT is a hash value obtained based on the bucket name and object name.

[0047] Files stored in object storage can also be uploaded in segments. In segmented uploads, the object file is divided into multiple segments, with each segment task having up to 10,000 segments. The size of each segment ranges from 5MB to 5GB, and the last segment can be less than 5MB. Each segment uploaded in segments is stored as an independent file with its own filename.

[0048] Segmented upload can be performed in three steps: Initialize the segment task, generating an UploadId to uniquely identify the segment task; Upload each segment, with each segment identified by a PartNumber; Merge the segments, combining all or some segments into a single object. Unselected segments will be deleted during the merge process.

[0049] It should be noted that, in order to improve upload efficiency, the uploaded segments are not merged into one file. Instead, the uploaded segments are logically combined into one file by indexing each segment using an index file.

[0050] Therefore, the objects uploaded in segments are divided into two parts, see [link / reference] Figure 2 As shown. The first part is the index file. The filename is a hash value obtained based on the bucket name, object name, and UploadId. The file data includes object metadata, object segment data information, and object user metadata. The object segment data information includes: the UploadId of the entire segment task and information about each segment, such as the segment number PartNumber, segment size, and the hash value corresponding to the segment data. The second part is the individual segment files. The filename is a hash value obtained based on the bucket name, object name, UploadId, and PartNumber. The file data is the object segment data. Since the filename of each segment data file is based on the hash value obtained from the bucket name, object name, UploadId, and PartNumber, the filename of each segment data file is unique, making it impossible to identify different segments of the same object data.

[0051] When multiple versions are enabled in a bucket, the system will randomly generate a version number for each uploaded object. The version number is used in the calculation when storing the object. The index file name is a hash value obtained based on the bucket name, object name, version number, and UploadId. The data file name is a hash value obtained based on the bucket name, object name, version number, UploadId, and PartNumber.

[0052] In summary, neither PUT uploads nor segmented uploads can distinguish files with the same data when dealing with duplicate data. This results in the repeated storage of duplicate data, making it impossible to utilize existing duplicate data, impacting the efficiency of object uploads, and wasting storage space.

[0053] This application proposes a method for storing objects, which uniquely identifies data files based on object data. This method can utilize already stored duplicate data, eliminating the need for repeated storage of duplicate data, thus improving the efficiency of object writing and reducing object storage costs. See [link to relevant documentation]. Figure 3 As shown. The method includes the following steps:

[0054] S301, Generate an index file and a data file for the object. The filename of the index file is obtained based on the object name. The data of the index file includes object metadata and the filename of the data file. The data of the data file includes object data. The filename of the data file is a unique identifier of the object data obtained based on the object data.

[0055] S302, Store the generated index file and the data file.

[0056] In this embodiment, an index file and a data file for the object are first generated. The filename of the index file is based on the object name, and the data of the index file includes object metadata and the filename of the data file. The data of the data file includes object data, and the filename of the data file is a unique identifier of the object data obtained based on the object data.

[0057] Since there are two ways to upload object files in object storage: PUT upload and segmented upload, this application adopts different implementation methods for object files uploaded using these two different methods.

[0058] For objects uploaded via PUT, first generate the object's index file and data file, see [link / reference]. Figure 4 As shown. The index file's filename is calculated based on the bucket name and object name. The index file's data includes object metadata and a data file location index, i.e., the data file's filename. The data file's filename is a unique identifier derived from the object data. This unique identifier allows direct location of the corresponding data file in the underlying storage environment. The data file's data includes the object data itself.

[0059] In some embodiments, the unique identifier obtained based on the object data is the hash value of the object data. Of course, it is understood that other methods that can uniquely identify object data are similar, and this application will not elaborate on them.

[0060] Furthermore, it is understood, but not limited to, that any algorithm that obtains a hash value from object data by any means can be applied to this object storage method. For example, algorithms such as MD5, SHA1, and SHA256 can all obtain the hash value of object data based on object data.

[0061] The object metadata of the index file uploaded via PUT using the above scheme remains unchanged; the file names are all based on the hash values ​​obtained from the bucket name and the object name.

[0062] When multiple versions are enabled in a bucket, to prevent objects with the same name from being overwritten, the system will randomly generate a version number for each uploaded object. The version number is used in the calculation when storing the object. The filename of the index file is a hash value obtained based on the bucket name, object name, and version number. The filename of the data file, after modification, is a hash value obtained based on the object data. If multiple versions are not enabled in a bucket or multiple versions are paused, the filename of the index file is a hash value obtained based on the bucket name and object name. The filename of the data file, after modification, is a hash value obtained based on the object data.

[0063] In some embodiments, to maximize the use of object attributes, the ETag value (i.e., MD5 value) of the PUT uploaded object is used as the filename of the index file.

[0064] For objects uploaded in segments, since segmented uploads inherently consist of both an index file and segment files, duplicate objects can be deleted by modifying the uploaded files. See [link to relevant documentation]. Figure 5 As shown.

[0065] The filename of the index file for segmented uploads is calculated based on the bucket name, object name, and UploadId identifier of the segmented task. The data in the segmented upload index file includes object metadata and a data file location index, i.e., the data file's filename. The data file's filename is a unique identifier derived from the object data. This unique identifier allows direct location of the underlying storage location of the data file corresponding to the object data. The data in the data file is the object data.

[0066] In some embodiments, the unique identifier obtained based on the object data is the hash value of the object data. Of course, it is understood that other methods that can uniquely identify object data are similar, and this application will not elaborate on them.

[0067] Furthermore, it is understood, but not limited to, that any algorithm that obtains a hash value from object data by any means can be applied to this object storage method. For example, algorithms such as MD5, SHA1, and SHA256 can all obtain the hash value of object data based on object data.

[0068] Furthermore, the index file also includes the identifier UploadId for the segmented task and the segmentation change PartNumber.

[0069] The object metadata remains unchanged in the segmented upload index file obtained by applying the above scheme; the file name is based on the hash value obtained from the bucket name, object name and the identifier UploadId of the segmented task.

[0070] When multiple versions are enabled in a bucket, the system will randomly generate a version number for each uploaded object. The version number is used in the calculation when storing the object. The filename of the index file is a hash value obtained based on the bucket name, object name, version number, and UploadId. The filename of the data file, after modification, is a hash value obtained based on the object data. If multiple versions are not enabled in a bucket or multiple versions are paused, the filename of the index file is a hash value obtained based on the bucket name, object name, and UploadId. The filename of the data file, after modification, is a hash value obtained based on the object data.

[0071] In some embodiments, to maximize the use of object attributes, the ETag value (i.e., MD5 value) of the segmented uploaded object is used as the filename of the index file.

[0072] After obtaining the object's index file and data file using the above method, the index file and data file are stored. Because the data in the index file includes the filename of the corresponding data file, and the filename of the data file is a unique identifier derived from the object data, the index file can directly locate the underlying storage location of the data file through the filename, thus logically forming an object from the index file and the data file.

[0073] Since the filename of a data file is a unique identifier derived from the object data, if two data files have the same object data, their filenames will also be the same. Therefore, two data files with identical object data are reduced to a single data file. Similarly, data files with identical object data are reduced to one, avoiding duplicate storage and reducing storage costs.

[0074] In some embodiments, the filename of the data file is based on the hash value of the object data obtained from the object data. Of course, it is understood that other methods that can uniquely identify object data are similar, and this application will not elaborate on them.

[0075] When storing index files and data files, the following four cases apply: 1. If no existing index file with the same name as the newly generated index file exists, and no existing data file with the same name also exists, store the newly generated index file and data file directly. 2. If no existing index file with the same name exists, but an existing data file with the same name exists, store the newly generated index file, without storing the data file repeatedly. 3. If an existing index file with the same name exists, but no existing data file exists, delete the existing index file, and then store the newly generated index file and data file. 4. If an existing index file with the same name exists, and an existing data file with the same name also exists, delete the existing index file, and then store the newly generated index file, without storing the data file repeatedly.

[0076] After storing index files and data files, the stored files will be updated. During the update process, some stored index files or data files may be deleted. Since there may be cases where one data file corresponds to multiple index files, it is necessary to obtain the number of index files corresponding to the data file.

[0077] To facilitate convenient querying of the index files corresponding to data files and to promptly delete data files without corresponding index files, one approach is to assign a value to each data file to record the number of index files corresponding to it. When a data file is first stored, it is assigned a default index count. To better track the number of index files, the index count increases by the same value when an index file is added, and decreases by one when an index file is deleted. When the number of index files corresponding to a data file falls below the default value, it indicates that no index file indexes the data file, and the data file is deleted.

[0078] Therefore, the index count of the data file is added to the data file to describe the number of index files corresponding to the data file. When the filename of the newly generated data file is the same as an already stored data file, the index count of the stored data file is incremented by one; when the filename of the newly generated data file is different from the stored data file, the newly generated data file is stored, and its index count is set to a preset value. When an object's index file is deleted, the index count of the corresponding data file is decremented by one. When the index count of a data file is less than the preset value, it indicates that the data file has not been indexed by any index file, and the data file is deleted.

[0079] Furthermore, when storing the first data file, it is stored directly, and the number of times the data file has been indexed is set to a preset value.

[0080] In some embodiments, the preset value for the number of times a data file is indexed is one.

[0081] In some embodiments, when the number of times a data file is indexed is less than a preset value, the data file is not deleted immediately. Instead, it is deleted when the operating system of the device is found to be idle in the background, thereby reducing the consumption of system resources by deleting data files in real time.

[0082] The methods for storing objects provided in this application are described in detail below.

[0083] First, an index file and a data file for the object are generated. The filename of the index file is a hash value derived from the bucket name, object name, and version number. The data in the index file includes object metadata and the filename of the data file. The data in the data file includes object data, and the filename of the data file is the hash value of the object data.

[0084] Since there are two types of objects, PUT uploaded and segmented uploaded, they will be explained separately below.

[0085] For objects uploaded via PUT, an index file and a data file are generated. The index file's filename is a hash value derived from the bucket name, object name, and version number. The index file's data consists of object metadata and a data file location index, i.e., the data file's filename. The data file's filename is the hash value of the object data, which directly locates the data file's underlying storage location; the data file's data is the object data itself.

[0086] For objects uploaded in segments, an index file and a data file are generated. The index file's filename is a hash value obtained based on the bucket name, object name, version number, and the segment task identifier UploadId. The index file's data includes object metadata, segment number PartNumber, and the data file's location index, which is the data file's filename. The data file's filename is the hash value of the object data; this hash value directly locates the data file's underlying storage location. The data file contains the object data itself.

[0087] After obtaining the object's index file and data file using the above method, store the generated index file and data file. Because the index file includes the filename of the corresponding data file, and the data file's filename is the hash value of the object's data, the index file can directly locate the underlying storage location of the data file through the data file's filename.

[0088] Furthermore, since the filename of a data file is the hash value of the object data, if two data files contain the same object data, their filenames will also be the same. Therefore, two data files with identical object data are reduced to a single data file. Similarly, the number of data files with identical object data is reduced to one, thus reducing the cost of object storage.

[0089] When storing index files and data files, the following four cases apply: 1. If no existing index file with the same name as the newly generated index file exists, and no existing data file with the same name also exists, store the newly generated index file and data file directly. 2. If no existing index file with the same name exists, but an existing data file with the same name exists, store the newly generated index file, without storing the data file repeatedly. 3. If an existing index file with the same name exists, but no existing data file exists, delete the existing index file, and then store the newly generated index file and data file. 4. If an existing index file with the same name exists, and an existing data file with the same name also exists, delete the existing index file, and then store the newly generated index file, without storing the data file repeatedly.

[0090] After storing index files and data files, the stored files will be updated. During the update process, some stored index files or data files may be deleted. To facilitate convenient querying of the number of index files corresponding to a data file, and to promptly delete data files without corresponding index files, an index count of one is assigned to the data file when it is first stored. To better count the number of index files corresponding to a data file, the index count of the data file is incremented by one when an index file corresponding to the data file is added; when an index file corresponding to a data file is deleted, the index count of the data file is decremented by one. When the number of index files corresponding to a data file is less than a preset value, it indicates that no index file indexes the data file, and the data file is deleted. However, the deletion of a data file is not immediate; instead, it is done in the background when the operating system of the device is idle, reducing the system resource consumption of real-time data file deletion.

[0091] The object storage method of this application is illustrated below with an application example. A set of objects and data x:1, y:1, z:2, x:3, y:4 is stored. The objects are stored sequentially from front to back. The entire process of storing the above objects using the method of this application is described in detail below. Figure 6 As shown; the preset value for the number of times the data file is indexed is set to 1.

[0092] First, process the first object x, generating an index file for object x and a data file with 1 data. At the same time, set the index count of the data file with 1 data to 1.

[0093] Then, the next object y is processed, generating an index file for object y and a data file containing the value 1. Object y differs from the already stored object x; its index file has not been stored before, so the newly generated index file for object y is stored directly. However, since a data file containing the value 1 has already been stored, it is not stored again. Instead, the index count of the data file containing the value 1 is incremented by one, meaning the index count of the data file containing the value 1 is now 2.

[0094] Next, process the next object z, generating an index file for object z and a data file with 2 data points. Object z differs from the already stored objects x and y; its index file has not been stored before. Since a data file with 2 data points has not been stored before, we directly store the newly generated index file and the data file with 2 data points, setting the index count in the data file with 2 data points to 1.

[0095] Next, process the next object x, generating an index file for object x and a data file with the value 3. Since an index file for object x already exists, but a data file with the value 3 has not been stored, delete the existing index file for object x. Decrement the index count of the data file with the value 1 by one, making its index count 1. Then, store the newly generated index file for object x and the data file with the value 3, setting the index count of the data file with the value 3 to 1.

[0096] Next, process the next object y, generating an index file for object y and a data file containing the value 4. Since an existing index file for object y already exists, delete it. Decrement the index count of the data file containing the value 1 by one, making its index count 0. Then, store the newly generated index file for object y. Since no data file containing the value 4 has been stored before, directly store it and set its index count to 1. Because the final index count of the data file containing the value 1 is 0, which is less than the initial preset value of 1, delete the data file containing the value 1 when the operating system of the device is detected to be idle.

[0097] The final storage results include index files for objects x, y, and z, as well as data files with data 2, 3, and 4; among them, the data files with data 2, 3, and 4 are all indexed once.

[0098] Without modifying the original storage algorithm, this application divides the original object file into two parts: an index file and a data file. The file name of the data file is set to the hash value of the object data. This supports the deletion of duplicate object data, maximizes the utilization of duplicate data in the system, improves the efficiency of object writing, and reduces the cost of object storage.

[0099] Corresponding to the above method implementation examples, the following are some examples of the above method steps in actual operation scenarios.

[0100] In the scenario of copying an object, first copy the object's index file and data file, and then store the copied index file and data file in the manner described in the above embodiments.

[0101] In the scenario of renaming an object, after modifying the object name in the filename of the object's index file, the modified index file and data file are stored in accordance with the methods described in the above embodiments.

[0102] In scenarios where object metadata is modified, after modifying the object metadata, the modified index file and data file are stored in accordance with the methods described in the above embodiments.

[0103] In the scenario of deleting an object, the index file corresponding to the object is first deleted, and the index count of the corresponding data file is decremented by one. Then, the index count of the data file is scanned in the background. If the index count of the data file is less than the preset value, the data file is deleted when the device's operating system is idle.

[0104] In the scenario of reading a stored object, the index file of the stored object is read first, and then the storage location of the data file is calculated based on the filename of the data file in the index file; the data file of the object is read from this storage location.

[0105] Corresponding to the above method embodiments, this application also provides a storage object device, see [link to relevant documentation]. Figure 7 As shown, it includes:

[0106] Processing module 710 generates an index file and a data file for the object. The filename of the index file is obtained based on the object name. The data of the index file includes object metadata and the filename of the data file. The data of the data file includes object data. The filename of the data file is a unique identifier of the object data obtained based on the object data.

[0107] Storage module 720 stores the generated index file and data file.

[0108] In addition, this application also provides an electronic device, see [link to application]. Figure 8 As shown, it includes:

[0109] processor;

[0110] Memory used to store processor-executable instructions;

[0111] The processor executes the executable instructions to implement the steps in the various embodiments of the storage object method described above.

[0112] In addition, this application also provides a computer-readable storage medium having computer instructions stored thereon, which, when executed by a processor, implement the steps in the various embodiments of the storage object method described above.

[0113] For ease of description, the above devices are described separately by function as various units. Of course, in implementing this application, the functions of each unit can be implemented in one or more software and / or hardware.

[0114] As can be seen from the above description of the embodiments, those skilled in the art can clearly understand that this application can be implemented by means of software plus necessary general-purpose hardware platforms. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in various embodiments or some parts of the embodiments of this application.

[0115] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the device embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments. The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0116] This application can be used in a wide range of general-purpose or special-purpose computing system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics devices, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, etc.

[0117] This application can be described in the general context of computer-executable instructions, such as program modules, that are executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform a specific task or implement a specific abstract data type. This application can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0118] The above description is only a specific embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. A method for storing objects, characterized in that, include: Generate an index file and a data file for the object. The filename of the index file is obtained based on the object name. The data of the index file includes object metadata and the filename of the data file. The data of the data file includes object data and the number of times the data file has been indexed. The number of times the data file has been indexed is used to describe the number of index files corresponding to the data file. The filename of the data file is a unique identifier of the object data obtained based on the object data. Store the generated index file and the data file; Where the filename of the newly generated data file is the same as the filename of the already stored data file, the index count of the already stored data file is incremented by one; When the filename of the newly generated data file is different from the filename of the already stored data file, the newly generated data file is stored, and the number of times the newly generated data file is indexed is set to a preset value. If a stored index file with the same name as the newly generated index file already exists, delete the stored index file, decrement the index count of the data file corresponding to the stored index file by one, and store the newly generated index file. If the number of times a data file is indexed is less than the preset value, the data file is deleted.

2. The method according to claim 1, characterized in that, The object is the one uploaded via PUT. The filename of the index file is obtained based on the bucket name, object name, and version number.

3. The method according to claim 1, characterized in that, The object is an object uploaded in segments; The filename of the index file is obtained based on the bucket name, object name, version number, and UploadId identifier of the segmented task; The index file also includes the segment task identifier UploadId and the segment number PartNumber.

4. The method according to claim 1, characterized in that, The unique identifier of the object data is the hash value of the object data.

5. The method according to claim 1, characterized in that, Deleting the data file includes: The data files are deleted when the operating system of the device is found to be idle.

6. A device for storing objects, characterized in that, include: The processing module generates an index file and a data file for the object. The filename of the index file is obtained based on the object name. The data of the index file includes object metadata and the filename of the data file. The data of the data file includes object data and the number of times the data file has been indexed. The number of times the data file has been indexed describes the number of index files corresponding to the data file. The filename of the data file is a unique identifier of the object data obtained based on the object data. The storage module stores the generated index file and the data file; Where the filename of the newly generated data file is the same as the filename of the already stored data file, the index count of the already stored data file is incremented by one; When the filename of the newly generated data file is different from the filename of the already stored data file, the newly generated data file is stored, and the number of times the newly generated data file is indexed is set to a preset value. If a stored index file with the same name as the newly generated index file already exists, delete the stored index file, decrement the index count of the data file corresponding to the stored index file by one, and store the newly generated index file. If the number of times a data file is indexed is less than the preset value, the data file is deleted.

7. An electronic device, comprising: processor; Memory used to store processor-executable instructions; The processor implements the method as described in any one of claims 1-5 by executing the executable instructions.

8. A computer-readable storage medium having stored thereon computer instructions that, when executed by a processor, implement the steps of the method as claimed in any one of claims 1-5.

Images