Object storage method, system and device

By introducing default column clusters and multiple extended column clusters in the object storage device, using column cluster identifiers to write data to different extended column clusters and controlling the merge operation, the problem of LSM-Tree merge operation taking up too much resources is solved, and the problem of LSM-Tree merge operation is improved. System stability.

CN113051301BInactive Publication Date: 2024-11-01CHINANETCENT TECH
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202110250266.1
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2021-03-08
Publication Date
2024-11-01
Estimated Expiration
Not applicable · inactive patent

AI Technical Summary

Technical Problem

When LSM-Tree performs merge operations, it consumes more and more IO resources, affecting normal business and causing system stability to decline.

Method used

Introduce default column clusters and multiple extended column clusters in the object storage device. Data is written into different extended column clusters through column cluster identification. Only the currently written column clusters are merged to avoid merging all data at the same time.

Benefits of technology

This greatly reduces the system resource usage during data merging and improves the overall stability of the system.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN113051301B_ABST
    Figure CN113051301B_ABST
Patent Text Reader

Abstract

The present invention discloses an object storage method, system and device, wherein the method is applied to an object storage device, wherein the object storage device includes a default column cluster and multiple extended column clusters, and the method includes: receiving a data write request sent by an object storage gateway, wherein the data write request includes key-value pair data to be written and a column cluster identifier corresponding to the key-value pair data; if a target extended column cluster having the column cluster identifier is included in the multiple extended column clusters, the mapping data of the key-value pair data is recorded in the default column cluster, and the key-value pair data is written into the target extended column cluster; wherein, among the multiple extended column clusters, only the extended column cluster currently performing data writing performs a data merge operation. The technical solution provided by the present application can effectively reduce the resources occupied during the merge operation.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of Internet technology, and in particular to an object storage method, system and device. Background Art

[0002] LSM-Tree (Log-Structured Merge Tree) is a storage engine designed for key-value pair storage. This storage engine can first store the latest data in memory, and when enough data is accumulated in the memory, it will use merge sort to merge (compact) the data in the memory and append it to the end of the queue on the hard disk. Currently, there are many databases based on LSM-Tree, such as Hbase, RocksDB, LevelDB, etc.

[0003] LSM-Tree data files are distributed by layers, and when the data in a layer reaches a certain amount, a merge operation will be automatically performed. When performing a merge operation, LSM-Tree needs to read the data to be merged, and then write the merged data into a new file. As the amount of data increases, the more IO (input and output) resources the merge operation occupies, the greater the impact on normal business will be. Summary of the invention

[0004] The purpose of the present application is to provide an object storage method, system and device that can effectively reduce the resources occupied during the merge operation.

[0005] To achieve the above-mentioned purpose, the present application provides an object storage method on the one hand, which is applied to an object storage device, wherein the object storage device includes a default column cluster and multiple extended column clusters, and the method includes: receiving a data write request sent by an object storage gateway, wherein the data write request includes key-value pair data to be written and a column cluster identifier corresponding to the key-value pair data; if a target extended column cluster with the column cluster identifier is included in the multiple extended column clusters, recording mapping data of the key-value pair data in the default column cluster, and writing the key-value pair data into the target extended column cluster; wherein, among the multiple extended column clusters, only the extended column cluster that is currently performing data writing performs a data merge operation.

[0006] To achieve the above-mentioned purpose, the present application further provides an object storage system on the other hand, which is applied to an object storage device, wherein the object storage device includes a default column cluster and multiple extended column clusters, and the object storage system includes: a request receiving unit, which is used to receive a data write request sent by an object storage gateway, wherein the data write request includes key-value pair data to be written and a column cluster identifier corresponding to the key-value pair data; a data writing unit, which is used to record mapping data of the key-value pair data in the default column cluster if a target extended column cluster with the column cluster identifier is included in the multiple extended column clusters, and write the key-value pair data into the target extended column cluster; wherein, among the multiple extended column clusters, only the extended column cluster that is currently performing data writing performs a data merge operation.

[0007] To achieve the above-mentioned purpose, the present application further provides an object storage device on the other hand, the object storage device includes a memory and a processor, the memory is used to store a computer program, and when the computer program is executed by the processor, the above-mentioned object storage method is implemented.

[0008] As can be seen from the above, the technical solution provided by one or more embodiments of the present application can include a plurality of extended column clusters in addition to the default column cluster (columnm family) in the object storage device. When receiving a data write request from a user, the key-value pair data to be written and the corresponding column cluster identifier can be identified. If there is a target extended column cluster pointed to by the column cluster identifier in multiple extended column clusters, then the actual key-value pair data can be written into the target extended column cluster, and in the default column cluster, only the mapping data of the key-value pair data can be recorded, and the actual key-value pair data can be queried in the target extended column cluster through the mapping data. In the entire object storage device, only the extended column cluster that is currently executing data writing will perform data merging operations, while other extended column clusters that have completed data writing will not perform data merging operations. In this way, different key-value pair data are written into different extended column clusters through column cluster identifiers, which can not only be compatible with the current default column cluster, but also avoid merging all written data at the same time, thereby greatly reducing the system resources occupied during data merging and improving the overall stability of the system. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0010] Figure 1is a schematic diagram of the system structure in an embodiment of the present invention;

[0011] Figure 2 is a flow chart of an object storage method in an embodiment of the present invention;

[0012] Figure 3 is a data processing flow chart within the maximum expiration period in an embodiment of the present invention;

[0013] Figure 4 It is a schematic diagram of the structure of an object storage device in an embodiment of the present invention. DETAILED DESCRIPTION

[0014] In order to make the purpose, technical solutions and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below in combination with the specific implementation methods of this application and the corresponding drawings. Obviously, the described implementation methods are only part of the implementation methods of this application, not all of the implementation methods. Based on the implementation methods in this application, all other implementation methods obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0015] See also Figure 1 The object storage method provided by this application can be applied to Figure 1 In the system architecture shown in FIG. 1 , the system architecture may include an object storage gateway (RadosGW, RGW), which may receive various requests from the user's client, and after pre-processing the requests, transmit the user's requests to the back-end object storage device (Object StorageDevice, OSD).

[0016] In object storage devices, multiple storage engines can be supported by plug-ins. For example, storage engines such as BlueStore, KStore, and FileStore can be supported. In this embodiment, KStore is used as an example for explanation. The KStore storage engine can use LSM-Tree as the storage form of data. In practical applications, among the databases developed based on LSM-Tree, RocksDB is the most commonly used one. In RocksDB, data is stored in the form of key-value pairs, and the data is first aggregated in memory and then written to disk uniformly.

[0017] In actual applications, RocksDB usually has only one default column family, but RocksDB can provide additional extended column families. According to actual scenario requirements, you can create different numbers of extended column families in RocksDB to store data through extended column families.

[0018] In a specific application example, there may be a large number of low-frequency storage objects in the object storage device. Low-frequency storage objects refer to data objects that are usually stored for a long time, but these data objects are not frequently accessed, and these data objects have the characteristics of centralized upload and centralized expiration. If such low-frequency storage objects are stored in the existing way, then due to the characteristics of centralized upload, the pressure on the object storage device during upload will be considerable. This part of data will be stored in the object storage device for a long time, and will be merged together with the data written later, which will cause the merge operation to occupy too many device resources, and the object storage device cannot provide stable read and write services.

[0019] In view of this, in one embodiment of the present application, multiple extended column clusters matching the maximum expiration period of the data to be written by the target user can be created in the object storage device in quantity. Among them, assuming that the maximum expiration period is in "days", the data to be written every day will correspond to an extended column cluster. For example, the maximum expiration period of the data to be written is 4 days, which means that the data will automatically expire and be deleted from the object storage device after 4 days from the start of writing. Then 4 corresponding extended column clusters can be created at this time, and the data of the first day can be written to the first extended column cluster, and the data of the second day can be written to the second extended column cluster, and so on. Then when the time moves to the fifth day (actually the first day of the next cycle), since the data written on the first day has expired and deleted, the first extended column cluster is in an idle state again, so the data of the fifth day can be written to the first extended column cluster. That is to say, within the same maximum expiration period, data of different unit time (the unit time can be, for example, one day as mentioned above) can be written to different extended column clusters respectively.

[0020] It should be noted that the above maximum expiration period can be for the same target user. If there is another user, another set of extended column clusters can be created for this user, and the extended column clusters of these two users can be isolated from each other and do not affect each other. For each user's extended column cluster, the same data storage standard can be used.

[0021] In this embodiment, after creating multiple extended column clusters that match the maximum expiration period in number, respective column cluster identifiers can also be set for each extended column cluster. In order to facilitate the subsequent data storage process, the column cluster identifier of the extended column cluster can be jointly set based on the above-mentioned maximum expiration period and the user identifier of the target user. Specifically, the combination of the user identifier of the target user and the upload time of the data can be used as the column cluster identifier of the extended column cluster. For example, the user identifier of the target user is B00, and the maximum expiration period is four days, then these four days can correspond to four extended column clusters. Among them, the column cluster identifier of the corresponding extended column cluster for the data uploaded on the first day can be B00_CF0001, and the column cluster identifier of the corresponding extended column cluster for the data uploaded on the second day can be B00_CF0002, and so on. The naming method of the column cluster identifier can be recorded in the object storage gateway. Later, the object storage gateway can automatically calculate the column cluster identifier corresponding to the data based on the upload time of the data and the user identifier corresponding to the data.

[0022] In the above manner, in addition to the default column cluster, the object storage device may also have multiple extended column clusters consistent with the maximum expiration period of the data to be written by the target user, and each extended column cluster may have its own column cluster identifier.

[0023] See also Figure 1 and Figure 2 , an object storage method provided by one embodiment of the present application may include the following multiple steps.

[0024] S1: receiving a data write request sent by an object storage gateway, wherein the data write request includes key-value pair data to be written and a column cluster identifier corresponding to the key-value pair data.

[0025] In this embodiment, the target user can send a storage request to the object storage gateway through the client, and the storage request can carry the user ID of the target user, the business data to be stored, and the specified upload time. Among them, the specified upload time can be the time node when the target user expects to upload the business data. In some scenarios, the storage request may not specify the upload time of the business data, then the object storage gateway can use the current moment of receiving the storage request as the upload time of the business data. Generally speaking, the format of the upload time needs to be consistent with the time representation format of the maximum expiration period. For example, if the maximum expiration period is in days, then the upload time is also in days.

[0026] In this embodiment, after receiving the storage request from the target user, the object storage gateway can identify the user ID and upload time (if not, the current moment of receiving the storage request is used as the upload time), and generate the corresponding column cluster identifier according to the naming method recorded in the previous text based on the user ID and the upload time. For example, the user ID of the target user is B00, and the upload time is the third day, then the generated column cluster identifier is B00_CF0003. It should be noted that if the upload time exceeds the time period limited by the maximum expiration cycle, the object storage gateway can automatically convert the upload time to the second or subsequent more cycles. For example, the maximum expiration cycle is 4 days, and the upload time specified in the user's storage request is the 7th day, then by comparing with the maximum expiration cycle, the object storage gateway can convert the upload time of the business data to the third day of the second cycle, and the generated column cluster identifier is B00_CF0003.

[0027] In this embodiment, after the object storage gateway generates the column cluster identifier corresponding to the user's storage request, it can construct a data write request. The data write request may include the generated column cluster identifier and the key-value pair data to be written. After generating the data write request, the object storage gateway can send the data write request to the object storage device.

[0028] S3: If the multiple extended column clusters include a target extended column cluster having the column cluster identifier, record the mapping data of the key-value pair data in the default column cluster, and write the key-value pair data into the target extended column cluster; wherein, among the multiple extended column clusters, only the extended column cluster currently performing data writing performs the data merging operation.

[0029] In this embodiment, after receiving the data write request sent by the object storage gateway, the object storage device can identify the key-value pair data and the column cluster identifier therein. Then, it can be queried whether the target extended column cluster with the column cluster identifier is included in the multiple extended column clusters created. If the target extended column cluster exists, the key-value pair data can be stored according to the upload time specified by the user. Specifically, since there is a target extended column cluster with a matching column cluster identifier in the extended column cluster, the key-value pair data in the data write request should eventually be written into the target extended column cluster. However, when RocksDB performs data storage and query, it usually first queries the data in the default column cluster. Therefore, in addition to writing the key-value pair data into the target extended column cluster, the relevant information should also be recorded in the default column cluster. Specifically, in the default column cluster, the mapping data of the key-value pair data can be recorded in the form of a key-value pair. Among them, the key data in the mapping data can be the key data in the key-value pair data, and the value data in the mapping data can be the column cluster identifier corresponding to the key-value pair data generated by the object storage gateway. Key data and value data refer to a pair of data with a mapping relationship. In this way, the key data and value data of the mapping data can be recorded in the default column family. However, in the default column family, the value data is not the actual value data of the key-value pair data, but the corresponding column family identifier.

[0030] In another embodiment, if there is no extended column cluster with the column cluster identifier among the created multiple extended column clusters, it means that the object storage device has not created a corresponding extended column cluster for the target user, or the created extended column cluster should not be used to store the key-value pair data. In this case, the key-value pair data can be directly written into the default column cluster, and the actual key-value pair data can be obtained from the default column cluster later.

[0031] After completing the data writing, the user may also send a data acquisition request, which may include the data identifier of the data to be read. After receiving the data acquisition request, the object storage gateway can extract the data identifier, construct a data read request containing the data identifier, and send the data read request to the object storage device.

[0032] After receiving the data read request from the object storage gateway, the object storage device can extract the data identifier, and then use the data identifier as the key data of the index to read the value data corresponding to the key data from the default column cluster. According to the previous description, the value data recorded in the default column cluster may be the real value data or the corresponding column cluster identifier. At this time, the read value data can be further analyzed to determine whether the value data is the real value data.

[0033] Specifically, if the structure of the read value data is consistent with the structure of the column cluster identifier of the extended column cluster, then it means that the read value data is actually the column cluster identifier, not the real value data. At this time, the actual value data can be read from the extended column cluster pointed to by the value data through the mapping relationship, and the read actual value data can be fed back to the object storage gateway to provide the real value data to the user through the object storage gateway.

[0034] If the composition of the read value data is inconsistent with the composition of the column cluster identifier of the extended column cluster, the value data is not identified as a column cluster identifier but real value data. Therefore, the value data can be fed back to the object storage gateway as actual value data to provide the real value data to the user through the object storage gateway.

[0035] In another embodiment, the user can also delete the data in the object storage device. Of course, the object storage device can also automatically delete the data according to the expiration time of the data. The following is an example of a user sending a data deletion request.

[0036] After the object storage gateway receives the data deletion request sent by the user, it can forward the data deletion request to the object storage device. The data deletion request can include the data identifier of the data to be deleted. Similar to the aforementioned data reading process, the object storage device can use the data identifier as key data to query the value data corresponding to the key data in the default column cluster. Then it can be determined whether the value data obtained by the query is consistent with the composition method of the column cluster identifier. If the composition method of the value data is inconsistent with the composition method of the column cluster identifier of the extended column cluster, it means that the value data obtained by the query is the real value data, then the key data and the corresponding value data can be directly deleted in the default column cluster. Since the actual value data is not stored in the extended column cluster at this time, there is no need to process the data in the extended column cluster.

[0037] If the value data is constructed in the same way as the column cluster identifier of the extended column cluster, it means that the column cluster identifier is actually stored in the default column cluster, and the real value data is in the extended column cluster. Then the key data and the corresponding value data can be deleted in the default column cluster, and the actual key-value pair data corresponding to the data identifier can be queried from the extended column cluster pointed to by the value data, and the actual key-value pair data can be deleted. In this way, the mapping data in the default column cluster can be deleted, and the actual key-value pair data in the extended column cluster can also be deleted, thereby completing the data deletion process.

[0038] It should be noted that when the object storage device processes the request sent by the object storage gateway, it can complete the data storage, reading and deletion process through the internal KStore storage engine and the RocksDB database. Specifically, the KStore storage engine can organize the data identifiers in the request into key data for indexing, and then pass in the key data by calling the interface of the RocksDB database, so that the value data pointed to by the key data can be obtained from the default column cluster, or the corresponding mapping data can be written to the default column cluster. Subsequently, according to the composition of the value data, the KStore storage engine can further read or delete the actual value data from the extended column cluster, or write the actual value data, thereby completing the data processing process.

[0039] See also Figure 3 In a specific application example, assuming that the maximum expiration period is 4, the data writing and deletion process and the execution process of the merge operation in the five unit time from T1 to T5 can be described as follows:

[0040] At time T1, the customer uploads object data. According to the rules, the current data will fall into B00_CF0001. As the amount of data in the column cluster increases, a merge operation will be triggered.

[0041] At T2, the data object uploaded by the customer is switched to B00_CF0002. At this time, no more data will be written to B00_CF0001. The merge operation only occurs in B00_CF0002, and the data in B00_CF0001 does not participate in the merge operation of B00_CF0002.

[0042] At T3, the data object uploaded by the customer is switched to B00_CF0003. At this time, no more data will be written to B00_CF0002, and the merge operation only occurs in B00_CF0003. When the expiration time of the data written at T1 arrives, the data deletion of B00_CF0001 will be initiated.

[0043] At T4, the data object uploaded by the customer is switched to B00_CF0004. At this time, no more data will be written to B00_CF0003, and the merge operation only occurs in B00_CF0004. The expiration time of the data written at T2 arrives, and the data deletion of B00_CF0002 will be started. The data written at T1 has been deleted, and B00_CF0001 is cleared.

[0044] When T5 reaches the next cycle, the data object uploaded by the customer is switched to B00_CF0001. At this time, no more data will be written to B00_CF0004, and the merge operation only occurs in B00_CF0001. Assuming that the expiration time of the data written at T4 is reached, the data deletion of B00_CF0004 will be started. The data at T2 has been deleted and B00_CF0002 is cleared.

[0045] From the above description, it can be seen that as time goes by, the objects uploaded by the front end will be stored in different extended column clusters (some will also be stored in the default column cluster). Overall, the data originally stored centrally is divided into different column clusters, and the historically uploaded data will not actively participate in the merge operation of the column cluster that is currently performing data writing, thereby greatly reducing the amount of data involved in the merge operation and improving the stability of the overall system.

[0046] The present application also provides an object storage system, which is applied to an object storage device, wherein the object storage device includes a default column cluster and multiple extended column clusters, and the object storage system includes:

[0047] A request receiving unit, configured to receive a data writing request sent by an object storage gateway, wherein the data writing request includes key-value pair data to be written and a column cluster identifier corresponding to the key-value pair data;

[0048] The data writing unit is configured to record mapping data of the key-value pair data in the default column cluster and write the key-value pair data into the target extended column cluster if the multiple extended column clusters include a target extended column cluster having the column cluster identifier; wherein, among the multiple extended column clusters, only the extended column cluster currently performing data writing performs the data merging operation.

[0049] See also Figure 4 The present application also provides an object storage device, which includes a memory and a processor, wherein the memory is used to store a computer program, and when the computer program is executed by the processor, the above-mentioned object storage method is implemented.

[0050] In the present application, the memory may include a physical device for storing information, which is usually a device that digitizes the information and then stores it in a medium using electrical, magnetic or optical methods. The memory may also include: a device that stores information using electrical energy, such as RAM or ROM; a device that stores information using magnetic energy, such as a hard disk, a floppy disk, a magnetic tape, a magnetic core memory, a magnetic bubble memory or a USB flash drive; a device that stores information using optical means, such as a CD or a DVD. Of course, there are other types of memory, such as quantum memory or graphene memory, etc.

[0051] In the present application, the processor may be implemented in any suitable manner. For example, the processor may take the form of a microprocessor or processor and a computer-readable medium storing a computer-readable program code (e.g., software or firmware) executable by the (micro)processor, a logic gate, a switch, an application specific integrated circuit (ASIC), a programmable logic controller, and an embedded microcontroller, etc.

[0052] As can be seen from the above, the technical solution provided by one or more embodiments of the present application can include a plurality of extended column clusters in addition to the default column cluster (columnm family) in the object storage device. When receiving a data write request from a user, the key-value pair data to be written and the corresponding column cluster identifier can be identified. If there is a target extended column cluster pointed to by the column cluster identifier in multiple extended column clusters, then the actual key-value pair data can be written into the target extended column cluster, and in the default column cluster, only the mapping data of the key-value pair data can be recorded, and the actual key-value pair data can be queried in the target extended column cluster through the mapping data. In the entire object storage device, only the extended column cluster that is currently executing data writing will perform data merging operations, while other extended column clusters that have completed data writing will not perform data merging operations. In this way, different key-value pair data are written into different extended column clusters through column cluster identifiers, which can not only be compatible with the current default column cluster, but also avoid merging all written data at the same time, thereby greatly reducing the system resources occupied during data merging and improving the overall stability of the system.

[0053] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the embodiments of the system and the device, the introduction of the embodiment of the aforementioned method can be referred to for comparative explanation.

[0054] It will be appreciated by those skilled in the art that the embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may be in the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may be in the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0055] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0056] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0057] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0058] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0059] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0060] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0061] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0062] The above is only the implementation mode of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.

Claims

1. An object storage method, characterized in that: The method is applied to an object storage device, wherein the object storage device includes a default column cluster and a plurality of extended column clusters, and the method includes: Receive a data write request sent by the object storage gateway, wherein the data write request includes key-value pair data to be written and a column cluster identifier corresponding to the key-value pair data; If the multiple extended column clusters include a target extended column cluster having the column cluster identifier, recording the mapping data of the key-value pair data in the default column cluster, and writing the key-value pair data into the target extended column cluster; wherein, among the multiple extended column clusters, only the extended column cluster currently performing data writing performs the data merging operation; Recording the mapping data of the key-value pair data in the default column cluster includes: using the key data in the key-value pair data as the key data of the mapping data, and using the column cluster identifier corresponding to the key-value pair data as the value data of the mapping data, and recording the key data and the value data of the mapping data in the default column cluster; The multiple extended column clusters are created in the following manner: a maximum expiration period of data to be written by a target user is identified, and a plurality of extended column clusters matching the maximum expiration period in number are created, and a respective column cluster identifier is set for each of the extended column clusters according to the maximum expiration period and a user identifier of the target user.

2. The method according to claim 1, characterized in that: The column cluster identifier corresponding to the key-value pair data is generated by the object storage gateway in the following manner: The object storage gateway receives a storage request sent by a user, identifies a user ID of the user and an upload time specified in the storage request, and generates a corresponding column cluster ID according to the user ID and the upload time.

3. The method according to claim 2, characterized in that If the storage request does not specify an upload time, the current time when the storage request is received is used as the upload time.

4. The method according to claim 1, characterized in that: The method further comprises: If there is no extended column cluster having the column cluster identifier in the multiple extended column clusters, the key-value pair data is written into the default column cluster.

5. The method according to claim 1, characterized in that The method further comprises: Receiving a data read request sent by the object storage gateway, wherein the data read request includes a data identifier of the data to be read; Using the data identifier as key data, reading value data corresponding to the key data from the default column cluster; If the composition mode of the value data is consistent with the composition mode of the column cluster identifier of the extended column cluster, the actual value data is read from the extended column cluster pointed to by the value data, and the read actual value data is fed back to the object storage gateway.

6. The method according to claim 5, characterized in that The method further comprises: If the composition mode of the value data is inconsistent with the composition mode of the column cluster identifier of the extended column cluster, the value data is fed back to the object storage gateway as actual value data.

7. The method according to claim 1, characterized in that The method further comprises: Receiving a data deletion request sent by the object storage gateway, wherein the data deletion request includes a data identifier of the data to be deleted; Using the data identifier as key data, querying the value data corresponding to the key data in the default column cluster; If the composition mode of the value data is inconsistent with the composition mode of the column cluster identifier of the extended column cluster, the key data and the corresponding value data are deleted in the default column cluster.

8. The method according to claim 7, characterized in that The method further comprises: If the composition mode of the value data is consistent with the composition mode of the column cluster identifier of the extended column cluster, the key data and the corresponding value data are deleted in the default column cluster, and actual key-value pair data corresponding to the data identifier is queried from the extended column cluster pointed to by the value data, and the actual key-value pair data is deleted.

9. An object storage system, characterized in that: The object storage system is applied to an object storage device, the object storage device includes a default column cluster and multiple extended column clusters, and the object storage system includes: A request receiving unit, configured to receive a data writing request sent by an object storage gateway, wherein the data writing request includes key-value pair data to be written and a column cluster identifier corresponding to the key-value pair data; A data writing unit, configured to record mapping data of the key-value pair data in the default column cluster and write the key-value pair data into the target extended column cluster if the multiple extended column clusters include a target extended column cluster having the column cluster identifier; wherein, among the multiple extended column clusters, only the extended column cluster currently performing data writing performs a data merging operation, and recording the mapping data of the key-value pair data in the default column cluster includes: using key data in the key-value pair data as key data of the mapping data, and using the column cluster identifier corresponding to the key-value pair data as value data of the mapping data, and recording the key data and value data of the mapping data in the default column cluster; the multiple extended column clusters are created in the following manner: identifying a maximum expiration period of data to be written by a target user, creating multiple extended column clusters matching the maximum expiration period in number, and setting a respective column cluster identifier for each of the extended column clusters according to the maximum expiration period and the user identifier of the target user.

10. An object storage device, characterized in that: The object storage device comprises a memory and a processor, wherein the memory is used to store a computer program, and when the computer program is executed by the processor, the method according to any one of claims 1 to 8 is implemented.