Snapshot object processing method and device, equipment and medium

Abstract

The embodiment of the invention provides a snapshot object processing method and device, equipment and a medium, and the method comprises the steps: receiving an access request which carries to-be-modified data and a snapshot identifier, obtaining a snapshot object and a forward relation of the snapshot object according to the snapshot identifier, and processing the snapshot object according to the forward relation. The forward relationship is used for pointing to the internal object corresponding to the snapshot object, creating a new snapshot object according to the to-be-modified data, and setting the forward relationship of the new snapshot object to point to the modified internal object, so that unnecessary total cloning is avoided, disk resource consumption is reduced, and the data modification efficiency is improved. And the influence on the front-end service performance is reduced.

Description

Technical Field

[0001] The present invention belongs to the field of storage technology, and in particular relates to a snapshot object processing method, device, equipment and medium. Background Art

[0002] With the rapid development of the Internet, the amount of Internet data is becoming larger and larger, and efficient data storage has become a core issue. In the current distributed all-flash system, the snapshot cloning technology has the following problems:

[0003] Each time a snapshot is updated, a full clone operation including reference relationships and data must be performed, which directly increases the write latency of the operation. This full clone operation not only increases the burden on the storage system, but also occupies a large amount of I / O resources, which causes performance fluctuations in the front-end business. In addition, since each snapshot update will clone the reference relationship, a large amount of reference relationship data will accumulate in the system. During the garbage collection process, the system needs to traverse and check each reference relationship to determine the validity of the data. This process will cause a huge consumption of disk resources and computing performance. Summary of the invention

[0004] In view of the related problems in the above background technology, the embodiments of the present invention are proposed to provide a snapshot object processing method, apparatus, device and medium that overcome the above problems or at least partially solve the above problems.

[0005] In order to solve the above problem, an embodiment of the present invention discloses a snapshot object processing method, the method comprising:

[0006] receiving an access request, wherein the access request carries the data to be modified and a snapshot identifier;

[0007] According to the snapshot identifier, acquiring a snapshot object and a forward relationship between the snapshot object, wherein the forward relationship is used to point to an internal object corresponding to the snapshot object;

[0008] Create a new snapshot object according to the data to be modified;

[0009] The forward relationship of the new snapshot object is set to point to the modified internal object.

[0010] Optionally, creating a new snapshot object according to the data to be modified includes:

[0011] When the snapshot identifier of the snapshot object is different from the snapshot identifier carried in the access request, a new snapshot object is created according to the snapshot identifier carried in the access request and the data to be modified, and the snapshot identifier carried in the access request is used as the snapshot identifier of the new snapshot object.

[0012] Optionally, the method further comprises:

[0013] When the snapshot identifier of the snapshot object is the same as the snapshot identifier carried in the access request, writing the data to be modified into the internal object;

[0014] The forward relationship of the snapshot object is set to point to the modified internal object.

[0015] Optionally, the method further comprises:

[0016] Acquire a reverse relationship between the modified internal object corresponding to the new snapshot object and the modified internal object corresponding to the new snapshot object, wherein the reverse relationship is used to point to the new snapshot object corresponding to the modified internal object corresponding to the new snapshot object;

[0017] The data of the internal object is modified according to the data to be modified, and the reverse relationship of the internal object is set to point to the new snapshot object.

[0018] Optionally, the method further comprises:

[0019] Acquire the reverse relationship between the modified internal object corresponding to the snapshot object and the modified internal object corresponding to the snapshot object, wherein the reverse relationship is used to point to the snapshot object corresponding to the modified internal object corresponding to the snapshot object;

[0020] The data of the internal object is modified according to the data to be modified, and the reverse relationship of the modified internal object corresponding to the snapshot object is set to point to the snapshot object.

[0021] Optionally, the method further comprises:

[0022] When the data storage pool reaches a preset storage threshold, a target internal object that needs to be recycled is determined, and a reverse relationship of the target internal object is obtained;

[0023] According to the reverse relationship of the target internal object, acquiring the target snapshot object corresponding to the target internal object and the forward relationship of the target snapshot object;

[0024] Performing positive verification on the data of the target internal object according to the positive relationship of the target snapshot object;

[0025] Reversely verify the data of the target internal object according to the reverse relationship of the target internal object;

[0026] Data that can pass both forward verification and reverse verification is taken as valid data, the valid data is aggregated into a new internal object, the reverse relationship of the new internal object is set to point to the snapshot object corresponding to the target internal object; and the forward relationship of the target snapshot object is set to point to the new internal object.

[0027] Optionally, the method further includes:

[0028] When the valid data cannot be aggregated into a new internal object, return to the step of acquiring a target snapshot object corresponding to the target internal object and a forward relationship of the target snapshot object according to a reverse relationship of the target internal object until the valid data is aggregated into a new internal object.

[0029] On the other hand, an embodiment of the present invention further discloses a snapshot object processing device, the device comprising:

[0030] A receiving module, used for receiving an access request, wherein the access request carries the data to be modified and a snapshot identifier;

[0031] A first forward relationship acquisition module, used for acquiring a snapshot object and a forward relationship between the snapshot object according to the snapshot identifier, wherein the forward relationship is used for pointing to an internal object corresponding to the snapshot object;

[0032] A creation module, used for creating a new snapshot object according to the data to be modified;

[0033] The first forward relationship processing module is used to set the forward relationship of the new snapshot object to point to the modified internal object.

[0034] Optionally, the creation module includes:

[0035] A snapshot object creation submodule is used to create a new snapshot object according to the snapshot identifier carried in the access request and the data to be modified when the snapshot identifier of the snapshot object is different from the snapshot identifier carried in the access request, and use the snapshot identifier carried in the access request as the snapshot identifier of the new snapshot object.

[0036] Optionally, the device further comprises:

[0037] A modification module, configured to write the to-be-modified data into the internal object when the snapshot identifier of the snapshot object is the same as the snapshot identifier carried in the access request;

[0038] The second forward relationship processing module is used to set the forward relationship of the snapshot object to point to the modified internal object.

[0039] Optionally, the device further comprises:

[0040] A first reverse relationship acquisition module, used for acquiring a reverse relationship between the modified internal object corresponding to the new snapshot object and the modified internal object corresponding to the new snapshot object, wherein the reverse relationship is used for pointing to the new snapshot object corresponding to the modified internal object corresponding to the new snapshot object;

[0041] The first reverse relationship processing module is used to modify the data of the internal object according to the data to be modified, and set the reverse relationship of the internal object to point to the new snapshot object.

[0042] Optionally, the device further comprises:

[0043] A second reverse relationship acquisition module, used for acquiring the reverse relationship between the modified internal object corresponding to the snapshot object and the modified internal object corresponding to the snapshot object, wherein the reverse relationship is used for pointing to the snapshot object corresponding to the modified internal object corresponding to the snapshot object;

[0044] The second reverse relationship processing module is used to modify the data of the internal object according to the data to be modified, and set the reverse relationship of the modified internal object corresponding to the snapshot object to point to the snapshot object.

[0045] Optionally, the device further comprises:

[0046] A third reverse relationship acquisition module is used to determine a target internal object that needs to be recycled when the data storage pool reaches a preset storage threshold, and to acquire a reverse relationship of the target internal object;

[0047] A third forward relationship acquisition module, configured to acquire a target snapshot object corresponding to the target internal object and a forward relationship of the target snapshot object according to a reverse relationship of the target internal object;

[0048] A positive verification module, used for performing positive verification on the data of the target internal object according to the positive relationship of the target snapshot object;

[0049] A reverse verification module, used for performing reverse verification on the data of the target internal object according to the reverse relationship of the target internal object;

[0050] A recycling processing module is used to take data that can pass both forward verification and reverse verification as valid data, aggregate the valid data into a new internal object, set the reverse relationship of the new internal object to point to the snapshot object corresponding to the target internal object; and set the forward relationship of the target snapshot object to point to the new internal object.

[0051] Optionally, the device further comprises:

[0052] The aggregation processing module is used to return to the step of obtaining the target snapshot object corresponding to the target internal object and the forward relationship of the target snapshot object according to the reverse relationship of the target internal object when the valid data cannot be aggregated into a new internal object, until the valid data is aggregated into a new internal object.

[0053] Accordingly, an embodiment of the present invention discloses an electronic device, including: a processor, a memory, and a computer program stored in the memory and capable of running on the processor, wherein the computer program implements the various steps of the above-mentioned snapshot object processing method embodiment when executed by the processor.

[0054] Accordingly, an embodiment of the present invention discloses a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, each step of the above-mentioned snapshot object processing method embodiment is implemented.

[0055] The embodiments of the present invention include the following advantages:

[0056] In the embodiment of the present invention, when updating a snapshot, instead of performing a full clone operation including reference relationships and data, a new snapshot object is created, the forward relationship of the new snapshot object is set to point to the modified internal object, and data is written to the internal object and the reference relationship is updated. This method avoids unnecessary full clones, thereby reducing write latency and the impact on front-end business performance, and since unnecessary full clone operations are reduced, redundant reference relationships accumulated in the system are also reduced. During garbage collection, the number of reference relationships that the system needs to traverse and check will be greatly reduced, thereby reducing disk resource consumption and computing performance consumption. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] Figure 1 is a flowchart of a method for processing a snapshot object according to an embodiment of the present invention;

[0058] Figure 2 is a flowchart of another embodiment of a snapshot object processing method of the present invention;

[0059] Figure 3 It is a schematic diagram of a storage pool snapshot processing mechanism of an embodiment of a snapshot processing method of the present invention;

[0060] Figure 4 is a cloning flow chart of an embodiment of a snapshot processing method of the present invention;

[0061] Figure 5 is a garbage collection flow chart of an embodiment of a snapshot processing method of the present invention;

[0062] Figure 6is a structural block diagram of a snapshot object processing device provided by an embodiment of the present invention;

[0063] Figure 7 is a structural block diagram of an electronic device provided by an embodiment of the present invention;

[0064] Figure 8 It is a structural block diagram of a computer-readable storage medium provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0065] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0066] One of the core concepts of an embodiment of the present invention is that, when updating a snapshot, instead of performing a full clone operation including reference relationships and data, a new object is created and data is written internally and reference relationships are updated. In this way, unnecessary full clones are reduced, thereby achieving the effect of reducing latency and alleviating the impact on front-end performance.

[0067] Reference Figure 1 , shows a flowchart of a snapshot object processing method embodiment of the present invention, which may specifically include the following steps:

[0068] Step 101, receiving an access request, wherein the access request carries the data to be modified and a snapshot identifier;

[0069] Under the existing technical framework, the distributed all-flash system usually uses a front-end and back-end separation architecture. The storage system at the back end needs to promptly receive and process access requests from the front end. The access requests include write, read, delete and other operations. The embodiments of the present invention are mainly used to process write operations in access requests.

[0070] A distributed all-flash system is a distributed storage system that uses all-flash storage technology and is used in scenarios such as virtualization platforms or big data that have high requirements for storage systems. Common flash memories include NAND (Not AND Flash) flash memories used for data storage, which will not be described here.

[0071] Step 102, obtaining a snapshot object and a forward relationship between the snapshot object according to the snapshot identifier;

[0072] After obtaining the access request, the access request is parsed to obtain the data to be modified and the snapshot identifier carried in the access request, and the snapshot object corresponding to the access request and the forward relationship of the snapshot object are obtained according to the correspondence between the preset snapshot identifier and the snapshot object;

[0073] A forward relationship indicates a relationship from a source object to a target object. In this embodiment, the forward relationship of the snapshot object indicates a relationship from a snapshot object to an internal object. The internal object of the snapshot object can be determined through the forward relationship of the snapshot object. The internal object is a data aggregation object within the storage pool.

[0074] Step 103: Create a new snapshot object according to the data to be modified.

[0075] Create new snapshot objects based on business needs and preset snapshot object creation rules.

[0076] Step 104: Set the forward relationship of the new snapshot object to point to the modified internal object.

[0077] When the data to be modified is written into the internal object, the storage system will modify the forward relationship of the new snapshot object so that the new snapshot object can correctly and completely reference the corresponding internal object;

[0078] In the embodiment of the present invention, when updating a snapshot, instead of performing a full clone operation including reference relationships and data, a new snapshot object is created, the forward relationship of the new snapshot object is set to point to the modified internal object, and data is written to the internal object and the reference relationship is updated. This method avoids unnecessary full clones, thereby reducing write latency and the impact on front-end business performance, and since unnecessary full clone operations are reduced, redundant reference relationships accumulated in the system are also reduced. During garbage collection, the number of reference relationships that the system needs to traverse and check will be greatly reduced, thereby reducing disk resource consumption and computing performance consumption.

[0079] refer to Figure 2 , shows a flowchart of another embodiment of a snapshot object processing method of the present invention, which may specifically include the following steps:

[0080] Step 201, receiving an access request, wherein the access request carries the data to be modified and a snapshot identifier;

[0081] Under the existing technical framework, the distributed all-flash system usually uses a front-end and back-end separation architecture. The storage system at the back end needs to promptly receive and process access requests from the front end. The access requests include write, read, delete and other operations. The embodiments of the present invention are mainly used to process write operations in access requests.

[0082] A distributed all-flash system is a distributed storage system that uses all-flash storage technology and is used in scenarios such as virtualization platforms or big data that have high requirements for storage systems. Common flash memories include NAND (Not AND Flash) flash memories used for data storage, which will not be described here.

[0083] Step 202, obtaining a snapshot object and a forward relationship between the snapshot object according to the snapshot identifier;

[0084] After obtaining the access request, the access request is parsed to obtain the data to be modified and the snapshot identifier carried in the access request, and the snapshot object corresponding to the access request and the forward relationship of the snapshot object are obtained according to the correspondence between the preset snapshot identifier and the snapshot object;

[0085] A forward relationship indicates a relationship from a source object to a target object. In this embodiment, the forward relationship of the snapshot object indicates a relationship from a snapshot object to an internal object. The internal object of the snapshot object can be determined through the forward relationship of the snapshot object. The internal object is a data aggregation object within the storage pool.

[0086] Step 203, when the snapshot identifier of the snapshot object is different from the snapshot identifier carried in the access request, a new snapshot object is created according to the snapshot identifier carried in the access request and the data to be modified, and the snapshot identifier carried in the access request is used as the snapshot identifier of the new snapshot object.

[0087] When creating a new snapshot object, it is necessary to first compare the snapshot identifiers; illustratively, the snapshot identifier of the snapshot object may be compared with the snapshot identifier carried in the access request.

[0088] When the two snapshot identifiers are inconsistent, the system will create a new snapshot object. At this time, the snapshot identifier carried in the access request will be used as the snapshot identifier of the new snapshot object. This snapshot object is used to record the modification information of the internal object corresponding to the snapshot object under the snapshot identifier. The modification information can specify relevant information standards according to business needs, and this embodiment does not limit this.

[0089] Step 204: Set the forward relationship of the new snapshot object to point to the modified internal object.

[0090] When the data to be modified is written into the internal object, the storage system will modify the forward relationship of the new snapshot object so that the new snapshot object can correctly and completely reference the corresponding internal object;

[0091] In the above embodiment, when the snapshot identifier changes, unlike the general snapshot object cloning process, a new snapshot object is created with the snapshot identifier carried by the access request to store necessary modification information, such as the snapshot identifier and related modification records. This method avoids full cloning and saves system resources.

[0092] In one embodiment, the method further comprises:

[0093] When the snapshot identifier of the snapshot object is the same as the snapshot identifier carried in the access request, writing the data to be modified into the internal object, and setting the forward relationship of the snapshot object to point to the modified internal object;

[0094] When the two snapshot identifiers are consistent, there is no need to create a snapshot object. At this time, the system will obtain the data to be modified carried in the access request and write it into the internal object. The writing process can be adjusted and optimized according to business needs, such as cache strategy optimization and security and consistency optimization.

[0095] In one example, a cache area of ​​reasonable size can be maintained locally based on business needs and performance goals to store recently accessed or modified data objects; the cache type of the cache area can be set to one of write-back cache, write-through cache, and write-through cache based on business needs; the cache strategy can also be selected as the least recently used strategy or the least frequently used strategy; an intelligent selection scheme can also be designed based on historical access patterns and current load conditions to identify data objects that are about to be accessed and place them in the cache. At this time, when the front-end access request needs to request to write data, the storage system first checks whether the data already exists in the cache. If the data is already cached, the data in the cache is directly updated to reduce disk I / O operations.

[0096] In another example, when data is written to internal objects, operations such as checksum verification and cyclic redundancy check can be performed on the data to ensure the integrity, consistency and correctness of the data. Based on business needs, sensitive data can be encrypted for privacy protection, access control and security auditing can be implemented, important data can be backed up regularly, and disaster recovery plans can be developed to prevent data loss or damage.

[0097] When the data to be modified is written into the internal object, the storage system will modify the forward relationship of the snapshot object so that the snapshot object can correctly and completely reference the corresponding internal object;

[0098] In the prior art, a database usually stores multiple forward relationships for snapshot objects of the same business type, which places a heavy burden on the database. In the present embodiment, each time the access request is processed, the forward relationship is not copied but directly modified, so that the database only stores one forward relationship for snapshot objects of the same business type, thereby reducing database pressure and improving storage efficiency.

[0099] In one embodiment, the method further comprises:

[0100] Obtain a reverse relationship between the modified internal object corresponding to the new snapshot object and the modified internal object corresponding to the new snapshot object, wherein the reverse relationship is used to point to the new snapshot object corresponding to the modified internal object corresponding to the new snapshot object; modify the data of the internal object according to the data to be modified, and set the reverse relationship of the internal object to point to the new snapshot object.

[0101] After finding the modified internal object corresponding to the new snapshot object based on the forward relationship of the new snapshot object, the storage system obtains the reverse relationship of the modified internal object; the reverse relationship is opposite to the forward relationship, and refers to the relationship from the target object to the source object. In this embodiment, the reverse relationship of the modified internal object refers to the relationship from the modified internal object to the new snapshot object. After determining the modified internal object, the reverse relationship of the modified internal object is updated so that the reverse relationship of the modified internal object can correctly and completely point to the new snapshot object.

[0102] Different from the implicit reverse relationship in general databases and programming architectures, the reverse relationship in this embodiment is explicitly defined in the new snapshot object, which makes it easier for the system to locate and operate the required reverse relationship when the reverse relationship is needed.

[0103] In one embodiment, the method further comprises:

[0104] Obtain the modified internal object corresponding to the snapshot object and the reverse relationship of the modified internal object corresponding to the snapshot object, wherein the reverse relationship is used to point to the snapshot object corresponding to the modified internal object corresponding to the snapshot object; modify the data of the internal object according to the data to be modified, and set the reverse relationship of the modified internal object corresponding to the snapshot object to point to the snapshot object.

[0105] The storage system finds the modified internal object corresponding to the snapshot object based on the forward relationship of the snapshot object, and then obtains the reverse relationship of the modified internal object, where the reverse relationship of the modified internal object refers to a relationship from the modified internal object to the snapshot object;

[0106] In this embodiment, the update strategy of the reverse relationship is to directly modify the existing reverse relationship stored in the snapshot object based on the modified content. Therefore, the modified internal object will not be repeatedly referenced by multiple snapshot objects of the same business type, simplifying the reverse relationship structure of the internal object.

[0107] In one embodiment, the method further comprises:

[0108] When the data storage pool reaches a preset storage threshold, determine the target internal object that needs to be recycled, and obtain the reverse relationship of the target internal object; obtain the target snapshot object corresponding to the target internal object and the forward relationship of the target snapshot object based on the reverse relationship of the target internal object; perform positive verification on the data of the target internal object based on the forward relationship of the target snapshot object; perform reverse verification on the data of the target internal object based on the reverse relationship of the target internal object; take the data that can pass both the forward verification and the reverse verification as valid data, aggregate the valid data into a new internal object, set the reverse relationship of the new internal object to point to the snapshot object corresponding to the target internal object; and set the forward relationship of the target snapshot object to point to the new internal object.

[0109] When the data storage pool of the storage system reaches the preset storage threshold, the system triggers a garbage collection operation; at this time, based on the preset rules, the target object that needs to be recycled for this garbage collection operation is determined, and the reverse relationship of the target object is obtained; based on the forward relationship of the target snapshot object, the data of the target internal object is checked to see if it conforms to the preset logic; illustratively, the preset rules may include data type verification, value range check, reference integrity verification, etc. Based on the reverse relationship of the target internal object, check whether the reference from the target internal object to the target snapshot object is still valid; illustratively, the check content may include whether the state of the target snapshot object matches the state of the target internal object, etc.; aggregate the verified data into a new internal object, and adaptively update the reverse relationship of the new internal object and the forward relationship of the target snapshot object, so that the target snapshot object can correctly and completely reference the corresponding internal object, and the new internal object can correctly and completely reference the corresponding snapshot object.

[0110] This embodiment ensures the forward consistency and reverse consistency of the data to be modified by performing forward and reverse checks on the data to be modified, so that the data inside the aggregated internal objects are all valid data, and by directly obtaining the reverse relationship without traversal, the garbage collection efficiency of the system is improved and the consumption of system resources is reduced.

[0111] In one embodiment, the method further comprises:

[0112] When the valid data cannot be aggregated into a new internal object, return to the step of acquiring a target snapshot object corresponding to the target internal object and a forward relationship of the target snapshot object according to a reverse relationship of the target internal object until the valid data is aggregated into a new internal object.

[0113] When the valid data cannot be aggregated into a new internal object based on the preset aggregation rule, return to the step of reading the reverse relationship of the target internal object, obtaining the target snapshot object corresponding to the target internal object and the forward relationship of the target snapshot object, until the valid data can meet the preset aggregation rule and be aggregated into a new internal object;

[0114] In this embodiment, by presetting aggregation rules and taking loop measures when the rules are not met, the data quality of the internal objects is guaranteed and the stability of the storage system is improved.

[0115] In the embodiment of the present invention, when updating a snapshot, instead of performing a full clone operation including reference relationships and data, a new snapshot object is created, the forward relationship of the new snapshot object is set to point to the modified internal object, and data is written to the internal object and the reference relationship is updated. This method avoids unnecessary full clones, thereby reducing write latency and the impact on front-end business performance, and since unnecessary full clone operations are reduced, redundant reference relationships accumulated in the system are also reduced. During garbage collection, the number of reference relationships that the system needs to traverse and check will be greatly reduced, thereby reducing disk resource consumption and computing performance consumption.

[0116] Reference Figure 3 , showing a schematic diagram of a storage pool snapshot processing mechanism according to an embodiment of a snapshot processing method of the present invention:

[0117] The top layer in the figure is the object layer (ObjA): ObjA.0, ObjA.1 and ObjA.2 in this layer are head objects under different snapshot versions. When a new snapshot is generated, the original head object is no longer cloned, but a new head object with the snapshot number is created.

[0118] The figure also includes the cache pool, where L0, L1, L2, and L3 are cache items, which are connected to the data items in the data pool through positive relationships. In the snapshot processing mechanism, these cache items will perform corresponding data caching operations according to different snapshot versions.

[0119] The diagram also includes the data pool, where A, B, C, D, BB, and DD are data items that are connected to cache items in the cache pool through positive relationships. The storage and management of data for these data items in different snapshot versions is an important part of the snapshot processing mechanism.

[0120] For example, L0->P0 means that in a specific snapshot version, the data item corresponding to cache item L0 is P0. This correspondence will be different in different snapshot versions.

[0121] For example, ObjA.0 is connected to L0, L1, L2, and L3 through a forward relationship, indicating that these cache items are associated with the head object under the initial snapshot version (ObjA.0). When new snapshots are generated (such as ObjA.1 and ObjA.2), the new head object will establish a relationship with the corresponding cache items based on the new snapshot operation.

[0122] When a new snapshot is generated, the original header object (such as ObjA.0) is no longer cloned, but a new header object with a snapshot number (such as ObjA.1 and ObjA.2) is created. Subsequent modification and write operations for the snapshot number will be written to the new header object.

[0123] Reference Figure 4 , showing a cloning flow chart of a snapshot processing method embodiment of the present invention;

[0124] First, the process starts when the storage pool receives a request from the client to modify an object. After receiving the request, the storage pool first reads the snapshot number stored locally on the object.

[0125] After reading the local snapshot number, the storage pool will verify whether the snapshot number carried in the request is consistent with the locally saved snapshot number. If the snapshot number has not changed, the modified data and the reverse relationship will be directly written to the internal object, and then the current object will be updated to the internal object forward relationship, and finally a success will be returned to the client.

[0126] When it is determined that the snapshot number carried in the request has been updated, the storage pool will create a new object named after the new snapshot number. After the new object is created, the internal object will point to the newly created object when writing data and reverse relations. Then, the forward relation of the new object is updated to point to the internal object data. Finally, regardless of whether the snapshot number has changed, the processing flow will return a successful response to the client.

[0127] Reference Figure 5 , showing a garbage collection flow chart of an embodiment of a snapshot processing method of the present invention;

[0128] First, when the storage pool water level reaches the GC (Garbage Collection) water level, the garbage collection process is started. When the storage pool water level reaches the GC water level, first check whether there is data redundancy.

[0129] Select an internal object for processing. After selecting an internal object, the reverse relationship of the internal object is read first. Based on the read reverse relationship, check the business object pointed to in the reverse relationship to determine whether the data is valid. If the data is invalid, reselect the internal object for processing. If the data is valid, read the data of the data slice. After reading the data of the data slice, check whether these valid data can be aggregated into an internal object. If not, reselect the internal object for processing. If yes, write the valid data to the internal object and update the reverse relationship. Then, update the forward relationship.

[0130] It should be noted that, for the sake of simplicity, the method embodiments are described as a series of action combinations, but those skilled in the art should be aware that the embodiments of the present invention are not limited by the order of the actions described, because according to the embodiments of the present invention, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily required by the embodiments of the present invention.

[0131] Reference Figure 6 , shows a structural block diagram of an embodiment of a snapshot object processing device of the present invention, which may specifically include the following modules:

[0132] The receiving module 301 is used to receive an access request, wherein the access request carries the data to be modified and a snapshot identifier;

[0133] A first forward relationship acquisition module 302, configured to acquire a snapshot object and a forward relationship between the snapshot object according to the snapshot identifier, wherein the forward relationship is used to point to an internal object corresponding to the snapshot object;

[0134] A creation module 303, used to create a new snapshot object according to the data to be modified;

[0135] The first forward relationship processing module 304 is configured to set the forward relationship of the new snapshot object to point to the modified internal object.

[0136] In the process of creating a new snapshot object in this embodiment, unlike the method of fully cloning the forward relationship of the snapshot object in the prior art, the forward relationship of the original snapshot object is not fully loaded and written, thereby reducing the impact on the front-end performance.

[0137] Optionally, the creation module includes:

[0138] A snapshot object creation submodule is used to create a new snapshot object according to the snapshot identifier carried in the access request and the data to be modified when the snapshot identifier of the snapshot object is different from the snapshot identifier carried in the access request, and use the snapshot identifier carried in the access request as the snapshot identifier of the new snapshot object.

[0139] In the above embodiment, when the snapshot identifier changes, unlike the general snapshot object cloning process, a new snapshot object is created with the snapshot identifier carried by the access request to store necessary modification information, such as the snapshot identifier and related modification records. This method avoids full cloning and saves system resources.

[0140] Optionally, the device further comprises:

[0141] A modification module, configured to write the to-be-modified data into the internal object when the snapshot identifier of the snapshot object is the same as the snapshot identifier carried in the access request;

[0142] The second forward relationship processing module is used to set the forward relationship of the snapshot object to point to the modified internal object.

[0143] In this embodiment, because the forward relationship is not copied but directly modified each time the access request is processed, the database only stores one forward relationship for the snapshot object of the same business type, which reduces database pressure and improves storage efficiency.

[0144] Optionally, the device further comprises:

[0145] A first reverse relationship acquisition module, used for acquiring a reverse relationship between the modified internal object corresponding to the new snapshot object and the modified internal object corresponding to the new snapshot object, wherein the reverse relationship is used for pointing to the new snapshot object corresponding to the modified internal object corresponding to the new snapshot object;

[0146] The first reverse relationship processing module is used to modify the data of the internal object according to the data to be modified, and set the reverse relationship of the internal object to point to the new snapshot object.

[0147] The reverse relationship in this embodiment is explicitly defined in the new snapshot object, which makes it easier for the system to locate and operate the required reverse relationship when the reverse relationship needs to be used.

[0148] In one embodiment, the device further comprises:

[0149] A second reverse relationship acquisition module, used for acquiring the reverse relationship between the modified internal object corresponding to the snapshot object and the modified internal object corresponding to the snapshot object, wherein the reverse relationship is used for pointing to the snapshot object corresponding to the modified internal object corresponding to the snapshot object;

[0150] The second reverse relationship processing module is used to modify the data of the internal object according to the data to be modified, and set the reverse relationship of the modified internal object corresponding to the snapshot object to point to the snapshot object.

[0151] In this embodiment, the update strategy of the reverse relationship is to directly modify the existing reverse relationship stored in the snapshot object based on the modified content. Therefore, the modified internal object will not be repeatedly referenced by multiple snapshot objects of the same business type, simplifying the reverse relationship structure of the internal object.

[0152] In one embodiment, the device further comprises:

[0153] A third reverse relationship acquisition module is used to determine a target internal object that needs to be recycled when the data storage pool reaches a preset storage threshold, and to acquire a reverse relationship of the target internal object;

[0154] A third forward relationship acquisition module, configured to acquire a target snapshot object corresponding to the target internal object and a forward relationship of the target snapshot object according to a reverse relationship of the target internal object;

[0155] A positive verification module, used for performing positive verification on the data of the target internal object according to the positive relationship of the target snapshot object;

[0156] A reverse verification module, used for performing reverse verification on the data of the target internal object according to the reverse relationship of the target internal object;

[0157] A recycling processing module is used to take data that can pass both forward verification and reverse verification as valid data, aggregate the valid data into a new internal object, set the reverse relationship of the new internal object to point to the snapshot object corresponding to the target internal object; and set the forward relationship of the target snapshot object to point to the new internal object.

[0158] This embodiment ensures the forward consistency and reverse consistency of the data to be modified by performing forward and reverse checks on the data to be modified, so that the data inside the aggregated internal objects are all valid data, and by directly obtaining the reverse relationship without traversal, the garbage collection efficiency of the system is improved and the consumption of system resources is reduced.

[0159] In one embodiment, the device further comprises:

[0160] The aggregation processing module is used to return to the step of obtaining the target snapshot object corresponding to the target internal object and the forward relationship of the target snapshot object according to the reverse relationship of the target internal object when the valid data cannot be aggregated into a new internal object, until the valid data is aggregated into a new internal object.

[0161] In this embodiment, by presetting aggregation rules and taking loop measures when the rules are not met, the data quality of the internal objects is guaranteed and the stability of the storage system is improved.

[0162] In the embodiment of the present invention, when updating a snapshot, instead of performing a full clone operation including reference relationships and data, a new snapshot object is created, the forward relationship of the new snapshot object is set to point to the modified internal object, and data is written to the internal object and the reference relationship is updated. This method avoids unnecessary full clones, thereby reducing write latency and the impact on front-end business performance, and since unnecessary full clone operations are reduced, redundant reference relationships accumulated in the system are also reduced. During garbage collection, the number of reference relationships that the system needs to traverse and check will be greatly reduced, thereby reducing disk resource consumption and computing performance consumption.

[0163] As for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0164] like Figure 7 , showing a structural block diagram of an electronic device 40 provided in an embodiment of the present invention, including:

[0165] The processor 401, the memory 402, and the computer program 4021 stored in the memory 402 and capable of running on the processor 401. When the computer program 4021 is executed by the processor 401, each process of the above-mentioned snapshot object processing method embodiment is implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0166] like Figure 8 , shows a structural block diagram of a computer-readable storage medium 50 provided in an embodiment of the present invention. A computer program 501 is stored on the computer-readable storage medium 50. When the computer program 501 is executed by the processor, each process of the above-mentioned snapshot object processing method embodiment is implemented, and the same technical effect can be achieved. To avoid repetition, it will not be described here.

[0167] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0168] It will be appreciated by those skilled in the art that the embodiments of the present invention may be provided as methods, devices, or computer program products. Therefore, the embodiments of the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the embodiments of the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0169] The embodiments of the present invention are described with reference to the flowcharts and / or block diagrams of the methods, terminal devices (systems), and computer program products according to the embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of the processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing terminal device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0170] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing terminal device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0171] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device so that a series of operating steps are executed on the computer or other programmable terminal device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable terminal device to implement the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0172] Although the preferred embodiments of the present invention have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the embodiments of the present invention.

[0173] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or terminal device including the elements.

[0174] The snapshot object processing method, device, equipment and medium provided by the present invention are described in detail above. The principle and implementation mode of the present invention are described in detail using specific examples herein. The description of the above embodiments is only used to help understand the method and core idea of ​​the present invention. At the same time, for those skilled in the art, according to the idea of ​​the present invention, there may be changes in the specific implementation mode and application scope. In summary, the content of this specification should not be understood as limiting the present invention.

Claims

1. A snapshot object processing method, characterized in that: The method comprises: receiving an access request, wherein the access request carries the data to be modified and a snapshot identifier; According to the snapshot identifier, acquiring a snapshot object and a forward relationship between the snapshot object, wherein the forward relationship is used to point to an internal object corresponding to the snapshot object; Create a new snapshot object according to the data to be modified; The forward relationship of the new snapshot object is set to point to the modified internal object.

2. The method according to claim 1, characterized in that The creating a new snapshot object according to the data to be modified includes: When the snapshot identifier of the snapshot object is different from the snapshot identifier carried in the access request, a new snapshot object is created according to the snapshot identifier carried in the access request and the data to be modified, and the snapshot identifier carried in the access request is used as the snapshot identifier of the new snapshot object.

3. The method according to claim 2, characterized in that The method further comprises: When the snapshot identifier of the snapshot object is the same as the snapshot identifier carried in the access request, Writing the data to be modified into the internal object; The forward relationship of the snapshot object is set to point to the modified internal object.

4. The method according to claim 1, characterized in that: The method further comprises: Acquire a reverse relationship between the modified internal object corresponding to the new snapshot object and the modified internal object corresponding to the new snapshot object, wherein the reverse relationship is used to point to the new snapshot object corresponding to the modified internal object corresponding to the new snapshot object; The data of the internal object is modified according to the data to be modified, and the reverse relationship of the internal object is set to point to the new snapshot object.

5. The method according to claim 3, characterized in that: The method further comprises: Get the modified internal object corresponding to the snapshot object and the modified internal object corresponding to the snapshot object a reverse relationship of the modified internal object, the reverse relationship being used to point to the snapshot object corresponding to the modified internal object corresponding to the snapshot object; Modify the data of the internal object according to the data to be modified, and The reverse relationship of the modified internal object is set to point to the snapshot object.

6. The method according to claim 4 or 5, characterized in that: The method further comprises: When the data storage pool reaches a preset storage threshold, a target internal object that needs to be recycled is determined, and a reverse relationship of the target internal object is obtained; According to the reverse relationship of the target internal object, acquiring the target snapshot object corresponding to the target internal object and the forward relationship of the target snapshot object; Performing positive verification on the data of the target internal object according to the positive relationship of the target snapshot object; Reversely verify the data of the target internal object according to the reverse relationship of the target internal object; Data that can pass both forward verification and reverse verification is taken as valid data, the valid data is aggregated into a new internal object, the reverse relationship of the new internal object is set to point to the snapshot object corresponding to the target internal object; and the forward relationship of the target snapshot object is set to point to the new internal object.

7. The method according to claim 6, characterized in that The method further comprises: When the valid data cannot be aggregated into a new internal object, return to the step of acquiring a target snapshot object corresponding to the target internal object and a forward relationship of the target snapshot object according to a reverse relationship of the target internal object until the valid data is aggregated into a new internal object.

8. A snapshot object processing device, characterized in that: The device comprises: A receiving module, used for receiving an access request, the access request carrying the data to be modified and a snapshot identifier; a first forward relationship acquisition module, used for acquiring a snapshot object and a forward relationship of the snapshot object according to the snapshot identifier, the forward relationship being used for pointing to an internal object corresponding to the snapshot object; A creation module, used for creating a new snapshot object according to the data to be modified; The first forward relationship processing module is used to set the forward relationship of the new snapshot object to point to the modified internal object.

9. An electronic device, characterized in that: include: A processor, a memory, and a computer program stored in the memory and capable of running on the processor, wherein when the computer program is executed by the processor, the steps of the snapshot object processing method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the snapshot object processing method according to any one of claims 1 to 7 are implemented.

Images