Data uploading method and device, gateway equipment, readable storage medium and program product

Abstract

The application relates to a data uploading method and device, a gateway device, a computer readable storage medium and a computer program product. The method comprises the following steps: creating a first lock object related to a target volume, wherein the first lock object comprises a target gateway identifier of a target storage gateway and a creation timestamp; uploading the first lock object to a cloud storage server under the condition that no other lock object exists in the cloud storage server, wherein the other lock object is a lock object related to the target volume and created by another storage gateway; and uploading data of a target volume to which the target storage gateway belongs to the cloud storage server under the condition that feedback information about successful uploading of the lock object is received from the cloud storage server, so that the data on the cloud can be effectively prevented from being damaged.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data processing, and in particular to a data uploading method and device, a gateway device, a computer readable storage medium and a computer program product. BACKGROUND

[0002] With the rapid development of cloud computing technology, hybrid cloud storage mode combines traditional storage arrays and cloud storage, and becomes the choice of more and more enterprises in digital transformation. Compared with a single storage mode, hybrid cloud storage is more flexible and reliable. Cloud storage can be used as a backup and expansion of local storage, and is also a powerful data security guarantee when local storage fails. Generally, a storage gateway is used as middleware between local storage and cloud storage to perform data uploading operations, for example, data is uploaded to a corresponding storage bucket in cloud storage through the storage gateway, and for another example, data is obtained from a corresponding storage bucket in cloud storage through the storage gateway.

[0003] In related technologies, most gateway products limit data recovery on a single storage gateway, but the availability is poor, and in practice, human errors may occur, which may cause multiple storage gateways to recover data on the cloud at the same time, which is likely to destroy data on the cloud. Therefore, there is a problem that availability and data destruction cannot be considered. SUMMARY

[0004] Therefore, it is necessary to provide a data uploading method, device, gateway device, computer readable storage medium and computer program product which can ensure availability and avoid data destruction in cloud storage servers during data uploading, so as to ensure data availability.

[0005] In a first aspect, the present application provides a data uploading method, comprising:

[0006] creating a first lock object related to a target volume, the first lock object comprising a target gateway identifier of the target storage gateway and a creation timestamp;

[0007] uploading the first lock object to the cloud storage server in a case where it is verified that no other lock object exists in the cloud storage server, the other lock object being a lock object created by another storage gateway and related to the target volume;

[0008] uploading data of a target volume to which the target storage gateway belongs to the cloud storage server in a case where feedback information indicating that the uploading of the lock object is successful is received from the cloud storage server.

[0009] In a second aspect, the present application further provides a data uploading device, comprising:

[0010] A creating module is configured to create a first lock object related to a target volume, the first lock object including a target gateway identifier of a target storage gateway and a creation timestamp;

[0011] A first uploading module is configured to upload the first lock object to a cloud storage server in a case where it is checked that no other lock object exists in the cloud storage server, the other lock object being a lock object created by another storage gateway and related to the target volume;

[0012] A second uploading module is configured to upload data of a target volume to which the target storage gateway belongs to the cloud storage server in a case where feedback information indicating that the lock object is successfully uploaded is received from the cloud storage server.

[0013] In a third aspect, the present application provides a gateway device, comprising a memory and a processor, the memory storing a computer program, and the processor implementing the following steps when executing the computer program:

[0014] A first lock object related to a target volume is created, the first lock object including a target gateway identifier of a target storage gateway and a creation timestamp;

[0015] The first lock object is uploaded to a cloud storage server in a case where it is checked that no other lock object exists in the cloud storage server, the other lock object being a lock object created by another storage gateway and related to the target volume;

[0016] Data of a target volume to which the target storage gateway belongs is uploaded to the cloud storage server in a case where feedback information indicating that the lock object is successfully uploaded is received from the cloud storage server.

[0017] In a fourth aspect, the present application provides a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the following steps:

[0018] A first lock object related to a target volume is created, the first lock object including a target gateway identifier of a target storage gateway and a creation timestamp;

[0019] The first lock object is uploaded to a cloud storage server in a case where it is checked that no other lock object exists in the cloud storage server, the other lock object being a lock object created by another storage gateway and related to the target volume;

[0020] Data of a target volume to which the target storage gateway belongs is uploaded to the cloud storage server in a case where feedback information indicating that the lock object is successfully uploaded is received from the cloud storage server.

[0021] In a fifth aspect, the present application further provides a computer program product comprising a computer program which, when executed by a processor, implements the following steps:

[0022] a first lock object related to the target volume is created, the first lock object comprising a target gateway identifier of the target storage gateway and a creation timestamp;

[0023] In a case where it is checked that no other lock object exists in the cloud storage server, the first lock object is uploaded to the cloud storage server, the other lock object being a lock object created by another storage gateway and related to the target volume;

[0024] In a case where feedback information indicating that the uploading of the lock object is successful is received from the cloud storage server, data of the target volume to which the target storage gateway belongs is uploaded to the cloud storage server.

[0025] The data uploading method, device, gateway equipment, computer readable storage medium and computer program product described above, by creating a first lock object related to the target volume, the first lock object comprising a target gateway identifier of the target storage gateway and a creation timestamp; in a case where it is checked that no other lock object exists in the cloud storage server, the first lock object is uploaded to the cloud storage server, the other lock object being a lock object created by another storage gateway and related to the target volume. That is, the other storage gateway does not preempt the lock of the data of the target volume in the cloud storage server, at this time, the first lock object is allowed to be uploaded to the cloud storage server, that is, the target storage gateway timely applies to preempt the lock. In a case where feedback information indicating that the uploading of the lock object is successful is received from the cloud storage server, it is indicated that the target storage gateway successfully preempts the lock of the data of the target volume, at this time, the data of the target volume to which the target storage gateway belongs can be uploaded to the cloud storage server, that is, only the target storage gateway which preempts the lock is allowed to upload the data of the target volume, avoiding the inconsistency of the data caused by the simultaneous data uploading of multiple storage gateways to the target volume, that is, avoiding the data of the target volume from being damaged. The whole process not only allows multiple storage gateways to simultaneously restore data to improve availability, but also keeps the data in the cloud storage server (i.e., the data in the cloud) from being damaged. BRIEF DESCRIPTION OF DRAWINGS

[0026] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the related art, the following will briefly introduce the drawings needed to be used in the description of the embodiments of the present application or the related art. Obviously, the drawings in the following description only some embodiments of the present application, and for those skilled in the art, other related drawings can be obtained without creative labor on the basis of these drawings.

[0027] Figure 1A comparison diagram of single storage gateway recovery process and multiple storage gateway simultaneous recovery process;

[0028] Figure 2 A diagram of prior art multiple storage gateway simultaneous upload;

[0029] Figure 3 An application environment diagram of the data upload method in an embodiment;

[0030] Figure 4 A flowchart of the data upload method in an embodiment;

[0031] Figure 5 A lock object verification diagram in an embodiment;

[0032] Figure 6 A lock object verification diagram in another embodiment;

[0033] Figure 7 A lock object verification diagram in another embodiment;

[0034] Figure 8 A structural block diagram of the data upload device in an embodiment;

[0035] Figure 9 An internal structure diagram of the gateway device in an embodiment. DETAILED DESCRIPTION

[0036] In order to make the purpose, technical scheme and advantages of the present application clearer, the present application will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and do not limit the present application.

[0037] Before introducing the embodiments of the present application, the following terms are explained:

[0038] Hybrid cloud storage: a storage model that combines cloud storage and local storage, allowing users to use both local and cloud resources. Local data can be uploaded to the cloud, and cloud data can be restored to the local, taking advantage of the elasticity and scalability of the cloud while retaining the advantages of local storage.

[0039] Exclusive lock: a lock mechanism that ensures that only one transaction or thread can hold the lock at the same time and access or modify the locked resources (such as data, files, code segments, etc.). This lock mechanism ensures data consistency and integrity through "exclusivity" and "mutual exclusivity".

[0040] Hybrid cloud storage allows users to store data on the most suitable platform according to the importance and access frequency of the data. For example, sensitive or hot data can be saved in local storage to ensure data security and privacy; while less sensitive or cold data can be saved in cloud storage to reduce storage costs. Hybrid cloud storage also provides a more reliable data backup and recovery solution. By backing up data to cloud storage, data loss caused by local storage device failure can be avoided. At the same time, the elasticity and scalability of cloud storage make the backup and recovery process more efficient and convenient.

[0041] Among them, the storage gateway + cloud storage service is a common hybrid cloud storage solution. The storage gateway is a middleware used to connect local storage devices and cloud storage services. It acts as a bridge between local storage devices and cloud storage, and realizes seamless integration of local storage and cloud storage by mapping local storage devices as object storage buckets or file storage shares in cloud storage services. The storage gateway can provide file storage or block storage services locally, and upload data to the cloud storage service, which can be recovered when needed, providing a reliable data protection mechanism. The storage gateway has good data protection measures when dealing with disaster handling, such as data backup, snapshot and recovery functions, to protect the security and integrity of data. Most gateway products currently limit data recovery to a single storage gateway, but using a single storage gateway to recover data on the cloud may have low recovery efficiency and low availability problems.

[0042] And in some scenarios, it is actually possible to manually operate multiple storage gateways to recover at the same time, which results in the destruction of data on the cloud. The data upload method provided by the embodiments of the present application can allow multiple storage gateways to recover at the same time while ensuring availability, and keep the data on the cloud from being destroyed. As shown in Figure 1 , it is a comparison diagram of single storage gateway recovery process and multiple storage gateways recovery process. Figure 1 In the middle, "Cloud" means on the cloud (cloud storage server), and "Error" means the storage gateway fails. Figure 1 The left half is to recover data by a single storage gateway, and then upgrade to multiple storage gateways for storage, as shown in the right part of Figure 1 . In the right part, two storage gateways have recovered the data of volume 1 from the cloud. If storage gateway 1 fails again after recovery, storage gateway 2 can provide services to increase the availability of local storage services.

[0043] As described above, when multiple storage gateways allow simultaneous recovery, there is a problem of data inconsistency. Specifically, after multiple storage gateways complete the recovery of a certain volume, they normally provide read and write backup services, but cannot perceive whether other storage gateways are using the recovered volume or also recovering the same volume, so there may be a case where multiple storage gateways upload data to the same storage bucket in the cloud, and each storage gateway thinks that the data in the cloud is the data uploaded by itself, which causes data inconsistency when reading data from the cloud, and the data in the cloud is destroyed. As shown in Figure 2 , it is a schematic diagram of the prior art when multiple storage gateways simultaneously upload. Referring to Figure 2 , both storage gateways recover successfully and provide services normally, so both storage gateways can normally upload and download data of volume 1 to the cloud. At this time, storage gateway 1 may have uploaded data of a specified offset of volume 1, and storage gateway 2 may have also uploaded data of the offset of volume 1. When storage gateway 1 needs to read data of the offset from the cloud, it will read the data uploaded by storage gateway 2, resulting in data inconsistency, and the data in the cloud is destroyed due to the simultaneous uploading of the two storage gateways.

[0044] In summary, the related art cannot balance availability and data destruction, and therefore, in the data uploading method provided in the embodiments of the present application, a first lock object related to a target volume is created, the first lock object includes a target gateway identifier of a target storage gateway and a creation timestamp; in a case where it is verified that no other lock object exists in a cloud storage server, the first lock object is uploaded to the cloud storage server, and the other lock object is a lock object related to the target volume and created by another storage gateway. That is, the other storage gateway does not preempt the lock of the data of the target volume in the cloud storage server, and at this time, the first lock object is allowed to be uploaded to the cloud storage server, that is, the target storage gateway timely applies to preempt the lock. In a case where feedback information indicating that the uploading of the lock object is successful is received from the cloud storage server, it is indicated that the target storage gateway successfully preempts the lock of the data of the target volume, and at this time, the data of the target volume to which the target storage gateway belongs is uploaded to the cloud storage server, that is, only the target storage gateway that preempts the lock is allowed to upload the data of the target volume, avoiding data inconsistency caused by the simultaneous data uploading of multiple storage gateways to the target volume, that is, avoiding the destruction of the data of the target volume. The entire process not only allows multiple storage gateways to simultaneously recover data to improve availability, but also keeps the data in the cloud storage server (i.e., the data in the cloud) from being destroyed.

[0045] The data uploading method provided in the embodiments of the present application can be applied to, for example, Figure 3In the application environment shown, the target storage gateway 302 communicates with the cloud storage server 304 (which provides cloud storage services) via a network. Other storage gateways 306 also communicate with the cloud storage server 304 (which provides cloud storage services) via a network.

[0046] In some embodiments, the target storage gateway 302 creates the first lock object associated with the target volume. The first lock object includes the target gateway identifier and creation timestamp of the target storage gateway. If no other lock object is found in the cloud storage server 304, the first lock object is uploaded to the cloud storage server 304. Other lock objects are lock objects associated with the target volume created by other storage gateways 306. Upon receiving feedback information from the cloud storage server indicating that the lock object upload was successful, the data of the target volume to which the target storage gateway belongs is uploaded to the cloud storage server 304.

[0047] Among them, the target storage gateway 302 and other storage gateways 306 are different storage gateways.

[0048] In one exemplary embodiment, such as Figure 4 As shown, a data upload method is provided, which can be applied to... Figure 3 Taking the target storage gateway 302 as an example, the explanation includes the following steps S402 to S406. Wherein:

[0049] Step S402: Create the first lock object associated with the target volume. The first lock object includes the target gateway identifier of the target storage gateway and the creation timestamp.

[0050] The storage gateway comprises multiple volumes, each of which can be considered a storage component. Essentially, the storage gateway provides read and write services at the volume level within local storage. For each volume, a corresponding bucket exists on the cloud storage server (a container within the cloud storage server, which includes multiple buckets). Data storage occurs through the volume and its corresponding bucket. The target volume is a volume within the target storage gateway. A lock object can be understood as an instance of a lock (preemptive lock) or a lock object (i.e., a lock object uploaded to the bucket), used to lock the data in the corresponding bucket on the cloud storage server. Only the storage gateway that created the lock object is allowed to modify the data in that bucket, preventing modifications from other storage gateways. For example, a lock object can also be considered lock information for a preemptive lock, one representation being (gateway identifier_timestamp). The gateway identifier can be represented by `gw_id`, and the timestamp by `last_modify_time`. The timestamp can be the creation timestamp for initial creation or a maintenance timestamp for maintenance purposes.

[0051] The first lock object refers to a lock object related to the target volume that is first created by the target storage gateway. The target gateway identifier is used to represent the target storage gateway, and the creation timestamp represents the creation time of the first lock object.

[0052] The target storage gateway is a storage gateway that is in the process of completing data recovery. The target storage gateway can be a storage gateway that has failed before the current time but has completed failure repair at the current time, or a storage gateway that has not failed until the current time.

[0053] Optionally, after completing data recovery, the target storage gateway determines the volume in which the data to be uploaded exists according to the data upload requirement, takes the volume as the target volume, and generates the first lock object corresponding to the target volume based on the target gateway identifier of the target storage gateway and the current time.

[0054] Illustratively, after the target storage gateway is started, an exclusive lock checking thread corresponding to the target storage gateway is started. After the target volume is started and the target storage gateway creates the first lock object related to the target volume, the target volume registers with the exclusive lock checking thread and records the volume information of the target volume in the queue of the exclusive lock checking thread. The exclusive lock checking thread is a global-level thread responsible for maintaining the exclusive lock of the target storage gateway. Illustratively, before uploading data, the target volume checks whether the target storage gateway still holds the exclusive lock through the exclusive lock checking thread.

[0055] In step S404, the first lock object is uploaded to the cloud storage server in the case where it is verified that no other lock object exists in the cloud storage server. The other lock object is a lock object related to the target volume that is created by another storage gateway.

[0056] The cloud storage server is a server for providing cloud storage services and can also be regarded as cloud storage. The other lock object refers to a lock object related to the target volume that is created by another storage gateway.

[0057] It should be noted that in the case where it is verified that no other lock object exists in the cloud storage server, it means that another storage gateway has not preempted the lock related to the target volume, i.e., the data in the storage bucket corresponding to the target lock is not locked at the current time. Uploading the first lock object to the cloud storage server can be understood as the target storage gateway applying to lock the data in the storage bucket corresponding to the target volume, i.e., the target storage gateway applies to allow only the data of its own target volume to be uploaded to the storage bucket.

[0058] Exemplarily, the target storage gateway checks whether there is other lock object in the cloud storage server, if there is, the first lock object is not uploaded, and the target storage gateway waits. If there is not, the target storage gateway uploads the first lock object to the storage bucket corresponding to the target volume in the cloud storage server, to indicate that the storage bucket corresponding to the target volume in the cloud storage server saves the first lock object.

[0059] Exemplarily, the exclusive lock checking thread cyclically detects the lock holding situation of each volume at a fixed interval, that is, if it is determined by the exclusive lock checking thread that there is no lock related to the target volume, the target storage gateway determines that there is no other lock object in the cloud storage server.

[0060] In the case that the feedback information of successful uploading of the lock object sent by the cloud storage server is received, the data of the target volume to which the target storage gateway belongs is uploaded to the cloud storage server.

[0061] The feedback information of successful uploading of the lock object sent by the cloud storage server indicates that the target volume in the target storage gateway successfully preempts the lock, that is, the storage bucket corresponding to the target volume only allows the target storage gateway to upload data to the storage bucket.

[0062] Exemplarily, in the case that the feedback information of successful uploading of the lock object sent by the cloud storage server is received by the target storage gateway, the target storage gateway determines that the lock related to the target volume is successfully preempted, and then the data of the target volume to which the target storage gateway belongs is uploaded to the storage bucket corresponding to the target volume in the cloud storage server.

[0063] In the above data uploading method, the first lock object related to the target volume is created, the first lock object includes the target gateway identifier of the target storage gateway and the creation timestamp; in the case that it is checked that there is no other lock object in the cloud storage server, the first lock object is uploaded to the cloud storage server, and the other lock object is a lock object related to the target volume and created by other storage gateway. That is, the other storage gateway does not preempt the lock of the data related to the target volume in the cloud storage server, at this time, the first lock object is allowed to be uploaded to the cloud storage server, that is, the target storage gateway timely applies to preempt the lock. In the case that the feedback information of successful uploading of the lock object sent by the cloud storage server is received, it indicates that the target storage gateway successfully preempts the lock of the data related to the target volume, at this time, the data of the target volume to which the target storage gateway belongs can be uploaded to the cloud storage server, that is, only the target storage gateway which preempts the lock is allowed to upload the data of the target volume, so as to avoid the inconsistency of the data caused by simultaneous data uploading of multiple storage gateways to the target volume, that is, to avoid the data of the target volume from being damaged. In the whole process, multiple storage gateways are allowed to simultaneously restore data, so as to improve the availability, and the data in the cloud storage server (that is, the data in the cloud) is not damaged.

[0064] In some embodiments, the method further includes: sending the volume identifier of the target volume to the cloud storage server to instruct the cloud storage server to query whether a corresponding lock object exists based on the volume identifier and generate a query result; and verifying whether other lock objects exist in the cloud storage server based on the query result returned by the cloud storage server.

[0065] For example, the target storage gateway sends the volume identifier of the target volume to the cloud storage server, instructing the cloud storage server to query the lock object storage information for a corresponding lock object based on the volume identifier, and return the query result to the target storage gateway.

[0066] For example, if a lock object exists, the query result includes the existing lock object. The target storage gateway then parses the query result to obtain the existing lock object and verifies whether the existing lock object is another lock object.

[0067] For example, if no lock object exists, the query result contains an indication that no lock was found. Thus, the target storage gateway parses the query result, obtains the indication, and can directly determine that no other lock object exists.

[0068] In this embodiment, by sending the volume identifier to the cloud storage server, the cloud storage server can quickly query whether there is a corresponding lock object and directly return the query result without requiring analysis and verification by the cloud storage server, thus reducing the processing pressure on the cloud storage server. Therefore, the target storage gateway can quickly determine whether another storage gateway has preempted the corresponding lock based on the query result, avoiding accidental restoration of a normally used volume due to human error, and enabling the target storage gateway to promptly preempt the lock, ensuring the validity of subsequent data uploads.

[0069] In some embodiments, based on the query results returned by the cloud storage server, verifying whether there are other lock objects in the cloud storage server includes: parsing the query results returned by the cloud storage server; if no lock object is parsed, determining that there are no other lock objects created by other storage gateways in the cloud storage server; if a lock object related to the target volume is parsed, obtaining the gateway identifier from the parsed lock object; if the gateway identifier is not the target gateway identifier, determining that there are other lock objects created by other storage gateways in the cloud storage server.

[0070] It should be noted that if the gateway identifier is not the target gateway identifier, it means that the storage bucket corresponding to the target volume in the cloud server has been preempted by another storage gateway. In this case, the target storage gateway will not allow its first lock object to be uploaded to the cloud storage server.

[0071] In the embodiment, the current whether other storage gateway occupies the corresponding lock can be checked in time through the query result returned from the cloud storage server, the artificial misoperation of repeatedly recovering a volume being normally used is avoided, the target storage gateway can timely occupy the lock, and the effectiveness of subsequent data uploading is ensured.

[0072] In some embodiments, the method further comprises: starting a lock maintenance operation on the target volume every predetermined time interval, and recreating a new lock object related to the target volume; uploading the new lock object to the cloud storage server, and in the case that feedback information of maintenance success sent by the cloud storage server is received, continuing to upload the data of the target volume to which the target storage gateway belongs to the cloud storage server.

[0073] The lock maintenance operation refers to continuing to occupy the data of the storage bucket corresponding to the target volume, and the new lock object is a lock object used for the lock maintenance operation.

[0074] Exemplarily, the target storage gateway acquires the lock object of the storage bucket corresponding to the target volume in the cloud storage server, in the case that the gateway identifier of the acquired lock object is verified by the target storage gateway as the gateway identifier of the target storage gateway, the target storage gateway uploads a new lock object to the cloud storage server to indicate that the cloud storage server replaces the first lock object with the new lock object, and after the replacement is completed, feedback information of maintenance success is sent.

[0075] Exemplarily, after the first lock object is uploaded to the cloud storage server, the target storage gateway checks whether the lock related to the target volume is held before uploading the data of the target volume through the exclusive lock checking thread, if the lock is held and the predetermined time interval is not reached, the data of the target volume is uploaded, if the lock is held and the predetermined time interval is reached, a new lock object is generated and uploaded to the cloud storage server, and in the case that feedback information of maintenance success sent by the cloud storage server is received, the data of the target volume to which the target storage gateway belongs is continued to be uploaded to the cloud storage server.

[0076] In the embodiment, through the lock maintenance operation, it can be ensured that the target storage gateway continues to have the qualification of uploading the data of the target volume to the corresponding storage bucket, and it can also avoid that other storage gateways snatch the uploaded data after the lock is released.

[0077] In some embodiments, the predetermined time interval determination step comprises: determining the corresponding predetermined time interval according to the current storage amount of the cloud storage server, and the storage amount and the predetermined time interval are positively correlated.

[0078] Exemplarily, the storage amount reflects the pressure of the cloud storage server, and the greater the storage amount, the greater the pressure, and the greater the predetermined time interval required, i.e., the frequency of data uploading is slowed down. Of course, in other embodiments, the pressure is also affected by the uploading and downloading frequency, etc. Therefore, the storage amount, uploading and downloading frequency can be integrated to determine the corresponding predetermined time interval.

[0079] In the present embodiment, the maintenance period of the maintenance operation is adaptively adjusted according to the current storage amount of the cloud storage server, i.e., a matching predetermined time interval is determined, to ensure that whether other gateways are using the corresponding volume is detected at a reasonable period.

[0080] In other embodiments, in addition to the storage amount, uploading and downloading frequency, other factors such as network, etc. can also be considered in the predetermined time interval determination process. Therefore, the predetermined time interval can also be determined according to the storage amount, uploading and downloading frequency, and network, which is not limited in specific embodiments. In some embodiments, the new lock object related to the target volume is re-created, including: taking the time when the lock maintenance operation is started as a maintenance timestamp, and generating a new lock object based on the maintenance timestamp and the target gateway identifier of the target storage gateway.

[0081] Exemplarily, for the target storage gateway, the gateway identifier is id1, and the first lock object related to the target volume is (id1_t0). After the predetermined time interval is reached, the time when the lock maintenance operation is started is t1, and the new lock object is (id1_t1).

[0082] In the present embodiment, the new lock object for maintaining the lock is automatically generated by the time when the lock maintenance operation is started, to ensure that the target storage gateway can continue to upload the data of the target volume.

[0083] The following illustrates the lock object verification process under different conditions:

[0084] Case 1: As shown in Figure 5 , it is a lock object verification diagram in an embodiment. Referring to Figure 5 the left half, the gateway identifier of the target storage gateway is id1, the creation timestamp is t0, and there is no lock object displayed in the cloud storage server. Therefore, the target storage gateway verifies that there is no other lock object, and uploads its first lock object (id1_t0). Then, referring to Figure 5 the right half, the lock maintenance operation is started, a new lock object (id1_t1) is generated based on the time t1 of the lock maintenance operation, and is uploaded to the cloud storage server in time.

[0085] Case 2: As shown in Figure 6 , it is a lock object verification diagram in another embodiment. First, referring to Figure 6The left half of the diagram illustrates scenario 2.1: The old storage gateway (gateway identifier id2) is another storage gateway, and the newly restored storage gateway is the target storage gateway. In the left half, the old storage gateway uploads its own lock object (id2, t0) at time t0. Subsequently, it maintains its lock promptly, i.e., it regenerates another lock object (id2, t1) for maintenance at time t1. Then, the newly restored storage gateway creates its first lock object (id1, t2). The newly restored storage gateway will verify that there is a lock object created by another storage gateway on the cloud storage server, and that the other storage gateway promptly creates another lock object, i.e., maintains the lock promptly. Therefore, the newly restored storage gateway cannot upload its own lock object. The maintenance process is explained later.

[0086] See Figure 6 The right half of the diagram illustrates situation 2.2: The old storage gateway uploads its own lock object (id2, t0) at time t0. Subsequently, the old storage gateway does not maintain its own lock in time, indicating that the old gateway has failed and is unusable. The newly restored storage gateway verifies that there are no other lock objects at time t1, and then uploads its own lock object (id1_t1) to preempt the lock.

[0087] Scenario 3: such as Figure 7 The diagram shown illustrates lock object verification in another embodiment. The old storage gateway was unable to maintain the lock due to network issues. The newly restored storage gateway preempted the lock (i.e., uploaded its first self-created lock object (id1_t0)) and uploaded data normally. The old storage gateway's network then recovered, and it generated a lock object (id2_t2). However, the new storage gateway maintained the lock promptly, i.e., it promptly generated a new lock object (id1_t1) for maintenance. At this point, the old storage gateway could not preempt the lock.

[0088] In one specific embodiment, the cloud storage server communicates with multiple storage gateways, randomly selects one of the multiple storage gateways as the target storage gateway, and designates the other storage gateways as the remaining storage gateways. The specific steps are as follows:

[0089] The target storage gateway creates the first lock object associated with the target volume. This first lock object includes the target gateway identifier and the creation timestamp. The target volume identifier is sent to the cloud storage server, instructing the cloud storage server to query whether a corresponding lock object exists based on the volume identifier, generate the query results, and return them to the target storage gateway.

[0090] The target storage gateway parses the query result returned by the cloud storage server; in the case where no lock object is parsed, it is determined that no other lock object created by other storage gateways exists in the cloud storage server; in the case where a lock object related to the target volume is parsed, the gateway identifier is obtained from the parsed lock object, and in the case where the gateway identifier is not the target gateway identifier, it is determined that other lock objects created by other storage gateways exist in the cloud storage server.

[0091] In the case where the target storage gateway checks that no other lock object exists in the cloud storage server, the first lock object is uploaded to the cloud storage server, and the other lock objects are other lock objects created by other storage gateways and related to the target volume; in the case where feedback information of successful uploading of the lock object sent by the cloud storage server is received, the data of the target volume to which the target storage gateway belongs is uploaded to the cloud storage server.

[0092] The lock maintenance operation related to the target volume is started every predetermined time interval, and a new lock object related to the target volume is re-created; the new lock object is uploaded to the cloud storage server, and in the case where feedback information of successful maintenance sent by the cloud storage server is received, the data of the target volume to which the target storage gateway belongs is continuously uploaded to the cloud storage server. The predetermined time interval determination step comprises: determining the corresponding predetermined time interval according to the current storage amount of the cloud storage server, and the storage amount and the predetermined time interval are positively correlated. The new lock object related to the target volume is re-created, which comprises: taking the time when the lock maintenance operation is started as a maintenance time stamp, and generating the new lock object based on the maintenance time stamp and the target gateway identifier of the target storage gateway.

[0093] In the embodiment, by creating a first lock object related to the target volume, the first lock object includes a target gateway identifier of the target storage gateway and a creation timestamp; in the case that it is checked that no other lock object exists in the cloud storage server, the first lock object is uploaded to the cloud storage server, the other lock object being a lock object created by other storage gateways and related to the target volume. That is, other storage gateways do not preempt the lock of the data about the target volume in the cloud storage server, at this time, the first lock object is allowed to be uploaded to the cloud storage server, that is, the target storage gateway timely applies for preemption of the lock. In the case that the feedback information of successful uploading of the lock object sent by the cloud storage server is received, it is indicated that the target storage gateway successfully preempts the lock of the data about the target volume, at this time, the data of the target volume to which the target storage gateway belongs can be uploaded to the cloud storage server, that is, only the target storage gateway preempts the lock to upload the data of the target volume, avoiding data inconsistency caused by simultaneous data uploading of multiple storage gateways on the target volume, that is, avoiding the data of the target volume being damaged, achieving simultaneous recovery of multiple storage gateways without damaging the data on the cloud to cause data loss. In addition, compared with the way of avoiding damage to the data on the cloud by limiting user operations in the prior art when the storage gateway recovers data, the embodiment of the application solves the problem of damaging the data on the cloud by means of a software built-in mechanism (used to perform the above-mentioned data uploading specific steps) without user perception and limiting user operations. Moreover, not only for the block storage gateway + object cloud storage mode, the software built-in mechanism can also be flexibly applied to other multiple local storage data recovery schemes from the cloud storage, without quantity limitation, flexibility and scalability. In addition, users can arbitrarily perform data recovery and local backup on multiple storage gateways according to specific scenes and needs; the software built-in mechanism can also be flexibly applied to any other type of hybrid cloud storage scheme, which has good practicability.

[0094] It should be understood that, although each step in the flowchart involved in each embodiment as described above is shown in sequence according to the arrow, these steps are not necessarily executed in sequence according to the arrow. Unless otherwise specified herein, the execution of these steps is not strictly limited in sequence, and these steps can be executed in other sequences. Moreover, at least part of the steps in the flowchart involved in each embodiment as described above can include multiple steps or stages, which are not necessarily executed at the same time, but can be executed at different times, and the execution sequence of these steps or stages is not necessarily sequential, but can be executed in rotation or alternation with at least part of other steps or steps or stages in other steps.

[0095] Based on the same inventive concept, the embodiments of the present application also provide a data uploading apparatus for implementing the data uploading method described above. The implementation scheme of the apparatus for solving the problem is similar to the implementation scheme described above in the method, so the specific limitations in one or more data uploading apparatus embodiments provided below can refer to the limitations of the data uploading method described above, which will not be repeated here.

[0096] In one exemplary embodiment, as shown in Figure 8 A data uploading apparatus 800 is provided, comprising a creating module 802, a first uploading module 804 and a second uploading module 806, wherein:

[0097] The creating module 802 is configured to create a first lock object related to the target volume, the first lock object comprising a target gateway identifier of the target storage gateway and a creation timestamp;

[0098] The first uploading module 804 is configured to upload the first lock object to the cloud storage server in the case that it is checked that there is no other lock object in the cloud storage server, the other lock object being a lock object created by other storage gateways and related to the target volume;

[0099] The second uploading module 806 is configured to upload the data of the target volume to which the target storage gateway belongs to the cloud storage server in the case that feedback information of successful uploading of the lock object sent by the cloud storage server is received.

[0100] In some embodiments, the apparatus further comprises a checking module configured to send a volume identifier of the target volume to the cloud storage server to instruct the cloud storage server to query whether there is a corresponding lock object based on the volume identifier and to generate a query result; and based on the query result returned by the cloud storage server, check whether there is other lock object in the cloud storage server.

[0101] In some embodiments, the checking module is configured to parse the query result returned by the cloud storage server; in the case that no lock object is parsed, it is determined that there is no other lock object created by other storage gateways in the cloud storage server; and in the case that a lock object related to the target volume is parsed, the gateway identifier is obtained from the parsed lock object, and in the case that the gateway identifier is not the target gateway identifier, it is determined that there is other lock object created by other storage gateways in the cloud storage server.

[0102] In some embodiments, the apparatus further comprises a maintaining module configured to start a lock maintaining operation related to the target volume every predetermined time interval and to recreate a new lock object related to the target volume; upload the new lock object to the cloud storage server, and in the case that feedback information of successful maintaining sent by the cloud storage server is received, continue to upload the data of the target volume to which the target storage gateway belongs to the cloud storage server.

[0103] In some embodiments, the apparatus further comprises a determining module configured to determine a corresponding predetermined time interval according to a current storage amount of the cloud storage server, the storage amount being positively correlated with the predetermined time interval.

[0104] In some embodiments, the creating module 802 is further configured to take a time point when the lock maintenance operation is started as a maintenance timestamp, and generate a new lock object based on the maintenance timestamp and a target gateway identifier of the target storage gateway.

[0105] The modules in the data uploading apparatus described above can be implemented by software, hardware, or a combination thereof, in whole or in part. The modules described above can be embedded in or independent of a processor in the gateway device in hardware form, or stored in a memory in the gateway device in software form, so as to be invoked and executed by the processor to perform the operations corresponding to the modules.

[0106] In one exemplary embodiment, a gateway device is provided, which is a storage gateway. An internal structure diagram of the gateway device can be as shown in Figure 9 The gateway device includes a processor, a memory, an input / output interface (I / O), and a communication interface. The processor, the memory, and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. The processor of the gateway device is configured to provide computing and control capabilities. The memory of the gateway device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operating system and the computer program in the non-volatile storage medium to run. The input / output interface of the gateway device is configured to exchange information between the processor and external devices. The communication interface of the gateway device is configured to communicate with external terminals through network connection. The computer program is executed by the processor to implement a data uploading method.

[0107] Those skilled in the art can understand that Figure 9 The structure shown in the figure is only a block diagram of part of the structure related to the scheme of the present application, and does not constitute a limitation on the gateway device to which the scheme of the present application is applied. A specific gateway device can include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0108] In one embodiment, a gateway device is also provided, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the steps in the method embodiments described above.

[0109] In an embodiment, a computer readable storage medium is provided, and a computer program is stored on the computer readable storage medium, and the computer program is executed by a processor to implement the steps in the above method embodiments.

[0110] In an embodiment, a computer program product is provided, and the computer program product comprises a computer program, and the computer program is executed by a processor to implement the steps in the above method embodiments.

[0111] It should be noted that the user information (including but not limited to user equipment information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in the present application are all information and data authorized by the user or authorized by all parties, and the collection, use and processing of the related data need to comply with relevant regulations.

[0112] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer readable storage medium, and when executed, can include the processes of the above-mentioned embodiment methods. Any reference to memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile memory and volatile memory. The non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical storage, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. The volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration but not limitation, the RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc. The database involved in the embodiments provided in the present application can include at least one of a relational database and a non-relational database. The non-relational database can include a distributed database based on a block chain, etc., without being limited thereto. The processor involved in the embodiments provided in the present application can be a general-purpose processor, a central processing unit, a graphics processing unit, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, an artificial intelligence (AI) processor, etc., without being limited thereto.

[0113] The technical features of the above embodiments can be combined in any manner. To make the description concise, not all possible combinations of the technical features in the above embodiments are described, but as long as the combinations of the technical features do not exist, they should be considered as the scope of the present application.

[0114] The above-described embodiments are merely illustrative of several embodiments of the present application, which are described in more detail and in a specific manner, but should not be construed as limiting the scope of the patent of the present application. It should be noted that, for those of ordinary skill in the art, several modifications and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the appended claims.

Claims

1. A data uploading method, characterized by, The method applied to a target storage gateway comprises: creating a first lock object related to a target volume, the first lock object comprising a target gateway identifier of the target storage gateway and a creation timestamp; uploading the first lock object to a cloud storage server in a case where it is checked that no other lock object exists in the cloud storage server, the other lock object being a lock object created by another storage gateway and related to the target volume; uploading data of a target volume to which the target storage gateway belongs to the cloud storage server in a case where feedback information of successful uploading of the lock object is received from the cloud storage server.

2. The method of claim 1, wherein, The method further comprises: sending a volume identifier of the target volume to the cloud storage server to instruct the cloud storage server to query whether a corresponding lock object exists based on the volume identifier and to generate a query result; checking whether the other lock object exists in the cloud storage server based on the query result returned by the cloud storage server.

3. The method of claim 2, wherein, The checking whether the other lock object exists in the cloud storage server based on the query result returned by the cloud storage server comprises: parsing the query result returned by the cloud storage server; determining that no other lock object created by another storage gateway exists in the cloud storage server in a case where no lock object is parsed; determining that the other lock object created by another storage gateway exists in the cloud storage server in a case where a lock object related to the target volume is parsed, and acquiring a gateway identifier from the parsed lock object, and determining that the other lock object created by another storage gateway exists in the cloud storage server in a case where the gateway identifier is not the target gateway identifier.

4. The method of claim 1, wherein, The method further comprises: starting a lock maintenance operation related to the target volume every predetermined time interval, and recreating a new lock object related to the target volume; uploading the new lock object to the cloud storage server, and continuing to upload data of the target volume to which the target storage gateway belongs to the cloud storage server in a case where feedback information of successful maintenance is received from the cloud storage server.

5. The method of claim 4, wherein, The determining of the predetermined time interval comprises: determining a corresponding predetermined time interval according to a current storage amount of the cloud storage server, the storage amount being positively correlated with the predetermined time interval.

6. The method of claim 4, wherein, The recreating of the new lock object related to the target volume comprises: taking a time point at which the lock maintenance operation is started as a maintenance timestamp, and generating the new lock object based on the maintenance timestamp and the target gateway identifier of the target storage gateway.

7. A data upload apparatus, characterized by, The apparatus comprises: a creating module configured to create a first lock object related to a target volume, the first lock object comprising a target gateway identifier of a target storage gateway and a creation timestamp; a first uploading module configured to upload the first lock object to a cloud storage server in a case where it is checked that no other lock object exists in the cloud storage server, the other lock object being a lock object created by another storage gateway and related to the target volume; a second uploading module configured to upload data of a target volume to which the target storage gateway belongs to the cloud storage server in a case where feedback information of successful uploading of the lock object is received from the cloud storage server.

8. A gateway device comprising a memory and a processor, the memory storing a computer program, characterized in that, The computer program is executed by the processor to implement the steps of the method of any one of claims 1 to 6.

9. A computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the steps of the method of any one of claims 1 to 6.

10. A computer program product comprising a computer program, characterized in that, The computer program is executed by the processor to implement the steps of the method of any one of claims 1 to 6.

Images