Data processing method and device
By adding lock attributes to the object storage server, supporting semantic operation objects based on lock attributes, the problem of data inconsistency between concurrent writes and concurrent readings in the prior art is solved, and the consistency of data in the object storage is achieved.
Patent Information
- Application Number
- CN202410371863.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Priority Date
- 2023-12-12
- Filing Date
- 2024-03-26
- Publication Date
- 2025-06-13
AI Technical Summary
The existing object storage may cause data inconsistent data to occur because the server does not support mutex access.
Added lock attributes to the object storage server, supporting semantic operation of objects based on lock attributes, thereby realizing mutually exclusive access to the same object by multiple clients and completing data synchronization of objects.
Through the use of lock attributes, the data consistency of concurrent writes and concurrent reads in object storage is ensured, and the problem of data inconsistency is avoided.
Smart Images
Figure CN120144043A_ABST
Abstract
Description
[0001] This application claims the priority of the Chinese patent application with the application number 202311704032.5 filed with the National Intellectual Property Administration on December 12, 2023, the entire content of which is incorporated herein by reference. Technical Field
[0002] Embodiments of this application relate to the storage field, and in particular, to a data processing method and apparatus. Background Art
[0003] In the digital age, data has become the core asset of all industries and needs to be stored and managed in an efficient, scalable, and highly reliable manner. Object storage is a storage method centered around objects, storing data as objects rather than files, and it has advantages such as high reliability, high scalability, and high performance.
[0004] However, in existing object storage, since the server side of object storage does not support mutually exclusive access, the client needs to follow the "last win" rule when accessing objects in object storage, that is, the last person to submit is the latest, not necessarily in chronological order. Therefore, the client needs to customize the access order when accessing objects, otherwise data inconsistency will occur in concurrent writing and concurrent reading. Summary of the Invention
[0005] This application provides a data processing method and apparatus, thereby preventing data inconsistency from occurring in concurrent writing and concurrent reading in object storage.
[0006] In a first aspect, a data processing method is provided, which is applied to the server side of an object storage service. In the process of the data processing method, the server first receives an object access request, and the object access request includes a target object. Then, the server determines the lock attribute corresponding to the object access request. The metadata of the target object includes the lock attribute of the target object, and the metadata of the target object is distributed and stored in at least one object storage device. The lock attribute is used to indicate the locking state of the target object. Next, the server obtains a lock according to the lock attribute. Finally, when the server successfully obtains the lock, it performs the data operation corresponding to the object access request on the data of the target object according to the lock.
[0007] Based on the above data processing method, the server adds a lock attribute to the object itself stored in the object storage service, supports the server to operate on the object based on the semantics of the lock attribute, so that it can perform the data operation of successfully obtaining the lock when the client accesses the same object, realizes the mutually exclusive access of multiple clients to the same object, completes the data synchronization of the object, and ensures the data consistency of concurrent writing and concurrent reading in object storage.
[0008] As a possible implementation, the lock attributes include a lock type and an effective feature. The lock type includes a read lock or a write lock. The effective feature is used to indicate the condition for successfully obtaining the lock.
[0009] Optionally, the effective feature includes a mutual exclusion feature. The mutual exclusion feature is used to indicate that the lock is successfully obtained when the lock is not held by any client.
[0010] Optionally, the effective feature includes a lock waiting timeout feature. The lock waiting timeout feature is used to indicate that the lock is successfully obtained after a preset duration since the object access request of the previous client arrives at the server, or after the object access request of the previous client is successfully executed.
[0011] Optionally, the effective feature includes a lock expiration feature. The lock expiration feature is used to indicate that the lock is successfully obtained when the object access request of the previous client is successfully executed, or when the current time reaches a preset time.
[0012] Optionally, the effective feature includes a lock effective times feature. The lock effective times feature is used to indicate that lock competition occurs among the clients accessing the target object when the number of times the target object is accessed is less than or equal to a preset number of times.
[0013] Optionally, the effective feature includes a lock remaining effective times matching feature. The lock remaining effective times matching feature is used to indicate that the lock is successfully obtained when the number of times the target object is accessed is equal to the preset number of times.
[0014] Optionally, the effective feature includes a lock remaining invalid times matching feature. The lock remaining invalid times matching feature is used to indicate that the lock is successfully obtained when the number of times the target object is accessed is not equal to the preset number of times.
[0015] Optionally, the effective feature includes a data lock feature. The data lock feature is used to indicate that lock competition occurs among the clients accessing the target object when accessing a preset attribute of the metadata or a preset data segment of the target object.
[0016] Based on the different lock types and effective features of the above lock attributes, the server identifies lock operations with different lock attribute semantics through the identifiers of different lock attribute operation protocols, and thus applies the locks corresponding to different types and effective features of lock attributes, which can achieve different effects and improve the applicability and flexibility of the data processing method.
[0017] As a possible implementation, before processing the target data according to the lock attribute of the target object, the server also needs to add a lock attribute to the target object. The server receives a lock request sent by the client. The lock request includes the target object and the lock attribute. The server records the lock attribute into the metadata of the target object. In this way, the client can manage the lock attribute of the object by sending a lock request to the server. Among them, the client that locks the target object and the client that sends the object access request can be the same client or different clients.
[0018] As a possible implementation, the server returns an object access response to the client. The object access response includes the lock attribute and an execution status code, and the execution status code is used to indicate the execution status of the data operation corresponding to the object access request. In this way, the client can know the specific execution status of the object access.
[0019] As a possible implementation, the server obtains a lock according to the lock attribute and the lock inflation mechanism. The secondary retry time of the optimistic lock in the lock inflation mechanism is obtained by using an exponential backoff algorithm based on the number of times the object access request fails. In this way, the polling times in the process of upgrading the lightweight lock to the heavyweight lock are optimized, and the hardware overhead caused by occupying the CPU during lock inflation is reduced.
[0020] In a second aspect, a data processing device is provided, including a transceiver module and a processing module. The transceiver module is configured to receive an object access request sent by a client of an object storage service; the object access request includes a target object. The processing module is configured to, in response to the object access request, determine the lock attribute of the target object; the metadata of the target object includes the lock attribute of the target object, and the metadata of the target object is distributed and stored in at least one object storage device, and the lock attribute is used to indicate the locked state of the target object. The processing module is further configured to obtain a lock according to the lock attribute. The processing module is further configured to perform a data operation corresponding to the object access request on the data of the target object according to the lock.
[0021] As a possible implementation, the lock attribute includes a lock type and an effective feature. The lock type includes a read lock or a write lock. The effective feature is used to indicate the condition for successfully obtaining the lock.
[0022] Optionally, the effective feature includes a mutual exclusion feature. The mutual exclusion feature is used to indicate that the lock is successfully obtained when the lock is not held by any client.
[0023] Optionally, the effective feature includes a lock waiting timeout feature. The lock waiting timeout feature is used to indicate that the lock is successfully obtained after a preset duration has elapsed since the object access request of the previous client reached the server, or after the object access request of the previous client has been successfully executed.
[0024] Optionally, the effective feature includes a lock expiration feature, which is used to indicate that when the object access request of the previous client is successfully executed, or when the current time reaches a preset time, the lock is obtained successfully.
[0025] Optionally, the effective feature includes a lock effective times feature. The lock effective times feature is used to indicate that when the number of times the target object is accessed is less than or equal to a preset number of times, lock competition occurs among the clients accessing the target object.
[0026] Optionally, the effective feature includes a remaining lock effective times matching feature. The remaining lock effective times matching feature is used to indicate that when the number of times the target object is accessed is equal to the preset number of times, the lock is obtained successfully.
[0027] Optionally, the effective feature includes a remaining lock expiration times matching feature. The remaining lock expiration times matching feature is used to indicate that when the number of times the target object is accessed is not equal to the preset number of times, the lock is obtained successfully.
[0028] Optionally, the effective feature includes a data lock feature. The data lock feature is used to indicate that when accessing a preset attribute of metadata or a preset data segment of a target object, lock competition occurs among the clients accessing the target object.
[0029] As a possible implementation, the transceiver module is further configured to: receive a lock request sent by a client, where the lock request includes a target object and lock attributes. The processing module is further configured to: record the lock attributes in the metadata of the target object.
[0030] As a possible implementation, the transceiver module is further configured to: return an object access response to the client. The object access response includes lock attributes and an execution status code, and the execution status code is used to indicate the execution status of the data operation corresponding to the object access request.
[0031] As a possible implementation, the processing module is specifically configured to: obtain a lock according to the lock attributes and a lock inflation mechanism. The secondary retry time of the optimistic lock in the lock inflation mechanism is obtained by using an exponential backoff algorithm based on the number of times the object access request fails.
[0032] As a possible implementation, the data processing device may further include other modules that perform the operation steps of the data processing method described in the second aspect.
[0033] In a third aspect, an object storage service system is provided, including a client and a server. The server is configured to execute the data processing method according to any one of the possible implementation manners in the first aspect above, and the client is configured to send an object access request to the server and receive an object access response sent by the server.
[0034] Regarding the technical principles and beneficial effects of the second and third aspects, reference may be made to the relevant descriptions of the foregoing first aspect, which will not be elaborated herein.
[0035] In a fourth aspect, there is provided a computing device cluster, including at least one computing device, and each computing device includes a processor and a memory. The processor of at least one computing device is configured to execute instructions stored in the memory of at least one computing device, so that the computing device cluster executes the data processing method described in any one of the possible implementation manners of the foregoing first aspect.
[0036] In a fifth aspect, there is provided a computer program product. The computer program product includes a computer program or instructions, and when the computer program or instructions run on a computer, the computer is caused to execute the data processing method described in any one of the possible implementation manners of the foregoing first aspect.
[0037] In a sixth aspect, there is provided a computer-readable storage medium. The readable storage medium includes: a computer program or instructions; when the computer program or instructions run on a computer, the computer is caused to execute the data processing method described in any one of the possible implementation manners of the foregoing first aspect. BRIEF DESCRIPTION OF THE DRAWINGS
[0038] Figure 1 FIG. is a schematic diagram of a scenario of an object storage service provided by the present application;
[0039] Figure 2 FIG. is a schematic diagram of the architecture of an object storage system provided by the present application;
[0040] Figure 3 FIG. is a schematic flowchart of a data processing method provided by the present application;
[0041] Figure 4 FIG. is a schematic diagram of the lock attribute in an object provided by the present application;
[0042] Figure 5 FIG. is a schematic flowchart of a locking step provided by the present application;
[0043] Figure 6 FIG. is a schematic diagram of a locked data segment provided by the present application;
[0044] Figure 7 FIG. is a schematic diagram of a lock inflation mechanism provided by the present application;
[0045] Figure 8 FIG. is a schematic diagram of the structure of a data processing device provided by the present application;
[0046] Figure 9 FIG. is a schematic diagram of the structure of a computing device provided by the present application;
[0047] Figure 10 A structural schematic diagram of a computing device cluster provided by this application;
[0048] Figure 11 A structural schematic diagram of network connection between computing devices provided by this application. Detailed implementation manners
[0049] The data processing method provided by the embodiments of this application can be applied to the object storage service scenario in the storage field. The following briefly introduces the technologies that this application may involve.
[0050] (1) Object Storage Service (OBS): OBS is an object-based storage service, which has the advantages of being massive, secure, highly reliable, and low-cost. OBS is a service oriented to Internet access. Tenants can establish a connection with the object storage service layer (computing nodes supporting OBS) through the object storage client, create a bucket in the storage nodes managed by the object storage service layer, and then access and manage the objects in the bucket. The object storage service layer defaults to using the sequential distribution method to store objects into the bucket, and the sequential distribution method can also be called lexicographical distribution or range distribution.
[0051] (2) Object: An object is the basic unit of data storage in an object storage system. An object is actually a collection of the data of a file and its related attribute information (metadata). The data uploaded by tenants to OBS is saved in the bucket in the form of objects. An object includes three parts: key, metadata, and data. The key, that is, the object name, is, for example, a character sequence encoded in utf-8 with a length greater than 0 and not exceeding 1024. Each object in a bucket has a unique object key.
[0052] (3) Bucket: A bucket is a container for storing objects in OBS. Object storage provides a flat storage method based on buckets and objects. All objects in the bucket are at the same logical level, removing the multi-level tree directory structure in the file system. Each bucket has its own storage class, access permission, region and other attributes. Tenants can create buckets with different storage classes and access permissions, and configure more advanced attributes to meet the storage requirements of different scenarios.
[0053] (4) Object Storage Device (OSD): OSD is the basic storage unit of an object storage system, which is set on a physical disk. Specifically, it is a fixed-size storage space of the physical disk. The object storage system manages the physical disks of multiple computing devices in the form of OSDs.
[0054] Tenants can log in to the cloud management platform on the public cloud access page through pre-registered account passwords. After successful login, on the public cloud access page, they can select and purchase corresponding public cloud services, such as OBS, virtual machine services, container services, etc. For OBS, tenants can further create multiple buckets through the configuration interface or application programming interface (API) on the public cloud access page provided by the cloud platform. There is no limit to the total number and size of objects stored in each bucket, and tenants do not need to consider data scalability. OBS is a service based on the Representational State Transfer API (REST API), Hypertext Transport Protocol (HTTP), and Hypertext Transfer Protocol Secure (HTTPS). Tenants can use the Uniform Resource Locator (URL), also known as the access domain name in this application and simply referred to as the domain name, to locate bucket resources. Among them, in OBS, the bucket name is globally unique and cannot be modified, that is, the buckets created by a tenant cannot have the same name as other buckets created by the tenant himself, nor can they have the same name as the buckets created by other tenants.
[0055] Figure 1 shows a schematic diagram of at least one application scenario in an embodiment of this application. As Figure 1 shown, each bucket can include multiple objects, and the objects between buckets are isolated from each other. Tenants log in to the cloud management platform through the object storage client, select and purchase the cloud service of the object storage service on the cloud management platform. After purchase, tenants can perform object storage based on the functions provided by the object storage service. Among them, the cloud management platform is mainly used to manage the infrastructure for running the object storage service. Exemplarily, the infrastructure for running the object storage service can include multiple data centers set in different regions, and each data center includes multiple servers. The data center can provide basic resources for the object storage service, such as computing resources, storage resources, etc. Therefore, when tenants purchase and use the object storage service, they mainly pay for the resources used. Specifically, the object storage service provides the domain name of the bucket, and tenants can access the domain name of the bucket through the object storage client, so as to upload data to the bucket or download data from the bucket. Among them, the uploaded data is stored in the bucket in the form of objects.
[0056] Figure 2The figure shows a schematic architecture diagram of an object storage system in an embodiment of the present application. This object storage system can also be referred to as an object storage service system, and includes a client of object storage, an object storage service layer, and an object storage device.
[0057] Among them, the client of object storage can be set in a terminal device, and the terminal device can be a mobile phone, a laptop computer, a tablet computer, a personal digital assistant, a wireless terminal in a smart city, or a wireless device in a smart home. A tenant logs in to the cloud management platform through the client, and selects and purchases cloud services for object storage on the cloud management platform.
[0058] The object storage service layer is used to manage one or more object storage devices. The object storage service layer includes one or more servers, and the one or more servers can be set in service devices of the service layer. Exemplarily, the service device is connected to the storage device through a switching device. Additionally, the service device is connected to the terminal device through a remote connection gateway.
[0059] The tenant logs in to the object storage service layer using an account on the client, and selects and purchases cloud services for object storage in the service layer, such as creating a bucket, configuring the bucket name, etc. After the service layer detects the tenant's operation (such as creating a bucket), it can send a creation command to the object storage device. The creation command includes information such as the bucket name and domain name, and is used to notify the storage device to create a bucket and save information such as the bucket name and bucket domain name. After creating the bucket, the tenant can access the bucket domain name through the client of object storage, locate the bucket in the storage device, and upload, download, or delete objects from the bucket.
[0060] The service layer includes a control unit and an operating system. The control unit runs on the operating system, and the operating system includes a disk driver and a physical network card driver. The control unit controls the disk controller through the disk driver to set a physical disk as multiple persistence log (PLOG) units.
[0061] The infrastructure for running the object storage service can be set up in multiple data centers in different regions (Regions). Exemplarily, the object storage devices are set in multiple data centers in different Regions, and each data center includes multiple object storage devices. Each object storage device includes at least one physical disk. Exemplarily, the disk controller sets physical disk 1 and physical disk 2 in Region 1 as 4 PLOGs.
[0062] After the service layer receives the creation instruction for bucket 1, it notifies the control unit to create bucket 1. The control unit, through the operating system, creates bucket 1 on physical disk 11 and physical disk 12 in the object storage device. The PLOG of bucket 1 is distributed in physical disk 11 and physical disk 12, and information such as the bucket name and bucket domain name of bucket 1 is saved.
[0063] When a tenant needs to upload an object to bucket 1 through the client, the tenant can trigger the object storage device to write the object into bucket 1 through the client. For example, the control unit in the object storage service layer can write the object into bucket 1.
[0064] When a tenant needs to download an object from bucket 1 through the client, the tenant can trigger the object storage device to read the object from bucket 1 through the client. For example, the control unit in the service layer can obtain the object from bucket 1 and feedback it to the client.
[0065] In the embodiments of the present application, the client of the object storage service uses the APIs for uploading and downloading objects to operate on the objects of the server, and the server returns the object lock attribute status and the execution status code to the client, and the communication and operations between the client and the server follow the operation protocol of the OBS lock attribute.
[0066] It should be noted that Figure 1 and Figure 2 are only schematic diagrams and should not be construed as limitations on the present application. Any server or object storage device of an object storage with object storage functions can apply the data processing method provided by the present application. On the other hand, the application scenarios of the object storage shown above Figure 1 and Figure 2 the architecture of the object storage system shown may also include other devices or architectures, which are not shown in Figure 1 or Figure 2 .
[0067] Next, the data processing method provided in this embodiment will be specifically described with reference to the accompanying drawings.
[0068] The steps of the data processing method provided by the present application are executed by any client in the client of the object storage shown in Figure 2 and any server in the object storage service layer. Next, with reference to Figure 3 , the data processing method provided by the present application will be described.
[0069] Step 301: The client sends an object access request to the server.
[0070] The client sends an object access request to the server in response to the tenant's operation.
[0071] As a possible implementation, the object access request can be an upload request or a download request, including the target object and the data operation.
[0072] Optionally, the target object in the object access request can be the unique identification name (identifier) of the object. The data operation can be data reading, data writing, etc.
[0073] In a possible embodiment, the client can call application programming interfaces corresponding to functions such as uploading, downloading, and deleting to implement the interaction of object access requests, which will not be elaborated here.
[0074] Step 302, the server receives an object access request sent by the client.
[0075] Step 303, the server determines the lock attribute corresponding to the object access request.
[0076] The server determines the lock attribute corresponding to the object access request according to the type of data operation. The lock attribute is used to indicate the locking state of the target object.
[0077] As a possible implementation, the server extracts the lock attribute corresponding to the type of data operation from the metadata of the target object.
[0078] Optionally, the target object includes metadata. The metadata includes the lock attribute of the target object. The locking state is used to indicate the locking policy when the target object is performing a certain type of data operation, such as the corresponding lock type and the conditions under which the lock is successfully acquired, and can also be referred to as the lock type and the effective characteristics.
[0079] Such as Figure 4 shown, at least one object is stored in a bucket of the object storage device. The object includes metadata information. In addition to traditional attributes such as life cycle expiration information, creation time, version number, server-side encryption method, and storage type, the metadata information also includes a lock attribute.
[0080] Exemplarily, the lock type can include a read lock (read), a write lock (write), an unlock (unlock), etc. When the data operation is data reading, the lock attribute corresponding to the object access request is the lock attribute with the lock type being a read lock. When the data operation is data writing, the lock attribute corresponding to the object access request is the lock attribute with the lock type being a write lock. Locks of different lock types can be stored in different data chunks.
[0081] Exemplarily, the effective characteristic is used to indicate the condition for successfully acquiring the lock. The effective characteristic can include a lock waiting timeout characteristic, a lock expiration characteristic, a lock effective times characteristic, a lock remaining effective times matching characteristic, or a data lock characteristic, etc.
[0082] The mutual exclusion characteristic is used to indicate that when the lock is not held by any thread, the lock is successfully acquired.
[0083] The lock waiting timeout characteristic is used to indicate that when a preset duration has passed since the object access request of the previous client reached the server, or after the object access request of the previous client has been successfully executed, the lock is successfully acquired.
[0084] The lock expiration feature is used to indicate that the lock is acquired successfully when the object access request of the previous client is executed successfully or the current time reaches the preset time.
[0085] The lock effective times feature is used to indicate that when the number of times the target object is accessed is less than or equal to the preset number of times, lock competition occurs among the clients accessing the target object.
[0086] The lock remaining effective times matching feature is used to indicate that the lock is acquired successfully when the number of times the target object is accessed is equal to the preset number of times.
[0087] The lock remaining invalid times matching feature is used to indicate that the lock is acquired successfully when the number of times the target object is accessed is not equal to the preset number of times.
[0088] The data lock feature is used to indicate that when accessing the preset attributes of the metadata or the preset data segment of the target object, lock competition occurs among the clients accessing the target object.
[0089] Step 304: The server obtains the lock according to the lock attributes.
[0090] The server attempts to obtain the lock according to the effective feature of the lock attributes.
[0091] As a possible implementation, the principle for the server to obtain the lock is similar to the principle for a thread in an existing operating system to attempt to obtain the lock, which will not be elaborated here.
[0092] In this embodiment, the difference between the process for the server to obtain the lock according to the lock attributes and the process for a thread in an existing operating system to obtain the lock is that the server determines whether the lock is obtained successfully according to the effective feature, and the judgment methods for whether the lock is obtained successfully corresponding to different effective features are different. For the lock acquisition scenarios corresponding to the above lock waiting timeout feature, lock expiration feature, lock effective times feature, lock remaining effective times matching feature, or data lock feature, etc., please refer to the exemplary descriptions of the specific scenarios of multiple clients competing for the lock attributes with different effective features later, which will not be elaborated here.
[0093] Step 305: The server performs the data operation corresponding to the object access request on the data of the target object according to the lock.
[0094] Step 306: The server returns an object access response to the client.
[0095] The object access response includes lock attributes and an execution status code.
[0096] As a possible implementation, the lock attributes are represented in the form of identifiers such as characters, strings, etc. For the identifiers of the locks with the above different lock types and effective features, please refer to the exemplary descriptions of the specific scenarios of multiple clients competing for the lock attributes with different effective features later, which will not be elaborated here.
[0097] As a possible implementation, the execution status code is used to indicate the execution status of the data operation corresponding to the object access request.
[0098] Optionally, the execution status code may include 200 ("OK"), 409 ("Conflict"), 412 ("PreconditionFailed"), etc., and may also include other response codes in the common Hypertext Transfer Protocol (HTTP), such as 400 ("Bad Request"), 500 ("Internal Server Error"), etc.
[0099] Among them, 200 indicates that everything is normal, and the document in the entity body (if any) is a representation of a certain resource. 409 indicates that this response code is sent when the client executes an operation that will cause one or more resources to be in an inconsistent state. 412 indicates that the client specifies some preconditions in the request header field and requires the server to process this request only when certain conditions are met. In addition, for other possible meanings of the execution status code, please refer to the existing HTTP, which will not be elaborated here.
[0100] Step 307: The client receives the object access response.
[0101] Based on the above data processing method, the server adds a lock attribute to the object itself stored in the object storage service, supports the server to operate on the object based on the semantics of the lock attribute, so that it can execute the successful data operation of obtaining the lock when the client accesses the same object, realizes the mutually exclusive access of multiple clients to the same object, completes the data synchronization of the object, and ensures the data consistency problem of concurrent writing and concurrent reading in the object storage.
[0102] The above text has elaborated in detail how the server obtains a lock for an object access request to execute the data operation corresponding to the object access request. Before the server extracts the lock attribute corresponding to the object access request from the metadata of the target object, it is also necessary to lock the target object. For example, the lock attributes corresponding to different data operations of an object are written into the metadata of the object. The following combines Figure 5 , to illustrate the process of adding a lock attribute to the target object in the data processing method provided by this application.
[0103] Step 501: The client sends a lock request to the server.
[0104] The client sends a lock request to the server in response to the operation of the tenant.
[0105] As a possible implementation, the lock request includes the target object and the lock attribute.
[0106] Optionally, the target object in the lock request may be the unique identification name (identifier) of the object. The lock attribute is the lock attribute corresponding to the target object.
[0107] Optionally, the lock attribute is carried in the header field of the lock request.
[0108] Step 502: The server receives the lock request.
[0109] Step 503: The server records the lock attribute into the metadata of the target object.
[0110] For the way of recording the lock attribute in the metadata, please refer to Figure 4 , which will not be elaborated here.
[0111] In this way, the client and the server can communicate based on the common lock attribute operation protocol, and implement related operations such as locking and lock competition of the target object based on the lock attribute in the header field of the request or the metadata of the target object.
[0112] The above text has given an overall description of the process of locking and obtaining a lock. Next, an exemplary description will be given of the specific scenario where multiple clients compete for locks with lock attributes of different effective features.
[0113] For an object access request with the data operation being object read, the corresponding lock type is a read lock. The read lock can be a lock corresponding to tasks (objects) such as GET / READ, etc. The read lock can include locks with effective features such as mutual exclusion feature, data lock feature, etc. In addition, the write lock can be a lock corresponding to tasks such as PUT / POST, etc.
[0114] The identifier of the lock with the mutual exclusion feature in the lock attribute operation protocol can be "X-obs-lock: read".
[0115] Client 1 sends a lock request carrying the lock attribute header field "X-obs-lock: read" to instruct the server to record the lock attribute into the metadata of the target object, and complete the setting of the read lock for the target object.
[0116] Client 2 sends an object access request to the server at the first moment, and Client 3 sends an object access request to the server at the second moment. If the first moment is earlier than the second moment, then Client 2 successfully obtains the lock, and Client 3 will successfully obtain the lock after Client 2's access ends.
[0117] In other possible embodiments, the write lock with the mutual exclusion feature has the same principle as the above-mentioned read lock with the mutual exclusion feature, which will not be elaborated here.
[0118] The identifier of the lock with the lock waiting timeout feature in the lock attribute operation protocol can be "X-obs-lock-await".
[0119] Client 1 sends a lock request carrying the lock attribute header field "X-obs-lock-await: 10s" to instruct the server to record the lock attribute in the metadata of the target object and complete the read lock setting of the target object.
[0120] Client 2 sends an object access request to the server at the first moment, and Client 3 sends an object access request to the server at the second moment. If the first moment is equal to the second moment, then Client 2 successfully acquires the lock, and Client 3 will successfully acquire the lock 10 seconds after Client 2's object access request reaches the server or after Client 2's access ends.
[0121] In other possible embodiments, the write lock of the lock waiting timeout feature has the same principle as the read lock of the above waiting timeout feature, which will not be elaborated here.
[0122] The identifier of the lock with the lock expiration feature in the lock attribute operation protocol can be "X-obs-lock-expire".
[0123] Client 1 sends a lock request carrying the lock attribute header field "X-obs-lock-expire: Mon, 06 Feb 2023 21:18:15 GMT" to instruct the server to record the lock attribute in the metadata of the target object and complete the read lock setting of the target object.
[0124] Client 2 sends an object access request to the server at the first moment, and Client 3 sends an object access request to the server at the second moment. If the first moment is equal to the second moment, then before the moment "Mon, 06 Feb 2023 21:18:15", Client 2 successfully acquires the lock, and Client 3 will successfully acquire the lock after Client 2's access ends; after the moment "Mon, 06 Feb 2023 21:18:15", both Client 2 and Client 3 can successfully acquire the lock simultaneously.
[0125] In other possible embodiments, the write lock of the lock expiration feature has the same principle as the read lock of the above lock expiration feature, which will not be elaborated here.
[0126] The identifier of the lock with the lock effective times feature in the lock attribute operation protocol can be "X-obs-lock-count-down".
[0127] Client 1 sends a lock request carrying the lock attribute header field "X-obs-lock-count-down: 3" to instruct the server to record the lock attribute in the metadata of the target object and complete the read lock setting of the target object.
[0128] Client 2 to Client 5 send object access requests to the server in ascending order of client numbers. Then, Client 2, Client 3, and Client 4 successfully acquire the lock in the order of sending object access requests, that is, the next client successfully acquires the lock after the previous client finishes data operations. Client 4 and Client 5 acquire the lock successfully at the same time.
[0129] In other possible embodiments, the write lock of the lock effective times feature has the same principle as the read lock of the above lock effective times feature, which will not be elaborated here.
[0130] The identifier of the lock with the lock remaining effective times matching feature in the lock attribute operation protocol can be "X-obs-if-match-lock-count".
[0131] Client 1 sends a lock request carrying the lock attribute header field "X-obs-if-match-lock-count: 2" to instruct the server to record the lock attribute in the metadata of the target object and complete the read lock setting of the target object.
[0132] Client 2 to Client 5 send object access requests to the server at the same time. Then, Client 2, Client 3, and Client 5 access fails, and the server returns an object access response carrying the execution status code 409 to Client 2, Client 3, and Client 5. Client 4 accesses successfully.
[0133] In other possible embodiments, the write lock of the lock remaining effective times matching feature has the same principle as the read lock of the above lock remaining effective times matching feature.
[0134] The identifier of the lock with the lock remaining invalid times matching feature in the lock attribute operation protocol can be "X-obs-if-none-match-lock-count", and its effective feature is opposite to that of the lock with the lock remaining effective times matching feature, which will not be elaborated here.
[0135] The identifier of the lock with the data lock feature in the lock attribute operation protocol can be "X-obs-lock-range".
[0136] Client 1 sends a lock request carrying the lock attribute header field "X-obs-lock-range: meta xxx|data 0-1199" to instruct the server to record the lock attribute in the metadata of the target object and complete the read lock setting of the target object.
[0137] If both Client 2 and Client 3 send object access requests to the server, there will be lock contention when Client 2 and Client 3 access the attributes of the metadata indicated by "meta xxx" or the data segment indicated by "data 0 - 1199". Here, xxx represents the identifier of the data segment, and 0 - 1199 represents the range of the data segment.
[0138] The above object segmentation-based locking can be referred to for illustration Figure 6 An object includes multiple segments of data. The lock attribute stored in the metadata of the object indicates adding a lock to the locked segment data, where the locked segment data is the data segment 0 - 1199 in the data segment 0 - 2399. For example, when any client accesses an object, it polls the stored lock attribute to query whether each segment of data of the object is locked segment data. In this way, the lock mechanism provided by the embodiments of the present application supports scenarios of segmented uploading or downloading, supports locking at different granularities, and improves the accuracy and efficiency of locking and data processing.
[0139] In a possible application scenario of this embodiment, the server identifies lock operations with different lock attribute semantics through the identifiers of different lock attribute operation protocols, and thus applies the locks corresponding to different types and effective characteristics of lock attributes, which can achieve different effects. For example, when multiple clients concurrently download a batch of objects, a lock attribute with a mutual exclusion feature can be added to the objects to implement data sharding. In this way, customers do not need to develop sharding strategies.
[0140] Combined with the application scenarios of the locks with the above various lock attributes, it can be seen that in an object storage system, a server can simultaneously receive and process object access requests from one or more clients. When any client fails to obtain a lock during lock contention and retries to obtain the lock for the second time, it may retry to obtain the lock for the second time. To improve the overall access efficiency of the object storage system, an exponential backoff inflation lock can also be introduced based on the lock attribute, that is, a lock inflation mechanism is used, and the second retry time of the optimistic lock in the lock inflation mechanism is obtained by an exponential backoff algorithm based on the number of times of object access request failures. The exponential backoff algorithm is an algorithm that gradually reduces the rate of a certain process by multiples through feedback to gradually find a suitable rate.
[0141] As Figure 7 shown, the lock inflation process in the object storage of the present application, also known as lock upgrade, refers to the process in which the state of the lock upgrades from no lock to biased lock, from biased lock to lightweight lock, and from lightweight lock to heavyweight lock. Among them, the biased lock and the lightweight lock can be regarded as optimistic locks, and the heavyweight lock can be regarded as a pessimistic lock.
[0142] During the process of upgrading from lightweight lock to heavyweight lock, the delay time between every two lock acquisition actions is determined based on exponential backoff.
[0143] Exemplarily, after the server locks the target object according to client 1, when receiving the object access request of client 3 for the target object before finishing processing the object access request of client 2 for the target object. At this time, the server will first try to acquire the lock, not yield the CPU, wait for the delay time, and then try to acquire the lock again. If the lock cannot be acquired more than 3 times, the CPU will be relinquished to execute other tasks. After the server finishes processing the object access request of client 2, it notifies the CPU, loads the context, and switches to the processing of client 3. When the server acquires the lock for the object access request of client 3, if the lock is acquired within 3 times, no CPU switching and context loading operations will occur. When the data length processed by client 2 is greater than 3 concurrent disk input / output (I / O), currently 3 * 512K * 4 (6M), a context switching behavior will directly occur and wait for scheduling.
[0144] Among them, the delay time refers to the duration determined based on exponential backoff, such as 10ms, etc., and it gets longer each time according to the exponential inflation mechanism.
[0145] Below, taking t = b c as the modeling function to illustrate the exponentially backoff inflated lock. Among them, t is the delay time between two actions (lock acquisition actions) (the secondary retry time of the optimistic lock), b is the base number, b = 2 when using the binary exponential backoff algorithm, c is the number of times of adverse events, which refers to the number of failures of the lock acquisition operation when the optimistic lock inflates to a pessimistic lock in this embodiment, and following the lock inflation mechanism of the JVM, c = 10.
[0146] The embodiments of the present application use the above-mentioned lock inflation mechanism based on exponential backoff to optimize the polling times during the process of upgrading the lightweight lock to the heavyweight lock, and reduce the hardware overhead caused by occupying the CPU during lock inflation.
[0147] To cooperate with the above Figure 3 shown data processing method provided by the present application, the present application also provides a data processing device 800, and this data processing device 800 can be used to implement the functions of the server in the above Figure 3 shown data processing method. As Figure 8 shown, the data processing device 800 includes a transceiver module 810 and a processing module 820.
[0148] The transceiver module 810 is used to receive the object access request sent by the client of the object storage service; the object access request includes the target object.
[0149] A processing module 820, configured to determine the lock attribute of a target object in response to an object access request; the metadata of the target object includes the lock attribute of the target object, and the metadata of the target object is distributed and stored in at least one object storage device, and the lock attribute is used to indicate the locking state of the target object.
[0150] The processing module 820 is further configured to obtain a lock according to the lock attribute.
[0151] The processing module 820 is further configured to perform a data operation corresponding to the object access request on the data of the target object according to the lock.
[0152] As a possible implementation, the lock attribute includes a lock type and an effective feature, the lock type includes a read lock or a write lock, and the effective feature is used to indicate the condition for successfully obtaining the lock.
[0153] As a possible implementation, the effective feature includes a mutual exclusion feature, and the mutual exclusion feature is used to indicate that the lock is successfully obtained when the lock is not held by any client.
[0154] As a possible implementation, the effective feature includes a lock waiting timeout feature, and the lock waiting timeout feature is used to indicate that the lock is successfully obtained after a preset duration since the object access request of the previous client reaches the server, or after the object access request of the previous client is successfully executed.
[0155] As a possible implementation, the effective feature includes a lock expiration feature, and the lock expiration feature is used to indicate that the lock is successfully obtained when the object access request of the previous client is successfully executed, or when the current time reaches a preset time.
[0156] As a possible implementation, the effective feature includes a lock effective times feature, and the lock effective times feature is used to indicate that lock competition occurs among clients accessing the target object when the number of times the target object is accessed is less than or equal to a preset number of times.
[0157] As a possible implementation, the effective feature includes a lock remaining effective times matching feature, and the lock remaining effective times matching feature is used to indicate that the lock is successfully obtained when the number of times the target object is accessed is equal to a preset number of times.
[0158] As a possible implementation, the effective feature includes a lock remaining invalid times matching feature, and the lock remaining invalid times matching feature is used to indicate that the lock is successfully obtained when the number of times the target object is accessed is not equal to a preset number of times.
[0159] As a possible implementation, the effective feature includes a data lock feature, and the data lock feature is used to indicate that lock competition occurs among clients accessing the target object when accessing a preset attribute of the metadata or a preset data segment of the target object.
[0160] As a possible implementation, the transceiver module 810 is further configured to: receive a locking request sent by a client; the locking request includes a target object and lock attributes. The processing module 820 is further configured to record the lock attributes into the metadata of the target object.
[0161] Optionally, the header field of the locking request carries the lock attributes.
[0162] As a possible implementation, the transceiver module 810 is further configured to: return an object access response to the client; the object access response includes lock attributes and an execution status code, and the execution status code is used to indicate the execution status of the data operation corresponding to the object access request.
[0163] As a possible implementation, the processing module 820 is further configured to: obtain a lock according to the lock attributes and a lock inflation mechanism; the secondary retry time of the optimistic lock in the lock inflation mechanism is obtained by using an exponential backoff algorithm based on the number of failures of the object access request.
[0164] Among them, both the transceiver module 810 and the processing module 820 can be implemented by software or by hardware. Exemplarily, next, taking the transceiver module 810 as an example, the implementation manner of the transceiver module 810 will be introduced. Similarly, the implementation manner of the processing module 820 can refer to the implementation manner of the transceiver module 810.
[0165] As an example of a software functional unit, the transceiver module 810 may include code running on a computing instance. Among them, the computing instance may include at least one of a physical host (computing device), a virtual machine, and a container. Further, the above computing instance may be one or more. For example, the transceiver module 810 may include code running on multiple hosts / virtual machines / containers. It should be noted that the multiple hosts / virtual machines / containers for running the code may be distributed in the same region, or may be distributed in different regions. Further, the multiple hosts / virtual machines / containers for running the code may be distributed in the same availability zone (AZ), or may be distributed in different AZs, and each AZ includes one data center or multiple geographically close data centers. Among them, generally, one region may include multiple AZs.
[0166] Similarly, the multiple hosts / virtual machines / containers for running the code may be distributed in the same virtual private cloud (VPC), or may be distributed in multiple VPCs. Among them, generally, one VPC is set within one region. For cross-region communication between two VPCs within the same region and between VPCs in different regions, a communication gateway needs to be set in each VPC, and the interconnection between VPCs is realized through the communication gateway.
[0167] As an example of a hardware functional unit, the transceiver module 810 may include at least one computing device, such as a server or the like. Alternatively, the transceiver module 810 may also be a device implemented using an application-specific integrated circuit (ASIC) or a programmable logic device (PLD). Among them, the above PLD may be implemented by a complex programmable logic device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL), or any combination thereof.
[0168] The multiple computing devices included in the transceiver module 810 may be distributed in the same region or in different regions. The multiple computing devices included in the transceiver module 810 may be distributed in the same availability zone (AZ) or in different AZs. Similarly, the multiple computing devices included in the transceiver module 810 may be distributed in the same virtual private cloud (VPC) or in multiple VPCs. Among them, the multiple computing devices may be any combination of computing devices such as servers, ASICs, PLDs, CPLDs, FPGAs, and GALs.
[0169] It should be noted that in other embodiments, any one of the transceiver module 810 and the processing module 820 may be used to execute any step in the data processing method. The steps to be implemented by the transceiver module 810 and the processing module 820 can be specified as needed. By implementing different steps in the data processing method through the transceiver module 810 and the processing module 820 respectively, all functions of the data processing apparatus 800 are realized.
[0170] This application also provides a computing device 900. As Figure 9 shown, the computing device 900 includes: a bus 902, a processor 904, a memory 906, and a communication interface 908. The processor 904, the memory 906, and the communication interface 908 communicate with each other through the bus 902. The computing device 900 may be a server or a terminal device. It should be understood that this application does not limit the number of processors and memories in the computing device 900.
[0171] The bus 902 can be a Peripheral Component Interconnect (PCI) bus, an Extended Industry Standard Architecture (EISA) bus, or the like. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 9 only one line is used in Figure 9 , but it does not mean that there is only one bus or one type of bus. The bus 902 can include a path for transmitting information between various components of the computing device 900 (e.g., the memory 906, the processor 904, the communication interface 908).
[0172] The processor 904 can include any one or more of a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP), or a digital signal processor (DSP).
[0173] The memory 906 can include volatile memory, such as random access memory (RAM). The memory 906 can also include non-volatile memory, such as read-only memory (ROM), flash memory, a hard disk drive (HDD), or a solid state drive (SSD).
[0174] The memory 906 stores executable program code, and the processor 904 executes the executable program code to respectively implement the functions of the various modules included in the foregoing data processing apparatus 800, thereby implementing the data processing method. That is, the memory 906 stores instructions for executing the data processing method.
[0175] Alternatively, the memory 906 stores executable code, and the processor 904 executes the executable code to respectively implement the functions of the foregoing client or server, thereby implementing the data processing method. That is, the memory 906 stores instructions for executing the data processing method.
[0176] The communication interface 908 uses a transceiver module such as, but not limited to, a network interface card, a transceiver, etc., to implement communication between the computing device 900 and other devices or a communication network.
[0177] Considering that the data processing method provided in this application is applied to an object storage system, the object storage system may include multiple clients or multiple servers. Therefore, this application also provides a cluster of computing devices. The cluster of computing devices includes at least one computing device. The computing device can be a server, such as a central server, an edge server, or a local server in a local data center. In some embodiments, the computing device can also be a terminal device such as a desktop computer, a laptop computer, or a smart phone.
[0178] As Figure 10 shown, the cluster of computing devices includes at least one computing device 900. Instructions for executing the data processing method that are the same can be stored in the memory 906 of one or more of the computing devices 900 in the cluster of computing devices.
[0179] In some possible implementation manners, partial instructions for executing the data processing method can also be stored separately in the memory 906 of one or more of the computing devices 900 in the cluster of computing devices. In other words, a combination of one or more computing devices 900 can jointly execute the instructions for executing the data processing method.
[0180] It should be noted that the memories 906 in different computing devices 900 in the cluster of computing devices can store different instructions, which are respectively used to execute partial functions of the data processing device 800. That is to say, the instructions stored in the memories 906 of different computing devices 900 can implement the functions of one or more modules included in the data processing device 800.
[0181] In some possible implementation manners, one or more of the computing devices in the cluster of computing devices can be connected through a network. Among them, the network can be a wide area network or a local area network, etc. Figure 11 shows a possible implementation manner. As Figure 11 shown, two computing devices 900A and 900B are connected through a network. Specifically, they are connected to the network through the communication interfaces in each computing device. In this type of possible implementation manner, instructions for executing the functions of one or more of the transceiver module 810 and the processing module 820 are stored in the memory 906 of the computing device 900A. Figure 11 Taking the example that instructions for executing the function of the transceiver module 810 are stored in the memory 906 of the computing device 900A. At the same time, instructions for executing the functions of one or more of the transceiver module 810 and the processing module 820 are stored in the memory 906 of the computing device 900B. Figure 11 Taking the example that instructions for executing the function of the processing module 820 are stored in the memory 906 of the computing device 900B.
[0182] It should be understood that Figure 11The functions of the computing device 900A shown can also be completed by multiple computing devices 900. Similarly, the functions of the computing device 900B can also be completed by multiple computing devices 900.
[0183] Embodiments of this application also provide a computer program product containing instructions. The computer program product can be software or a program product that contains instructions and can run on a computing device or be stored in any available medium. When the computer program product runs on at least one computing device, it causes at least one computing device to execute the data processing method as Figure 3 shown, or the steps executed by the client or the server in the data processing method as Figure 3 shown.
[0184] Embodiments of this application also provide a computer-readable storage medium. The computer-readable storage medium can be any available medium that a computing device can store or a data storage device such as a data center that contains one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state drive), etc. The computer-readable storage medium includes instructions that direct the computing device to execute the data processing method as Figure 3 shown, or the steps executed by the client or the server in the data processing method as Figure 3 shown.
[0185] The above embodiments can be implemented in whole or in part by software, hardware (such as circuits), firmware, or any arbitrary combination thereof. When implemented using software, the above embodiments can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, the processes or functions described in the embodiments of this application are generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (such as infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that contains one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium. The semiconductor medium can be a solid-state drive.
[0186] Those of ordinary skill in the art can realize that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be implemented by electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are executed in a hardware or software manner depends on the specific application and design constraints of the technical solution. Professional technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of this application.
[0187] Those skilled in the art can clearly understand that for the convenience and conciseness of description, the specific working processes of the systems, devices, and units described above can refer to the corresponding processes in the foregoing method embodiments, and will not be elaborated herein.
[0188] In several embodiments provided in this application, it should be understood that the disclosed systems, devices, and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed couplings or direct couplings or communication connections to each other can be through some interfaces. The indirect couplings or communication connections of the devices or units can be electrical, mechanical, or other forms.
[0189] The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they can be located in one place, or distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.
[0190] In addition, the functional units in each embodiment of this application can be integrated in a processing unit, or each unit can exist physically alone, or two or more units can be integrated in one unit.
[0191] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in the various embodiments of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory, a random access memory, a magnetic disk, or an optical disk.
[0192] In the present application, "at least one" means one or more, and "plurality" means two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone, where A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can mean: a, b, c, a and b, a and c, b and c, or a and b and c, where a, b and c can be single or multiple.
[0193] It should be noted that, in this application, words such as "exemplary" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary" or "for example" in this application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific way.
[0194] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the protection scope of the technical solutions of the embodiments of the present invention.
Claims
1. A data processing method, characterized in that: A server end applied to an object storage service, the server end is used to access at least one object storage device managed by a cloud platform, the at least one object storage device is located in multiple data centers in different regions, each data center includes multiple servers, and the method includes: Receiving an object access request sent by a client of the object storage service; the object access request includes a target object; In response to the object access request, determining a lock attribute of the target object; the metadata of the target object includes the lock attribute of the target object, the metadata of the target object is distributed and stored in the at least one object storage device, and the lock attribute is used to indicate a locked state of the target object; Acquire a lock according to the lock attribute; According to the lock, a data operation corresponding to the object access request is performed on the data of the target object.
2. The method according to claim 1, characterized in that: The lock attributes include a lock type and a validation feature, the lock type includes a read lock or a write lock, and the validation feature is used to indicate a condition for successfully acquiring the lock.
3. The method according to claim 2, characterized in that The validation feature includes a mutually exclusive feature, and the mutually exclusive feature is used to indicate that when the lock is not held by any client, the lock is successfully acquired.
4. The method according to claim 2, characterized in that: The effectiveness feature includes a lock waiting timeout feature, and the lock waiting timeout feature is used to indicate that the lock is successfully acquired after a preset time has passed since the object access request of the previous client arrived at the server, or after the object access request of the previous client is successfully executed.
5. The method according to claim 2, characterized in that: The effectiveness feature includes a lock expiration feature, and the lock expiration feature is used to indicate that the lock is successfully acquired when the object access request of the previous client is successfully executed or the current time reaches a preset time.
6. The method according to claim 2, characterized in that The effectiveness feature includes a lock effectiveness times feature, and the lock effectiveness times feature is used to indicate that when the target object is accessed less than or equal to a preset number of times, lock competition occurs between clients accessing the target object.
7. The method according to claim 2, characterized in that The effectiveness feature includes a lock remaining effectiveness times matching feature, and the lock remaining effectiveness times matching feature is used to indicate that when the target object is accessed a number of times equal to a preset number of times, the lock is successfully acquired.
8. The method according to claim 2, characterized in that: The effectiveness feature includes a lock remaining failure count matching feature, and the lock remaining failure count matching feature is used to indicate that when the number of times the target object is accessed is not equal to a preset number of times, the lock is successfully acquired.
9. The method according to claim 2, characterized in that: The validation feature includes a data lock feature, and the data lock feature is used to indicate that when accessing a preset attribute of metadata or a preset data segment of a target object, lock competition occurs between clients accessing the target object.
10. The method according to any one of claims 1 to 9, characterized in that Before receiving the object access request sent by the client of the object storage service, the method further includes: receiving a lock request sent by the client; the lock request includes the target object and the lock attribute; The lock attribute is recorded in metadata of the target object.
11. The method according to claim 10, characterized in that The header field of the lock request carries the lock attribute.
12. The method according to claim 11, characterized in that After performing the data operation corresponding to the object access request on the data of the target object according to the lock, the method further includes: An object access response is returned to the client; the object access response includes the lock attribute and an execution status code, and the execution status code is used to indicate the execution status of the data operation corresponding to the object access request.
13. The method according to any one of claims 1 to 12, characterized in that The method further comprises: The lock is acquired according to the lock attribute and the lock expansion mechanism; the second retry time of the optimistic lock in the lock expansion mechanism is obtained by using an exponential backoff algorithm based on the number of failed object access requests.
14. A data processing device, characterized in that: include: A transceiver module, configured to receive an object access request sent by a client of the object storage service; The object access request includes a target object; A processing module, configured to determine a lock attribute of the target object in response to the object access request; The metadata of the target object includes a lock attribute of the target object, the metadata of the target object is distributed and stored in the at least one object storage device, and the lock attribute is used to indicate a locking state of the target object; The processing module is further used to obtain the lock according to the lock attribute; The processing module is further configured to execute a data operation corresponding to the object access request on the data of the target object according to the lock.
15. An object storage service system, characterized in that: include: The server is used to receive object access requests from clients of the object storage service; The object access request includes a target object, and the target object is stored in a target bucket of at least one object storage device; In response to the object access request, determining a lock attribute of the target object; metadata of the target object includes the lock attribute of the target object, the metadata of the target object is distributed and stored in at least one object storage device, and the lock attribute is used to indicate a locked state of the target object; acquiring a lock according to the lock attribute; According to the lock, performing a data operation corresponding to the object access request on the data of the target object; The client is used to send an object access request to the server and receive an object access response sent by the server; the object access response includes the lock attribute and an execution status code, and the execution status code is used to indicate the execution status of the data operation corresponding to the object access request.
16. A computing device cluster, characterized in that: comprising at least one computing device, each computing device comprising a processor and a memory; The processor of the at least one computing device is configured to execute instructions stored in the memory of the at least one computing device, so that the computing device cluster executes the method according to any one of claims 1 to 13.
17. A computer program product comprising instructions, characterized in that When the instructions are executed by a computing device cluster, the computing device cluster executes the method according to any one of claims 1 to 13.
18. A computer-readable storage medium, characterized in that: The method comprises computer program instructions. When the computer program instructions are executed by a computing device cluster, the computing device cluster performs the method according to any one of claims 1 to 13.