Data access method, device, system and program product

Abstract

The invention discloses a data access method, device and system and a program product, and relates to the technical field of storage. The method and the device are applied to a storage system which stores a plurality of objects and metadata of each object in the plurality of objects. And in the process of concurrent access request, obtaining a first access request of the first data access device for the first object, and matching the first verification field in the first access request with the metadata of the first object. And if the first verification field is the first value associated with the element of the first object, executing an atomic operation of updating the first object, and returning a second verification field to the first data access device. Therefore, in the process of processing the concurrent access, the first verification field is used as a condition of the atomic operation update object, the concurrent access conflict of the plurality of data access devices to the first object is identified, and the first object does not need to be locked or a plurality of versions of the first object do not need to be created. Resource consumption caused by locking or creating multiple versions can be reduced. And the overhead of the storage system can be reduced.

Description

Technical Field

[0001] The present application relates to the field of storage technology, and in particular to a data access method, device, system and program product. Background Art

[0002] Object storage refers to storing data in the form of objects, and each object has a unique identifier. With the development of cloud computing and big data technology, object storage, as a scalable and cost-effective storage solution, has been widely used in scenarios such as data lakes, backup and archiving, and multimedia data processing. Taking the distributed storage system using object storage as an example, when multiple users or processes modify the same data object concurrently, the distributed storage system uses distributed locks or multi-version concurrency control mode to avoid access conflicts of the data object. However, the server overhead required by the distributed lock or multi-version concurrency control mode is large, and the computing resources and storage resources are consumed more, resulting in reduced performance of the distributed storage system. Summary of the invention

[0003] The present application provides a data access method, device, system and program product to solve the problem of large resource consumption of object storage under concurrent operations.

[0004] In a first aspect, the present application provides a data access method, which is applied to a storage system that stores multiple objects and metadata of each of the multiple objects. Each object in the storage system supports concurrent access by multiple data access devices. In the process of the storage system processing concurrent access to a first object by multiple data access devices, the storage system obtains a first access request from the first data access device to the first object. The storage system matches a first verification field in the first access request with the metadata of the first object. If the first verification field is consistent with a first value associated with an element of the first object, the storage system performs an atomic operation to update the first object and returns a second verification field to the first data access device.

[0005] Compared with the distributed lock that adds a global lock to an object or publishes multiple versions of an object in a multi-version concurrent control mode, the method provided in the first aspect uses the first verification field included in the access request as a condition for atomic operation to update the object during the process of processing concurrent access to the first object by multiple data access devices, identifies concurrent access conflicts to the first object by multiple data access devices, and does not need to lock the first object or create multiple versions of the first object. This can reduce the resource consumption caused by locking or creating multiple versions. In turn, it can reduce the overhead of the storage system.

[0006] In an optional implementation manner, the specific implementation is: the metadata of the first object includes a first value, and / or the metadata of the second object includes a second value.

[0007] When an object changes, the metadata of the object will also change. Therefore, when the first object changes, the first value included in the metadata of the first object will also change. In this way, the storage system can determine whether the first object has been modified based on the comparison relationship between the first verification field and the first value. In this way, it is identified whether there is an access conflict between multiple data access devices on the first object.

[0008] In an optional implementation, the specific implementation is: the first value includes one or two of the following: a timestamp and a control field in the metadata of the first object; the timestamp is used to indicate the last modification time of the first object; the control field is used to indicate the version of the first object.

[0009] In this optional implementation, since the first value includes a timestamp or a control field in the metadata of the first object, and the timestamp or the control field in the metadata can record changes in the object, it is possible to identify whether there is an access conflict between multiple data access devices on the first object based on the first value.

[0010] In an optional implementation, the specific implementation is as follows: after the storage system matches the first verification field in the first access request with the metadata of the first object, if the first verification field is not the first value, the storage system queries the metadata of the first object in the storage system to obtain the third verification field of the first object. The storage system matches the third verification field with the metadata of the first object. If the third verification field is the third value associated with the metadata of the first object, the storage system executes the first atomic operation of updating the first object.

[0011] Based on this optional implementation, when the first verification field is not the first value, the storage system performs an access retry process to identify whether there is an access conflict between multiple data access devices to the first object. In this way, the reliability of data access is improved by using the retry mechanism.

[0012] In an optional implementation, the specific implementation is: after the storage system matches the third verification field with the metadata of the first object, if the third verification field is not a third value, the storage system returns a prompt message to the first data access device, and the first data access device re-initiates an access request.

[0013] Based on this optional implementation, when the storage system fails to retry, the first data access device initiates a retry. In this way, the reliability of data access is improved by using a multi-level retry mechanism, and the network bandwidth occupied by the data access device retry alone is reduced, thereby reducing the consumption of network resources.

[0014] In an optional implementation, the specific implementation is: after the storage system sends a first access response to the first data access device, the storage system obtains a second access request from the second data access device. The second access request includes a fourth verification field associated with the metadata of the first object. The storage system matches the fourth verification field with the metadata of the second object. If the fourth verification field is not the second value, the storage system updates the fourth verification field based on the metadata of the second object to obtain the fifth verification field. The storage system matches the metadata of the second object based on the re-acquired fifth verification field. When the fourth verification field is the second value, the storage system performs a second atomic operation to update the second object to obtain a third object. The storage system also returns the sixth verification field associated with the metadata of the third object to the second data access device.

[0015] Optionally, when the second data access device obtains the fourth verification field before the storage system responds to the first access request, the fourth verification field is the same as the first verification field.

[0016] Optionally, when the second data access device obtains the fourth verification field after the storage system responds to the first access request, the fourth verification field is different from the first verification field.

[0017] Based on this optional implementation, when multiple data access devices access the first object, the storage system performs an atomic operation on each access request, and after performing the atomic operation of updating the object, the storage system updates the metadata and verification field of the object. In addition, when there is a conflict in access to the first object by multiple data access devices, the storage system initiates a retry. If the storage system fails to retry, the data access device initiates a retry. In this way, the reliability of data access is improved by using the retry mechanism.

[0018] In an optional implementation, the storage system obtains a verification request from the first data access device before obtaining the first access request from the first data access device. The verification request includes an object identifier. In response to the verification request, the storage system obtains metadata of a first object corresponding to the object identifier from among multiple objects. The storage system returns a first verification field to the first data access device based on the metadata of the first object.

[0019] Based on this optional implementation, the first data access device obtains the first verification field from the storage system before sending the first access request. Since the first verification field is associated with the metadata of the first object, the storage system can determine whether the metadata of the first object has changed, and whether there is a conflict in access to the first object by multiple data access devices based on the first verification field carried in the first access request.

[0020] In an optional implementation manner, the specific implementation is: the first access request includes one or more first verification fields.

[0021] Optionally, when the first access request is used to indicate access to a first object in the storage system, the first access request includes a first verification field.

[0022] Optionally, when the first access request is used to indicate batch access to multiple first objects in the storage system, the first access request includes multiple first verification fields, and different first verification fields in the multiple first verification fields correspond to different first objects.

[0023] In this way, the storage system can implement an update of a single first object or a batch update of multiple first objects based on the first access request.

[0024] In an optional implementation manner, the specific implementation is: the header of the first access request includes the first verification field, or the data content of the first access request includes the first verification field.

[0025] In this way, the first access request can carry the first verification field in the header or data content. The way of carrying the first verification field is highly flexible, simple and convenient.

[0026] In an optional implementation, the storage system performs the first atomic operation of updating the first object as follows: the storage system performs a modification operation on the first object based on the first access request, and updates metadata of the first object.

[0027] Optionally, the modification operation includes one or more of the following: data writing, data deletion, and data modification.

[0028] Based on this optional implementation, access requests of different request types can carry the first verification field, so that first access requests of different request types can identify access conflicts based on the first verification field, and the data access method has high applicability.

[0029] In a second aspect, the present application provides a data access device, which includes a communication module, a processing module and a storage module.

[0030] The storage module stores a plurality of objects, metadata of each object, executable program code, data of the data access device in access request processing, and the like.

[0031] The communication module is used to obtain a first access request from a first data access device and send a first access response to the first data access device.

[0032] The processing module is used to execute the executable program code stored in the storage module, and match the first verification field included in the first access request with the metadata of the first object among the multiple objects stored in the storage module. When the first verification field is a first value associated with the metadata of the first object, the processing module is used to perform a first atomic operation to update the first object to obtain the second object.

[0033] Optionally, the first verification field is associated with metadata of the first object, and the first access response includes a second verification field including a second value associated with metadata of the second object.

[0034] In an optional implementation, the specific implementation is: the metadata of the first object stored by the storage module includes a first value; and / or the metadata of the second object stored by the storage module includes a second value.

[0035] In an optional implementation, the first value includes a timestamp or a control field in metadata of the first object, wherein the timestamp is used to indicate the last modification time of the first object, and the control field is used to indicate the version of the first object.

[0036] In an optional implementation, when the first verification field is not the first value, the processing module is further used to query the metadata of the first object in the storage module to obtain a third verification field of the first object. And match the third verification field with the metadata of the first object. When the third verification field is a third value associated with the metadata of the first object, the processing module is further used to perform a first atomic operation to update the first object.

[0037] In an optional implementation, when the third verification field is not the third value, the communication module is further configured to return prompt information to the first data access device, wherein the prompt information is configured to instruct the first data access device to re-initiate an access request.

[0038] In an optional implementation, after the communication module sends a first access response to the first data access device, the processing module is further used to obtain a second access request from a second data access device, and match a fourth verification field included in the second access request with metadata of the second object in the storage module.

[0039] When the fourth verification field is not the second value, the processing module is further configured to update the fourth verification field based on the metadata of the second object to obtain the fifth verification field, and match the fifth verification field with the metadata of the second object.

[0040] When the fourth verification field is the second value, the processing module is further used to execute a second atomic operation to update the second object to obtain a third object, and the communication module is further used to return a second access response to the second data access device.

[0041] The fourth verification field is associated with metadata of the first object. The first access response includes a sixth verification field, and the sixth verification field includes a third value associated with metadata of the third object.

[0042] Optionally, when the second data access device obtains the fourth verification field before the storage system responds to the first access request, the fourth verification field is the same as the first verification field.

[0043] Optionally, when the second data access device obtains the fourth verification field after the storage system responds to the first access request, the fourth verification field is different from the first verification field.

[0044] In an optional implementation, before obtaining the first access request of the first data access device, the communication module is further used to obtain a verification request of the first data access device, and return a first verification field to the first data access device based on the metadata of the first object. The verification request includes: an object identifier, and the verification request is used to request the verification field. The processing module is further used to obtain metadata of a first object corresponding to the object identifier among multiple objects in response to the verification request.

[0045] In an optional implementation manner, the specific implementation is: the first access request carries one or more first verification fields.

[0046] Optionally, when the first access request is used to indicate access to a first object in the storage system, the first access request includes a first verification field.

[0047] Optionally, when the first access request is used to indicate batch access to multiple first objects in the storage system, the first access request includes multiple first verification fields, and different first verification fields in the multiple first verification fields correspond to different first objects.

[0048] In an optional implementation manner, the specific implementation is: the header of the first access request includes the first verification field, or the data content of the first access request includes the first verification field.

[0049] In an optional implementation, the processing module is further configured to perform a modification operation on the first object based on the first access request, and update metadata of the first object.

[0050] Optionally, the modification operation includes one or more of the following: data writing, data deletion, and data modification.

[0051] In a third aspect, the present application provides a storage system, including a processor and a memory. The memory is used to store multiple objects and metadata of each of the multiple objects, and the processor is used to execute the method in the first aspect or any implementation of the first aspect to access the first object of the multiple objects stored in the memory.

[0052] In a fourth aspect, the present application provides a data access system, comprising a plurality of data access devices and the storage system provided in the third aspect.

[0053] Among them, a first data access device among the multiple data access devices sends a first access request to the storage system, and the storage system executes the method in the above-mentioned first aspect or any implementation manner of the first aspect according to the first access request.

[0054] Optionally, the first access request includes a first verification field associated with metadata of a first object, and the first object is one of multiple objects stored in the storage system.

[0055] In a fifth aspect, the present application provides a storage controller, comprising an interface circuit and a control circuit. The interface circuit is used to receive a first access request; the control circuit is used to cooperate with the interface circuit to execute the method in the first aspect or any implementation of the first aspect according to the first access request.

[0056] In a sixth aspect, the present application provides a computer program product, which, when the instructions are executed by a storage controller, enables the storage controller to execute the method in the above-mentioned first aspect or any optional implementation of the first aspect.

[0057] In a seventh aspect, the present application provides a computer-readable storage medium, comprising computer program instructions. When the computer program instructions are executed by a storage controller, the storage controller executes the instructions in the computer program stored in the computer-readable storage medium to perform the method in the first aspect or any optional implementation of the first aspect.

[0058] The technical effects brought about by any one of the implementations in the second to seventh aspects may refer to the technical effects brought about by the first aspect or any optional implementation in the first aspect. No further details will be given here. Based on the implementations provided in the above aspects, this application can also be further combined to provide more implementations. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] Figure 1 A schematic diagram of the structure of the data access system provided for this application;

[0060] Figure 2A Schematic diagram of the interaction between the data access device and the storage system provided in this application Figure 1 ;

[0061] Figure 2B Schematic diagram 2 of the interaction between the data access device and the storage system provided in this application;

[0062] Figure 3 Schematic diagram of the data access method provided for this application Figure 1 ;

[0063] Figure 4A Schematic diagram of the process of obtaining the first verification field by the data access device 1 provided in this application Figure 1 ;

[0064] Figure 4B Schematic diagram 2 of a process for the data access device 1 provided in the present application to obtain a first verification field;

[0065] Figure 5 A schematic diagram of the format of the first access request provided for this application;

[0066] Fig. 6A Schematic diagram of the first access request provided by this application carrying the first verification field Figure 1 ;

[0067] Figure 6B Schematic diagram 2 of the first access request provided by this application carrying the first verification field;

[0068] Figure 7 Flowchart 2 of the data access method provided for this application;

[0069] Figure 8 Schematic diagram of the data access method provided for this application Figure 3 ;

[0070] Fig. 9 Flowchart 4 of the data access method provided for this application;

[0071] Fig.10 A schematic diagram of the structure of the storage system provided for this application;

[0072] Fig.11 A schematic diagram of the structure of the data access device provided in this application. DETAILED DESCRIPTION

[0073] When multiple users or multiple processes modify the same file at the same time, object storage currently mainly uses locking or multi-version concurrency control to solve the problem of concurrent modification. Among them, locking is to directly add a global lock to the object when requesting read and write operations on the object until the transaction is completed or explicitly unlocked. This method may cause the performance of the object storage system to degrade or deadlock problems, and increase the server overhead. Multi-version concurrency control requires creating a new version of the object for each modification to handle concurrency, which increases storage costs and management complexity. It can be seen that locking or multi-version concurrency control methods may not be able to efficiently handle such concurrent operations, causing distributed locks, multi-version control and other methods to have serious server overhead problems.

[0074] Based on this, the present application provides a data access method to solve the problem of high resource consumption of object storage in concurrent access scenarios. In the multi-threaded concurrent access processing, the method uses the first verification field as an update condition to identify concurrent access conflicts, without locking the object or creating multiple versions of the object. This can reduce the resource consumption caused by locking or creating multiple versions. In turn, the overhead of the storage system can be reduced. In addition, the present application uses atomic operations to implement concurrent operations, ensures the atomicity of object modifications, and can efficiently handle object concurrency issues.

[0075] Specifically, in a multi-threaded concurrent access process, a first access request of a first data access device is obtained, and the first access request includes a first verification field associated with metadata of a first object. The storage system matches the first verification field in the first access request with the metadata of the first object. If the first verification field is consistent with a first value associated with an element of the first object, the storage system performs an atomic operation of updating the first object and returns a second verification field to the first data access device.

[0076] The technical solutions involved in this application may not only be applied to current object storage systems, databases or storage servers, but may also be applied to future databases, object storage systems or storage servers. For example, the above data access method can be executed by a storage system or a storage server in a storage system. The terms used in the implementation method section of this application are only used to explain the specific embodiments of this application and are not intended to limit this application. The following is a brief introduction to some concepts that may be involved in this application.

[0077] File storage: A file system provided by the object storage service for storing and managing files.

[0078] File system: refers to the software component used to organize and manage files and directories in a computer system.

[0079] Object Storage System: A data storage architecture that stores data in the form of objects. Object storage systems are often used in cloud storage services.

[0080] Object Storage: Data objects stored using an object storage system. Each object contains: the data itself, metadata, and a unique identifier. Each object is accessed and operated through its identifier. Object storage systems are usually used to store unstructured data, such as pictures, videos, audio, documents, etc.

[0081] Metadata: It can also be called metadata of object storage, which refers to the descriptive information about the object stored in the object storage system, used to describe the properties and characteristics of the object. In some optional ways, the descriptive information of the object can also be called the attribute information of the object.

[0082] Multi-Version Concurrency Control (MVCC): Each time data is updated, a new version is created and appended to the version chain. By maintaining multiple versions of data, multiple transactions are allowed to access the data concurrently without locking.

[0083] Locking: It is a commonly used concurrency control mechanism used to ensure the isolation and consistency of transactions when reading or modifying data.

[0084] Atomic operation: refers to an operation or a series of operations in a multi-threaded environment, which are executed as an indivisible unit. They are either completely executed or not executed at all, with no intermediate state.

[0085] The technical solution provided by this application is described in detail below in conjunction with specific drawings.

[0086] First, the system architecture of the data access method provided in this application is exemplified.

[0087] See also Figure 1 , Figure 1 The data access system 100 includes a storage system 110, a plurality of data access devices for accessing the storage system 110, and a plurality of data access devices for accessing the storage system 110. Figure 1 The data access device 1 and the data access device 2 are shown. Among them, one or more servers in the storage system 110 may also be connected to a computing device, which can be used to provide more computing resources for the server. Alternatively, the computing function on the server can be offloaded to an external acceleration device to improve the data access performance of the storage system 110.

[0088] The data access device can access the server in the storage system 110 to access data through a network, and the communication function of the network can be implemented by a switch or a router. In an optional example, the data access device can also communicate with the server through a wired connection, such as a peripheral component interconnect express (PCIe) high-speed bus, a compute express link (CXL), a universal serial bus (USB) protocol, or a bus of other protocols.

[0089] The data access device includes a host and a computing device, such as data access device 1 includes a host 1 and a computing device 131, and data access device 2 includes a host 2 and a computing device 132. Figure 1 As shown, the computing device is represented by a central processing unit (CPU), but it should not be understood as a limitation of the present application. The computing device may include one or more processing units, which may not only be a CPU, but also a data processing unit (DPU), other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, transistor logic devices, hardware components or any combination thereof. A general-purpose processor may be a microprocessor or any conventional processor. The computing device may also be a dedicated processor for artificial intelligence (AI), such as a neural processing unit (NPU) or a graphic processing unit (GPU). In physical form, the one or more processing units included in the computing device may be packaged as a card, such as Figure 1 The CPU can be connected to the host through a PCIe interface, a CXL interface, a unified bus (UB) interface, an NVlink interface or other communication interfaces.

[0090] Exemplarily, the host is a computer running an application. For example, if the computer running the application is a physical computing device, the physical computing device may be a server or a terminal. The terminal may also be referred to as a terminal device, user equipment (UE), mobile station (MS), mobile terminal (MT), etc. The terminal may be a mobile phone, a tablet computer, a laptop computer, a desktop computer, a personal communication service (PCS) phone, a desktop computer, a wireless terminal in a smart city, a wireless terminal in a smart home, etc. The embodiments of the present application do not limit the specific technology and specific device form adopted by the host. In some optional implementations, Figure 1 The host shown may also be referred to as a client.

[0091] The storage system 110 provided in the embodiment of the present application may be a distributed storage system or a centralized storage system.

[0092] In one possible scenario, Figure 1 The storage system 110 shown may be a distributed storage system. Figure 1 As shown, the distributed storage system provided in this embodiment includes a storage cluster integrating computing and storage (storage and computing in one). The storage cluster includes one or more servers (such as Figure 1 The servers 110A and 110B are shown, and the servers can communicate with each other.

[0093] In some optional implementations, the server included in the storage system 110 is also referred to as a storage server. Figure 1 The server 110A shown is used for illustration. The server 110A is a device having both computing capability and storage capability, such as a server, a desktop computer, etc. For example, an advanced reduced instructionset computer machine (ARM) server or an X86 server can be used as the server 110A here. In terms of hardware, Figure 1As shown, server 110A at least includes processor 112, memory 113, network card 114 and hard disk 105. Processor 112, memory 113, network card 114 and hard disk 105 are connected by bus. Wherein, processor 112 and memory 113 are used to provide computing resources. Specifically, processor 112 is a CPU, used to process data access requests (such as write data request or read data request, etc.) from the outside of server 110A (application server or other server), and also used to process the request generated inside server 110A. Exemplary, when processor 112 receives write log request, the data in these write log requests will be temporarily stored in memory 113. When the total amount of data in memory 113 reaches a certain threshold, processor 112 sends the data stored in memory 113 to hard disk 105 for persistent storage. In addition, processor 112 is also used for data calculation or processing, etc. Figure 1 Only one processor 112 is shown in the figure. In actual applications, there are often multiple processors 112, wherein one processor 112 has one or more CPU cores. This embodiment does not limit the number of CPUs and the number of CPU cores.

[0094] The memory 113 refers to an internal memory that directly exchanges data with the processor 112. It can read and write data at any time, and the speed is very fast. It serves as a temporary data storage for the operating system or other running programs. The memory includes at least two types of memories. For example, the memory can be a random access memory. For example, the random access memory is a dynamic random access memory (DRAM) or a storage class memory (SCM). DRAM is a semiconductor memory, and like most random access memories (RAM), it is a volatile memory device. SCM is a composite storage technology that combines the characteristics of traditional storage devices and memory. Storage class memory can provide faster read and write speeds than hard disks, but the access speed is slower than DRAM, and the cost is cheaper than DRAM. However, DRAM and SCM are only exemplary descriptions in this embodiment, and the memory can also include other random access memories, such as static random access memory (SRAM).

[0095] In addition, the memory 113 may also be a dual in-line memory module or a dual-line memory module (DIMM), that is, a module composed of a dynamic random access memory (DRAM), or a solid state disk (SSD). In actual applications, multiple memories 113 and different types of memories 113 may be configured in the storage server 110A. This embodiment does not limit the number and type of memory 113. In addition, the memory 113 may be configured to have a power-saving function. The power-saving function means that when the system loses power and then powers on again, the data stored in the memory 113 will not be lost. A memory with a power-saving function is called a non-volatile memory.

[0096] The hard disk 105 is used to provide storage resources, such as information such as storage data and data access status of each data access device (or host). For example, data can be stored in the form of an object (object) or a file (file) in the storage hard disk 105 or the memory 113. The hard disk can be a disk or other types of storage media, such as a solid-state hard disk or a shingled magnetic recording hard disk. For example, the hard disk 105 can be a solid-state hard disk based on the non-volatile memory host controller interface specification (Non-Volatile Memory Express, NVMe), such as an NVMe SSD.

[0097] In an optional implementation, the hard disk 105 may store data in an object storage manner. Figure 1 As shown, the hard disk 105 of the server 110A stores objects 1, 2, 3, and 4. Here, object 1 is taken as an example. Figure 1 As shown, object 1 includes the object identifier of object 1, user data, related metadata, and other attributes of the data (such as access characteristics, etc.). The object identifier of each object is a unique identity document (ID). In an optional scenario, the object identifier can be generated using a dedicated algorithm (such as a hash value of data), so as to ensure that the object identifier of each object is unique.

[0098] The network card 114 in the server 110A is used to communicate with a host or other application servers (such as Figure 1 The server 110B shown communicates with the server.

[0099] In one embodiment, the functions of the processor 112 can be offloaded to the network card 114. In other words, in this embodiment, the processor 112 does not perform the processing operation of the service data, but the network card 114 completes the processing of the service data, address conversion and other computing functions.

[0100] In some application scenarios, the network card 114 may also have a persistent memory medium, such as persistent memory (PM), or non-volatile random access memory (NVRAM), or phase change memory (PCM). The CPU is used to perform operations such as address conversion and reading and writing logs. The memory is used to temporarily store data to be written to the hard disk 105, or data read from the hard disk 105 to be sent to the controller. It can also be a programmable electronic component, such as a data processing unit (DPU). The DPU has the versatility and programmability of the CPU, but is more specialized and can run efficiently on network data packets, storage requests or analysis requests. The DPU is distinguished from the CPU by a greater degree of parallelism (needing to process a large number of requests). Optionally, the DPU here can also be replaced with a processing chip such as a GPU or NPU. There is no ownership relationship between the network card 114 and the hard disk 105. The network card 114 can access any hard disk 105 in the server 110B where the network card 114 is located. Therefore, it is more convenient to expand the hard disk when the storage space is insufficient.

[0101] Figure 1 This is only an example provided for the embodiment of the present application. The storage system 110 may also include more servers, memories, hard disks and other devices. The present application does not limit the number and specific form of servers, memories and hard disks.

[0102] In another possible scenario, the storage system 110 provided in the embodiment of the present application may also be a storage cluster with separated computing and storage, which includes a computing device cluster and a storage device cluster, and the computing device cluster includes one or more computing devices, and each computing device can communicate with each other. The computing device may be a computing device, such as a server, a desktop computer, or a controller of a storage array. In hardware, the computing device may include a processor, a memory, and a network card, etc. Among them, the processor is a CPU for processing data access requests from outside the computing device, or requests generated inside the computing device. Exemplarily, when the processor receives a write request sent by a user, the data carried in these write requests will be temporarily stored in the memory. When the total amount of data in the memory reaches a certain threshold, the processor sends the data stored in the memory to the storage device for persistent storage. In addition, the processor is also used for computing or processing data, such as metadata management, deduplication, data compression, virtualized storage space, and address conversion.

[0103] As an optional implementation, the storage system 110 provided in the embodiment of the present application may also be a centralized storage system. The characteristic of a centralized storage system is that it has a unified entrance, and all data from external devices must pass through this entrance. This entrance is the engine of the centralized storage system. The engine is the core component of the centralized storage system, and many advanced functions of the storage system are implemented in it.

[0104] For example, there may be one or more controllers in the engine. In one possible example, if the engine has multiple controllers, there may be a mirror channel between any two controllers to achieve the function of backing up any two controllers to each other, thereby avoiding the unavailability of the centralized storage system due to hardware failure. The engine also includes a front-end interface and a back-end interface, wherein the front-end interface is used to communicate with the computing device in the centralized storage system to provide storage services for the computing device. The back-end interface is used to communicate with the hard disk to expand the capacity of the centralized storage system. Through the back-end interface, the engine can connect more hard disks to form a very large storage resource pool (referred to as: memory pool).

[0105] The following Figure 1 Taking the provided data access device 1 and the storage system 110 as examples, a data access process between the data access device and the storage system 110 is exemplarily described.

[0106] First, taking the data access device 1 as an example, the interaction process of the data access device storing objects in the storage system 110 is described.

[0107] like Figure 2A As shown, the interactive process of the data access device storing objects in the storage system 110 includes stages ① to ③.

[0108] In stage ①, data access device 1 sends file 1. Data access device 1 sends file 1 to storage system 110, requesting to store file 1.

[0109] In stage ②, the storage system 110 stores file 1. The storage system 110 receives file 1, divides file 1 into two parts: user data and metadata, and generates object identifier ID1 of file 1 using a dedicated algorithm. The storage system 110 associates object identifier ID1 of file 1 with metadata of file 1 and stores user data of file 1 with object identifier ID1 of file 1, generating object ID1, which stores object identifier ID1 of file 1, user data, and metadata.

[0110] In stage ③, the storage system 110 sends confirmation information to the data access device 1. The confirmation information is used to indicate that the file 1 has been stored.

[0111] Next, taking the data access device 1 as an example, the interaction process of the data access device requesting the storage system 110 to access an object is described.

[0112] like Figure 2B As shown, the interactive process of the data access device requesting the storage system 110 to access an object includes stages ① to ⑥.

[0113] In stage ①, data access device 1 sends a read request to storage system 110 to read file 1.

[0114] Phase ②: the storage system 110 obtains the object identifier D1 corresponding to the file 1. The storage system 110 receives the read request and obtains the object identifier D1 corresponding to the file 1 based on the metadata.

[0115] In stage ③, the storage system 110 returns the object identifier ID1 corresponding to the file to the data access device 1.

[0116] Phase ④: Data access device 1 sends an access request to storage system 110. Data access device 1 receives object identifier ID1, and sends an access request to read file 1 to storage system 110 based on object identifier ID1.

[0117] Phase ⑤: the storage system 110 responds to the access request and reads the user data corresponding to the object identifier ID1 from the object ID1.

[0118] Phase ⑥: the storage system 110 returns the user data corresponding to the object identifier ID1 to the data access device 1.

[0119] In an optional scenario, multiple data access devices can concurrently access the same object in the storage system 110 and concurrently modify the same object. For example, data access device 1 and data access device 2 both request the storage system 110 to modify the data of object 1. In order to resolve the access conflict of multiple data access devices to the same object, the storage system 110 can use the data access method provided by the present application to process multi-threaded concurrent access requests.

[0120] Based on the interaction logic between the data access device and the storage system 110, the data access device and the storage system 110 can also collaboratively process multi-threaded concurrent requests. The data access method provided by the present application can be implemented in the processing flow of multi-threaded collaborative concurrent requests. Figures 3 to 9 An exemplary description is given.

[0121] like Figure 3 As shown, Figure 3 A flow chart of the data access method provided in this application. The data access method can be applied to Figure 1The data access system shown, or the data access method can be applied to Figures 1 to 2B The storage system 110 is shown. For example, the data access method is applied to a data access system as an example to introduce the implementation of the data access method provided in the embodiment of the present application.

[0122] For example, Figure 1 Taking the data access device 1, data access device 2 and storage system 110 in the example, data access device 1 and data access device 2 concurrently request to modify object 1 in storage system 110. In some optional implementations, data access device 1 may also be referred to as a first data access device, a source node or other names, which are not limited in the embodiments of the present application. Similarly, in some optional methods, data access device 2 may also be referred to as a second data access device, a source node or other names, which are not limited in the embodiments of the present application. Correspondingly, in some optional methods, object 1 may also be referred to as a first object, a target object or other names, etc., which are not limited in the embodiments of the present application.

[0123] like Figure 3 As shown, the storage system 110 executes Figure 3 Taking the method as an example, the data access method includes steps S310 to S340.

[0124] S310 , the storage system 110 obtains a first access request sent by the data access device 1 .

[0125] Corresponding to the implementation process of S310 , the data access device 1 sends a first access request to the storage system 110 .

[0126] In an optional scenario, the first access request may be any IO request in an IO data stream. An IO data stream may also be referred to as an IO stream, which includes multiple IO requests, each IO request including a verification field and an object identifier. Different IO requests in the IO data stream include the same verification field and the same object identifier, or different IO requests in the IO data stream include different verification fields but the same object identifier, or different IO requests in the IO data stream include different verification fields and different object identifiers.

[0127] Three specific examples are provided below to illustrate the first access request.

[0128] In a first optional example, the first access request may be any IO request in an IO data stream sent by the same data access device, which may be a host, a terminal, a user device, a server or other types of devices.

[0129] In a second optional example, the first access request may be any IO request in an IO data stream from the same application.

[0130] For example, the application may be deployed on a single device, such as the host or data access device mentioned above.

[0131] For another example, the application can be deployed on a distributed system, which includes multiple devices, each of which is deployed with a complete application, or each device is deployed with a portion of the program code of the application. For example, the application may include but is not limited to: artificial intelligence applications, distributed applications, etc. For example, a distributed application means that the application is distributed on different computers and completes a task together through the network.

[0132] In the third optional example, the first access request may be any IO request in the IO data stream from the same task. For example, the task may be a read task or other data access task, etc. The task may be a data access task issued by a single application or a data access task of multiple applications managed by a data access interface, which is not limited in this application.

[0133] The above three possible examples are only optional ways of providing the first access request in the embodiments of the present application. The data indicated by the multiple first access requests belonging to the same IO stream flow from one storage location to another storage location. The direction of the IO stream can be input (reading data from the storage system 110) or output (writing data to the storage system 110). In some optional situations, the IO stream can also be called a request sequence, a data request stream or other names, etc., which are not limited in the present application.

[0134] The following describes the first verification field included in the first access request.

[0135] In an optional implementation, the first verification field is associated with metadata of object 1.

[0136] The metadata is described below by taking the metadata of object 1 as an example.

[0137] In an optional example, the metadata of the object 1 includes: system metadata and user metadata of the object 1. The user metadata may also be referred to as user-defined metadata.

[0138] The system metadata refers to the metadata automatically generated and maintained by the object storage system. Generally, the system metadata includes: the object key, creation time, last modification time, version control, etc. of the object 1.

[0139] User metadata refers to metadata customized by users and used to store additional information related to the business.

[0140] The object key, creation time, last modification time, and version control of object 1 are exemplarily described below.

[0141] An object key, also known as an object's unique identifier or object ID, is used to locate and access an object.

[0142] Creation time refers to the timestamp when object 1 is created.

[0143] The last modification time refers to the timestamp of the last time object 1 was modified.

[0144] Version control is used to indicate the version of object 1. The version control includes the version identifier of object 1, the last modification time of the version, etc.

[0145] In combination with the exemplary description of the metadata of object 1 above, two specific examples are provided below to illustrate the association relationship between the first verification field and the metadata of object 1.

[0146] In a first optional example, the metadata of object 1 includes a first verification field, that is, the first verification field may include one or more attributes in the metadata of object 1.

[0147] For example, the first validation field may be the last modification time in the metadata of Object 1.

[0148] For another example, the first validation field may be version control in the metadata of object 1 .

[0149] As another example, the first validation field may be the last modification time and version control in the metadata of object 1 .

[0150] In a second optional example, the first verification field may be obtained by encoding the metadata of the object 1 in an encoding manner. That is, the first verification field is an encoded value.

[0151] The encoding method includes one or more of the following: hash encoding, message digest algorithm version 5 (Message-Digest Algorithm 5, MD5) encoding, Uniform Resource Locator (Uniform Resource Locator, URL) encoding, variable length character encoding (Unicode Transformation Format-8bit, UTF-8) method, etc.

[0152] For example, the first verification field may be obtained by encoding the system metadata in the metadata of the object 1 .

[0153] For another example, the first verification field may also be obtained by encoding the last modification time in the metadata of object 1.

[0154] For another example, the first verification field may also be obtained by encoding the version control in the metadata of object 1 .

[0155] The above two optional examples are only different optional ways of providing the first verification field in the embodiments of the present application. In other embodiments, the first verification field may also have other forms. For example, the first verification field may also be a password generated based on the metadata of object 1. This is not limited in the embodiments of the present application.

[0156] The following describes the implementation method of the data access device 1 acquiring the first verification field.

[0157] In an optional implementation, the first verification field may be obtained by the data access device 1 requesting the storage system 110. The implementation of the data access device 1 requesting the first verification field from the storage system 110 may refer to the following: FIG. 4A to FIG. 4B , the embodiments of the present application are not described in detail here.

[0158] S320 , the storage system 110 matches the first verification field in the first access request with the metadata of object 1 , and detects whether the first verification field is a first value associated with the metadata of object 1 .

[0159] Since the first verification field is obtained by the data access device 1 before sending the first access request, and the first verification field is associated with the metadata of the object 1. During the period from when the data access device 1 obtains the first verification field to when the first access request occurs, if other data access devices modify, delete, or write data to the object 1, the metadata of the object 1 will change. Therefore, in the embodiment of the present application, the storage system 110 matches the first verification field in the first access request with the metadata of the object 1 to determine whether there is an access conflict between multiple data access devices on the object 1.

[0160] The first value may be obtained by the storage system 110 based on metadata of the object 1 at the current moment. The current moment may be the moment when the first access request is received.

[0161] The first value is exemplarily described below.

[0162] In a first optional implementation, the metadata of object 1 includes a first value. For example, the first value may be the latest update time in the metadata of object 1. For another example, the first value may be the control version in the metadata of object 1. For another example, the first value may also be the latest time and control version in the metadata of object 1.

[0163] In a second optional implementation, the first value may also be an encoded value of metadata of object 1. For example, the first value may be an encoded value of system metadata in the metadata of object 1. For another example, the first value may also be an encoded value of the last modification time in the metadata of object 1. For another example, the first value may also be an encoded value of version control in the metadata of object 1.

[0164] The above two optional implementations are only optional methods for providing different first values ​​in the embodiments of the present application. In other embodiments, the first value may also have other implementations, which are not limited in the embodiments of the present application.

[0165] The following uses first values ​​in different forms as examples to exemplify the matching method between the first verification field and the metadata of object 1.

[0166] In an optional implementation, the metadata of object 1 includes a first verification field, and the control version in the metadata of object 1 includes a first value. The storage system 110 obtains the attribute field corresponding to the first verification field in the metadata of object 1 at the current moment, and compares the attribute field with the first verification field. If the first verification field is consistent with the attribute field corresponding to the first verification field in the metadata at the current moment, it is determined that the first verification field is the first value associated with the metadata of object 1. If the first verification field is inconsistent with the attribute field corresponding to the first verification field in the metadata at the current moment, it is determined that the first verification field is not the first value associated with the metadata of object 1.

[0167] For example, when the first verification field includes the last modification time in the metadata of object 1, the storage system 110 compares the last modification time in the metadata of object 1 with the first verification field.

[0168] For another example, when the first verification field includes the version control in the metadata of object 1, the storage system 110 compares the version control in the metadata of object 1 with the first verification field.

[0169] The above is an example of the first verification field included in the metadata of object 1, and the matching method of the first verification field and the metadata of object 1 is explained. In other embodiments, the first verification field and the metadata of object 1 can also adopt other matching methods. For example, in the case where the first verification field is obtained by encoding the system metadata in the metadata of object 1, the storage system 110 can also match the reference coding value of the metadata of object 1 with the first verification field. The embodiments of the present application are not limited to this.

[0170] In the following, the example in which the first verification field is obtained by encoding the system metadata in the metadata of the object 1 is taken to exemplarily illustrate the matching method between the first verification field and the metadata of the object 1 .

[0171] In an optional implementation, the first value is a reference coding value of the metadata of object 1. The storage system 110 obtains the metadata of object 1 at the current moment, encodes the metadata of object 1 at the current moment, and obtains the reference coding value. The storage system 110 compares the reference coding value with the first verification field. When the first verification field is consistent with the reference coding value, the first verification field is determined to be the first value associated with the metadata of object 1. When the first verification field is inconsistent with the reference coding value, it is determined that the first verification field is not the first value associated with the metadata of object 1.

[0172] For example, the storage system 110 encodes the last modification time in the metadata of the object 1 at the current moment to obtain a first value, and compares the first value with the first verification field.

[0173] For another example, the storage system 110 encodes the version control in the metadata of the object 1 at the current moment to obtain a first value, and compares the first value with the first verification field.

[0174] For another example, the storage system 110 encodes the last modification time and version control in the metadata of the object 1 at the current moment to obtain a first value, and compares the first value with the first verification field.

[0175] S330 , if the first verification field is the first value associated with the metadata of object 1 , the storage system 110 executes a first atomic operation to update object 1 to obtain an updated object 1 .

[0176] In an optional implementation, the updated object 1 may be referred to as a second object or other names, which is not limited in the embodiments of the present application.

[0177] In an optional implementation, "the first verification field is the first value associated with the metadata of object 1" may mean that during the time period from when data access device 1 obtains the first verification field to when it sends the first access request, there is no access conflict between multiple data access devices on object 1.

[0178] The following is an exemplary description of the implementation of the first atomic operation of updating object 1 in the storage system 110.

[0179] In a first optional example, the first atomic operation of updating the object 1 includes: the storage system 110 performs a modification operation on the data content of the object 1 according to the first access request. The modification operation includes one or more of the following: data deletion, data modification, and data writing.

[0180] In a second optional example, the first atomic operation of updating object 1 includes: the storage system 110 performs a modification operation on the metadata of object 1 according to the first access request.

[0181] In a third optional example, the first atomic operation of updating object 1 includes: the storage system 110 performs a modification operation on the data content of object 1 and the metadata of object 1 according to the first access request.

[0182] The above three optional examples are only different optional ways of updating the first atomic operation of object 1. In other embodiments, the first atomic operation of updating object 1 can also be performed in other ways, which is not limited by the embodiments of the present application. In addition, the storage system 110 performs the first atomic operation of updating object 1 to update the metadata of object 1.

[0183] In an optional example, “updating metadata of object 1” may include one or more of the following:

[0184] (1) The storage system 110 changes the last modification time of the metadata of object 1 to the timestamp when object 1 was modified.

[0185] For example, the last modification time in the metadata of object 1 in storage system 110 is: XXXX00, and storage system 110 deletes the data of object 1 according to the first access request at 16:57:23 of xxx, and the timestamp corresponding to 16:57:23 of xxx is XXXX10. Then storage system 110 updates the last modification time in the metadata of object 1 from XXXX00 to XXXX10.

[0186] (2) The storage system 110 modifies the last modification time in the version control of object 1 according to the timestamp when object 1 is modified. For example, the storage system 110 deletes the data of object 1 according to the first access request at 16:57:23 in xxx year, and the storage system 110 updates the last modification time in the version control of object 1 in the metadata of object 1 to XXXX10.

[0187] (3) The storage system 110 generates a new version identifier for object 1, and updates the version identifier in the version control of object 1 to the new version identifier of object 1.

[0188] For example, the version identifier in the version control of object 1 is 8.0.01.50. After the storage system 110 updates object 1, the storage system 110 generates a new version identifier 8.0.01.60 for object 1. Then, the storage system 110 updates the version identifier in the version control of object 1 from 8.0.01.50 to 8.0.01.60.

[0189] (4) The storage system 110 updates the meaning of the data in the metadata of object 1. For example, the storage system 110 updates the meaning of field X in the data of object 1 from A to B according to the first access request, and the storage system 110 updates the meaning of field X in the metadata of object 1 to B.

[0190] S340 , sending a first access response to data access device 1 .

[0191] Corresponding to the implementation process of S340 , the data access device 1 receives the second verification field.

[0192] The first access response is used to indicate that the data access device 1 successfully accesses the object 1 .

[0193] In an optional implementation, the first access response includes a second verification field, and the second verification field includes: a second value associated with the updated metadata of object 1.

[0194] In an optional implementation, similar to the first value, the metadata of the updated object 1 includes a second value, or the second value includes an encoded value of the metadata of the updated object 1.

[0195] Accordingly, the storage system 110 may refer to the optional method of the first verification field to obtain the second verification field. The embodiment of the present application will not be described in detail here.

[0196] based on Figure 3 In the provided embodiment, in the multi-thread concurrent access processing, the first verification field is used as an update condition to identify concurrent access conflicts, without locking the object or creating multiple versions of the object. This can reduce the resource consumption caused by locking or creating multiple versions. In turn, the overhead of the storage system 110 can be reduced. In addition, the present application uses atomic operations to implement concurrent operations, ensures the atomicity of object modification, and can efficiently handle object concurrency issues.

[0197] It is worth noting that the above Figure 3The data access method provided in the present application is described by taking the storage system 110 as an example. In some optional implementations, the data access method provided in the embodiments of the present application may also be executed by other storage devices. For example, the storage device may include but is not limited to: a centralized storage system, a storage resource pool with computing functions, a storage server, a computing node or computing device with storage functions, or other electronic devices or clusters that can execute the data access method provided in the embodiments of the present application.

[0198] The following combination FIG. 4A to FIG. 4B An exemplary description is given of the implementation manner in which the data access device 1 obtains the first verification field.

[0199] In an optional implementation, after acquiring the object identification of the object 1 , the data access device 1 requests the storage system 110 to acquire the first verification field of the object 1 based on the object identification of the object 1 .

[0200] like Figure 4A As shown, Figure 4A Schematic diagram of the process of obtaining the first verification field by the data access device 1 provided in the embodiment of the present application Figure 1 . In the process of data access device 1 obtaining the first verification field, data access device 1 obtains the object identification ID1 of object 1 returned by storage system 110 (stage ①). Data access device 1 sends a verification request to storage system 110 based on object identification ID1 of object 1 (stage ②). Storage system 110 receives and responds to the verification request. Storage system 110 obtains metadata of object 1 corresponding to object identification ID1 among multiple objects (stage ③). Storage system 110 returns a verification response to the first data access device based on the metadata of object 1 (stage ④), and the verification response includes the first verification field. Corresponding to Figure 4A In the implementation process of stage ④, the data access device 1 receives the verification response returned by the storage system 110.

[0201] Above Figure 4A The data access device 1 requests the first verification field from the storage system 110 after obtaining the object identification of the object 1 as an example to illustrate the implementation of the data access device 1 obtaining the first verification field. In another optional implementation, the data access device 1 can also obtain the first verification field in the process of obtaining the object identification of the object 1.

[0202] Exemplarily, when data access device 1 requests the object identifier of object 1 from storage system 110, storage system 110 sends the object identifier of object 1 and the first verification field to data access device 1. Storage system 110 can determine whether the metadata of object 1 has changed and whether there is an access conflict between multiple data access devices to object 1 based on the first verification field carried in the first access request. In this way, concurrent access conflicts can be quickly identified without the need for distributed locks and multi-version concurrency control.

[0203] like Figure 4B As shown, Figure 4B Schematic diagram 2 of the process of obtaining the first verification field by the data access device 1 provided in the embodiment of the present application. Figure 4A In the process of obtaining the first verification field provided, data access device 1 sends a read request to storage system 110 to read object 1 (stage ①). Storage system 110 receives the read request and obtains object identifier ID1 and the first verification field corresponding to object 1 based on the metadata (stage ②). Storage system 110 returns a read response to data access device 1 based on the object identifier ID1 and the first verification field corresponding to object 1 (stage ③), and the read response includes the object identifier and the first verification field corresponding to object 1. Figure 4B In the implementation process of stage ③, the data access device 1 receives a read response returned by the storage system 110.

[0204] based on Figure 4A and Figure 4B In the provided embodiment, it can be known that before sending an access request, the data access device obtains the verification field from the storage system 110. Since the verification field is associated with the metadata of the object, the storage system 110 can determine whether the metadata of the object 1 has changed, and whether there is an access conflict between multiple data access devices to the object 1, based on the first verification field carried in the first access request.

[0205] Above Figure 4A and Figure 4B The illustrated implementations are merely different implementations of the data access device 1 requesting the first verification field from the storage system 110. In other embodiments, the data access device 1 may obtain the first verification field in other ways. For example, when the storage system 110 detects that any object has been modified, the storage system 110 sends the verification field of the modified object to one or more data access devices connected to the storage system 110 based on the metadata of the modified object. This is not limited in the embodiments of the present application. After the data access device 1 obtains the first verification field in the manner described above, it sends a first access request for the object 1 to the storage system 110, and carries the first verification field in the first access request.

[0206] First, the request type of the first access request in the above S310 is exemplarily described.

[0207] In an optional implementation, the request type of the first access request includes but is not limited to: object deletion, object modification or data addition, and the corresponding first access request may specifically be an object deletion request (DELETE) or an object modification request (PUT) or a data addition request (POST).

[0208] The object deletion request is used to request the storage system 110 to delete the data of the object. The object modification request is used to request the storage system 110 to update the data of the object. The data addition request is used to write data to the storage system 110.

[0209] In one example, the data of an object may be text, images, audio, video, or other forms of content data stored in the object, which is not limited in the embodiments of the present application.

[0210] In the case where the first access request is an object deletion request, the first access request includes: an object identifier of object 1 and a first verification field. The first access request can be used to instruct the storage system 110 to delete user data of object 1. The object identifier is used to indicate the object to be accessed by the first access request. The first verification field is used by the storage system 110 to detect access conflicts. Specifically, the implementation method of the storage system 110 detecting access conflicts based on the first verification field can refer to the following Fig. 9 The embodiments provided are not described in detail here.

[0211] When the first access request is an object modification request, the first access request includes: the object identifier of object 1, the first verification field, and the data content to be updated. The first access request is used to instruct: the storage system 110 updates the user data of object 1 to the data content to be updated.

[0212] When the first access request is a data adding request, the first access request includes: the object identifier of object 1, the first verification field, and the data content to be written. The first access request is used to instruct the storage system 110 to write the data content to be written into object 1.

[0213] Next, an exemplary description is given of an implementation method of the first access request carrying the first verification field in combination with the request type of the first access request.

[0214] For example, the following takes the first access request as an object deletion request as an example. Figures 5 to 6BThe implementation method of the first access request in S310 above carrying the first verification field is exemplarily described. Similarly, in the scenario where the first access request is an object modification request or a data addition request, the implementation method of the first access request carrying the first verification field can refer to the implementation method of the first access request carrying the first verification field in the scenario of an object deletion request.

[0215] First, the number of first verification fields carried in the first access request is illustrated by way of example.

[0216] In an optional implementation manner, the first access request may carry one or more first verification fields.

[0217] In a first optional example, the first access request carries a first verification field, and accordingly, the first access request carries an object identifier. The first access request is used to indicate access to an object in the storage system 110 .

[0218] In a second optional example, the first access request carries multiple first verification fields, and the first access request carries multiple object identifiers. Different first verification fields correspond to different object identifiers. The first access request is used to indicate batch access to multiple objects in the storage system 110. Wherein, when the first access request performs batch access to multiple objects in the storage system 110, the multiple objects accessed in batch include the above-mentioned object 1.

[0219] Next, the format of the first access request is exemplarily described, such as Figure 5 As shown, Figure 5 A schematic diagram of the format of a first access request provided in an embodiment of the present application. The first access request includes: a header, a data field, and a data integrity field (DIF).

[0220] The header is used to carry the operation type of the first access request, the object identifier, the access address of the storage system 110, the verification information of the data access device 1, the time when the request is sent, and the transmission protocol of the first access request. The data field is used to carry the data to be operated (the data content to be updated or the data content to be written) of the first access request. The DIF field is used to detect the integrity of the data to be operated, such as Figure 5 The DIF field shown includes a check area, a version area, an application software area, and an address area. The sizes of the header, data field, and DIF field can be customized by the user or the system, such as using 512+8 bytes, 4096+8, and 4096+64 storage methods, etc., which are not limited in this application.

[0221] The following is a brief description of the DIF domain including the check area, version area, application software area and address area.

[0222] The check area is used to carry check data, such as cyclic redundancy check (CRC) data, etc. The byte size occupied by the check area can be customized by the user or the system, such as 2 bytes.

[0223] The version area is used to indicate the version number of the DIF, and the size occupied can be customized by the user or the system, for example, 1 byte.

[0224] The address area is used to indicate the logical address LBA corresponding to the data to be operated. The size occupied by the address area can be set by the user or the system, for example, 4 bytes.

[0225] The application software area is composed of a reserved field and an LBA valid indication field. The size occupied by the application software area can be customized by the user or the system. The LBA valid indication field is used to indicate whether the LBA logical address carried by the address area is valid. Specifically, when the LBA valid indication field is the first preset character (such as 1), it indicates that the LBA logical address carried in the address area is valid; when the LBA valid indication field is the second preset character (such as 0), it indicates that the LBA logical address carried in the address area is invalid. The size occupied by the LBA valid indication field can be customized by the user or the system, for example, 1 bit.

[0226] The reserved fields are fields to be defined, and the size they occupy can also be customized by the user or the system, such as 7 bits.

[0227] Finally, based on the format of the first access request and the number of first verification fields carried in the first access request, an exemplary description is given of an implementation method of carrying the first verification field in the first access request.

[0228] In an optional implementation, the header of the first access request carries a first verification field. Fig. 6A Figures (a) and (b) in the figure illustrate an example of the implementation method of the first verification field carried in the header of the first access request.

[0229] First, take the first access request carrying a first verification field as an example, such as Fig. 6AAs shown in Figure (a), the header of the first access request includes the following fields: "DELETE / file? Verification field = xxxxx HTTP / 1.1", "ID: ID1", "Verification header field: xxxx", "Host: xxx.com", "Authorization: authorization" and "Date: xx / xx / 8 16:13:21".

[0230] Here, “DELET” is used to indicate that the request type of the first access request is object deletion.

[0231] “HTTP / 1.1” is used to indicate that the transmission protocol of the first access request is HTTP / 1.1.

[0232] “Verification header field: xxxx” is used to indicate that the first verification field is “xxxx”.

[0233] “file? Verification field=xxxxx” is used to instruct the storage system 110 to detect the first verification field “xxxx” carried in the first access request.

[0234] “ID: s1” is used to indicate the object identifier s1. “Host: xxx.com t” is used to indicate that the access address of the storage system 110 is xxx.com.

[0235] “Authorization: authorization” is used to indicate that the authentication information of the data access device 1 is authorization.

[0236] “Date: xx / xx / 8 16:13:21” is used to indicate that the time when the first access request was sent was xx / xx / 8 16:13:21.

[0237] It is worth noting that the above Fig. 6A Figure (a) in FIG. 1 is only an example of the header of the first access request and should not be understood as limiting the header of the first access request. In other optional implementations, the header of the first access request may also include more than the above Fig. 6A In addition, when the first access request is an object modification request or a data addition request, the content included in the header of the first access request may be different from the above Fig. 6A The content shown in Figure (a) in .

[0238] Secondly, taking the first access request carrying multiple first verification fields as an example, the check field in the header of the first access request includes multiple first verification fields. For example, the check field includes the following information: object identifier 1: first verification field 1; object identifier 2: first verification field 2; ... object identifier n: first verification field n. Among them, n is the number of first verification fields carried by the first access request, and the specific value of n is not limited in the embodiment of the present application.

[0239] like Fig. 6A As shown in Figure (b), relative to Fig. 6A In the middle (a) figure, Fig. 6A In the middle (b) figure, "Verification header field: ID1: xxxx; ID2: xxx0; ID3: xxx1" is used to indicate that the first verification field of object identifier ID1 is xxxx, the first verification field of object identifier ID2 is xxx0, and the first verification field of object identifier ID3 is xxx1. "ID: ID1; ID2; ID3" is used to indicate object identifiers ID1, ID2, and ID3.

[0240] It is worth noting that the above Fig. 6A Figure (b) in FIG. 1 is only an example of a header when the first access request is an object deletion request, and should not be understood as a limitation on the header of the first access request. In other optional implementations, the header of the first access request may also include more than the above Fig. 6A In addition, when the first access request is an object modification request or a data addition request, the content included in the header of the first access request may be different from the above-mentioned Fig. 6A The content shown in Figure (b) in .

[0241] The above two examples are mainly described by taking the first access request header carrying the first verification field as an implementation mode for the first access request to carry the first verification field. In other embodiments, the first access request data content carries the first verification field.

[0242] In the following, the implementation method of carrying the first verification field in the data content of the first access request is described by taking the case where the first verification field is carried in the data content of the first access request as an example.

[0243] In a first optional implementation, the first access request carries a first verification field, and the first access request carries the first verification field in the data field or the DIF field. For example, the first verification field is included in the data field. For another example, the first verification field is included in the reserved field of the application software area of ​​the DIF field.

[0244] In a second optional implementation manner, the first access request carries multiple first verification fields, and the data field of the first access request carries multiple first verification fields.

[0245] like Figure 6B As shown, Figure 6B Schematic diagram 2 of a first access request carrying a first verification field provided in an embodiment of the present application. Fig. 6A In Figure (a), Figure 6B In the example, the header of the first access request includes the fields: “POST / multi_deleteHTTP / 1.1”, “Content-MD5:MD5”, and “Content-Length:length”.

[0246] Among them, "POST / multi_delete HTTP / 1.1" is used to indicate that the first access request is a request to delete batch objects. "Content-MD5:MD5" is used to indicate that the verification data of the data access device 1 is MD5. "Content-Length:length" is used to indicate that the length of the first access request is length.

[0247] like Figure 6B As shown, in the data field of the first access request, <delete> … <delete>"Used to indicate that the first access request performs a batch object deletion operation. "<Object>…< / Object>" is used to indicate the object identifier of the object and the corresponding verification field." <quiet> true< / quiet> "Used to indicate that the storage system returns the object identifier for which the deletion fails to the book access device 1. "<Identifier>ID1< / Identifier>" is used to indicate that the object identifier to be deleted is ID1. "<Verification field>xxx< / Verification field>" is used to indicate that the first verification field for the object identifier ID1 is xxx. "<Identifier>ID2< / Identifier>" is used to indicate that the object identifier to be deleted is ID2. "<Verification field>xxx1< / Verification field>" is used to indicate that the first verification field for the object identifier ID1 is xxx1."

[0248] It should be noted that the above Figure 6B takes the first access request as a batch object deletion request as an example to illustrate the implementation manner of the first access request carrying the first verification field. In some other embodiments, the first access request may perform deletion operations on more objects. In addition, when the first access request is used to implement batch data addition or batch object modification, the form of the first access request may be different from the above Figure 6B shown form. The embodiments of the present application do not limit this."

[0249] Based on FIG. 6A to FIG. 6B it can be known that the first access request can carry the first verification field in the header or the data content. The manner of carrying the first verification field is highly flexible and simple and convenient."

[0250] In some other embodiments, the first access request may also adopt other implementation manners to carry the first verification field. For example, the first verification field is carried in the parameters of the first access request. The embodiments of the present application do not limit this."

[0251] After the data access device 1 sends the first access request to the storage system 110 with reference to the above implementation manner of carrying the first verification field, the storage system 110 executes the above Figure 3 S320. When the first verification field is not the first value associated with the metadata of object 1, the storage system 110 refers to the following implementation manner for access retry or data error reporting."

[0252] Next, in combination with Figure 7 the implementation manner of the data access method is described when the first verification field is not the first value associated with the metadata of object 1."

[0253] In an optional implementation, when the first verification field is not the first value associated with the metadata of object 1, the storage system 110 or the data access device 1 can re-acquire a new verification field, and the storage system 110 determines whether there is an access conflict between multiple data access devices on object 1 based on the new verification field.

[0254] The following is an exemplary description of the implementation manner in which the storage system 110 obtains a new verification field.

[0255] In an optional implementation, during the period from when data access device 1 obtains the first verification field to when the first access request occurs, object 1 in storage system 110 has been accessed by other data access devices, causing the metadata of object 1 to change, thereby causing the first verification field carried by the first access request to not be the first value associated with the metadata of object 1. In addition, when multiple data access devices have access conflicts to object 1, the metadata of object 1 will also change.

[0256] In order to detect the concurrent access conflicts of multiple data access devices to the object 1 and reduce the consumption of network resources, the storage system 110 can process the data access process of the object 1 by access retry or data error reporting. The access retry and data error reporting are respectively described as examples below.

[0257] In the first embodiment, the storage system 110 performs an access retry process. Exemplarily, when the first verification field is not the first value, the storage system 110 obtains the latest metadata of object 1, updates the first verification field based on the latest metadata of object 1, and obtains a new verification field. The storage system 110 matches the metadata of object 1 again based on the new verification field. In this way, the retry mechanism is used to improve the reliability of data access, and reduce the occupation of network bandwidth by the retry of the data access device, thereby reducing the consumption of network resources.

[0258] In the second embodiment, the storage system 110 performs a data error reporting process. Exemplarily, if the first verification field is not the first value, the storage system 110 returns a prompt message to the data access device 1. After receiving the prompt message, the data access device 1 re-acquires a new verification field and sends an access request to the object 1 to the storage system 110 again based on the new verification field. In this way, when the first verification field is not the first value, the data access device 1 initiates a retry to improve the reliability of data access.

[0259] It should be noted that the above two embodiments are only different retry methods when the first verification field is not the first value. In other embodiments, there may be other retry methods when the first verification field is not the first value, and the embodiments of the present application do not limit this. For example, when the first verification field is not the first value, the storage system 110 uses a multi-level retry mechanism, and when the storage system 110 fails to perform an access retry, it performs a data error reporting process.

[0260] For the multi-level retry mechanism, the following is combined Figure 7 For example, Figure 7 As shown, Figure 7 Schematic diagram 2 of the data access method provided in this application. Figure 3 The process of providing data access methods is Figure 7 In the embodiment, the storage system 110 further includes steps S350 to S380 after step S320.

[0261] S350 , if the first verification field is not the first value, the storage system 110 queries the metadata of the object 1 in the storage system 110 to obtain the third verification field of the object 1 .

[0262] In an optional implementation, “querying the metadata of object 1 in storage system 110” may mean that storage system 110 obtains the latest metadata of object 1.

[0263] In an optional implementation, the third verification field may be obtained by referring to the implementation of the above-mentioned storage system 110 for obtaining the first verification field or obtaining the second verification field, which is not described in detail in the embodiment of the present application.

[0264] S360 , the storage system 110 matches the third verification field with the metadata of object 1 .

[0265] In an optional implementation, the storage system 110 may match the third verification field with the metadata of the object 1 with reference to the above step S320, which is not described in detail in this embodiment of the present application.

[0266] S370 , if the third verification field is a third value associated with the metadata of object 1 , the storage system 110 performs a first atomic operation to update object 1 .

[0267] In an optional implementation, the third value may be obtained by the storage system 110 based on the metadata of the re-acquired object 1. Similar to the first value described above, the third value associated with the metadata of the object 1 may be the third value included in the metadata of the object 1, or the third value may be an encoded value of the metadata of the object 1. The embodiment of the present application does not elaborate on the method for obtaining the third value.

[0268] In an optional implementation, when other data access devices modify the data of object 1 during the time period from when data access device 1 obtains the first verification field to when the first access request occurs, the metadata of object 1 is stable, and the updated verification field obtained at the previous moment matches the metadata of object 1 at the current moment, that is, the third verification field is a third value associated with the metadata of object 1.

[0269] In an optional implementation, the storage system 110 may refer to the above S340 to perform the first atomic operation of updating the object 1, which is not described in detail in the embodiment of the present application.

[0270] S380 , if the third verification field is not the third value, the storage system 110 returns prompt information to the data access device 1 .

[0271] The prompt information is used to instruct the data access device 1 to re-initiate the access request.

[0272] In a first optional example, the prompt information may be an identification code. For example, the storage system 110 sends the identification code "409" to the data access device 1.

[0273] In the second optional example, the prompt information may also be a field. For example, the storage system 110 sends a prompt information of "access conflict" to the data access device 1. For another example, the storage system 110 sends a prompt information of "access failed, failure reason: access conflict" to the data access device 1.

[0274] The above two optional examples are only different optional forms of prompt information, and should not be understood as limitations of the data access method provided by the embodiment of the present application. In other embodiments, the prompt information may also have other forms, which are not limited by the embodiment of the present application.

[0275] In an optional implementation, when multiple data access devices concurrently access object 1, the metadata of object 1 changes dynamically, and the updated verification field obtained at the previous moment does not match the metadata of object 1 at the current moment, that is, the third verification field is not the third value associated with the metadata of object 1.

[0276] Therefore, when the third verification field is not the third value, the storage system 110 fails to retry, and the storage system 110 prompts the data access device 1 to initiate a retry, and the data access device 1 re-requests the storage system 110 for a new verification field of the object 1, and re-sends an access request to the storage system 110 based on the new verification field. When the storage system 110 receives the new access request from the data access device 1, the storage system 110 executes the above steps S320 to S340 to re-update the object 1.

[0277] In an optional implementation, if the storage system 110 fails in the retry, the data access device 1 initiates a retry again. The data access device 1 records the number of times the access request is resent. Before the data access device 1 initiates a retry, the data access device 1 compares the number of times the access request is resent with the number threshold. If the number of times the access request is resent is greater than the number threshold, the data access device 1 stops initiating a retry to the storage system 110. If the number of times the access request is resent is less than the number threshold, the data access device 1 continues to initiate a retry to the storage system 110.

[0278] The number threshold may be set by a user, or the number threshold may be configured by the storage system 110. This embodiment of the present application does not limit this.

[0279] In an optional scenario, when the number of times the access request is resent is equal to the number threshold, the data access device 1 may stop retrying the storage system 110, or the data access device 1 may continue to retry the storage system 110, which is not limited in the embodiment of the present application.

[0280] In an optional scenario, the data access device 1 stops initiating a retry to the storage system 110, which may mean that the data access device 1 stops sending an access request to the storage system 110 for accessing the object 1, and after the preset time period, the data access device 1 obtains the verification field again, and sends an access request to the storage system 110 for accessing the object 1. The preset time period may be set by the user, or configured by the data access device 1 itself. Or the storage system 110 configures itself. This embodiment of the present application does not limit this.

[0281] In an optional implementation, in order to reduce the number of times the data access device 1 re-sends the access request to the storage system 110, when the third verification field is not the third value, the storage system 110 can execute the above S350 to S360 to retry again and record the number of retries. When the number of retries is greater than the retry number threshold, the storage system 110 sends a prompt message to the data access device 1. In addition, when the number of retries is less than the retry number threshold, the storage system 110 continues to execute the above S350 to S360 to retry again. In addition, when the number of retries is equal to the retry number threshold, the storage system 110 sends a prompt message to the data access device 1, or the storage system 110 can continue to execute the above S350 to S360 to retry again, which is not limited in the embodiments of the present application.

[0282] In an optional situation, the retry number threshold may be set by a user, or the retry number threshold may be configured by the storage system 110. This embodiment of the present application does not limit this. In addition, the specific value of the retry number threshold is not limited in the embodiment of the present application.

[0283] based on Figure 7 In the provided embodiment, when the first verification field is not the first value associated with the metadata of object 1, the storage system 110 initiates a retry, and when the storage system 110 fails in the retry, the data access device initiates a retry. In this way, the reliability of data access is improved by using a multi-level retry mechanism, and the occupation of network bandwidth by the retry of the data access device alone is reduced, thereby reducing the consumption of network resources.

[0284] Above Figures 3 to 7 The processing of the first access request by the storage system 110 is mainly described by taking the data access device 1 as an example. In other embodiments, multiple data access devices may concurrently send access requests to the same object to the storage system 110, and the storage system 110 performs an atomic operation for each access request, and after processing the access request sent by one data access device, processes the access request sent by another data access device. Among them, "processing completed" may mean that the data of object 1 is updated successfully, or the storage system 110 fails in retrying.

[0285] Below, taking the data access device 2 accessing the object 1 as an example, the implementation manner in which the storage system 110 processes the access request of the data access device 2 to the object 1 is exemplarily described.

[0286] like Figure 8 As shown, Figure 8 Schematic diagram of the data access method provided in the embodiment of the present application Figure 3 . Relative to Figure 3 The process of providing data access methods is Figure 8 In the embodiment, after the storage system 110 sends the first access response to the data access device 1, the process of the data access method further includes steps S810 to S860.

[0287] S810 , the storage system 110 obtains a second access request from the data access device 2 .

[0288] Corresponding to the implementation process of S810 , the data access device 2 sends a second access request to the storage system 110 .

[0289] In an optional implementation manner, the implementation manner of the second access request may refer to the implementation manner of the first access request mentioned above, and the embodiment of the present application will not be described in detail here.

[0290] In an optional implementation, similar to the above-mentioned first access request, the second access request includes: associating a fourth verification field with the metadata of object 1.

[0291] Three specific examples are provided below to illustrate the relationship between the fourth verification field and the first verification field.

[0292] In a first optional example, when data access device 1 and data access device 2 obtain the verification field in the same time period, and metadata of object 1 does not change in the time period, the fourth verification field is the same as the first verification field.

[0293] In a second optional example, when data access device 2 obtains the fourth verification field after data access device 1 obtains the first verification field, if metadata of object 1 changes after data access device 1 obtains the first verification field, the fourth verification field is different from the first verification field. If metadata of object 1 does not change after data access device 1 obtains the first verification field, the fourth verification field is the same as the first verification field.

[0294] In a third optional example, after the data access device 2 sends the first access request, the data access device 2 obtains the fourth verification field, and the fourth verification field is different from the first verification field.

[0295] The above three examples are only different associations between the fourth verification field and the first verification field when the data access device 2 obtains the fourth verification field in different time periods, and should not constitute a limitation on the data access method provided in the embodiment of the present application.

[0296] It is worth noting that Figure 8 The fourth verification field is the same as the first verification field, and the data access device 1 and the data access device 2 simultaneously access the object 1. In the case where the data access device 1 and the data access device 2 do not simultaneously access the object 1, the storage system 110 can refer to the above after receiving the second access request sent by the data access device 2. Figure 3 or Figure 7 The provided embodiment processes the second access request, and the embodiment of the present application will not be described in detail here.

[0297] S820 , the storage system 110 matches the fourth verification field with the updated metadata of the object 1 .

[0298] In an optional implementation, the storage system 110 may match the fourth verification field with the updated metadata of the object 1 with reference to the above S320, which is not described in detail in this embodiment of the present application.

[0299] S830: If the fourth verification field is not the second value, the storage system 110 updates the fourth verification field based on the updated metadata of the object 1 to obtain a fifth verification field.

[0300] In an optional implementation, the storage system 110 may update the fourth verification field with reference to the above S350 to obtain the fifth verification field, which is not described in detail in this embodiment of the present application.

[0301] Among them, the implementation method of the fifth verification field can refer to the implementation method of the first verification field mentioned above, and the embodiment of the present application will not be repeated here.

[0302] In an optional implementation, when the fourth verification field is the second value, the storage system 110 may perform a second atomic operation to update the updated object 1.

[0303] The second atomic operation is similar to the first principle operation mentioned above, and the second atomic operation includes one or both of the following: the storage system 110 performs a modification operation on the data content of the updated object 1 according to the second access request, and the storage system 110 performs a modification operation on the metadata of the updated object 1 according to the second access request.

[0304] S840 , the storage system 110 matches the fifth verification field with the updated metadata of the object 1 .

[0305] In an optional implementation, the storage system 110 may match the fifth verification field with the updated metadata of the object 1 with reference to the above S320.

[0306] S850, if the fifth verification field is the second value, the storage system 110 executes a second atomic operation to update the updated object 1 to obtain the updated object 1.

[0307] In an optional manner, the object 1 after being updated again may also be called a third object, a new object 1 or other names, which is not limited in the embodiment of the present application.

[0308] In an optional implementation, if the fourth verification field is not the second value, the storage system 110 may refer to the above Figure 7 Alternatively, the storage system 110 may also send a prompt message to the data access device 2 so that the data access device 2 resends the access request.

[0309] S860: Return a second access response to data access device 2.

[0310] The first access response includes a sixth verification field, and the sixth verification field includes: a fourth value associated with the metadata of the object 1 that is updated again.

[0311] In an optional implementation, the storage system 110 obtains a fourth value associated with the metadata of the updated object 1. The storage system 110 may obtain the fourth value in a manner similar to the manner in which the storage system 110 obtains the first value, which will not be described in detail in this embodiment of the present application.

[0312] In an optional implementation, the method for obtaining the sixth verification field may refer to the method for obtaining the second verification field described above, and the embodiment of the present application will not be described in detail here.

[0313] based on Figure 8 It can be known from the provided embodiment that when multiple data access devices access the same object, the storage system 110 performs an atomic operation on each access request, and after performing the atomic operation of updating the object, updates the metadata and verification field of the object. And when there is an access conflict, the storage system 110 initiates a retry. In this way, the reliability of data access is improved by using the retry mechanism.

[0314] To better illustrate the data access method provided by the present application, the implementation of the data access method is described by taking the storage system 110 including a server and a database, and the first access request and the second access request simultaneously updating the object 1 as an example. Figure 1 The database may be a storage resource constituted by the hard disk 105 in the server (server 110A or server 110B), or the database may be an additional storage device ( Figure 1 The server may be the storage resource provided by Figure 1 Server 110A or server 110B in.

[0315] like Fig. 9 As shown, Fig. 9 Schematic diagram 4 of the flow chart of the data access method provided in the embodiment of the present application. The data access method comprises steps S901 to S908.

[0316] S901 , data access device 1 obtains a first verification field of object 1 , and data access device 2 obtains a fourth verification field of object 1 .

[0317] like Fig. 9 As shown, data access device 1 obtains the first verification field of object 1 through a head request: lastMtime=100. Data access device 2 obtains the fourth verification field of object 1 through a head request: lastMtime=100.

[0318] S902, data access device 1 sends a first access request to the server, and data access device 2 sends a second access request to the server based on the fourth verification field lastMtime=100.

[0319] like Fig. 9 As shown, the first access request carries a first verification field lastMtime=100.

[0320] S903, the server receives the first access request, uses the first verification field lastMtime=100 for matching, and performs a first atomic operation of updating object 1 based on the first access request.

[0321] like Fig. 9 As shown, the server uses the first verification field lastMtime=100 as the update condition, and carries the update condition lastMtime=100 and the updated value 200 to perform the first atomic operation of updating object 1 on the database.

[0322] The value of lastMtime in the metadata of object 1 in the database is 100, the first verification field is the first value associated with the metadata of object 1, the database executes the first atomic operation to update object 1, and the value of lastMtime in the metadata of object 1 is updated from 100 to 200.

[0323] S904 , the server returns an update success to the data access device 1 , and returns the updated second verification field lastMtime=200 to the data access device 1 .

[0324] S905: The server receives a second access request.

[0325] S906, the server updates object 1 with the fourth check value lastMtime=100, and the update of object 1 fails.

[0326] like Fig. 9 As shown, since data access device 1 has updated the value of lastMtime in the metadata of object 1 from 100 to 200, the fourth verification field is not the second value lastMtime=200 associated with the updated metadata of object 1, so the update of updated object 1 fails.

[0327] S907, the server re-reads the fifth verification field lastMtime=200, the server re-matches based on the fifth verification field lastMtime=200, and executes the second atomic operation of updating object 1 based on the second access request.

[0328] like Fig. 9 As shown, the server uses the fifth verification field lastMtime=200 as the update condition, and carries the update condition lastMtime=200 and the updated value 300 to perform the second atomic operation of updating object 1 on the database.

[0329] The value of lastMtime in the metadata of object 1 in the database is 200, the fifth verification field is the second value associated with the metadata of object 1, the database executes the second atomic operation to update object 1, and the value of lastMtime in the metadata of object 1 is updated from 200 to 300.

[0330] S908 , the server returns an update success to the data access device 2 , and returns the updated sixth verification field lastMtime=300 to the data access device 2 .

[0331] In an optional implementation, if the server fails in the retry, the server returns a 409 error to the data access device 2. The data access device 2 retries again.

[0332] based on Fig. 9 In the provided embodiment, in the multi-thread concurrent access processing, the verification field is used as an update condition to identify concurrent access conflicts of multiple data access devices to the same object, without locking the object or creating multiple versions of the object. This can reduce the resource consumption caused by locking or creating multiple versions. In turn, the overhead of the storage system 110 can be reduced. In addition, the present application uses atomic operations to implement concurrent operations, ensures the atomicity of object modifications, and can efficiently handle concurrency issues of objects. And in the case of an access conflict, a retry is initiated by the storage system 110. If the storage system 110 fails to retry, a retry is initiated by the data access device. In this way, a multi-level retry mechanism is used to improve the reliability of data access, and reduce the occupation of network bandwidth by retrying only by the data access device, thereby reducing the consumption of network resources.

[0333] Above Figures 3 to 9 The technical solution provided by the present application is mainly introduced from the perspective of the interaction between the storage system 110 and the data access device. It is understandable that the storage system 110 includes hardware structures and / or software modules corresponding to the execution of various functions. It is easy for those skilled in the art to realize that, in combination with the algorithm steps of each example described in the embodiments disclosed in the present application, the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a certain function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present application.

[0334] The embodiment of the present application can group the functional modules of the storage system 110 according to the above method example. For example, each functional module can be grouped according to each function, or two or more functions can be integrated into one processing module. The above integrated module can be implemented in the form of hardware or in the form of software functional modules. It should be noted that the naming and grouping of the module groups in the embodiment of the present application are schematic, which is only a logical functional grouping. There may be other grouping methods in actual implementation.

[0335] For example, Fig.10 As shown, Fig.10 The schematic diagram of the structure of the storage system 110 provided in the embodiment of the present application. The storage system 110 can be divided into a processor 144 and a memory 146. It should be understood that the present application does not limit the number of processors 144 and memories 146 in the storage system 110.

[0336] The processor 144 may include any one or more of a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP), or a digital signal processor (DSP).

[0337] In the present application, the processor 144 may execute the above Figure 3 A data access method is provided. For example, a first access request of data access device 1 is obtained. A first verification field in the first access request is matched with metadata of object 1. If the first verification field is consistent with a first value associated with an element of object 1, an atomic operation of updating object 1 is performed, and a second verification field is returned to data access device 1.

[0338] The memory 146 may include a volatile memory, such as a random access memory (RAM). The memory may also include a non-volatile memory, such as a read-only memory (ROM), a flash memory, a hard disk drive (HDD), or a solid state drive (SSD).

[0339] The memory 146 stores multiple objects and metadata of each of the multiple objects. The processor 144 can read or write the multiple objects and metadata of each of the multiple objects in the memory 146 based on an access request.

[0340] In addition, the memory 146 also stores executable program codes, and the processor 144 executes the executable program codes to respectively implement the aforementioned data access methods, that is, the memory 146 stores instructions for executing the data access methods.

[0341] The data access method disclosed in the above method embodiment may be applied to the processor 144, or implemented by the processor 144. The processor 144 may be an integrated circuit chip having signal processor capabilities.

[0342] In the implementation process, each step of the above method can be completed by the hardware integrated logic circuit or software instructions in the processor 144. The above processor 144 can be a general-purpose processor, including a CPU, a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete electron tubes or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present application can be implemented or executed. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in the embodiment of the present application can be directly embodied as a hardware decoding processor to execute, or the hardware and software modules in the decoding processor are combined to execute. The software module can be located in a mature storage medium in the field such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory 146, and the processor 144 reads the information in the memory 146 and completes the steps of the above method in combination with its hardware.

[0343] In a possible implementation, the processor 144 may also be used to execute a data access method. For a specific implementation, reference may be made to the embodiments provided in the above data access method, and the embodiments of the present application will not be described in detail herein.

[0344] In addition, in some optional implementations, the storage system 110 may further include a bus 142 and a communication interface 148. Fig.10 As shown, the processor 144 , the memory 146 , and the communication interface 148 communicate via a bus 142 .

[0345] The communication interface 148 uses a transceiver module such as, but not limited to, a network interface card or a transceiver to implement communication between the storage system 110 and other devices or a communication network.

[0346] In addition, the storage system 110 can also be named as a data access device 130. The data access device 130 has different Figure 1 or Fig.10 The structure of the storage system 110 is shown. Fig.11 As shown, the data access device 130 includes a processing module 152 , a communication module 151 and a storage module 153 .

[0347] The storage module 153 is used to store multiple objects, metadata of each object, executable program code, and data of the data access device 130 in access request processing, etc.

[0348] The communication module 151 is used to obtain a first access request from a first data access device and send a first access response to the first data access device. The first access request includes: a first verification field associated with metadata of a first object, and the first object is one of the multiple objects stored in the storage module 153. The first access response includes a second verification field, and the second verification field includes: a second value associated with metadata of a second object. For example, the communication module 151 is used to perform the above Figure 3 S310 and S340 in.

[0349] The processing module 152 is used to match the first verification field in the first access request with the metadata of the first object, and if the first verification field is a first value associated with the metadata of the first object, perform a first atomic operation to update the first object to obtain the second object. Figure 3 S320 to S330 in.

[0350] Among them, the communication module 151, the processing module 152 and the storage module 153 can be implemented by software or by hardware. Exemplarily, the following takes the processing module 152 as an example to introduce the implementation of the processing module 152. Similarly, the communication module 151 and the storage module 153 can refer to the implementation of the processing module 152.

[0351] As an example of a software functional unit, the processing module 152 may include code running on a computing instance. The computing instance may include at least one of a physical host (computing device), a virtual machine, and a container. Further, the computing instance may be one or more. For example, the processing module 152 may include code running on multiple hosts / virtual machines / containers. It should be noted that the multiple hosts / virtual machines / containers used to run the code may be distributed in the same region or in different regions. Furthermore, the multiple hosts / virtual machines / containers used to run the code may be distributed in the same availability zone (AZ) or in different AZs, each AZ including one data center or multiple data centers with similar geographical locations. Generally, a region may include multiple AZs.

[0352] Similarly, multiple hosts / virtual machines / containers used to run the code can be distributed in the same virtual private cloud (VPC) or in multiple VPCs. Usually, a VPC is set up in a region. For cross-region communication between two VPCs in the same region and between VPCs in different regions, a communication gateway needs to be set up in each VPC to achieve interconnection between VPCs through the communication gateway.

[0353] As an example of a hardware functional unit, the processing module 152 may include at least one computing device, such as a server, etc. Alternatively, the processing module 152 may also be a device implemented using an application-specific integrated circuit (ASIC) or a programmable logic device (PLD). The PLD may be a complex programmable logical device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL), or any combination thereof.

[0354] The multiple computing devices included in the processing module 152 can be distributed in the same region or in different regions. The multiple computing devices included in the communication module 151 can be distributed in the same AZ or in different AZs. Similarly, the multiple computing devices included in the processing module 152 can be distributed in the same VPC or in multiple VPCs. The multiple computing devices can be any combination of computing devices such as servers, ASICs, PLDs, CPLDs, FPGAs, and GALs.

[0355] It should be noted that, in other embodiments, the processing module 152 can be used to execute any step in the data access method. The communication module 151 can be used to execute any step in the data access method. The storage module 153 can be used to execute any step in the data access method. The communication module 151, the processing module 152 and the storage module 153 can be used to execute any step in the data access method. The steps that the communication module 151, the processing module 152 and the storage module 153 are responsible for implementing can be specified as needed. The full functions of the data access device 130 are realized by respectively implementing different steps in the data access method through the communication module 151, the processing module 152 and the storage module 153.

[0356] It should be understood that the above-mentioned data access device 130 can be implemented by a storage controller. The data access device 130 according to the embodiment of the present application can correspond to the method described in the embodiment of the present application, and the above-mentioned and other operations and / or functions of each unit and module in the data access device 130 are respectively for implementing the corresponding processes of each method in the aforementioned figures, and for the sake of brevity, they are not repeated here.

[0357] For example, the storage controller includes a control circuit and an interface circuit. The interface circuit is used to receive data from other devices outside the storage controller and transmit it to the control circuit, or to send data from the control circuit to other devices outside the storage controller. For example, the interface circuit performs the above Figure 3 The control circuit performs the functions of the storage system 110 in the aforementioned data access method through the logic circuit or the execution code instruction, and the interface circuit, for example, performing the above Figure 3 S320 to S330 in.

[0358] The method steps in this embodiment can be implemented by hardware or by a processor executing software instructions. The software instructions can be composed of corresponding software modules, which can be stored in random access memory (RAM), flash memory, read-only memory (ROM), programmable ROM (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), registers, hard disks, mobile hard disks, CD-ROMs, or any other form of storage medium known in the art. An exemplary storage medium is coupled to the processor so that the processor can read information from the storage medium and write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and the storage medium can be located in an ASIC. In addition, the ASIC can be located in a computing device. Of course, the processor and the storage medium can also exist as discrete components in a network device or a terminal device.

[0359] The embodiment of the present application also provides a computer program product including instructions. The computer program product may be a software or program product including instructions that can be run on a computing device or stored in any available medium. When the computer program product is run on at least one computing device, the at least one computing device executes the above data access method.

[0360] For example, when the computer program product is run on at least one computing device, the at least one computing device is caused to execute Figure 3 The data access method shown.

[0361] The embodiment of the present application also provides a computer-readable storage medium. All or part of the processes in the above method embodiments can be completed by a computer program to instruct the relevant hardware, and the program can be stored in the above computer-readable storage medium. When the program is executed, it can include the processes of the above method embodiments. The computer-readable storage medium can be a terminal of any of the above embodiments, such as: an internal storage unit including a data transmission end and / or a data receiving end, such as a hard disk or memory of a terminal. The above computer-readable storage medium can also be an external storage device of the above terminal, such as a plug-in hard disk, a smart memory card (smart media card, SMC), a secure digital (securedigital, SD) card, a flash card (flash card), etc. equipped on the above terminal. Further, the above computer-readable storage medium can also include both the internal storage unit of the above terminal and an external storage device. The above computer-readable storage medium is used to store the above computer program and other programs and data required by the above terminal. The above computer-readable storage medium can also be used to temporarily store data that has been output or is to be output.

[0362] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented by software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer programs or instructions. When the computer program or instruction is loaded and executed on a computer, the process or function described in the embodiment of the present application is executed in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, a network device, a user device or other programmable device. The computer program or instruction may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer program or instruction may be transmitted from one website site, computer, server or data center to another website site, computer, server or data center by wired or wireless means. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium may be a magnetic medium, for example, a floppy disk, a hard disk, a tape; it may also be an optical medium, for example, a digital video disc (DVD); it may also be a semiconductor medium, for example, a solid state drive (SSD).

[0363] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed in the present application, and these modifications or replacements should be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.< / delete> < / delete>

Claims

1. A data access method, characterized in that: Applied to a storage system, wherein the storage system stores a plurality of objects and metadata of each of the plurality of objects, and one object supports concurrent access by a plurality of data access devices; The method comprises: Obtaining a first access request from a first data access device; Wherein, the first access request includes: associating a first verification field with metadata of a first object, the first object being one of the plurality of objects; Matching a first verification field in the first access request with metadata of the first object, and if the first verification field is a first value associated with the metadata of the first object, performing a first atomic operation of updating the first object to obtain a second object; A first access response is sent to the first data access device; the first access response includes a second verification field, and the second verification field includes: a second value associated with the metadata of the second object.

2. The method according to claim 1, characterized in that The metadata of the first object includes the first value; and / or the metadata of the second object includes the second value.

3. The method according to claim 1 or 2, characterized in that: The first value includes one or two of the following: a timestamp and a control field in the metadata of the first object; the timestamp is used to indicate the last modification time of the first object; and the control field is used to indicate the version of the first object.

4. The method according to any one of claims 1 to 3, characterized in that After matching the first verification field in the first access request with the metadata of the first object, the method further includes: If the first verification field is not the first value, querying metadata of the first object in the storage system to obtain a third verification field of the first object; The third verification field is matched with the metadata of the first object, and if the third verification field is a third value associated with the metadata of the first object, a first atomic operation of updating the first object is performed.

5. The method according to claim 4, characterized in that After matching the third verification field with the metadata of the first object, the method further includes: If the third verification field is not the third value, a prompt message is returned to the first data access device, where the prompt message is used to instruct the first data access device to re-initiate an access request.

6. The method according to any one of claims 1 to 5, characterized in that After sending the first access response to the first data access device, the method further includes: Obtaining a second access request from a second data access device; the second access request comprising: associating a fourth verification field with metadata of the first object; the fourth verification field being the same as the first verification field; Matching the fourth verification field with the metadata of the second object, and if the fourth verification field is not the second value, updating the fourth verification field based on the metadata of the second object to obtain a fifth verification field; Matching the fifth verification field with the metadata of the second object, and if the fourth verification field is the second value, performing a second atomic operation of updating the second object to obtain a third object; A second access response is returned to the second data access device; the first access response includes a sixth verification field, and the sixth verification field includes: a fourth value associated with the metadata of the third object.

7. The method according to any one of claims 1 to 6, characterized in that Before obtaining the first access request of the first data access device, the method further includes: Obtaining a verification request of the first data access device, the verification request comprising: an object identifier, the verification request being used to request a verification field; In response to the verification request, obtaining metadata of a first object corresponding to the object identifier among the multiple objects; A verification response is returned to the first data access device based on the metadata of the first object; the verification response includes a first verification field.

8. The method according to any one of claims 1 to 7, characterized in that The first access request carries one or more first verification fields; different first verification fields in the multiple first verification fields correspond to different first objects.

9. The method according to any one of claims 1 to 8, characterized in that The header of the first access request includes the first verification field, or the data content of the first access request includes the first verification field.

10. The method according to any one of claims 1 to 9, characterized in that The performing of the first atomic operation of updating the first object comprises: A modification operation is performed on the first object based on the first access request, and metadata of the first object is updated; the modification operation includes one or more of the following: data writing, data deletion, and data modification.

11. A data access device, characterized in that: The device comprises: A communication module, configured to obtain a first access request from a first data access device, wherein the first access request includes: a first verification field associated with metadata of a first object, the first object being one of a plurality of objects; a processing module, configured to match a first verification field in the first access request with metadata of the first object, and if the first verification field is a first value associated with the metadata of the first object, perform a first atomic operation of updating the first object to obtain a second object; The communication module is further used to send a first access response to the first data access device; the first access response includes a second verification field, and the second verification field includes: a second value associated with the metadata of the second object.

12. A storage system, characterized in that: include: Processor and Memory The memory is used to store a plurality of objects and metadata of each of the plurality of objects; The processor is configured to execute the method of any one of claims 1 to 10, and access a first object among the plurality of objects stored in the memory.

13. A data access system, characterized in that: The data access system comprises: A plurality of data access devices and the storage system according to claim 12; A first data access device among the multiple data access devices sends a first access request to the storage system, wherein the first access request includes a first verification field associated with metadata of a first object, and the first object is one of multiple objects stored in the storage system; The storage system executes the method according to any one of claims 1 to 10 according to the first access request.

14. A storage controller, characterized in that: include: Interface circuit and control circuit; The interface circuit is used to receive a first access request; The control circuit is used to cooperate with the interface circuit to execute the method according to any one of claims 1 to 10 according to the first access request.

15. A computer program product comprising instructions, characterized in that When the instructions are executed by a storage controller, the storage controller is caused to perform the method according to any one of claims 1 to 10.

Images