Data processing method and device in distributed system

Abstract

The invention discloses a data processing method and device in a distributed system. The method comprises the following steps: acquiring an identifier of a target object to be accessed by a first computing node in a plurality of computing nodes; wherein the target object is stored in a first storage pool or a second storage pool outside the first storage pool; the first computing node obtains metadata of the target object from a first storage pool based on the identifier of the target object; the metadata of the target object is used for the first computing node to perform first access operation on the target object; and the first computing node updates metadata of the target object in the first storage pool based on the first access operation. According to the method, the access conflict of a plurality of computing nodes to the same object can be avoided while the time delay is reduced and the protocol stack calling overhead is saved.

Description

Technical Field

[0001] The present application relates to the field of data storage technology, and in particular to a data processing method and device in a distributed system. Background Art

[0002] In a distributed system with storage and computing separation, compute nodes and storage pools are separated. The storage pool is used to store objects accessed by the compute nodes, such as files and directories. A distributed system typically includes multiple compute nodes, which share objects stored in the storage pool. Compute nodes access objects stored in the storage pool using remote direct memory access (RDMA) technology.

[0003] Compute nodes, based on Remote Direct Access (RDMA) technology, can bypass the storage pool's CPU and directly access objects stored in the pool. This reduces latency and saves protocol stack call overhead. However, because RDMA bypasses the storage pool's CPU, it relies on the compute nodes themselves to maintain information about objects in the pool, such as metadata. Objects in the pool are accessible to multiple compute nodes in a distributed system. Each compute node independently maintains information about objects in the pool, lacking a global perspective and prone to conflicts when multiple compute nodes attempt to access the same object. Summary of the Invention

[0004] The present application provides a data processing method and device in a distributed system, which can reduce latency, save protocol stack call overhead, and avoid access conflicts between multiple computing nodes to the same object.

[0005] In a first aspect, a data processing method in a distributed system is provided. The distributed system includes a first storage pool and multiple computing nodes separated from the first storage pool. The first storage pool is used to store metadata of objects accessed by the multiple computing nodes. The method includes: a first computing node among the multiple computing nodes obtains an identifier of a target object to be accessed. The target object is stored in the first storage pool or a second storage pool outside the first storage pool. The first computing node obtains metadata of the target object from the first storage pool based on the identifier of the target object. The metadata of the target object is used by the first computing node to perform a first access operation on the target object. The first computing node updates the metadata of the target object in the first storage pool based on the first access operation. The updated metadata of the target object can be used by other computing nodes among the multiple computing nodes, such as the second computing node, to perform a second access operation on the target object.

[0006] In this method, whenever any of the multiple compute nodes needs to access a target object, it retrieves the target object's metadata from a storage pool, accesses the target object based on the metadata, and updates the resource pool based on the access to the target object. This allows a compute node to both access the target object using the metadata and update the metadata, allowing other compute nodes to access the target object using the updated metadata, thus avoiding access conflicts between multiple compute nodes accessing the same object.

[0007] Furthermore, in this method, the metadata is updated by the computing node accessing the target object, thereby avoiding the problem of metadata operation performance degradation that may occur when the metadata server maintains the metadata.

[0008] Furthermore, in this method, the compute node can obtain the target object's metadata from the first storage pool through a Remote Direct Data Access (RDMA) operation based on the target object's identifier. The compute node can also update the target object's metadata in the first storage pool through RDMA operations. This reduces metadata access latency and protocol stack call overhead.

[0009] In one possible implementation, a first storage pool stores a data structure, which includes multiple storage areas, wherein each storage area in the multiple storage areas is associated with a mapping value, and the storage area is used to store metadata of an object corresponding to the mapping value associated with the storage area; the first computing node obtains the metadata of the target object from the first storage pool based on an identifier of the target object, including: the first computing node obtains the mapping value corresponding to the target object based on the identifier of the target object; the first computing node obtains the target storage area associated with the mapping value corresponding to the target object in multiple storage areas; the first computing node obtains the metadata of the target object from the target storage area based on the identifier of the target object.

[0010] Different storage areas are independent of each other, and a specific storage area can be locked without affecting other storage areas. Therefore, when a computing node accesses metadata in a storage area, it does not affect other computing nodes accessing metadata in other storage areas.

[0011] In one possible implementation, the target storage area includes multiple storage units, wherein the multiple storage units correspond to the multiple objects on a one-to-one basis, and each storage unit in the multiple storage units is used to store metadata of the object corresponding to the storage unit; the first computing node obtains the metadata of the target object from the target storage area based on the identifier of the target object, including: the first computing node obtains the target storage unit storing the metadata of the target object from the multiple storage units based on the identifier of the target object; the first computing node obtains the metadata of the target object from the target storage unit.

[0012] Different storage units are independent of each other, and a single storage unit can be locked without affecting other storage units. Therefore, when a computing node accesses metadata in a storage unit, it does not affect other computing nodes accessing metadata in other storage units.

[0013] In a possible implementation, the metadata of the target object includes: historical access information of multiple computing nodes to the target object; the method further includes: the first computing node deleting the target object from a storage pool storing the target object based on the historical access information.

[0014] In this method, a computing node can delete objects from the storage pool based on historical access information of multiple computing nodes to the objects. This frees up storage space in the storage pool by deleting objects with low access counts or access frequency.

[0015] In one possible implementation, the first computing node deletes the target object from the storage pool storing the target object based on historical access information, including: the first computing node confirms that the target object is an object to be deleted based on the historical access information; when the access information of the target object within a preset time period after confirming that the target object is an object to be deleted meets a preset condition, the first computing node deletes the target object from the storage pool storing the target object.

[0016] In this implementation, after confirming that the target object is to be deleted, the target object is observed for a period of time to confirm that no computing node has accessed the target object, and then the target object is deleted. In this way, the accidental deletion of useful objects can be prevented.

[0017] In a second aspect, a data processing device is provided, characterized in that a distributed system includes a first storage pool and multiple computing nodes separated from the first storage pool; wherein the first storage pool is used to store metadata of objects for access by the multiple computing nodes, and the device is configured at the first computing node among the multiple computing nodes; the device includes: a first acquisition unit, used to obtain an identifier of a target object to be accessed; wherein the target object is stored in the first storage pool or a second storage pool outside the first storage pool; a second acquisition unit, used to obtain metadata of the target object from the first storage pool based on the identifier of the target object; the metadata of the target object is used by the first computing node to perform a first access operation on the target object; and an update unit, used to update the metadata of the target object in the first storage pool based on the first access operation.

[0018] In one possible implementation, a first storage pool stores a data structure, which includes multiple storage areas, wherein each of the multiple storage areas is associated with a mapping value, and the storage area is used to store metadata of an object corresponding to the mapping value associated with the storage area; the second acquisition unit is used to: obtain the mapping value corresponding to the target object based on the identifier of the target object; obtain a target storage area associated with the mapping value corresponding to the target object in multiple storage areas; and obtain the metadata of the target object from the target storage area based on the identifier of the target object.

[0019] In one possible implementation, the target storage area includes multiple storage units, wherein the multiple storage units correspond one-to-one to the multiple objects, and each storage unit in the multiple storage units is used to store metadata of the object corresponding to the storage unit; the second acquisition unit is used to: based on the identification of the target object, obtain the target storage unit storing the metadata of the target object from the multiple storage units; and obtain the metadata of the target object from the target storage unit.

[0020] In a possible implementation, the metadata of the target object includes: historical access information of multiple computing nodes to the target object; the device also includes: a deletion unit, configured to delete the target object from a storage pool storing the target object based on the historical access information.

[0021] In one possible implementation, the deletion unit is used to: confirm that the target object is an object to be deleted based on historical access information; when the access information of the target object within a preset time period after the target object is confirmed as an object to be deleted meets a preset condition, delete the target object from the storage pool storing the target object.

[0022] In a possible implementation, the second acquiring unit is configured to acquire metadata of the target object from the first storage pool through a remote direct data access (RDMA) operation based on an identifier of the target object.

[0023] In a possible implementation, the updated metadata of the target object is used by a second computing node among the multiple computing nodes to perform a second access operation on the target object.

[0024] In a third aspect, a data processing device is provided, comprising: a memory for storing an executable program; and a processor for executing the method provided in the first aspect by running the executable program.

[0025] In a fourth aspect, a computer-readable storage medium is provided, comprising: computer program instructions. When the computer program instructions are executed by a computer device, the computer device executes the method provided in the first aspect.

[0026] In a fifth aspect, a computer program product comprising instructions is provided, wherein when the instructions are executed by a computer device, the computer device is caused to execute the method provided in the first aspect.

[0027] The beneficial effects of the second to fifth aspects can be referred to the above introduction to the beneficial effects of the first aspect, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 is a schematic diagram of a distributed system provided in an embodiment of the present application;

[0029] Figure 2 is a schematic diagram of a resource pool provided in an embodiment of the present application;

[0030] Figure 3 is a schematic diagram of a data structure provided in an embodiment of the present application;

[0031] Figure 4 is a schematic diagram of a computing node provided in an embodiment of the present application;

[0032] Figure 5 is a schematic diagram of a metadata management module provided in an embodiment of the present application;

[0033] Figure 6 This is a flow chart of a data processing method provided by an embodiment of the present application;

[0034] Figure 7 This is a flow chart of a data processing method provided by an embodiment of the present application;

[0035] Figure 8 This is a flow chart of a data processing method provided by an embodiment of the present application;

[0036] Figure 9 This is a flow chart of a data processing method provided by an embodiment of the present application;

[0037] Figure 10 A schematic diagram of the structure of a data processing device provided in an embodiment of the present application;

[0038] Figure 11 A schematic diagram of the structure of a data processing device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0039] The technical solutions provided by this application will be described in detail below in conjunction with the accompanying drawings. Although certain embodiments of the present application are shown in the accompanying drawings, it should be understood that the present application can be implemented in various forms and should not be construed as being limited to the embodiments described herein. Instead, these embodiments are provided to provide a more thorough and complete understanding of the present application. It should be understood that the drawings and embodiments of the present application are for illustrative purposes only and are not intended to limit the scope of protection of the present application.

[0040] In the description of the embodiments of this application, the term "including" and similar terms should be understood as open inclusion, that is, "including but not limited to." The term "based on" should be understood as "based at least in part on." The term "one embodiment" or "the embodiment" should be understood as "at least one embodiment." The terms "first," "second," etc. can refer to different or the same objects. Other explicit and implicit definitions may also be included below.

[0041] In this application, "at least one" means one or more, and "more" means two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can mean: including the existence of A alone, the existence of A and B at the same time, and the existence of B alone, where A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following items" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c can mean: a, b, c, ab, ac, bc, or abc, where a, b, c can be single or multiple.

[0042] In order to make the technical solution provided by this application clearer, before describing the technical solution provided by this application in detail, the relevant terms are first explained.

[0043] Cache: Loading data into memory or other high-performance storage that can be accessed by the processor at high speed. Cache is temporary storage of data, not persistent storage.

[0044] Storage pool: A resource pool with storage space used to provide data storage services. Typically, a storage pool consists of multiple storage nodes, and the storage space of the storage pool is composed of one or more types of memory within the multiple storage nodes. The memory within the multiple storage nodes is uniformly addressed, so that the memory within the multiple storage nodes is pooled to form the storage space of the storage pool. Storage pools can be divided into storage pools for persistent storage of data and cache pools for temporary storage of data.

[0045] Cache pool: A storage pool used for temporary data storage. A common cache pool is the memory pool. The storage space in a cache pool typically consists of the memory and / or high-performance storage of one or more storage nodes. Memory can include dynamic random access memory (DRAM), storage class memory (SCM), and persistent memory (PMEM). High-performance storage can include solid-state drives (SSDs).

[0046] Persistent storage pool: refers to the storage pool used for persistent storage of data.

[0047] Compute nodes are nodes that perform computing tasks. Typically, compute nodes are located on the user side, hosting applications and clients. The client receives access requests triggered by the application and interacts with the node or storage pool where the requested object resides to obtain the requested object. The client then forwards the requested object to the application, allowing the application to process it.

[0048] Remote direct memory access (RDMA) is a high-speed interconnect technology that allows a local processor (such as a CPU) to directly access remote data without going through the remote CPU. Common RDMA technologies include InfiniBand (IB), RDMA over converged Ethernet (RoCE), Compute Express Link (CXL), and Unified Bus (UB).

[0049] Object: This refers to the processing object or computational object of a compute node. Objects are typically directories or files. Objects reside in directories, meaning they are objects within a directory.

[0050] Object metadata: Data used to describe an object. File metadata includes the file's name, identifier (ID), last modification time, size, and directory. Directory metadata includes the directory's name, ID, creation time, and the ID of the parent directory.

[0051] Persistent storage: also known as disk flushing, refers to storing data in memory on the hard disk to achieve permanent data preservation.

[0052] A mapped value is a value obtained by mapping an object's identifier (e.g., its name) according to a predefined mapping algorithm (e.g., a hash range partitioning algorithm or a key range partitioning algorithm). The mapped value for an object is also referred to as the object's mapped value. A mapped value has a size and can be represented by a positive integer.

[0053] Mapping algorithm: refers to an algorithm that maps an object's key to a mapped value. Common mapping algorithms include hash range partitioning algorithm and key range partitioning algorithm.

[0054] Consistent hashing algorithm: a hash algorithm for calculating the mapping value of an object, which belongs to the hash range partitioning algorithm. In an embodiment of the present application, the consistent hashing algorithm is an algorithm that performs hash calculation on the identifier (such as the name) of the object based on the total number of assignable mapping values, wherein the obtained hash value is used as the mapping value of the object. The commonly used consistent hashing algorithm is: taking the remainder based on the total number of assignable mapping values. For example, the name of the object is hashed, the obtained hash value is divided by the total number of assignable mapping values, and the remainder obtained is used as the mapping value of the object.

[0055] Data centers can be categorized as either monolithic or decoupled architectures. In a monolithic architecture, each compute node in the data center has independent computing and storage resources. Compute nodes access each other's stored objects via remote procedure calls (RPCs). RPC technology is based on the Transmission Control Protocol / Internet Protocol (TCP / IP) network. Therefore, when compute nodes access objects in the resource pool using RPC, the storage pool's central processing unit (CPU) must process data packets between the compute node and the resource pool. This results in significant latency and protocol stack call overhead. In a decoupled architecture, compute and storage resources are separated. These resources can be pooled and allocated on demand. In this decoupled architecture, compute nodes access objects in the storage pool using Remote Direct Access (RDMA) technology, reducing latency and protocol stack call overhead. However, since RDMA bypasses the storage pool's CPU, the compute nodes themselves must maintain information about the objects in the storage pool, such as metadata. The objects in the storage pool are accessed by multiple computing nodes in the distributed system. Each computing node maintains information about the objects in the storage pool independently, lacking a global perspective. This makes it easy for multiple computing nodes to have access conflicts to the same object.

[0056] In related computing, for a separated architecture, one or more metadata servers are set up to store the metadata of objects. Each metadata server is responsible for storing and managing the metadata of objects in one or more directories. The data of the object is stored in a storage pool. When a computing node accesses an object, the metadata server corresponding to the object processes the metadata of the object. The objects in the storage pool are shared by multiple computing nodes, and there may be a situation where objects in a certain directory are accessed frequently. For example, multiple computing nodes access objects in the directory simultaneously or frequently. This requires the metadata server corresponding to the directory to process the metadata of the objects in the directory frequently, which puts pressure on the performance of the metadata server, resulting in access hotspots and reduced metadata operation performance.

[0057] Furthermore, in related technologies, metadata for multiple objects is maintained using a linked list data structure, meaning that multiple metadata items are recorded in the same data table. Modifying the metadata for one object requires locking the entire table, delaying operations on metadata for other objects and impacting metadata performance.

[0058] In addition, when objects are eliminated from the cache pool, an elimination list needs to be maintained, which requires increasing the version number and requiring multiple cycles of interaction, making the operation cumbersome.

[0059] The present invention provides a distributed system and data processing method. Through this method, each computing node in the distributed system can maintain the metadata of objects from a global perspective, reducing latency and saving protocol stack call overhead while avoiding access conflicts between multiple computing nodes to the same object and achieving higher metadata operation performance.

[0060] The distributed system provided in the embodiment of the present application is a system using a separate architecture. Figure 1 As shown, the distributed system includes a storage pool 110 and a plurality of computing nodes separated from the storage pool 110, such as computing node 210, computing node 220, etc. Among them, the plurality of computing nodes can constitute a computing pool 200. The computing nodes and storage pools in the distributed system can be connected through a network. In some embodiments, the distributed system can also include at least one storage pool such as storage pool 120. Exemplarily, one of the storage pools 110 and 120 can be a cache pool, and the other can be a persistent storage pool. Exemplarily, the storage pools 110 and 120 can both be cache pools, or can both be persistent storage pools. At least one storage pool such as storage pool 110 and storage pool 120 can constitute a storage cluster 100.

[0061] In the following description, when computing nodes 210 and computing nodes 220 are not specifically distinguished, they may be referred to as computing nodes. When storage pools 110 and storage pools 120 are not specifically distinguished, they may be referred to as storage pools.

[0062] In some embodiments, as Figure 2 As shown, a storage pool may include multiple storage nodes. A storage node is a device with storage resources, such as a server, desktop computer, or storage array controller, hard disk enclosure, etc. Each storage node may include a network card and one or more memories. The memories in the multiple storage nodes are uniformly addressed, that is, the address of the storage unit in each memory of each storage node in the multiple storage nodes is unique in the storage pool, so that the memories in the multiple storage nodes logically form a linear address space, resulting in the storage space of the storage pool. Among them, a storage unit is the basic unit of memory, and each storage unit can store a certain number of binary bits.

[0063] In some embodiments, the storage pool may be a cache pool, and the storage of the storage nodes in the storage pool may be a memory or a high-performance hard disk, such as DRAM, SCM, PMEM, SSD, etc. In some embodiments, the storage pool may be a storage pool for persistent storage of data, and the storage of the storage nodes in the storage pool 110 may be a hard disk, such as an SSD, a hard disk drive (HDD), etc.

[0064] The network card of the storage node is used to communicate with other storage nodes in the same storage pool and with the computing nodes for data exchange.

[0065] The storage pool 110 is used to store metadata of objects that are accessible to multiple computing nodes in the distributed system. In other words, the storage pool 110 is used to store metadata of objects that are accessible to each of the multiple computing nodes.

[0066] In some embodiments, as Figure 3 As shown, storage pool 110 stores a data structure. The data structure includes multiple storage areas, each of which is associated with a mapping value. The storage area is used to store metadata of an object corresponding to the mapping value associated with the associated storage area. The mapping value corresponds to the object and is obtained by mapping the object identifier according to a mapping algorithm.

[0067] In one example, different storage areas are independent of each other, and a storage area can be locked individually without affecting other storage areas.

[0068] The same mapping value may correspond to multiple objects, and the storage area associated with the mapping value can store the metadata of the multiple objects. Figure 3 As shown, a storage area may include multiple storage units, which are used to store metadata of objects corresponding to the mapping values ​​associated with the storage area. Each storage unit is used to store metadata for one object. Different storage units are independent of each other, and a single storage unit can be locked without affecting other storage units.

[0069] The metadata of an object may include the object's address, lock information, cache status, size, and historical access information. The historical access information includes the object's write time, the object's most recent access time, and the number of hits.

[0070] The address of an object refers to the storage address of the object, through which the object can be accessed. The address of an object can be a pointer to the object.

[0071] The lock information of an object includes the lock status (status) and the lock count (count). Among them, locks include shared locks (S locks) and exclusive locks (X locks). The lock status refers to whether the current object has a lock applied, and whether it is a shared lock or an exclusive lock. When an object is exclusively locked by a computing node, other computing nodes cannot access the object and need to delay access to the object. Shared locks allow multiple computing nodes to read the object at the same time. The lock count refers to the number of computing nodes that read the object at the same time when the shared lock is applied. If a shared lock is applied to the object, the lock count is increased when a computing node reads the object. When a computing node finishes reading the object, the lock count is decreased.

[0072] An object's cache status includes three states: deletable, cacheable, and persistent. These states can be represented by flags: 0 for deletable, 1 for cacheable, and 3 for persistent. For example, if an object is marked as persistent, it will not be selected for deletability.

[0073] The size of an object refers to the amount of data contained in the object itself.

[0074] The object's write time, last access time, and number of accesses are collectively referred to as the object's access history. The object's write time, also known as the object's insertion time, refers to the time the object was stored in the storage pool. The object's last access time refers to the time the object was most recently accessed. The object's access count refers to the total number of historical accesses to the object. The object's access count is the sum of all accesses to the object by all computing nodes in the distributed system.

[0075] In one embodiment, the data structure is specifically a hash table structure. Exemplarily, when the object is specifically a cache object (ie, an object stored in a cache pool), the data structure is a global metadata cache hash table structure.

[0076] The storage area in the data structure is specifically a bucket, and the storage unit in the storage area is specifically a slot. In an example, see Figure 3 ,Each storage unit can be divided into atomic area, check area and statistical area.

[0077] The atomic area consists of 8 bytes. The upper byte records the object's ID fingerprint (IFP). The object's ID fingerprint is the last byte of the object's ID. The middle 6 bytes of the atomic area record the object's address. The last byte of the atomic area records lock information. The upper 2 bits of this byte record the lock status, and the lower 6 bits record the lock count.

[0078] The checksum area, also known as the metadata area, is used to record the object's ID, cache status, and size. The object's ID occupies an integer multiple of 8 bytes, the object's cache status occupies 1 byte, and the object's size occupies 7 bytes.

[0079] The statistics area is used to record the object's write time, the object's most recent access time, and the object's access count. Each of the object's write time, the object's most recent access time, and the object's access count occupies one byte.

[0080] A computing node can be a computing device located at the user side, such as a server, computer, mobile terminal, etc. Figure 4 As shown, the computing node includes a processor (such as a CPU), memory, and a network card. The memory is used as the local cache of the computing node and can store applications, metadata management module programs, and other programs for the processor to run. The memory can also be used to store data obtained by the computing node from the storage pool for processing by the processor. The memory can also be used to store the data processing results of the computing node so that the processor can send the data processing results to the storage pool through the network card. The processor is used to run related programs loaded into the memory and process related data in the memory. The network card is used for the computing node to communicate with the storage pool and other computing nodes to exchange data.

[0081] In some embodiments, as Figure 5 As shown, the metadata management module includes a adding submodule, a querying submodule, and a deleting submodule. The adding submodule is used to add metadata to the storage pool. For example, when a new object is stored in the storage pool, the metadata of the object is written to the storage pool. The querying submodule is used by computing nodes to query the metadata of an object when they need to access the object, so as to access the object using the object's metadata, and to modify the object's metadata based on the access to the object. The deleting submodule is used by computing nodes to delete objects (such as objects with low access frequency) from the storage pool to save storage pool space.

[0082] The above describes a distributed system provided by an embodiment of the present application. Next, in conjunction with the distributed system, the data processing method provided by an embodiment of the present application is introduced.

[0083] In this method, a storage pool 110 in a distributed system is used to store metadata of objects accessible to multiple computing nodes in the distributed system. In other words, the storage pool 110 is used to store metadata of objects that are accessible to each of the multiple computing nodes.

[0084] The method is executed by a computing node in a distributed system. Each of the multiple computing nodes in the distributed system can execute the method. Next, taking computing node 210 as an example, Figure 6 , the execution process of this method is introduced.

[0085] In step 601 , the computing node 210 obtains an identifier of a target object to be accessed; wherein the target object is stored in the storage pool 110 or a storage pool 120 other than the storage pool 110 .

[0086] Computing node 210 may need to access one or more objects stored in storage pool 110 or storage pool 120. This object may be referred to as a target object to be accessed by computing node 210. Computing node 210 obtains an identifier for the target object. Object identifiers are globally unique in a distributed system; that is, identifiers for different objects are unique within the distributed system. Object identifiers can be used to identify the object.

[0087] Computing node 210 may obtain an access request. The access request may be triggered by an application in computing node 210 or received by computing node 210 from outside the distributed storage system. The object of the access request is a target object. The access request includes an identifier of the target object.

[0088] In some embodiments, the storage pool where the target object is located may be a cache pool. That is, the target object is a cache object stored in the cache pool. In some embodiments, the storage pool where the target object is located may be a persistent storage pool.

[0089] In some embodiments, the target object is stored in storage pool 110. That is, in this embodiment, the target object's own data and the target object's metadata can be stored in the same storage pool. In some embodiments, the target object is stored in storage pool 120. That is, in this embodiment, the target object's own data and the target object's metadata can be stored in different storage pools.

[0090] Before step 601, the target object may be stored in the storage pool 110 or the storage pool 120, and the metadata of the target object may be stored in the storage pool 110. Storing the target object in the storage pool 110 or the storage pool 120, and storing the metadata of the target object in the storage pool 110, may be performed under the operation of any computing node in the distributed system.

[0091] In some embodiments, it may be assumed that the computing node 210 stores the target object in the storage pool 110 or the storage pool 120, and stores the metadata of the target object in the storage pool 110. The specific process is as follows.

[0092] See Figure 7 , the application in the computing node 210 can trigger a storage request in step 701. The storage request includes the identifier of the target object. The metadata management module in the computing node 210 can respond to the storage request and obtain the target object based on the identifier of the target object, for example, from the memory or from the persistent storage pool. When the metadata management module obtains the target object, it can allocate storage space for the target object in the storage pool 110 or the storage pool 120 in step 702 and obtain the address of the storage space. Then, the target object is stored in the storage space. The address of the storage space becomes the address of the target object. If storage space has been allocated for the target object before step 702, the address of the storage space is directly obtained in step 702. The address of the storage space is the address of the target object.

[0093] The metadata management module stores the metadata of the target object in the storage pool 110 , as follows.

[0094] In one example, the metadata management module can calculate the mapping value corresponding to the target object based on the identifier of the target object. Then, in step 703, based on the mapping value corresponding to the target object, the storage area associated with the mapping value is located in the data structure stored in the storage pool 110. Then, in step 704, a free storage unit is queried from the storage area associated with the mapping value. Exemplarily, the computing node 210 can use a read primitive to pull a storage area associated with the mapping value corresponding to the target object to the computing node 210 locally. Then, the free storage unit is queried in the storage area pulled to the computing node 210 locally. Here, a free storage unit refers to a storage unit that does not yet store the metadata of the object. If there is no free storage unit in the storage area, another storage area associated with the mapping value corresponding to the target object in the storage pool 110 is pulled locally and a free storage unit is queried. And so on and so forth until a free storage unit is queried.

[0095] After finding a free storage unit, the metadata management module stores the metadata of the target object in the free storage unit in the storage pool 110 in step 705 .

[0096] In one example, the storage unit includes an atomic area, a check area, and a statistical area. The metadata management module writes the target object's identification fingerprint, address, and lock information into the atomic area. The lock information records that an exclusive lock has been applied to the target object, prohibiting other computing nodes from modifying the target object.

[0097] If writing the relevant information to the atomic area fails, the metadata management module re-executes step 704. If writing the relevant information to the atomic area succeeds, the metadata management module writes the object ID, the object's cache status, and the object's size to the check area. Upon completing writing the relevant information to the check area, the metadata management module may change the lock state of the target object from an exclusive lock to a shared lock, allowing other computing nodes to access the target object.

[0098] The metadata management module can record the writing time of the target object, the last access time of the target object and the number of accesses to the target object in the statistical area.

[0099] In the above manner, the target object can be stored in the storage pool 110 or the storage pool 120 , and the metadata of the target object can be stored in the storage pool 110 .

[0100] Back to Figure 6 In step 602 , the computing node 210 obtains metadata of the target object from the storage pool 110 based on the identifier of the target object; the metadata of the target object is used by the computing node 210 to perform an access operation A1 on the target object.

[0101] The computing node 210 may obtain metadata of the target object from the storage pool 120 through an RDMA operation based on the identifier of the target object. The RDMA operation may be an RDMA atomic operation. The RDMA operation may be an operation based on InfiniBand, RoCE, CXL, or a unified bus technology.

[0102] In some embodiments, as described above, the storage pool 110 stores a data structure, which includes multiple storage areas, wherein each storage area in the multiple storage areas is associated with a mapping value, and the storage area is used to store metadata of the object corresponding to the mapping value associated with the storage area. In step 602, the computing node 210 can obtain the mapping value corresponding to the target object based on the identifier of the target object. For example, according to the mapping algorithm, the identifier of the target object is calculated to obtain the mapping value corresponding to the target object. Then, based on the mapping value of the target object, a storage area associated with the mapping value corresponding to the target object is obtained from the multiple storage areas. For convenience of description, the storage area associated with the mapping value corresponding to the target object is referred to as the target storage area. Then, based on the identifier of the target object, the computing node 210 can obtain the metadata of the target object from the target storage area.

[0103] In one example of this embodiment, the target storage area includes multiple storage units, wherein the multiple storage units correspond one-to-one to multiple objects, and each storage unit in the multiple storage units is used to store metadata of the object corresponding to the storage unit, wherein the multiple objects correspond to the same mapping value.

[0104] In step 602, computing node 210 obtains a storage unit storing metadata of the target object from the multiple storage units in the target storage area based on the identifier of the target object. For ease of description, the storage unit storing metadata of the target object is referred to as the target storage unit. Next, computing node 210 obtains the metadata of the target object from the target storage unit.

[0105] In one example, step 602 may be performed by Figure 8 The method shown is implemented as follows.

[0106] An application on computing node 210 may trigger an access request. The metadata management module in computing node 210 may obtain the access request in step 801. Computing node 210 may obtain the identifier of the target object from the access request and, in step 802, locate the storage area in storage pool 110. Based on the identifier of the target object, computing node 210 may obtain a mapping value corresponding to the target object. Then, based on the mapping value of the target object, the target storage area may be obtained.

[0107] In step 803, computing node 210 may query the target storage unit from the target storage area. The metadata management module of computing node 210 may pull all storage units in the target storage area to the local computer of computing node 210. The identification fingerprint stored in the storage unit is then compared with the last byte of the identification of the target object to see if they are consistent.

[0108] If they are inconsistent, the identification fingerprint stored in the next storage unit is compared with the last byte of the identification of the target object to see if they are consistent.

[0109] If they are consistent, then determine whether the identifier of the object stored in the storage unit is consistent with the identifier of the target object. If they are not consistent, then compare the identifier fingerprint stored in the next storage unit with the last byte of the identifier of the target object to see if they are consistent. If they are consistent, then determine that the storage unit is the target storage unit.

[0110] Comparing the identification fingerprint with the last byte of the target object's identification requires less computing power than comparing the identification stored in the storage unit with the identification of the target object. Comparing the identification fingerprint with the last byte of the target object's identification first, and then comparing the identification stored in the storage unit with the target object's identification if the identification fingerprint and the target object's identification match, can save computing power and improve the efficiency of querying the target storage unit.

[0111] If the target storage unit is not found in the target storage area, the next storage area associated with the mapping value corresponding to the target object is determined as the target storage area. Then, the target storage unit is searched in the newly determined target storage area. This process is repeated until the target storage unit is found.

[0112] After finding the target storage unit, the computing node 210 executes step 804 to obtain metadata of the target object, such as the address of the target object, lock information of the target object, etc., from the target storage unit.

[0113] After acquiring the metadata of the target object, the computing node 210 may perform an access operation A1 on the target object based on the metadata of the target object. The access operation A1 may be an operation such as modification or reading.

[0114] The metadata of the target object includes the address of the target object. The computing node 210 may perform an access operation A1 on the target object based on the address of the target object.

[0115] The metadata of the target object also includes the lock information of the target object. Based on the lock information, the computing node 210 can determine whether the access operation A1 can be performed on the target object at present. The lock information includes the lock status. If the lock status indicates that an exclusive lock is applied to the target object, it means that a computing node other than the computing node 210, such as the computing node 220, is modifying the target object. In this case, the computing node 210 delays the access operation A1 on the target object, and when the exclusive lock of the target object A1 is released, the computing node 210 performs the access operation A1 on the target object again. After the computing node 220 completes the modification of the target object, it can modify the lock status of the target object in the storage pool 110 to release the exclusive lock of the target object.

[0116] If the lock state indicates that a shared lock is applied to the target object, and the access operation A1 is a read operation, the computing node 210 may perform the access operation A1 on the target object. When performing the access operation A1 on the target object, the computing node 210 may increase the lock count of the target object (e.g., by one).

[0117] If the lock state indicates that a shared lock is applied to the target object, and the access operation A1 is a modification operation. The computing node 210 can delay the access operation A1. After the shared lock of the target object is released, the access operation A1 is performed on the target object. The release of the shared lock of the target object indicates that the access of other nodes to the target object has ended, and no other node accesses the target object. In this case, the computing node 210 modifies the lock state of the target object to a state in which an exclusive lock is applied, and performs the access operation A1 on the target object. In the case that a shared lock is applied to the target object, each time a computing node ends the access operation to the target object, the lock count of the target object can be reduced (for example, the lock count is reduced by one). When the lock count of the target object is restored to the default value (for example, zero), it can be determined that the shared lock of the target object has been released.

[0118] In step 603 or step 805 , the computing node 210 updates the metadata of the target object in the storage pool 110 based on the access operation A1 .

[0119] In some embodiments, as described above, if the lock state indicates that a shared lock is applied to the target object, and the access operation A1 is a read operation, the computing node 210 increases the lock count of the target object when performing the access operation A1 on the target object. Additionally, when the computing node 210 completes the access operation A1, the lock count of the target object is decremented.

[0120] If access operation A1 is a modification operation, computing node 210 changes the lock state of the target object to an exclusive lock state and performs access operation A1 on the target object. In addition, when computing node 210 finishes access operation A1, it releases the exclusive lock of the target object.

[0121] In some embodiments, if access operation A1 is a modification operation, computing node 210 may modify the metadata of the target object in storage pool 110 based on the modification operation. For example, if the modification operation inserts data into the target object, computing node 210 may modify the size of the target object based on the amount of the inserted data. For another example, if the modification operation changes the storage location of the target object, computing node 210 may update the address of the target object in storage pool 110 based on the address of the new storage location of the target object.

[0122] In some embodiments, the computing node 210 may update the last access time of the target object in the storage pool 110 based on the time when the computing node 210 performs the access operation A1 on the target object. The computing node 210 performs the access operation A1 on the target object, and the computing node 210 may update the number of accesses to the target object in the storage pool 120. In one example, the metadata management module may cache the write time of the target object and the last access time of the object in the computing node 210. The cache duration may be specified by the application. After the cache duration is reached, the metadata management module updates the time when the access operation A1 is performed as the last access time of the target object to the storage pool 110, and updates the number of accesses to the target object. The clocks of the computing nodes and storage pools in the distributed system are synchronized, so there is no need to lock when updating the last access time of the target object. However, an exclusive lock is applied when updating the number of accesses to the target object.

[0123] In this way, the computing node 210 may update the metadata of the target object in the resource pool 110 based on the access operation of the computing node 210 on the target object.

[0124] Any computing node in the distributed storage system can perform an access operation on the target object based on the metadata of the target object after the above update. For example, after step 603, computing node 220 may need to perform access operation A2 on the target object. Computing node 220 can obtain the metadata of the target object from resource pool 110 based on the identifier of the target object. The metadata of the target object at this time is the metadata updated after step 603. Then, computing node 220 performs access operation A2 on the target object based on the updated metadata. Computing node 220 can also update the metadata of the target object in resource pool 110 based on access operation A2. For details, please refer to the above introduction to steps 601-603, which will not be repeated here.

[0125] In some embodiments, as described above, the metadata of the target object includes: historical access information of multiple computing nodes to the target object. The method also includes: the computing node 210 deletes the target object from the storage pool storing the target object based on the historical access information of the target object. The storage space of the storage pool is limited, and in particular, the storage space of the cache pool is prone to shortage. As the business proceeds, objects are continuously stored in the storage pool. Therefore, it is necessary to delete invalid objects from the storage pool, such as objects that have not been accessed by computing nodes for a long time, objects with low access frequency, etc. In this embodiment, it is possible to determine whether the target object is an object to be deleted based on the access history information of the target object. If the target object is an object to be deleted, the target object can be deleted from the storage pool where it is stored, and the metadata of the target object can be deleted from the storage pool 110 to save storage space.

[0126] The computing node 210 may determine whether the target object is an object to be deleted based on the historical information of the target object and the deletion policy. For example, the deletion policy may be configured or specified by the user.

[0127] In one example, the deletion policy may include identifying objects whose time difference between the most recent access time and the current time is greater than a threshold value Y1 as objects to be deleted. As described above, the object's access history information includes the object's most recent access time. Computing node 210 may determine whether the time difference between the target object's most recent access time and the current time is greater than threshold value Y1. If the time difference is greater than threshold value Y1, computing node 210 determines that the target object is an object to be deleted. Threshold value Y1 may be pre-set, for example, by a user or distributed system administrator based on experience or experimentation.

[0128] In another example, the deletion policy may be to determine an object whose access frequency is less than a threshold value Y2 as an object to be deleted. As described above, the access history information of an object includes the write time of the object and the number of times the object is accessed. The access frequency of the target object can be calculated based on the current time, the write time of the target object, and the number of times the target object is accessed. Specifically, the time difference between the current time and the write time of the target object can be calculated. The number of times the target object is accessed is divided by the time difference to obtain the access frequency of the target object. If the access frequency of the target object is less than the threshold value Y2, the computing node 210 confirms that the target object is an object to be deleted. The threshold value Y2 can be pre-set, for example, set by a user or an administrator of the distributed system based on experience or experiments.

[0129] In some embodiments, computing node 210 can identify a target object as an object to be deleted based on historical access information about the target object. If access information about the target object meets a preset condition within a preset period of time, T1, after the target object is identified as an object to be deleted, computing node 210 deletes the target object from the storage pool storing the target object. In other words, after identifying the target object as an object to be deleted, computing node 210 observes the target object for a period of time to confirm that no computing nodes have accessed the target object before deleting the target object. This prevents the accidental deletion of useful objects.

[0130] In some embodiments, when the number of objects stored in the storage pool where the target object resides exceeds a threshold value Y3, computing node 210 deletes the target object from the storage pool storing the target object based on historical access information of the target object. Exemplarily, threshold value Y3 may be determined by the storage space of the storage pool and the average size of the objects stored in the storage pool.

[0131] In some embodiments, objects are accessed by multiple computing nodes in a distributed system. When the number of objects in the storage pool exceeds the threshold value Y3, when the computing node 210 confirms that an object is an object to be deleted, the object may be being accessed by other computing nodes, but the metadata modifications of the object by other computing nodes are only saved locally and have not yet been updated to the storage pool 110. In addition, the load of the storage pool where the object to be deleted is located is also changing (for example, other objects in the storage pool are deleted by other computing nodes), and the information obtained by the computing node 210 for deleting the object may not be completely accurate. In view of this, the computing node 210 can Figure 9 The method shown is used to delete an object. The details are as follows.

[0132] In step 901, the computing node 210 confirms the storage pool trigger threshold where the target object is located. Exemplarily, the threshold may be threshold Y3, and the storage pool trigger threshold means that the number of objects in the storage pool exceeds threshold Y3. In step 902, the computing node 210 pulls a storage area from the storage pool 110 to the local computing node 210 by random sampling. Then, in step 903, the computing node 210 determines an object to be deleted based on the above-mentioned deletion policy and the metadata of the objects in the storage area. Among them, when the storage pool where the target object is located is a cache pool, objects with a cache status of persistent storage will not be determined as objects to be deleted.

[0133] After determining the object to be deleted, the computing node 210 caches the historical access information of the object to be deleted locally, and in step 904, sets the cache status of the object to be deleted to be deletable. In order to avoid the situation where the computing node needs to access the object again after the object is deleted, the computing node 210 delays the deletion of the object to be deleted. Specifically, Figure 9 As shown, in step 905, it is determined whether a computing node has accessed the object to be deleted within the duration T1. At the end of the duration T1, historical access information of the object to be deleted can be obtained again from the storage pool 110. The historical access information of the object to be deleted obtained from the storage pool 110 at the end of the duration T1 is compared with the historical access information of the object to be deleted locally cached by the computing node 210. If they are inconsistent, it means that a computing node has accessed the object to be deleted within the duration T1. If they are consistent, it means that no computing node has accessed the object to be deleted within the duration T1.

[0134] If a computing node accesses the object to be deleted within the T1 duration, the cache status of the object to be deleted is changed to cacheable in step 906, and step 903 is executed again. If no computing node accesses the object to be deleted within the T1 duration, the computing node executes step 907 to delete the object to be deleted.

[0135] In summary, in the data processing method provided in the embodiments of the present application, the metadata of an object is stored in a storage pool. Whenever any of the multiple computing nodes needs to access a target object, the target object's metadata is obtained from the storage pool, and the target object is accessed based on the metadata. Furthermore, the metadata is updated in the resource pool based on the access to the target object. Thus, a computing node can both access the target object using the metadata and update the metadata, allowing other computing nodes to access the target object using the updated metadata, thus avoiding access conflicts between multiple computing nodes accessing the same object.

[0136] Furthermore, in this method, the metadata is updated by the computing node accessing the target object, thereby avoiding the problem of metadata operation performance degradation that may occur when the metadata server maintains the metadata.

[0137] In addition, when any computing node in the distributed storage system accesses an object, it obtains and maintains the object's metadata in the storage pool, thereby avoiding problems such as access hotspots and decreased metadata operation performance caused by the metadata server maintaining the object's metadata at a single point.

[0138] See Figure 10 , an embodiment of the present application provides a data processing device 1000. The device 1000 can be configured in a first computing node among a plurality of computing nodes included in a distributed system. The distributed system further includes a first storage pool. The first storage pool is set separately from the plurality of computing nodes. The first storage pool is used to store metadata of objects accessed by the plurality of computing nodes. Figure 10 As shown, the device 1000 includes:

[0139] A first acquiring unit 1010 is configured to acquire an identifier of a target object to be accessed; wherein the target object is stored in the first storage pool or a second storage pool other than the first storage pool;

[0140] A second acquiring unit 1020 is configured to acquire metadata of the target object from the first storage pool based on the identifier of the target object; the metadata of the target object is used by the first computing node to perform a first access operation on the target object;

[0141] The updating unit 1030 is configured to update the metadata of the target object in the first storage pool based on the first access operation.

[0142] In some embodiments, the first storage pool stores a data structure, and the data structure includes multiple storage areas, wherein each storage area in the multiple storage areas is associated with a mapping value, and the storage area is used to store metadata of an object corresponding to the mapping value associated with the storage area; the second acquisition unit 1020 is used to: obtain the mapping value corresponding to the target object based on the identifier of the target object; obtain the target storage area associated with the mapping value corresponding to the target object in the multiple storage areas; and obtain the metadata of the target object from the target storage area based on the identifier of the target object.

[0143] In an example of this embodiment, the target storage area includes multiple storage units, wherein the multiple storage units correspond one-to-one to multiple objects, and each storage unit in the multiple storage units is used to store metadata of the object corresponding to the storage unit; the second acquisition unit 1020 is used to: based on the identification of the target object, obtain the target storage unit that stores the metadata of the target object from the multiple storage units; and obtain the metadata of the target object from the target storage unit.

[0144] In some embodiments, the metadata of the target object includes: historical access information of the multiple computing nodes to the target object; the device also includes: a deletion unit for deleting the target object from the storage pool storing the target object based on the historical access information.

[0145] In an example of this embodiment, the deletion unit is used to: confirm that the target object is an object to be deleted based on the historical access information; when the access information of the target object within a preset time period after confirming that the target object is an object to be deleted meets a preset condition, delete the target object from the storage pool storing the target object.

[0146] In some embodiments, the second acquisition unit is configured to: acquire metadata of the target object from the first storage pool through a remote direct data access (RDMA) operation based on an identifier of the target object.

[0147] In some embodiments, the updated metadata of the target object is used by a second computing node among the multiple computing nodes to perform a second access operation on the target object.

[0148] The functions of the functional units of the apparatus 1000 may be implemented with reference to the above description of the operations performed by the computing node 210 .

[0149] This embodiment of the present application provides a data processing device 1100. Figure 11As shown, the data processing device 1100 includes a processor 1110 and a memory 1120. The memory 1120 is used to store executable programs. The processor 1110 is used to execute the executable programs stored in the memory 1120, so that the data processing device 1100 can perform the operations performed by the computing node 210 above, such as Figure 6 The operations performed by the computing node 210 in FIG.

[0150] The present application also provides a computer program product including instructions. The computer program product may be a software or program product including instructions that can be run on a computer device or stored in any available medium. When the computer program product is run on a computer device, the computer device performs the operations performed by the computing node 210 above, such as Figure 6 The operations performed by the computing node 210 in FIG.

[0151] The embodiment of the present application also provides a computer-readable storage medium. The computer-readable storage medium can be any available medium that can be stored by a computer device or a data storage device such as a data center that contains one or more available media. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive). The computer-readable storage medium includes instructions that instruct the computer device to perform the operations performed by the computing node 210 above, such as Figure 6 The operations performed by the computing node 210 in FIG.

[0152] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the protection scope of the technical solutions of the embodiments of the present application.

Claims

1. A data processing method in a distributed system, characterized in that: The distributed system includes a first storage pool and a plurality of computing nodes that are separated from the first storage pool; wherein the first storage pool is used to store metadata of objects accessed by the plurality of computing nodes; and the method includes: A first computing node among the plurality of computing nodes obtains an identifier of a target object to be accessed; wherein the target object is stored in the first storage pool or a second storage pool other than the first storage pool; The first computing node obtains metadata of the target object from the first storage pool based on the identifier of the target object; the metadata of the target object is used by the first computing node to perform a first access operation on the target object; The first computing node updates the metadata of the target object in the first storage pool based on the first access operation.

2. The method according to claim 1, characterized in that The first storage pool stores a data structure, the data structure including a plurality of storage areas, wherein each storage area in the plurality of storage areas is associated with a mapping value, and the storage area is used to store metadata of an object corresponding to the mapping value associated with the storage area; The first computing node obtains metadata of the target object from the first storage pool based on the identifier of the target object, including: The first computing node obtains a mapping value corresponding to the target object based on the identifier of the target object; The first computing node obtains a target storage area associated with the mapping value corresponding to the target object from the multiple storage areas; The first computing node obtains metadata of the target object from the target storage area based on the identifier of the target object.

3. The method according to claim 2, characterized in that The target storage area includes a plurality of storage units, wherein the plurality of storage units correspond to a plurality of objects on a one-to-one basis, and each storage unit in the plurality of storage units is used to store metadata of the object corresponding to the storage unit; The first computing node obtains metadata of the target object from the target storage area based on the identifier of the target object, including: The first computing node obtains, based on the identifier of the target object, a target storage unit storing metadata of the target object from the plurality of storage units; The first computing node obtains metadata of the target object from the target storage unit.

4. The method according to any one of claims 1 to 3, characterized in that The metadata of the target object includes: historical access information of the multiple computing nodes to the target object; the method further includes: The first computing node deletes the target object from a storage pool storing the target object based on the historical access information.

5. The method according to claim 4, characterized in that The first computing node deleting the target object from a storage pool storing the target object based on the historical access information includes: The first computing node confirms, based on the historical access information, that the target object is an object to be deleted; When access information of the target object within a preset time period after the target object is confirmed to be an object to be deleted meets a preset condition, the first computing node deletes the target object from the storage pool storing the target object.

6. The method according to any one of claims 1 to 5, characterized in that The first computing node obtains metadata of the target object from the first storage pool based on the identifier of the target object, including: The first computing node obtains metadata of the target object from the first storage pool through a remote direct data access (RDMA) operation based on the identifier of the target object.

7. The method according to any one of claims 1 to 6, characterized in that The updated metadata of the target object is used by a second computing node among the multiple computing nodes to perform a second access operation on the target object.

8. A data processing device, characterized in that: The distributed system includes a first storage pool and a plurality of computing nodes disposed separately from the first storage pool; wherein the first storage pool is used to store metadata of objects accessed by the plurality of computing nodes, and the apparatus is configured on a first computing node among the plurality of computing nodes; the apparatus includes: A first acquiring unit is configured to acquire an identifier of a target object to be accessed; wherein the target object is stored in the first storage pool or a second storage pool other than the first storage pool; A second acquiring unit is configured to acquire metadata of the target object from the first storage pool based on the identifier of the target object; the metadata of the target object is used by the first computing node to perform a first access operation on the target object; An updating unit is configured to update metadata of the target object in the first storage pool based on the first access operation.

9. The device according to claim 8, characterized in that The first storage pool stores a data structure, the data structure including a plurality of storage areas, wherein each of the plurality of storage areas is associated with a mapping value, and the storage area is used to store metadata of an object corresponding to the mapping value associated with the storage area; the second acquisition unit is used to: Based on the identifier of the target object, obtaining a mapping value corresponding to the target object; Obtaining a target storage area associated with the mapping value corresponding to the target object from the plurality of storage areas; Based on the identifier of the target object, metadata of the target object is acquired from the target storage area.

10. The device according to claim 9, characterized in that The target storage area includes a plurality of storage units, wherein the plurality of storage units correspond to a plurality of objects in a one-to-one manner, and each of the plurality of storage units is used to store metadata of the object corresponding to the storage unit; the second acquisition unit is used to: Based on the identifier of the target object, obtaining a target storage unit storing metadata of the target object from the multiple storage units; The metadata of the target object is obtained from the target storage unit.

11. The device according to any one of claims 8 to 10, characterized in that The metadata of the target object includes: historical access information of the multiple computing nodes to the target object; the device also includes: A deleting unit is configured to delete the target object from a storage pool storing the target object based on the historical access information.

12. The device according to claim 11, characterized in that The deletion unit is used for: Based on the historical access information, confirming that the target object is an object to be deleted; When access information of the target object within a preset time period after the target object is confirmed to be an object to be deleted meets a preset condition, the target object is deleted from the storage pool storing the target object.

13. The device according to any one of claims 8 to 12, characterized in that The second acquiring unit is configured to acquire metadata of the target object from the first storage pool through a remote direct data access (RDMA) operation based on an identifier of the target object.

14. The device according to any one of claims 8 to 13, characterized in that The updated metadata of the target object is used by a second computing node among the multiple computing nodes to perform a second access operation on the target object.

15. A data processing device, characterized in that: include: a memory for storing executable programs; A processor, configured to execute the method according to any one of claims 1 to 7 by running the executable program.

16. A computer-readable storage medium, characterized in that The method comprises computer program instructions, and when the computer program instructions are executed by a computer device, the computer device performs the method according to any one of claims 1 to 7.

17. A computer program product comprising instructions, characterized in that When the instructions are executed by a computer device, the computer device is caused to perform the method according to any one of claims 1 to 7.

Images