Metadata acquisition method, network device and system

Abstract

This application provides a metadata acquisition method applied to a network device used to connect a client and a server. The server caches metadata information of some application data in its memory. The method includes the following steps: receiving an access request for target metadata sent by the client; acquiring metadata information at the location specified in the access request in memory; comparing the acquired metadata information with the target metadata information to determine whether the target metadata is stored in memory; when the target metadata is stored in memory, acquiring the target metadata and transmitting the target metadata data to the client; when the target metadata is not found in memory, the network card will continue to request the server's processor to query the target metadata without notifying the client, which then sends another query request to the server's processor, thereby reducing the client's load, saving network bandwidth, and reducing metadata query latency.

Description

Technical Field

[0001] This application relates to the field of communications, and in particular to a method for obtaining metadata, a network device, and a system. Background Technology

[0002] With the continuous development of science and technology, the massive amounts of data generated in the information explosion era have permeated every industry and business function. Distributed storage systems are widely used due to their high availability, high reliability, and high access efficiency. In a distributed storage system, data is distributed and stored across multiple servers. The address information of the data is written into the data's metadata. Therefore, each time a user uses a client to read data from a server, the client must first obtain the data's metadata from the server, then determine the data's address information based on the metadata, and retrieve the data from that address.

[0003] However, metadata stored on the server typically has a two-layer storage structure. The first layer is a cache layer, where the client can send a one-sided operation request to the server to read metadata stored in the cache layer. The second layer is a full index layer, where the client can send a two-sided operation request to the server to read metadata stored in the full index layer. Based on this, currently, when a client reads metadata from the server, it usually first sends a one-sided operation request to the server to access the target metadata. After receiving the metadata obtained from the server in response to the one-sided operation request, the client determines whether the received metadata is the target metadata. If it is not the target metadata, the client then sends a two-sided operation request to the server to access the target metadata, thereby obtaining the target metadata. This method requires the client to not only determine whether the received metadata is the target metadata, but also to send multiple access requests to the server, resulting in high communication overhead between the client and the server, high processing pressure on the client, and low metadata reading efficiency. Summary of the Invention

[0004] This application provides a metadata acquisition method, network device, and system that can solve the problems of high client processing pressure, high communication overhead between clients, and low metadata reading efficiency during the metadata reading process.

[0005] Firstly, a method for obtaining metadata is provided. This method can be applied to a network device used to connect a client and a server. The client runs an application, and the server caches metadata information of some application data in its memory. The method may include the following steps: the network device receives an access request for target metadata sent by the client, wherein the access request carries information about the target metadata and a location specified in the access request; the network device obtains the metadata information at the location specified in the access request in memory; then compares the obtained metadata information with the information of the target metadata to determine whether the target metadata is stored in memory; when the target metadata is stored in memory, the target metadata is obtained and the target metadata data is transmitted to the client.

[0006] Implementing the method described in the first aspect, the client first sends an access request for the target metadata to the server's network device. The server's network device queries the target metadata in the server's memory. When the target metadata is stored in memory, the network device can read the target metadata and then return it to the client. This eliminates the need for the client to determine whether the received data is the target metadata, thereby reducing the client's processing pressure and improving the efficiency of metadata reading.

[0007] In one possible implementation, the method may further include the following steps: when the target metadata is not stored in memory, the network device sends the target metadata information to the server's processor, so that the server's processor obtains the target metadata based on the target metadata information; the network device receives the target metadata sent by the server's processor and transmits the target metadata data to the client.

[0008] By implementing the above method, when the target metadata is not stored in memory, the server will not return non-target metadata to the client. The client also does not need to confirm whether the received metadata is the target metadata. Instead, it notifies the server's processor of the target metadata access request through the network device. The server's processor then globally queries the target metadata in memory and storage media. This not only reduces the processing pressure on the client but also reduces the number of communications between the client and the server, saves network bandwidth, and reduces the latency of metadata query.

[0009] In one possible implementation, the metadata information cached in memory includes a metadata identifier and metadata, or a metadata identifier and metadata retrieval information, whereby the metadata retrieval information is used to retrieve the metadata. Optionally, the metadata information is stored in memory in the form of key-value pairs, where the metadata identifier is the key in the key-value pair, and the metadata or metadata retrieval information is the value in the key-value pair.

[0010] Specifically, the value in the metadata information can be the metadata itself or the information on how the metadata was obtained. For example, metadata a's key = 1, value = metadata a, or metadata a's key = 1, value = the storage address of metadata a. The meaning of the value in the key-value pair can be determined according to the actual situation, and this application does not impose any specific limitations.

[0011] By implementing the above method, the metadata information cached in memory is stored in the form of key-value pairs. This storage method can be quickly queried using single keys, combination keys, range keys, etc., making the reading of metadata information simpler and faster.

[0012] In one possible implementation, metadata information for a portion of the application's data is cached in the server's memory using a hash table. The hash value obtained by the server performing a hash operation on the data identifier of the metadata indicates the storage location of the metadata information in the hash table. The location specified in the access request is the hash value obtained by the client performing a hash operation on the target metadata, where the hash function used by the client is the same as the hash function used by the server. Specifically, before the client sends the access request for the target metadata to the server, the server can send this hash function to the client. For example, after the client and server establish a connection, the server sends the hash function used to store the metadata information to the client. Furthermore, if the hash function used by the server changes, the server can send the changed hash function to the client.

[0013] In specific implementation, the target metadata information is the target metadata identifier. The step of comparing the acquired metadata information with the target metadata information to determine whether the target metadata is stored in memory may include the following steps: The network device compares the target metadata identifier with the key in the key-value pair to determine whether the target metadata is stored in memory. If the target metadata is stored in memory, the target metadata is acquired and the target metadata data is transmitted to the client. If the target metadata identifier and the key match, then the metadata corresponding to the key is the target metadata, or the metadata acquisition information corresponding to the key is the target metadata acquisition information. The network device acquires the target metadata based on the target metadata acquisition information and transmits the target metadata data to the client.

[0014] By implementing the above method, the server can perform a hash operation on the metadata key to obtain a hash value, and then store the metadata information at the storage location corresponding to the hash value. The hash function used for the hash operation is sent to the client in advance, allowing the client to predict the storage location of the target metadata on the server using the hash function. Thus, when the client initiates an access request for the target metadata, it can include the predicted storage location in the access request. The server reads the metadata information from the location specified in the access request. If the key in the metadata information matches the key in the target metadata, it means that the metadata information is the target metadata information, and the target metadata can be retrieved based on the read metadata information. Otherwise, the server can notify the processor to perform a two-way read operation, so that the client does not need to confirm the received metadata, reducing the client's processing pressure, reducing the number of communications between the client and the server, saving network bandwidth, and reducing the latency of metadata query.

[0015] In one possible implementation, the access request for the target metadata may also carry version information of the target metadata. The metadata information obtained by the server from the location specified in the access request may also include the version number of the metadata. When comparing the target metadata information with the metadata information, the server's network device may first compare the key in the metadata information obtained from the location specified in the access request with the key in the target metadata information. If they match, it will compare the version information in the metadata information obtained from the location specified in the access request with the version information in the target metadata information. If they also match, it will then read the target metadata from memory based on the metadata information. Otherwise, the network device will notify the server processor to perform a two-way operation and read the target metadata based on the key of the target metadata.

[0016] By implementing the above method, access to specific version metadata can be achieved, avoiding the return of one of the target metadata versions of the wrong version, and improving the accuracy of metadata reading.

[0017] Optionally, the access request may also include a data length L, wherein the read length is used for the server to read a data segment of length L from the location specified in the access request.

[0018] Optionally, the above access request may also include a write address, which is the address where the target metadata is written to the client. In this way, after the server's network device obtains the target metadata, it can write the target metadata to the write address through RDMA one-sided operation, so that the writing of the target metadata does not require the participation of the server's CPU, thereby improving the reading efficiency of the target metadata.

[0019] Optionally, the access request may further include a descriptor for the access request. This descriptor is used by the server to determine the meaning of each field in the read request, such as which field is the key, which field is the location specified in the access request, and which field is the data length. For example, the read request may be a message, and the descriptor may be located in the message header. After the server reads the descriptor from the message header, it determines that fields 1-4KB are the descriptor, fields 5-8KB are the key, fields 9-16 are the location specified in the access request, and fields 17-24 are the data length. It should be understood that the above examples are for illustrative purposes only. The access request may also include other information, such as keys or verification codes used to improve data transmission security. This application does not impose specific limitations on this. Furthermore, the descriptor may also be in other forms, such as using newline characters, spaces, colons, etc., as delimiters to split multiple fields in the access request. This application does not impose limitations on this.

[0020] In specific implementations, the aforementioned network devices can be network interface cards (NICs) with remote direct memory access (RDMA) functionality, such as RDMA-enabled NICs (RNICs), or NICs or switches that support InfiniBand (IB) functionality. Therefore, the network devices can implement the above-mentioned solutions provided in this application through programmable chips such as field-programmable gate arrays (FPGAs) and application-specific integrated circuits (ASICs).

[0021] In a second aspect, a computer system is provided, wherein the computer system may include a client and a server, the server including a network device for connecting the client and the server, wherein the client is used to run an application, the server is used to cache metadata information of some data of the application in memory, the network device is used to obtain metadata information from the location specified by the access request of the client to access the target metadata, and to determine whether the target metadata is stored in memory by comparing the metadata information with the information of the target metadata, and if the target metadata is stored in memory, to obtain the target metadata and transmit the target metadata data to the client, wherein the access request of the target metadata includes the information of the target metadata.

[0022] In one possible implementation, the network device is further configured to send the target metadata information to the server's processor when the target metadata is not stored in memory; the server's processor is configured to obtain the target metadata based on the target metadata information; and the network device is further configured to transmit the target metadata obtained by the processor to the client.

[0023] In one possible implementation, the metadata information cached in memory includes the metadata identifier and metadata, or the metadata identifier and metadata retrieval information, wherein the metadata retrieval information is used to retrieve the metadata.

[0024] In one possible implementation, the metadata information is stored in memory in the form of key-value pairs, wherein the identifier of the metadata is the key in the key-value pair, and the metadata or the information for obtaining the metadata is the value in the key-value pair.

[0025] In one possible implementation, metadata information of some application data is cached in the server's memory using a hash table. The hash value obtained by the server performing a hash operation on the data identifier of the metadata is the storage location of the metadata information in the hash table.

[0026] In one possible implementation, the location specified in the access request is the hash value obtained by the client performing a hash operation on the target metadata, wherein the hash function used by the client to perform the hash operation is the same as the hash function used by the server.

[0027] In one possible implementation, the target metadata information includes the identifier of the target metadata. The network device compares the identifier of the target metadata with the key in the key-value pair. When the identifier of the target metadata matches the key, the metadata corresponding to the key is the target metadata or the metadata retrieval information corresponding to the key is the target metadata retrieval information. The network device retrieves the target metadata based on the target metadata retrieval information.

[0028] Thirdly, a network device is provided for use in a computer system, the computer system including a client and a server, the network device for connecting the client and the server, the client running an application, and the server caching metadata information of some application data in its memory. The network device includes: a receiving unit for receiving an access request for target metadata sent by the client, the access request carrying information of the target metadata and a location specified in the access request; an information acquisition unit for acquiring metadata information at the location specified in the access request in memory; a determining unit for comparing the acquired metadata information with the information of the target metadata to determine whether the target metadata is stored in memory; and a metadata acquisition unit for acquiring the target metadata and transmitting the target metadata data to the client when the target metadata is stored in memory.

[0029] In one possible implementation, the network device also includes a transmitting unit.

[0030] The sending unit is used to send the target metadata information to the server's processor when the target metadata is not stored in memory, so that the processor can obtain the target metadata based on the target metadata information and transmit the target metadata data to the client.

[0031] In one possible implementation, the metadata information cached in memory includes the metadata identifier and metadata, or the metadata identifier and metadata retrieval information, wherein the metadata retrieval information is used to retrieve the metadata.

[0032] In one possible implementation, metadata information is stored in memory in the form of key-value pairs, where the identifier of the metadata is the key in the key-value pair, and the metadata or metadata retrieval information is the value in the key-value pair.

[0033] In one possible implementation, metadata information of some application data is cached in the server's memory using a hash table. The hash value obtained by the server performing a hash operation on the data identifier of the metadata is the storage location of the metadata information in the hash table.

[0034] In one possible implementation, the location specified in the access request is the hash value obtained by the client performing a hash operation on the target metadata, wherein the hash function used by the client to perform the hash operation is the same as the hash function used by the server.

[0035] In one possible implementation, the target metadata information includes an identifier for the target metadata; a determining unit, used to compare the identifier of the target metadata with the key in the key-value pair to determine whether the target metadata is stored in memory; a metadata acquisition unit, used to acquire the target metadata based on the acquisition information of the target metadata when the identifier of the target metadata matches the key, and transmit the target metadata data to the client; and a sending unit, used to send the target metadata information to the server's processor when the identifier of the target metadata does not match the key, so that the processor can acquire the target metadata based on the target metadata information and transmit the target metadata data to the client.

[0036] Fourthly, a network device is provided, including a processor and a communication interface, wherein the communication interface is used to receive an access request from a client to access target metadata, and the processor is used to execute the method described in the first aspect to process the access request.

[0037] Fifthly, a computer program product is provided, including a computer program that, when read and executed by a computing device, implements the method described in the first aspect.

[0038] In a sixth aspect, a computer-readable storage medium is provided, including instructions that, when executed on a computing device, cause the computing device to implement the method described in the first aspect. Attached Figure Description

[0039] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below.

[0040] Figure 1 It is a schematic diagram of a computer system architecture;

[0041] Figure 2 It is a schematic diagram of the hardware structure of a computer system;

[0042] Figure 3 This is a schematic diagram of the data structure of metadata;

[0043] Figure 4 This is a flowchart illustrating the steps of a metadata acquisition method provided in this application;

[0044] Figure 5 This is a flowchart illustrating another data acquisition method provided in this application;

[0045] Figure 6 This is a schematic diagram of the structure of a network device provided in this application;

[0046] Figure 7 This is a schematic diagram of the hardware structure of a network device provided in this application. Detailed Implementation

[0047] To facilitate understanding of the technical solutions of this invention, some of the terms involved in this invention will first be explained. It should be understood that the terminology used in the embodiments section of this application is only used to explain specific embodiments of this application and is not intended to limit this application.

[0048] One-sided operation: One-sided operation typically refers to a scenario where the receiving CPU does not participate in the data reading operation; instead, the receiving network card reads the data from memory. Specifically, in one-sided operation, after the sending side specifies the source and destination addresses of the data, the receiving CPU does not need to be aware of this communication. Data reading and writing are both completed through the network cards of the sending and receiving sides. This scenario is typically suitable for batch data transmission. For example, taking data reading as an example, after client A and server B establish a connection, server B stores data X in a memory space and sends the address information VB of that memory space to client A. When client A reads data X from server B, client A can encapsulate the local address VA (destination address) used to store data X, along with the aforementioned address information VB (source address), in a one-sided read request and send this request to server B's network card. Server B's network card responds to this one-sided read request, writing data X from VB into VA. The entire process does not require the participation of either party's CPU. Therefore, one-sided operation can reduce the CPU utilization of both the client and server sides, improve communication efficiency, and achieve high bandwidth and low latency communication. For example, the one-sided operation could be an RDMA one-sided operation.

[0049] Bilateral operation: Bilateral operation typically refers to data reading and writing performed by the receiving CPU, such as RDMA bilateral operation. Specifically, in bilateral operation, the receiving CPU needs to be aware of the bilateral operation to complete the transmission and reception. This scenario is typically applicable to the transmission and reception of control messages. Upon receiving a control message, the receiving CPU executes the corresponding operation steps and returns a completion message to the sending CPU. Although both CPUs participate in the entire process, it does not require traversing multiple layers of network protocol stacks, achieving zero-copy data transmission across nodes.

[0050] Key-value pairs (key, value): Key-value pairs are strings. In the storage field, metadata can be stored as key-value pairs (K, V), where K represents the key, which can be the name or identifier of the metadata, and V represents the value corresponding to that key. The value can be defined according to business needs; for example, the value can be the metadata itself, the location information of the metadata, or the storage location of the metadata's location information in a hash table. Each key corresponds to at least one value. For example, K = the name of metadata x, and V = the storage address of metadata x. This storage method allows for simple and fast retrieval of the data required by the business through single-key queries, composite key queries, and range key queries.

[0051] To facilitate understanding of the embodiments of this application, the application scenario "metadata reading" involved in this application will be briefly described first.

[0052] The modern internet has entered the era of big data, with massive amounts of data generated daily. Traditional network computer systems use centralized storage servers to store all data, making the performance of these servers a bottleneck and unable to meet the needs of large-scale storage applications. Therefore, distributed storage systems have emerged. Distributed storage systems distribute data across multiple independent devices, using multiple storage servers to share the storage load and metadata servers to locate the data's position. Compared to traditional network storage systems, distributed storage systems not only improve system reliability, availability, and access efficiency but are also easier to expand.

[0053] Figure 1 This is a schematic diagram of the architecture of a computer system 100, such as... Figure 1 As shown, computer system 100 may include client 100 and server 200, wherein client 100 and server 200 are connected by network 300, which is a network suitable for RDMA technology, such as Ethernet. The number of clients 100 and servers 200 can be one or more. It should be understood that... Figure 1 The example described uses two clients (client 100a and client 100b) and two servers (server 200a and server 200b), but this application does not limit the number of clients 100 and servers 200.

[0054] The client 100 provides an interactive interface to the user and, based on various user operations within the interface, sends data read requests to the server 200 via the RDMA network 300. The client 100 can be a web browser or a locally running application; this application does not impose any specific limitations. Furthermore, the client 100 is highly flexible in its deployment. It can be deployed on a separate physical server, such as an x86 server or a bare metal server (BMS); it can also be deployed on a terminal, such as a smartphone or tablet; or it can be deployed on a virtual machine (VM) based on a general-purpose physical server combined with network functions virtualization (NFV) technology. A VM refers to a complete computer system simulated by software, possessing full hardware system functionality and running in a completely isolated environment, such as a VM in a cloud data center. The client and server can also be deployed on the same physical server; this application does not impose any specific limitations.

[0055] Server 200 is a cluster composed of multiple storage devices. These storage devices can be used to store data or data metadata. This application does not limit the content of the data stored on server 200. Metadata here refers to data about data, primarily describing the data's properties to support functions such as indicating storage location, historical data, resource lookup, and file records. Specifically, data metadata may include: access permissions, file owner, and data location information. Location information may include the address of the data on server 200, the partition of the storage medium, the starting position on the storage medium, the data length, and the data's index information, etc. In short, if a user needs to read data from server 200, the user must first obtain the data's metadata to locate its location and then retrieve the data from that location.

[0056] In specific implementations, the server 200 can be a physical device or a virtual device. When the server 200 is a physical device, it can have a processor, hard disk, memory, network card, and system bus, etc. When the server 200 is a virtual device, it can simulate a computing device with complete hardware system functions running in a completely isolated environment through software. For example, the server 200 can be a virtual machine in a cloud data center, running an application on the computing device, simulating a standard x86-based PC environment. This environment, like a real computing device, has one or more of the following devices: chipset, CPU, memory, graphics card, sound card, network card, floppy drive, hard disk, optical drive, serial port, parallel port, USB controller, small computer system interface (SCSI) controller, etc. In some application scenarios, the server 200 can also exist in other forms. For example, in enterprise data storage scenarios, the server 200 can exist in the form of a storage array or a hard disk enclosure; this application does not make specific limitations.

[0057] It is worth noting that each server may store only data, only metadata, or both. The data and its metadata may reside on the same server or on different servers; this application does not impose any limitations on this. For example, such as... Figure 1As shown, server 200a stores data b1, data b2, and data b3, as well as metadata a. Server 200b stores data a, data b4, and data c, as well as metadata b. Metadata a is the metadata of data a, describing the storage location of data a. Metadata b is the metadata of data b1 to data b5, describing the storage location of data b1 to data b5 (assuming data b is fragmented into data b1 to data b5).

[0058] exist Figure 1 In the computer system shown, the process of a user reading data A through client 100a can be divided into the following two steps: Step 1: Client 100a sends a request to the server to read the target metadata a and obtains the metadata a; Step 2: Based on the metadata a, client 100a determines that the server where data A is located is 200c, sends a request to the server 200c to read data A and obtains data A.

[0059] It should be understood that in step 1, since the server 200 includes at least one storage device, when the client 100 sends a read request for target metadata to the server 200, one of the storage devices in the server 200 can receive the read request. Then, based on the key of the target metadata in the read request, it determines whether the target metadata exists on the local machine. If it exists on the local machine, the read request is processed; if it does not exist on the local machine, the read request is forwarded to the target storage device where the target metadata is located for processing. The storage device that first receives the read request can be a default storage device, the storage device geographically closest to the client 100, or a random storage device; this application does not impose any specific limitations.

[0060] Storage devices can identify the target storage device storing the target metadata in various ways. Optionally, the storage device can determine the target storage device by taking the modulo between the key and the total number of storage devices. The modulo can be the remainder obtained by dividing two numbers. For example, if there are 3 storage devices with instance IDs 0, 1, and 2, and the key of metadata A0 is 90, then when a storage device receives a write request for metadata, metadata A0 can be stored on the storage device with instance ID 0. If the key of metadata A1 is 88, then data A1 can be stored on the storage device with instance ID 1. If the key of metadata A1 is 86, then data A1 can be stored on the storage device with instance ID 2, and so on, storing the metadata. Similarly, when a storage device receives a read request for the target metadata, it can take the modulo of the key of the target metadata in the read request to obtain the instance ID of the storage device. Each storage device can store the network address corresponding to each instance ID, thereby identifying the network address of the target storage device storing the target metadata, and forwarding the read request to that network address for reading the metadata.

[0061] Optionally, the storage device can also determine the target storage device for storing the target metadata by reading the index. The indexes between storage devices will be synchronized periodically. Each storage device maintains one index, which may contain multiple routing information. Each routing information includes a key and the instance ID of the storage device where the key resides. Specifically, when metadata a (key k0) is written to storage device Y, storage device Y can create a route A' in its maintained index. This route A' records that the instance ID corresponding to "key = k0" is "0". Then, storage device Y synchronizes this route A' to the index maintained by each storage device. In this way, when any storage device receives an access request carrying key k0, the storage device can obtain the instance ID corresponding to "key = k0" as "0" based on the information recorded in route A' in the index. That is, the storage device storing metadata a (key k0) has an instance ID of 0. Then, based on the network address corresponding to each instance ID, the network address of the target storage device storing the target metadata is determined, and the read request is forwarded to that network address for the storage device to read the metadata.

[0062] It should be noted that the two methods for determining the storage device storing the target metadata described above are only for illustrative purposes. This application does not limit the specific way in which the storage device determines the target storage device storing the target metadata.

[0063] In summary, each time a user uses a client to read data A from the server, the client first needs to obtain the metadata of data A from the server, then determine the storage location of data A based on the metadata, and finally retrieve data A from that storage location. The solution provided in this application is applicable to the scenario where the client reads metadata (step 1). In this scenario, the solution provided in this application can improve the reading speed of metadata, thereby improving the efficiency of data reading in the distributed storage system and enhancing the user experience.

[0064] Figure 2 This is a schematic diagram of the hardware structure of the aforementioned computer system 100. For example, the client 100 may include a processor 120, a memory 110, and a network device 130, while the server 200 may include a processor 220, memory 230, storage medium 240, and a network device 210. It should be understood that the unit modules of the client 100 and server 200 can be divided in various ways; for example, the client 100 and server 200 may also include a communication module, expansion interfaces, etc. Figure 2 This is merely an exemplary division method, and this application does not impose any specific limitations. Furthermore, Figure 2 The positional relationships between the devices and modules shown do not constitute any limitation; for example, in Figure 2 In the server 200a shown, the storage medium is located outside the server 200a. In other cases, the storage medium may also be the internal memory of the server. This application does not make specific limitations.

[0065] The processor 120 of the client 100 and the processor 220 of the server 200 can be implemented by an application-specific integrated circuit (ASIC) or a programmable logic device (PLD). The PLD can be a complex programmable logical device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL), or any combination thereof.

[0066] The memory 110 of client 100 may include volatile memory, such as random access memory (RAM); the memory 1030 may also include non-volatile memory, such as read-only memory (ROM), flash memory, hard disk drive (HDD) or solid-state drive (SSD), etc., and may also include combinations of the above types. This application does not make specific limitations.

[0067] The memory 230 of the server 200 can be volatile memory, random access memory (RAM), dynamic random access memory (DRAM), static random access memory (SRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR), read-only memory (ROM), cache, etc., and may also include combinations of the above types. This application does not make specific limitations.

[0068] The storage medium 240 of the server 200 can be any medium used for storing data, such as floppy disks, optical discs, solid-state drives (SSDs), flash memory, storage media, memory sticks, etc., and this application does not make a specific limitation. The storage medium 240 is used for persistent storage of data in the memory 230 of the server 200. It is worth noting that the number of storage media of the server 200 can be one or more, and this application does not make a specific limitation.

[0069] The network device 130 of client 100 and the network device 210 of server 200 are network interface cards (NICs) that support RDMA functionality, such as RNICs, or they can be IB NICs or switches; this application does not specifically limit them. The network devices are used to connect client 100 and server 200, enabling communication between them. Specifically, when client 100 reads metadata A stored in server 200, client 100's network device 130 can send an access request for metadata A to server 200's network device 210. Server 200's network device 210 can then read metadata A based on this access request and return it to client 100's network device 130.

[0070] The following explains the data structure of the metadata stored in server 200.

[0071] like Figure 3 As shown, the metadata in the server 200 is usually split into two data structures. The first layer is the cache layer. The client can quickly read the metadata of the cache layer data structure through one-sided operations. The cache layer usually uses a hash method to store the metadata in memory as a hash table in the form of key-value pairs. The hash method can store the metadata in memory according to the set hash function and the collision handling method. Specifically, when the server stores the metadata of the cache layer data structure, it can first use a hash function to perform a hash operation on the key of the metadata to obtain a hash value V, and then store the metadata according to the hash value V. For example, the hash value V can be used as the storage address of the metadata, and the metadata and key-value pairs can be stored. Alternatively, the hash value V can be the metadata itself, or a mapping can be made between the hash value V and the server's memory address, and the metadata and key-value pairs can be stored in the memory address ADD0 corresponding to the hash value V. If other data has already been written to the hash value V of the metadata, then a conflict handling method can be used to process the other data and metadata, such as writing the other data to the storage medium for persistent storage, while writing the metadata and key-value pairs to the hash value V.

[0072] For example, such as Figure 3 As shown, when storing metadata 1 with key K1, the server can first use the hash function f(x) to determine the value V1 = f(K1) corresponding to key K1, and obtain the key-value pair (K1, f(K1)) of metadata 1. Assuming that there is a mapping relationship between multiple hash addresses and a continuous set or range of memory addresses, such as hash value 1 corresponding to memory address ADD1, hash value 2 corresponding to memory address ADD2, and so on, the server can obtain the memory address ADDf(K1) corresponding to hash value f(K1). The server can store the key-value pair of metadata 1 and metadata 1 together at memory address ADDf(K1). Simultaneously, if metadata 2 is already stored at memory address ADDf(K1), and the key of metadata 2 is the same as the key of metadata 1 (K1), it indicates that metadata 1 is a modified version of metadata 2, and metadata 2 can overwrite metadata 1. If the key of metadata 2 is K2, which is different from the key K1 of metadata 1, then metadata 2 already written to memory address ADDf(K1) can be written to storage medium 240 for persistent storage, and metadata 1 and its key-value pair can be written to memory address ADDf(K1). It should be understood that the above examples are for illustration, and the methods for handling conflicts can also be chaining, open addressing, etc. This application does not limit the methods for handling conflicts.

[0073] Similarly, when the server reads the target metadata of the cache layer data structure, the server's network interface card (NIC) can first obtain the key-value pair of the target metadata based on the key K, then retrieve the metadata from the server's memory based on the hash value V in the key-value pair, and return it to the client. Therefore, when querying the metadata of the cache layer data structure, the server's CPU does not need to participate in this query process, and the client can quickly read the metadata of the cache layer data structure through a one-way operation, resulting in a faster reading speed.

[0074] The second layer is the full index layer, where clients can read metadata from the full index layer's data structure via bilateral operations. The full index layer stores metadata using a search tree data structure, such as a log-structured merge-tree (LSM-tree). Specifically, in the full index layer, metadata is first written to the tree in memory. When the metadata stored in the tree in memory reaches a certain threshold, the tree in memory is flushed to the storage medium for persistent storage. The released tree in memory will continue to be used for writing new metadata. Simultaneously, the tree in the storage medium can also be merged periodically to generate new trees, such as... Figure 3 As shown, metadata can first be written to the C0 tree in memory, then flushed to the C1 tree in the storage medium, and then merged into the C2 tree, and so on. Therefore, when reading metadata 'a' from the full index layer structure, the server's CPU can first search the tree in memory, and then sequentially search the trees in the storage medium, for example, from the C0 tree to the C2 tree, until the metadata 'a' is obtained. It should be understood that when querying metadata from the full index layer data structure, the server's CPU needs to participate in the search of multiple trees. Therefore, the client 100 reads the metadata from the full index layer data structure through bilateral operations. Typically, frequently accessed metadata (also known as hot data) will be stored using the cache layer data structure, while less frequently accessed metadata (also known as cold data) will be stored using the full index layer data structure. Specifically, the server can use some memory management algorithms, such as the least recently used (LRU) algorithm, to distinguish between frequently accessed and less frequently accessed metadata. This application does not limit this.

[0075] Therefore, when client 100 sends an access request for target metadata 'a' to server 200 using the aforementioned two-layer data structure, due to the faster metadata reading speed of the cache layer, client 100 typically assumes that target metadata 'a' is stored using the cache layer data structure and sends a one-sided read request to server 200. Server 200 responds to this one-sided read request by executing the metadata reading process of the cache layer data structure through server 200's network interface card and returning the read data to client 100. This data is then used for reference... Figure 3 As can be seen from the example, the target metadata may be stored in memory or in a storage medium. If the target metadata is stored in a storage medium, then the data read by the network card of the server 200 in response to the one-sided read request is not the target metadata. In other words, the one-sided read will fail. Therefore, the client 100 will determine whether the received data is the target metadata a. If not, the client 100 will send a two-sided read request to the server 200. The server 200 will execute the two-sided read process, that is, the CPU will sequentially traverse and search the tree in memory and storage medium until the metadata is obtained.

[0076] In summary, in the current distributed storage system, when client 100 queries metadata from server 200, it first initiates a one-sided operation to search for metadata in the metadata of hot data cached in the server's memory. If the metadata is not found in memory, the client needs to initiate a two-sided operation, whereby the server's CPU searches for metadata in the global metadata. That is, if the metadata is not found in memory, the client needs to initiate another query request to the server, which increases the client's load, network bandwidth, and metadata query latency.

[0077] To address the aforementioned problems in current distributed storage systems, this application provides a metadata acquisition method, which is applied to, for example... Figure 2 In the computer system shown, in this method, the client first sends an access request for the target metadata to the server's network device. The server's network device queries the target metadata in the server's memory. If the target metadata is not found in memory, the network device can send an access request for the target metadata to the server's processor. The server's processor then searches for the target metadata in the global metadata. In this way, when the target metadata is not found in memory, the network device will continue to request the server's processor to query the target metadata without notifying the client to send another query request to the server's processor. This reduces the client's load, saves network bandwidth, and reduces the metadata query latency.

[0078] like Figure 4 As shown, the method may include the following steps:

[0079] Step S210: The client's network device sends an access request for the target metadata to the server's network device. This access request carries information about the target metadata.

[0080] In a specific implementation, the information of the target metadata may include the identifier of the target metadata and the location specified in the access request. The identifier of the target metadata may be the key of the target metadata, and the location specified in the access request is the location of the key-value pair of the target metadata calculated by the client in the server's memory.

[0081] In one embodiment, the location specified in the access request can be obtained by the client performing a hash operation on the key of the target metadata using a hash function f(x). This is the location where the client predicts the storage location of the key-value pairs of the target metadata on the server. (Refer to...) Figure 3 As can be seen from the example, if the server has already stored the target metadata in memory using a hash function f(x), then the location specified by the access request is the storage location of the key-value pairs of the target metadata. If the target metadata is not stored in memory, then the location specified by the access request is not the storage location of the key-value pairs of the target metadata. The hash function is sent by the server to the client before step S210.

[0082] Specifically, the location specified in the access request can be a hash value obtained by the client after performing a hash operation on the key of the target metadata using a hash function, or it can be a memory address determined based on the hash value. For example, if the identifier of the target metadata is K=3 and the hash function f(K)=2K+1, then the location specified in the access request could be the hash value V=7. If the server sends the mapping relationship between memory addresses and hash values ​​to the client in advance, the client can determine the memory address ADD7 corresponding to the hash value V=7 based on this mapping relationship. The location specified in the access request could also be the storage address ADD7 where the key-value pair of the target metadata is located. It should be understood that the above examples are for illustration only, and this application does not limit the specific form of the location specified in the above access request.

[0083] In one embodiment, the hash function can be sent by the server to the client after the client and server establish a connection. It should be understood that when the server caches metadata information of some data of the application in memory, if the hash function used for the key-value pairs in the metadata information changes, the server can send the changed hash function to the client so that the client can calculate the location specified by the access request based on the changed hash function.

[0084] In one embodiment, the key can be determined based on the target metadata accessed by the client. Specifically, the client can store the correspondence between metadata and keys in a database. For example, metadata a corresponds to key = 11100, metadata b corresponds to key = 11101, and so on. Thus, when the client sends an access request to the server for metadata a, the identifier of the target metadata carried in the access request can be determined based on the above correspondence as key = 11100, and the location specified in the access request is the hash value V corresponding to key 11100 obtained according to the hash function f(K). It should be understood that the above examples are for illustrative purposes only and are not intended to limit the scope of the application.

[0085] In one embodiment, the access request for the target metadata may further include other information. Optionally, the access request may further include a data length L, wherein the read length is used for the server to read metadata information of length L from the location specified in the access request. Optionally, the access request may further include a write address, which is the address where the target metadata is written to the client. In this way, after the server's network device obtains the target metadata, it can write the target metadata to the write address through RDMA one-sided operation, so that the writing of the target metadata does not require the participation of the server's CPU, thereby improving the reading efficiency of the target metadata. Optionally, the access request may further include a descriptor for the access request. This descriptor is used by the server to determine the meaning of each field in the read request, such as which field is the key, which field is the location specified in the access request, and which field is the data length. For example, the read request may be a message, and the descriptor may be located in the message header. After the server reads the descriptor from the message header, it determines that fields 1-4KB are the descriptor, fields 5-8KB are the key, fields 9-16 are the location specified in the access request, and fields 17-24 are the data length. It should be understood that the above examples are for illustrative purposes only. The access request may also include other information, such as keys or verification codes used to improve data transmission security. This application does not impose specific limitations on this. Furthermore, the descriptor may also be in other forms, such as using newline characters, spaces, colons, etc., as delimiters to split multiple fields in the access request. This application does not impose limitations on this.

[0086] S220: The server's network device retrieves metadata information at the location specified in the access request.

[0087] The server caches metadata information in its memory. This metadata information includes the metadata identifier and the metadata itself, or the metadata identifier and metadata retrieval information. The metadata identifier refers to the key, and the metadata retrieval information is used by the server to retrieve the metadata; specifically, it can be the value V in the metadata key-value pair. It should be understood that since the location specified in the access request is the storage location of the target metadata predicted by the client using a hash function, if the target metadata is stored in the server's memory, the metadata information read by the server from that location is the target metadata information. If the target metadata is not stored in the server's memory, the metadata information read by the server from that location is not the target metadata information.

[0088] As mentioned above, when the server stores metadata in memory, it first uses a hash function to hash the metadata key to obtain the hash value V corresponding to the key. Then, it stores the metadata based on the hash value V. For example, the hash value might be the memory address where the metadata is written to the server, or there might be a mapping relationship between the hash value and the memory address where the metadata is written to the server. Therefore, if the target metadata has already been stored in the server's memory, the client uses the same hash function to hash the target metadata key K0 to obtain the hash value V0, then encapsulates it in the target metadata access request and sends it to the server. The server can then retrieve the target metadata based on the hash value V0; that is, the metadata information obtained in step S220 is the target metadata information. However, referring to... Figure 3 As can be seen from the embodiments, regardless of whether the target metadata is a cache layer data structure or a full index layer data structure, the target metadata may be stored in the server's memory or in the server's storage medium. Therefore, if the target metadata is not stored in the server's memory but in the server's storage medium, the metadata information obtained in step S220 will not be the target metadata information. In this case, the server's network device cannot read the target metadata through a one-sided operation, and the server's network device can notify the server's CPU to read the target metadata through a two-sided operation.

[0089] S230: The server-side network device compares the acquired metadata information with the target metadata information to determine whether the target metadata is stored in memory.

[0090] refer to Figure 3As illustrated in the example, the metadata of the cache layer data structure is the metadata of hot data. When the server writes the metadata into memory, if the memory address corresponding to the hash value obtained by using a hash function to calculate the key of the metadata already contains old data with the same key, then that old data will be written to the storage medium and stored using the full index layer data structure. Therefore, the location information obtained by the client using a hash function to calculate the key K of the target metadata may not be stored at that location by the server. Therefore, after the server's network device obtains the metadata information from the location indicated by this location information, it can first compare the target metadata information with the obtained metadata information. If they match, it means that the metadata information read by the network device is the target metadata information requested by the client, and this metadata is stored in memory and can be read through a one-sided operation. If they do not match, it means that the target metadata is stored in the storage medium, and the server's network device cannot read it through a one-sided operation. The server's CPU can then be notified to perform a two-sided operation to obtain the target metadata.

[0091] In specific implementation, the server-side network device can compare the key K in the metadata information obtained from the location specified in the access request with the key K in the target metadata information. By comparing whether the two are consistent, it determines whether the metadata can be read through a one-sided operation. If they are consistent, it means that the target metadata is stored in memory, and step S240 can be executed to read the target metadata through a one-sided operation. If they are inconsistent, it means that the target metadata is stored in the storage medium, and step S250 can be executed to notify the CPU to perform a two-sided operation to read the target metadata. In this way, the client only needs to send an access request to the server once, and the server determines on its own whether this metadata reading needs to be changed from a one-sided operation to a two-sided operation, without notifying the client. This eliminates the need for the client to send multiple access requests to the server, improving communication efficiency, saving network bandwidth, reducing client processing pressure, and reducing metadata query latency.

[0092] Taking the example in step S210 as an example, assuming that the identifier of the target metadata carried in the access request is K=3, and the location specified by the access request calculated by the client is the memory address ADD7 corresponding to the hash value V=7, the communication device of the server can read a key-value pair (K=3, V=f(3)) from the memory address ADD7. By comparing the identifier of the target metadata with the key in the key-value pair obtained from the specified location, both are 3, and they are consistent. Therefore, the key-value pair read from ADD7 is the key-value pair of the target metadata, and the server can obtain the target metadata according to the value V=f(3) in the key-value pair. Similarly, if the key-value pair read from ADD7 is (K=5, V=f(5)), the identifier of the target metadata is K=3, and the key of the metadata read from the specified location is K=5, and they are inconsistent, it means that the information of the metadata read from the specified location is not the information of the target metadata. This one-sided operation read fails, and the network device can notify the CPU of the server to perform a two-sided read operation to obtain the target metadata according to the identifier of the target metadata. It should be understood that the above examples are for illustrative purposes only and are not intended to impose specific limitations in this application.

[0093] S240: When the target metadata is stored in memory, the server's network device retrieves the target metadata and returns it to the client.

[0094] Referring to step S220, the metadata information can specifically be a (K,V) key-value pair, where the value V corresponding to the key K can be metadata itself or metadata retrieval information. Therefore, if the value in the metadata key-value pair stored on the server is defined as the metadata itself, the server's network device can directly return the value V in the key-value pair as the target metadata to the client. If the value in the metadata key-value pair stored on the server is defined as metadata retrieval information, the network device can retrieve the target metadata based on this retrieval information and then return the target metadata to the client. For the specific process of obtaining metadata based on the value in the key-value pair, please refer to [link to relevant documentation]. Figure 3 Examples are not repeated here.

[0095] S250: When the target metadata is not stored in memory, the server's network device sends the target metadata information to the server's CPU.

[0096] In one embodiment, when the target metadata is not stored in memory, the network device can notify the server's CPU, which then initiates a two-way operation to obtain the metadata. Specifically, the target metadata information can be an identifier of the target metadata carried in the access request sent by the client to the server, such as the target metadata key.

[0097] S260: The server's CPU obtains the target metadata based on the target metadata information.

[0098] refer to Figure 3 As illustrated in the example, the CPU can first search the tree in memory to see if the target metadata is stored, based on the key of the target metadata. Then, it searches the C1 tree and C2 tree in the storage medium, and so on, until the target metadata is found. See the detailed description for further information. Figure 3 The description of how to read metadata through bilateral operations in the embodiments will not be repeated here.

[0099] S270: The server's CPU returns the target metadata to the network device, which then transmits the target metadata to the client.

[0100] In one embodiment, at steps S240 and S270, after the server obtains the target metadata, it can return the target metadata to the client through a network device, and then the client decides whether to initiate an access request for the target data based on the target metadata; alternatively, it can obtain the target data based on the target metadata and return the target data to the client; or, if it is determined that the target data is stored locally after obtaining the target metadata, it can obtain the target data from the local machine and return it to the client, thereby further reducing the processing pressure on the client and improving the data reading efficiency of the distributed system. It should be understood that after the server obtains the target metadata, it can determine the subsequent processing flow of the target metadata according to the actual application scenario. This application does not limit the specific processing flow after the server obtains the target metadata. In one embodiment, if the version of metadata stored on the server is constantly updated, the version of metadata returned by the server may not be the target version of metadata requested by the client. For example, at time T1, the version of metadata A is V1, and the client requests to access the latest version of metadata A at time T1. At time T2, the version of data A becomes V2, but at time T2, the server returns the V1 version of data A to the client. Alternatively, the client requests to access the V1 version of metadata A, but the server returns the latest V2 version of metadata A, causing the client to obtain the wrong version of the target metadata. To address the aforementioned issues, in step S210, the access request for the target metadata may further include version information of the target metadata. In step S220, the metadata information obtained by the server from the location specified in the access request may further include the version number of the metadata. In step S230, when comparing the target metadata information with the metadata information, the server's network device may first compare the key in the metadata information obtained from the location specified in the access request with the key in the target metadata information. If they match, it will compare the version information in the metadata information obtained from the location specified in the access request with the version information in the target metadata information. If they also match, step S240 will be executed, and the network device will read the target metadata through a one-sided operation. Otherwise, steps S250 to S270 will be executed to notify the CPU to perform a two-sided operation to read the target metadata.

[0101] For example, such as Figure 5 As shown, server 200 receives an access request for target metadata sent by client 100. This access request may include the key of the target metadata, version information of the target metadata, and the location specified in the access request. Server 200 stores the key-value pairs of the metadata using a hash function. The location specified in the access request is obtained by client 100 using the same hash function and the key of the target metadata. The steps for server 200 to process this read request can be as follows:

[0102] 1. Network device 210 receives an access request for target metadata sent by client 100.

[0103] 2. Network device 210 obtains metadata information from the location specified in the access request. This metadata information may include metadata key-value pairs and the metadata version. It should be understood, as referred to the foregoing, that if the target metadata is stored in memory 230, then the key-value pair obtained by the network device from the location specified in the access request is the target metadata key-value pair, and the key in this key-value pair is the same as the key of the target metadata. If the target metadata is not stored in memory 230, then the key in this key-value pair is different from the key of the target metadata.

[0104] 3. Network device 210 determines whether the key in the key-value pair of metadata obtained from the specified location from the location request is consistent with the key in the information of the target metadata.

[0105] 4. If the key in the key-value pair matches the key in the target metadata information, network device 210 determines whether the version information of the target metadata matches the version information obtained from the location specified in the location request. It is worth noting that this application does not restrict the order of steps 3 and 4 above.

[0106] 5. When the version information of the target metadata is consistent with the version information of the metadata, the network device 210 can obtain the target metadata based on the value of the key-value pair in the metadata information obtained at the specified location in the location request. Where the value of the metadata in the server represents the metadata itself, the value can be directly returned to the client as the target metadata. Where the value of the metadata represents the metadata acquisition information, the target metadata can be obtained based on the value and then returned to the client. In specific implementation, the method of obtaining the metadata can be determined according to the definition of the value. This application does not make specific limitations.

[0107] 6. Network device 210 returns target metadata to the client.

[0108] 7. If the key in the key-value pair does not match the key in the target metadata information, or if the version information of the target metadata does not match the version information of the metadata, the network device 210 sends the target metadata information to the processor 220 to request data query. Specifically, the CPU can query the target metadata sequentially in the tree of memory and storage medium according to the key value of the target metadata, as detailed above. Figure 3 Examples are not repeated here.

[0109] In one embodiment, if the target metadata does not carry version information, but the server determines that there are multiple versions of the target metadata, the server can also report the existence of multiple versions of the target metadata to the client. The client can then re-initiate an access request for the target metadata carrying version information, and the server can execute steps 1 to 7 above or the target metadata, thereby improving the user experience.

[0110] In one embodiment, as can be seen from the foregoing, the client network device 130 and the server network device 210 can be network cards with RDMA functionality, or network cards or switches that support IB. Therefore, network devices 130 and 210 can implement the above-mentioned solution provided in this application through programmable chips such as field programmable gate arrays (FPGAs) and application-specific integrated circuits (ASICs).

[0111] In summary, this application provides a data reading method. In this method, the client first sends an access request for target metadata to the server's network device. The server's network device queries the target metadata in the server's memory. If the target metadata is not found in memory, the network device can send an access request for the target metadata to the server's processor. The server's processor then searches for the target metadata in the global metadata. In this way, when the target metadata is not found in memory, the network card will continue to request the server's processor to query the target metadata without notifying the client to send another query request to the server's processor. This reduces the client's load, saves network bandwidth, and reduces the metadata query latency.

[0112] The methods of the embodiments of this application have been described in detail above. In order to facilitate better implementation of the above-described solutions of the embodiments of this application, relevant equipment for cooperating in implementing the above solutions is also provided below.

[0113] Figure 6 This is a schematic diagram of the structure of a network device 210 provided in this application. The network device 210 is used in applications such as... Figure 2 In the distributed storage system shown, network device 210 is connected to server 200 and client 100. The client runs an application, and the server caches metadata information of some application data in its memory, such as... Figure 6 As shown, the network device 210 may include a receiving unit 610, an information acquisition unit 620, a determining unit 630, a data acquisition unit 640, and a sending unit 650.

[0114] The receiving unit 610 is used to receive an access request for target metadata sent by the client. The access request carries information about the target metadata and the location specified in the access request. For specific implementation details, please refer to the above. Figure 4 Step S210 and the above in the illustrated embodiment Figure 5 The detailed description of step 1 in the embodiments will not be repeated here.

[0115] Information acquisition unit 620 is used to retrieve metadata information from the specified location in memory. For details on its implementation, please refer to the above. Figure 4 Step S220 and the above in the illustrated embodiment Figure 5 The detailed description of step 2 in the embodiments will not be repeated here.

[0116] The determining unit 630 is used to compare the acquired metadata information with the target metadata information to determine whether the target metadata is stored in memory. For specific implementation details, please refer to the above. Figure 4 Step S230 and the above in the illustrated embodiment Figure 5 The detailed descriptions of steps 3 and 4 in the embodiments are not repeated here.

[0117] Metadata acquisition unit 640 is used to acquire target metadata and transmit it to the client when the target metadata is stored in memory. For specific implementation details, please refer to the above. Figure 4 In the illustrated embodiment, step S240 and the above-mentioned Figure 5 The detailed descriptions of steps 5 and 6 in the embodiments are not repeated here.

[0118] In one embodiment, the network device 210 further includes a sending unit 650, which is configured to send target metadata information to the server's processor when the target metadata is not stored in memory, so that the processor can obtain the target metadata based on the target metadata information and transmit the target metadata data to the client. For specific implementation details, please refer to the above. Figure 4 Step S250 and the above in the illustrated embodiment Figure 5 The detailed description of step 7 in the embodiments will not be repeated here.

[0119] In one embodiment, the metadata information cached in memory includes the metadata identifier and metadata, or the metadata identifier and metadata retrieval information, wherein the metadata retrieval information is used to retrieve metadata.

[0120] In one embodiment, metadata information is stored in memory in the form of key-value pairs, wherein the identifier of the metadata is the key in the key-value pair, and the metadata or metadata retrieval information is the value in the key-value pair.

[0121] In one embodiment, metadata information of some application data is cached in the server's memory using a hash table. The hash value obtained by the server performing a hash operation on the data identifier of the metadata is the storage location of the metadata information in the hash table.

[0122] In one embodiment, the location specified in the access request is the hash value obtained by the client performing a hash operation on the target metadata, wherein the hash function used by the client to perform the hash operation is the same as the hash function used by the server.

[0123] In one embodiment, the target metadata information includes an identifier of the target metadata. The determining unit 630 is used to compare the identifier of the target metadata with the key in the key-value pair to determine whether the target metadata is stored in memory. The metadata acquisition unit 620 is used to acquire the target metadata based on the target metadata acquisition information when the identifier of the target metadata matches the key, and then transmit the target metadata data to the client. The sending unit 650 is used to send the target metadata information to the server's processor when the identifier of the target metadata does not match the key, so that the processor can acquire the target metadata based on the target metadata information and transmit the target metadata data to the client.

[0124] In summary, this application provides a network device that connects to a client and a server. This network device can receive access requests for target metadata sent by the client and query the target metadata in the server's memory. If the target metadata is not found in memory, the network device can send an access request to the server's processor, which then searches for the target metadata in the global metadata. Thus, when the target metadata is not found in memory, the network card will continue to request the server's processor to query the target metadata without notifying the client to send another query request to the server's processor. This reduces the client's load, saves network bandwidth, and lowers the metadata query latency.

[0125] Figure 7 This is a schematic diagram of the hardware structure of a network device 210 provided in this application, such as... Figure 7As shown, network device 210 includes a processor 710, a communication interface 720, and a memory 730. The processor 710, communication interface 720, and memory 730 can be interconnected via an internal bus 740, or they can communicate via wireless transmission or other means. This embodiment uses a connection via bus 740 as an example. Bus 740 can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. Bus 740 can be divided into address bus, data bus, control bus, etc. For ease of illustration, Figure 7 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0126] Processor 710 may consist of at least one general-purpose processor, such as a central processing unit (CPU), or a combination of a CPU and hardware chips. The hardware chips may be application-specific integrated circuits (ASICs), programmable logic devices (PLDs), or combinations thereof. The PLDs may be complex programmable logic devices (CPLDs), field-programmable gate arrays (FPGAs), generic array logic (GALs), or any combination thereof. Processor 710 executes various types of digital storage instructions, such as software or firmware programs stored in memory 730, enabling network device 210 to provide a wide range of services.

[0127] The memory 730 is used to store program code, and its execution is controlled by the processor 710 to perform the above-mentioned tasks. Figures 1-7 The processing steps of network device 210 in any embodiment. The program code may include one or more software modules. These one or more software modules can be Figure 6The software modules provided in the illustrated embodiment, such as a receiving unit, an information acquisition unit, a determining unit, a metadata acquisition unit, and a sending unit, are as follows: the receiving unit is used to receive an access request for target metadata sent by a client; the information acquisition unit is used to acquire metadata information at the location specified in the access request in memory; the determining unit is used to compare the acquired metadata information with the target metadata information to determine whether the target metadata is stored in memory; the metadata acquisition unit is used to acquire the target metadata and transmit the target metadata data to the client when the target metadata is stored in memory; and the sending unit is used to send the target metadata information to the server's processor when the target metadata is not stored in memory, so that the processor can acquire the target metadata based on the target metadata information and transmit the target metadata data to the client. Specifically, it can be used for execution... Figure 4 Steps S210-S270 in the embodiment Figure 5 Steps 1-7 and their optional steps in the embodiments can also be used to perform Figures 1-7 Other steps performed by network device 210 as described in the embodiments will not be repeated here.

[0128] The memory 730 may include volatile memory, such as random access memory (RAM); the memory 730 may also include non-volatile memory, such as read-only memory (ROM), flash memory, hard disk drive (HDD), or solid-state drive (SSD); the memory 730 may also include combinations of the above types. The memory 730 may store program code, specifically code for execution by the processor 710. Figures 1-6 The program code for the other steps described in the embodiments will not be repeated here.

[0129] The communication interface 720 can be a wired interface (e.g., an Ethernet interface), an internal interface (e.g., a Peripheral Component Interconnect Express (PCIe) bus interface), a wired interface (e.g., an Ethernet interface), or a wireless interface (e.g., a cellular network interface or a wireless LAN interface), used to communicate with other servers or modules. In specific implementations, the communication interface 720 can be used to receive messages for the processor 710 to process.

[0130] It needs to be explained that, Figure 7This is merely one possible implementation of an embodiment of this application. In practical applications, network devices may include more or fewer components, and this is not a limitation. For content not shown or described in the embodiments of this application, please refer to the foregoing. Figures 1-6 The relevant descriptions in the embodiments will not be repeated here.

[0131] This application also provides a computer-readable storage medium storing instructions that, when executed on a processor. Figures 1-6 The method and flow shown are thus implemented.

[0132] This application also provides a computer program product that, when run on a processor, provides a solution for... Figures 1-6 The method and flow shown are thus implemented.

[0133] The above embodiments can be implemented, in whole or in part, by software, hardware, firmware, or any other combination thereof. When implemented using software, the above embodiments can be implemented, in whole or in part, as a computer program product. A computer program product includes at least one computer instruction. When the computer program instruction is loaded or executed on a computer, all or part of the flow or function according to the embodiments of the present invention is generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, Digital Subscriber Line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes at least one set of available media. The available media can be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., high-density digital video discs (DVDs)), or semiconductor media. Semiconductor media can be SSDs.

[0134] The above are merely specific embodiments of the present invention, but the scope of protection of the present invention is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in the present invention, and these modifications or substitutions should all be covered within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A computer system comprising a client and a server, wherein the server includes a network device, the network device includes a network interface card (NIC), and the network device is used to connect the client and the server, characterized in that, The client is used to run applications; The server is used to identify metadata for caching a portion of the application's data in memory; The client is used to send an access request for target metadata to the server. The access request includes the identifier of the target metadata and the location specified in the access request. The network device is used to obtain the identifier of the first metadata from the location specified by the access request, and compare the identifier of the first metadata with the identifier of the target metadata to determine whether the target metadata is stored in the memory; The network device is configured to retrieve the target metadata from the location specified in the access request and transmit the target metadata to the client when the target metadata is stored in the memory; The network device is also configured to send the target metadata information to the server's processor when the target metadata is not stored in the memory; The server-side processor is used to obtain the target metadata based on the information in the target metadata; The network device is also used to transmit the target metadata acquired by the processor to the client.

2. The system according to claim 1, characterized in that, The metadata information cached in memory includes the identifier and metadata of the metadata; or, it includes the identifier and metadata acquisition information of the metadata, wherein the metadata acquisition information is used to acquire the metadata.

3. The system according to claim 2, characterized in that, The metadata information is stored in the memory in the form of key-value pairs, wherein the identifier of the metadata is the key in the key-value pair, and the metadata or the information for obtaining the metadata is the value in the key-value pair.

4. The system according to claim 2, characterized in that, Metadata information of some data of the application is cached in the memory of the server in a hash table. The hash value obtained by the server performing a hash operation on the data identifier of the metadata is the storage location of the metadata information in the hash table.

5. The system according to claim 4, characterized in that, The location specified in the access request is the hash value obtained by the client performing a hash operation on the target metadata information, wherein the hash function used by the client to perform the hash operation is the same as the hash function used by the server when determining the storage location of the metadata information.

6. The system according to claim 3, characterized in that, The target metadata information includes the identifier of the target metadata. The network device is used to compare the identifier of the target metadata with the key in the key-value pair. When the identifier of the target metadata is consistent with the key, the metadata corresponding to the key is the target metadata or the metadata acquisition information corresponding to the key is the acquisition information of the target metadata. The network device acquires the target metadata according to the acquisition information of the target metadata.

7. A method for obtaining metadata, applied to a network device on a server side, the network device including a network interface card (NIC), the network device being used to connect a client and the server, the client running an application, and the server having metadata identifiers of some data of the application cached in its memory; characterized in that, The method includes: Receive an access request for target metadata sent by a client, wherein the access request for target metadata carries the identifier of the target metadata and the location specified in the access request; The identifier of the first metadata is obtained at the location specified by the access request in the memory; The identifier of the first metadata is compared with the identifier of the target metadata to determine whether the target metadata is stored in the memory; When the target metadata is stored in the memory, the target metadata is retrieved from the location specified in the access request and the target metadata is transmitted to the client; When the target metadata is not stored in the memory, the network device sends the target metadata information to the server's processor, so that the server's processor can obtain the target metadata based on the target metadata information. The network device receives the target metadata sent by the server's processor and transmits the target metadata to the client.

8. The method according to claim 7, characterized in that, The metadata information cached in memory includes the identifier and metadata of the metadata; or, it includes the identifier and metadata acquisition information of the metadata, wherein the metadata acquisition information is used to acquire the metadata.

9. The method according to claim 8, characterized in that, The metadata information is stored in the memory in the form of key-value pairs, wherein the identifier of the metadata is the key in the key-value pair, and the metadata or the information for obtaining the metadata is the value in the key-value pair.

10. The method according to claim 8, characterized in that, Metadata information of some data of the application is cached in the memory of the server in a hash table. The hash value obtained by the server performing a hash operation on the data identifier of the metadata is the storage location of the metadata information in the hash table.

11. The method according to claim 10, characterized in that, The location specified in the access request is the hash value obtained by the client performing a hash operation on the target metadata, wherein the hash function used by the client to perform the hash operation is the same as the hash function used by the server when determining the storage location of the metadata information.

12. The method according to claim 9, characterized in that, The information of the target metadata is the identifier of the target metadata, and the step of comparing the acquired metadata information with the information of the target metadata to determine whether the target metadata is stored in the memory includes: The network device compares the identifier of the target metadata with the key in the key-value pair to determine whether the target metadata is stored in the memory; The step of acquiring the target metadata and transmitting it to the client when the target metadata is stored in the memory includes: When the identifier of the target metadata matches the key, the metadata corresponding to the key is the target metadata or the metadata acquisition information corresponding to the key is the acquisition information of the target metadata. The network device acquires the target metadata according to the acquisition information of the target metadata and transmits the target metadata to the client.

13. A network device for connecting a client and a server, wherein the client runs an application, the server includes the network device and memory, the network device includes a network interface card (NIC), and the memory caches identifiers of metadata containing partial data of the application, characterized in that... The network device includes: A receiving unit is configured to receive an access request for target metadata sent by a client, wherein the access request for target metadata carries the identifier of the target metadata and the location specified in the access request; An information acquisition unit is used to acquire the identifier of the first metadata at the location specified by the access request in the memory; A determining unit is configured to compare the identifier of the first metadata with the identifier of the target metadata to determine whether the target metadata is stored in the memory; The metadata acquisition unit is used to acquire the target metadata and transmit the target metadata to the client when the target metadata is stored in the memory; The sending unit is configured to send the target metadata information to the server's processor when the target metadata is not stored in the memory, so that the processor can obtain the target metadata based on the target metadata information and transmit the target metadata data to the client.

14. The network device according to claim 13, characterized in that, The metadata information cached in memory includes the identifier and metadata of the metadata; or, it includes the identifier and metadata acquisition information of the metadata, wherein the metadata acquisition information is used to acquire the metadata.

15. The network device according to claim 14, characterized in that, The metadata information is stored in the memory in the form of key-value pairs, wherein the identifier of the metadata is the key in the key-value pair, and the metadata or the information for obtaining the metadata is the value in the key-value pair.

16. The network device according to claim 14, characterized in that, Metadata information of some data of the application is cached in the memory of the server in a hash table. The hash value obtained by the server performing a hash operation on the data identifier of the metadata is the storage location of the metadata information in the hash table.

17. The network device according to claim 16, characterized in that, The location specified in the access request is the hash value obtained by the client performing a hash operation on the target metadata, wherein the hash function used by the client to perform the hash operation is the same as the hash function used by the server when determining the storage location of the metadata information.

18. The network device according to claim 15, characterized in that, The information in the target metadata includes the identifier of the target metadata. The determining unit is configured to compare the identifier of the target metadata with the key in the key-value pair to determine whether the target metadata is stored in the memory; The metadata acquisition unit is configured to, when the identifier of the target metadata is consistent with the key, either the metadata corresponding to the key is the target metadata or the metadata acquisition information corresponding to the key is the acquisition information of the target metadata, acquire the target metadata according to the acquisition information of the target metadata, and transmit the target metadata to the client; The sending unit is configured to send the target metadata information to the server's processor when the identifier of the target metadata does not match the key, so that the processor can obtain the target metadata based on the target metadata information and transmit the target metadata data to the client.

19. A network device, characterized in that, The device includes a processor and a communication interface, the communication interface being used to receive an access request from a client to access target metadata, and the processor being used to execute the method of any one of claims 7 to 12 to process the access request.

Images