A method and apparatus for storing metadata

Abstract

The application provides a metadata storage method and device, relates to the technical field of storage, and realizes efficient and high-reliability metadata storage. The scheme comprises the following steps: obtaining first metadata, the first metadata being used for indicating the storage position of a data file stored in a disk; writing the first metadata into a first address of a first area for temporarily storing metadata in the disk by means of appending writing; the appending writing is used for storing after the most newly written metadata, and the first address is a next address after the most newly written metadata in the first area. A first mapping relationship of the first metadata is recorded in the memory, the first mapping relationship of the first metadata comprises the identification of the first metadata and address information of the first address, and the first mapping relationship of the first metadata is used for acquiring the first address when the first metadata is queried.

Description

Technical Field

[0001] This invention relates to the field of storage technology, and in particular to a method and apparatus for storing metadata. Background Technology

[0002] With the advancement of modern science and technology, the number of data files that users expect to store is increasing exponentially. As we all know, when reading data in a storage system, a fixed amount of data must first be read from the disk into memory, and then the desired data can be read from it. Due to the massive amount of data in the storage system, reading a large amount of data into memory not only leads to low data file reading efficiency, but also consumes a large amount of system transmission bandwidth.

[0003] To quickly access stored data files, metadata describing inherent attributes such as file type, size, and creation time, as well as the disk storage location, can be recorded. When querying a data file, the recorded metadata is first read into memory to obtain the storage location of the desired file. This allows for direct access to the expected data. Because the metadata is small, the query time is significantly reduced, effectively improving data file reading efficiency while avoiding excessive consumption of system bandwidth.

[0004] Currently, storage systems record metadata for data files in memory. Only when the amount of metadata exceeds a set threshold is the metadata recorded in memory persisted to disk. A B+ tree data structure is typically used when persisting metadata from memory to disk.

[0005] In the aforementioned metadata storage solutions, metadata is recorded in memory. However, data loss occurs when the hardware loses power, compromising the reliability of metadata storage. Furthermore, the B+ tree data structure's restrictions on storage order lead to amplified writes during metadata storage on disk, resulting in long processing times, low metadata storage efficiency, and reduced system performance. Summary of the Invention

[0006] This application provides a method and apparatus for storing metadata, enabling reliable and efficient storage of metadata.

[0007] To achieve the above objectives, this application adopts the following technical solution:

[0008] In a first aspect, this application provides a method for storing metadata, which may include: obtaining first metadata, the first metadata being used to indicate the storage location of a data file stored on a disk; writing the first metadata to a first address of a first region on the disk used for temporarily storing metadata by appending; the append write being used to store after the most recently written metadata, the first address being the next address after the most recently written metadata in the first region; and recording a first mapping relationship of the first metadata in memory, the first mapping relationship of the first metadata including the identifier of the first metadata and the address information of the first address, the first mapping relationship of the first metadata being used to obtain the first address when querying the first metadata.

[0009] The solution provided in this application directly writes metadata to the disk in an append-only manner, and records the mapping relationship between the metadata and the storage address on the disk in memory. On the one hand, since the performance of sequential disk writes is far superior to that of random writes, directly writing metadata to the disk in an append-only manner ensures efficient metadata writing. By recording the initial mapping relationship in memory, it can be ensured that when reading metadata, the target metadata is accurately read from the disk using the address in the mapping relationship, without needing to read large amounts of data from the disk, thus ensuring efficient metadata reading. On the other hand, storing metadata on the disk ensures that the metadata will not be lost even if the hardware loses power, guaranteeing the reliability of metadata storage.

[0010] In one possible implementation, if the size of the metadata stored in the first region is greater than or equal to a first threshold, the first region is cleared to ensure that it stores recently written metadata without consuming excessive disk space. The first mapping relationship of the second metadata (metadata written to the second region) is deleted from memory. In this implementation, by configuring a second region on disk for persistent storage of metadata, space in the first region is promptly freed up for storing newly written metadata. Since newly written metadata has a higher access probability, and the metadata stored in the first region records the first mapping relationship in memory, the efficiency of metadata access can be further improved.

[0011] In one possible implementation, multiple metadata sets are configured in the second region. Each metadata set has a reference information range, and the reference information ranges of different metadata sets do not overlap. Metadata stored in the first region is appended to the metadata set in the second region on the disk. This includes appending third metadata stored in the first region to the first metadata set in the second region, where the reference information of the third metadata lies within the reference information range of the first metadata set. The third metadata is a metadata instance in the first region. In this way, metadata from the first region is appended to the corresponding metadata set in the second region, and the metadata set to which the metadata is written is determined by the reference information of the metadata, ensuring efficient metadata transfer.

[0012] In one possible implementation, the reference information range includes the lexicographical order of the metadata identifiers from smallest to largest. Sort the metadata set in the second region lexicographically to speed up access efficiency when reading metadata from the second region.

[0013] In one possible implementation, multiple metadata sets are configured in the second region. The metadata stored in the first region is written to the metadata set in the second region on the disk by appending. Specifically, the data in the first region is written to the metadata set in the second region in the order of storage in the first region. This application does not limit which metadata set the metadata is written to.

[0014] In one possible implementation, the method for storing metadata provided in this application may further include: recording a second mapping relationship of second metadata that meets certain conditions in memory; the second mapping relationship includes the identifier of the metadata and the address information of the storage address of the metadata in the second region. By recording the second mapping relationship in memory, it can be ensured that when reading metadata from the second region, the target metadata can be accurately read from the disk through the address in the second mapping relationship, without having to read a large amount of data from the disk, thus ensuring the efficiency of metadata reading.

[0015] In one possible implementation, the above condition is that the access popularity is greater than or equal to a third threshold.

[0016] In one possible implementation, the third threshold can be 0 or any other arbitrary value.

[0017] In one possible implementation, the method for storing metadata provided in this application may further include: recording a reference information range of the metadata set in memory. It should be noted that the reference information range of the metadata set can be stored in a second region on the disk or other disk regions capable of persistently storing data. The solution provided in this application, by recording the reference information range of the metadata set in memory, allows for the rapid determination of the metadata set where the metadata is stored in memory when reading metadata from the second region, based on the reference information. Since the addresses of each metadata set in the second region are known, data within the address range of the metadata set containing the desired metadata can be read into memory to retrieve the desired metadata. This eliminates the need to read large amounts of data from the disk into memory, ensuring efficient metadata retrieval.

[0018] In one possible implementation, the method for storing metadata provided in this application may further include: if the size of the metadata in the second metadata set in the second region is greater than or equal to a second threshold, splitting the second metadata set into multiple metadata subsets. The union of the reference information ranges of the metadata subsets obtained from the splitting of the second metadata set is the reference information range of the second metadata set. In this way, by limiting the size of the metadata set in the second region, it is ensured that when reading the metadata set in a certain second region, the number of metadata traversed and queried will not be excessive, thereby improving the access efficiency of reading metadata from the second region.

[0019] In one possible implementation, the first or second mapping relationship can be a hash table.

[0020] In one possible implementation, the method for storing metadata provided in this application may further include: obtaining a query request, the query request including an identifier of fourth metadata; if a first mapping relationship of the fourth metadata exists in memory, obtaining the second address information corresponding to the identifier of the fourth metadata in the first mapping relationship of the fourth metadata, and reading data from the location indicated by the second address information on the disk as the fourth metadata. By using the first mapping relationship in memory, metadata can be quickly queried, avoiding excessive disk read and write operations.

[0021] In one possible implementation, the method for storing metadata provided in this application may further include: obtaining a query request, the query request including an identifier of the fifth metadata; if a first mapping relationship of the fifth metadata exists in memory, obtaining the third address information corresponding to the identifier of the fifth metadata in the first mapping relationship of the fifth metadata, and reading data from the location indicated by the third address information on the disk as the fifth metadata. If a first mapping relationship of the fifth metadata does not exist in memory, reading the fifth metadata in a second area according to the identifier of the fifth metadata.

[0022] In one possible implementation, a second mapping relationship of metadata is recorded in memory. Based on the identifier of the fifth metadata, the fifth metadata is read from the second region. This includes: obtaining the fourth address information corresponding to the identifier of the fifth metadata in the second mapping relationship; and reading data from the location indicated by the fourth address information on the disk as the fifth metadata. By using the first mapping relationship in memory, metadata can be quickly queried, avoiding excessive disk read / write operations.

[0023] In one possible implementation, a reference information range for the metadata set is recorded in memory. The identifier of the fifth metadata falls within this reference information range. Based on the identifier of the fifth metadata, the fifth metadata is read in the second region, including reading the fifth metadata based on the address range of the third metadata set on disk. In this way, by recording the reference information range for the metadata set in memory, if the read metadata (the fifth metadata) is located within this set, only the space belonging to that set can be accessed, improving access efficiency.

[0024] Secondly, this application also provides a method for querying metadata. This method may include: obtaining a query request, the query request including an identifier of fourth metadata; the fourth metadata indicating the storage location of a data file stored on a disk; if a first mapping relationship of the fourth metadata exists in memory, obtaining second address information corresponding to the identifier of the fourth metadata in the first mapping relationship; and reading data from the location indicated by the second address information on the disk as the fourth metadata. The first mapping relationship of the fourth metadata includes the identifier of the fourth metadata and the storage address information of the fourth metadata in a first region on the disk, wherein the metadata in the first region is written in an append-only manner. Append-only writing is used to store metadata after the most recently written metadata.

[0025] The solution provided in this application directly writes metadata to the disk in an append-only manner, and records the mapping relationship between the metadata and the storage address on the disk in memory. On the one hand, since the performance of sequential disk writes is far superior to that of random writes, directly writing metadata to the disk in an append-only manner ensures efficient metadata writing. By recording the initial mapping relationship in memory, it can be ensured that when reading metadata, the target metadata is accurately read from the disk using the address in the mapping relationship, without needing to read large amounts of data from the disk, thus ensuring efficient metadata reading. On the other hand, storing metadata on the disk ensures that the metadata will not be lost even if the hardware loses power, guaranteeing the reliability of metadata storage.

[0026] In one possible implementation, the method for storing metadata provided in this application may further include: if the first mapping relationship of the fourth metadata does not exist in memory, reading the fourth metadata in a second area on the disk according to the identifier of the fourth metadata. Wherein, when the size of the metadata stored in the first area is greater than or equal to a first threshold, the metadata stored in the first area is appended to the metadata set in the second area on the disk. In this implementation, by configuring a second area on the disk for persistent storage of metadata, space in the first area can be freed up in a timely manner to store newly written metadata. Since the access probability of newly written metadata is higher, and the metadata stored in the first area records the first mapping relationship of the metadata in memory, the access efficiency of metadata can be further improved.

[0027] In one possible implementation, a second mapping relationship for metadata that meets certain conditions is recorded in memory. This second mapping relationship includes the identifier of the metadata and the address information of the metadata's storage address in the second region. Based on the identifier of the fourth metadata, the fourth metadata is read from the second region. This includes: obtaining the fourth address information corresponding to the identifier of the fourth metadata in the second mapping relationship; and reading the data from the disk at the location indicated by the fourth address information, using this data as the fourth metadata. By recording the second mapping relationship in memory, it can be ensured that when reading metadata from the second region, the target metadata is accurately read from the disk using the address in the second mapping relationship, without needing to read large amounts of data from the disk, thus ensuring efficient metadata reading.

[0028] In another possible implementation, multiple metadata sets are configured in the second region. Each metadata set has a reference information range, and the reference information ranges of different metadata sets do not overlap. The reference information ranges of the metadata sets are recorded in memory. The identifier of the fourth metadata is located within the reference information range of the third metadata set. Based on the identifier of the fourth metadata, the fourth metadata is read in the second region, including reading the fourth metadata based on the address range of the third metadata set on disk. In this way, when reading the fourth metadata, the reference information range of the metadata sets recorded in memory can be used to quickly locate the reference information range of the third metadata set to which the fourth metadata belongs, and then the fourth metadata can be read based on the address range of the third metadata set on disk, ensuring read speed.

[0029] Thirdly, this application provides an apparatus for storing metadata, which may include: an acquisition module for acquiring first metadata, the first metadata being used to indicate the storage location of a data file stored on a disk; a writing module for writing the first metadata to a first address of a first region on the disk used for temporarily storing metadata by appending the first metadata; the appending write is used to store the metadata after the most recently written metadata, and the first address is the next address after the most recently written metadata in the first region; and a recording module for recording a first mapping relationship of the first metadata in memory, the first mapping relationship of the first metadata including the identifier of the first metadata and the address information of the first address, the first mapping relationship of the first metadata being used to obtain the first address when querying the first metadata.

[0030] In one possible implementation, the writing module is further configured to, if the size of the metadata stored in the first region is greater than or equal to a first threshold, write the metadata stored in the first region to the metadata set in the second region of the disk by appending the metadata; the device may also include a deletion module for clearing the first region; the deletion module is further configured to delete the first mapping relationship of the second metadata in memory, wherein the second metadata is the metadata written to the second region.

[0031] In one possible implementation, multiple metadata sets are configured in the second region. Each metadata set has a reference information range, and the reference information ranges of different metadata sets do not overlap. The metadata stored in the first region is written to the metadata set in the second region on the disk by appending. The writing module is also used to write the third metadata stored in the first region to the first metadata set in the second region by appending, wherein the reference information of the third metadata is located within the reference information range of the first metadata set.

[0032] In one possible implementation, the range of reference information includes the lexicographical order of the metadata identifiers from smallest to largest.

[0033] In one possible implementation, the recording module is further configured to record in memory a second mapping relationship of the second metadata that meets the conditions; the second mapping relationship includes the identifier of the metadata and the address information of the storage address of the metadata in the second region; or, the recording module is further configured to record in memory a reference information range of the metadata set.

[0034] In one possible implementation, the device may further include a splitting module, used to split the second metadata set into multiple metadata subsets if the size of the metadata in the second metadata set in the second region is greater than or equal to a second threshold, wherein the union of the reference information ranges of the metadata subsets obtained by splitting the second metadata set is the reference information range of the second metadata set.

[0035] In one possible implementation, the first or second mapping relationship is a hash table.

[0036] In one possible implementation, the acquisition module is further configured to acquire a query request, the query request including an identifier of the fourth metadata; the device may also include a reading module, configured to, if a first mapping relationship of the fourth metadata exists in memory, acquire second address information corresponding to the identifier of the fourth metadata in the first mapping relationship of the fourth metadata, and read data at the location indicated by the second address information on the disk as the fourth metadata.

[0037] In one possible implementation, the acquisition module is further configured to acquire a query request, the query request including the identifier of the fifth metadata; the reading module is further configured to, if a first mapping relationship of the fifth metadata exists in memory, acquire the third address information corresponding to the identifier of the fifth metadata in the first mapping relationship of the fifth metadata, and read data at the location indicated by the third address information on the disk as the fifth metadata; the reading module is further configured to, if a first mapping relationship of the fifth metadata does not exist in memory, read the fifth metadata in the second area according to the identifier of the fifth metadata.

[0038] In one possible implementation, the reading module is further configured to record a second mapping relationship of metadata in memory, and read the fifth metadata in the second area according to the identifier of the fifth metadata; the reading module is further configured to obtain the fourth address information corresponding to the identifier of the fifth metadata in the second mapping relationship of the fifth metadata, and read the data at the location indicated by the fourth address information on the disk as the fifth metadata.

[0039] In one possible implementation, the reference information range of the metadata set is recorded in memory, the identifier of the fifth metadata is within the reference information range of the third metadata set, and the fifth metadata is read in the second area according to the identifier of the fifth metadata; the reading module is specifically used to read the fifth metadata according to the address range of the third metadata set on the disk.

[0040] Fourthly, this application also provides an apparatus for querying metadata, which may include: an acquisition module for acquiring a query request, the query request including an identifier of fourth metadata; the fourth metadata indicating the storage location of a data file stored on a disk; the acquisition module further for acquiring, if a first mapping relationship of the fourth metadata exists in memory, second address information corresponding to the identifier of the fourth metadata in the first mapping relationship; and a reading module for reading data from the location indicated by the second address information on the disk as the fourth metadata. The first mapping relationship of the fourth metadata includes the identifier of the fourth metadata and the storage address information of the fourth metadata in a first region on the disk, wherein the metadata in the first region is written in an append-only manner. The append-only write is used to store data after the most recently written metadata.

[0041] In one possible implementation, the method for storing metadata provided in this application may further include: a reading module, further configured to read the fourth metadata in a second region on the disk according to the identifier of the fourth metadata if a first mapping relationship for the fourth metadata does not exist in memory. Wherein, when the size of the metadata stored in the first region is greater than or equal to a first threshold, the metadata stored in the first region is written to the metadata set in the second region on the disk by appending.

[0042] In one possible implementation, a second mapping relationship for metadata that meets certain conditions is recorded in memory. This second mapping relationship includes the identifier of the metadata and the address information of the metadata's storage address in a second region. The reading module is further configured to obtain the fourth address information corresponding to the identifier of the fourth metadata in the second mapping relationship, and read data from the location indicated by the fourth address information on the disk as the fourth metadata.

[0043] In another possible implementation, multiple metadata sets are configured in the second region. Each metadata set has a reference information range, and the reference information ranges of different metadata sets do not overlap. The reference information ranges of the metadata sets are recorded in memory, and the identifier of the fourth metadata is within the reference information range of the third metadata set. The read module is also used to read the fourth metadata based on the address range of the third metadata set on the disk.

[0044] Fifthly, this application provides a server comprising one or more processors and one or more memories; the one or more memories are coupled to the one or more processors, and the one or more memories are used to store computer program code, the computer program code including computer instructions, and when the one or more processors execute the computer instructions, the server performs a method of storing metadata as described in the first aspect and any possible thereof, or performs a method of querying metadata as described in the second aspect and any possible thereof.

[0045] In a sixth aspect, this application provides a computer-readable storage medium storing computer instructions that, when executed on a server, cause the server to perform a method for storing metadata as described in the first aspect and any possible thereof, or a method for querying metadata as described in the second aspect and any possible thereof.

[0046] In a seventh aspect, this application provides a computer program product comprising computer instructions that, when executed on a server, cause the server to perform a method for storing metadata as described in the first aspect and any possible thereof, or a method for querying metadata as described in the second aspect and any possible thereof. Attached Figure Description

[0047] Figure 1 A storage system framework diagram provided for an embodiment of this application;

[0048] Figure 2 A flowchart illustrating a method for storing metadata provided in an embodiment of this application;

[0049] Figure 3a This is a schematic diagram of disk storage data provided in an embodiment of this application;

[0050] Figure 3b A flowchart illustrating another method for storing metadata provided in an embodiment of this application;

[0051] Figure 4 A flowchart illustrating a method for metadata querying provided in an embodiment of this application;

[0052] Figure 5 A flowchart illustrating another method for metadata querying provided in an embodiment of this application;

[0053] Figure 6 This application provides a schematic diagram of the structure of a management device for storing metadata.

[0054] Figure 7 This application provides a schematic diagram of the structure of a metadata management device.

[0055] Figure 8 This application provides a schematic diagram of the structure of another management device for querying metadata;

[0056] Figure 9 This application provides a schematic diagram of the structure of a server. Detailed Implementation

[0057] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0058] In this article, the term "and / or" is merely a description of the relationship between related objects, indicating that there can be three relationships. For example, A and / or B can represent three situations: A exists alone, A and B exist simultaneously, and B exists alone.

[0059] The terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this application, unless otherwise stated, "multiple" means two or more. Furthermore, "at least one" means one or more, and "multiple" means two or more.

[0060] It should be noted that, in this application, the terms "exemplary" or "for example" are used to indicate that something is being described as an example, illustration, or illustration. Any embodiment or design described as "exemplary" or "for example" in this application should not be construed as being more preferred or advantageous than other embodiments or design solutions. Specifically, the use of terms such as "exemplary" or "for example" is intended to present the relevant concepts in a concrete manner.

[0061] First, some of the terms used in this application will be explained to facilitate understanding by those skilled in the art.

[0062] Data file: In computer science, a data file refers to the general term for all media containing symbols that can be input into a computer and processed by computer programs. These symbols are used to input data into an electronic computer for processing and can be meaningful numbers, letters, symbols, analog quantities, etc.

[0063] Metadata refers to data used to describe the attributes of data files stored on disk. The metadata used to describe the attributes of a particular data file is called the metadata of that data file. Data file attributes may include, but are not limited to, inherent attributes such as file type, file size, and file creation time, as well as information about the disk location where the file is stored. Storing metadata for a large number of data files allows for more efficient writing and querying of metadata, thereby enabling efficient access to the data files. A collection of metadata constructed according to a unified metadata storage format is called a metadata index.

[0064] Memory (RAM): A crucial component of a computer, also known as internal memory or main memory, it temporarily stores data processed by the central processing unit (CPU) and data exchanged with external storage devices such as hard drives. It acts as a bridge between other modules and the CPU. All programs run within memory, and the performance of the memory directly impacts the overall performance of the computer.

[0065] Disk: It is an external storage device for a computer that can store large amounts of data and retain the data even when power is off.

[0066] As described in the background section, storing metadata on disk using a B+ tree data structure suffers from several drawbacks. The B+ tree structure's restriction on storage order leads to amplified writes, resulting in prolonged processing time and low efficiency, thus reducing the performance of the storage system. Furthermore, since metadata is first stored in memory and then persistently written to disk only after certain conditions are met, data loss is inevitable in the event of a power outage, leading to poor reliability of metadata storage.

[0067] This application proposes a method for storing metadata. Metadata is written to disk via append-only operations, and a mapping relationship between metadata identifiers and disk storage addresses is recorded in memory. This ensures the reliability of metadata storage because the metadata is stored on disk, preventing data loss even during power outages. Furthermore, the efficiency of writing metadata to disk is guaranteed because sequential disk writes (append-only writes) are significantly faster than random writes (written to disk in B+ tree data format). Finally, the efficient access to the metadata on the disk is ensured because the metadata mapping relationship is stored in memory.

[0068] The technical solutions in the embodiments of this application will now be described with reference to the accompanying drawings.

[0069] The solution provided in this application can be applied to... Figure 1 The storage system 10 shown is an example. Figure 1 As shown, the storage system 10 includes a memory unit 101 and a disk unit 102.

[0070] The storage system 10 can be a distributed storage system, a centralized storage system, or other types. This application embodiment does not limit the type of storage system 10.

[0071] Memory unit 101 can be used to temporarily store data that interacts with disk unit 102. It should be noted that memory unit 101 serves as a bridge for interaction (read / write) between other external modules and disk unit 102, and is an important component for maintaining the normal operation of storage system 10. All programs in storage system 10 run within memory unit 101, and the data in memory unit 101 is lost when power is off.

[0072] Disk unit 102 is an external storage device that provides storage space for storage system 10. The capacity of disk unit 102 determines the size of the storage space that storage system 10 can provide for storage services. Figure 1 As shown, disk unit 102 can be divided into data disk unit 1021 and metadata disk unit 1022.

[0073] The data disk unit 1021 can be used to store the original data file, and the metadata disk unit 1022 can be used to store metadata describing the attributes of the original data file stored in the data disk unit 1021, so as to facilitate the subsequent retrieval of the original data file in the data disk unit 1021.

[0074] This application provides a method for storing metadata, applied to a management device, the management device being used to manage metadata, and the management device can be deployed in... Figure 1 In the illustrated storage system, Figure 1 The memory unit 101 and disk unit 102 shown can be partitioned to allow the management device to manage metadata.

[0075] It should be noted that the storage process for each piece of metadata is the same for the management device. The following embodiments of this application describe the process of the management device storing metadata once as an example, and other examples will not be repeated.

[0076] Furthermore, such as Figure 1 As shown, it is possible to Figure 1 The illustrated storage system 10 deploys a management device 103 for managing the metadata of the storage system 10. For example, the management device 103 can manage the storage and retrieval of metadata.

[0077] For example, the management device 103 can execute the solution provided in this application to manage the storage of metadata, and the specific implementation is described in the following method embodiments.

[0078] For example, the management device 103 may be referred to as an index manager (IM) module.

[0079] On the one hand, this application provides a method for storing metadata, which is applied to a management device that can provide... Figure 1The management device 103 in the middle. The storage process is the same for each metadata acquired by the management device. In the following embodiments of this application, the storage of one metadata is described as an example, and other examples will not be described in detail.

[0080] like Figure 2 As shown, the method for storing metadata provided in this application embodiment may include:

[0081] S201, The management device obtains the first metadata.

[0082] The first metadata is used to indicate the storage location of the data file stored on the disk. The first metadata can be any metadata obtained by the management device.

[0083] As one possible implementation, a module deployed in the storage system that stores data files can generate metadata for the data files after storing them on the storage system's disk, and send this metadata to the management device. That is, in S201, the management device receives and obtains the first metadata.

[0084] As another possible implementation, the management device receives and saves data files requested for storage from the outside, stores the data files on the disk, and generates the first metadata of the data files.

[0085] Of course, the management device can also obtain metadata through other means, which is not limited in this application embodiment.

[0086] S202, the management device writes the first metadata to the first address of the first area in the disk used for temporarily storing metadata by appending the first metadata.

[0087] The append write is used to store data after the most recently written metadata, and the first address is the next address after the most recently written metadata in the first region.

[0088] As one possible implementation, the first region can be deployed in the data storage area of ​​the disk in the storage system.

[0089] For example, the first area can be deployed in Figure 1 In the data disk unit 1021 of the storage system shown, using the data disk unit 1021 as the storage area for the first metadata can avoid the problem that when the amount of metadata is large, some metadata cannot be stored due to the small space of the metadata disk unit 1022.

[0090] As one possible implementation, the first region can be deployed in the metadata storage area of ​​the disk in the storage system. For example, the first region can be deployed in... Figure 1 The data disk unit 1022 is shown in the diagram.

[0091] For example, the first region can be viewed as a file.

[0092] It should be noted that the order of addresses in the disk during append writing can be configured according to actual needs, and this application embodiment does not limit it.

[0093] For example, such as Figure 3a The first region of the disk shown is assumed to have been written with metadata 15, metadata 11, metadata 14, metadata 16, and metadata 12. The management device obtains metadata 13 and appends it to the location indicated by the next free location address 0110 in the disk.

[0094] S203. The management device records the first mapping relationship of the first metadata in memory. The first mapping relationship of the first metadata includes the identifier of the first metadata and the address information of the first address.

[0095] The first mapping relationship of the first metadata is used to obtain the first address when querying the first metadata.

[0096] As one possible implementation, the first mapping could be a hash table. The key value (the identifier of the first metadata and the address information of the first address) can be mapped to a location in the hash table.

[0097] For example, the first mapping relationship of the metadata stored in the first area illustrated in Table 1 can be shown as one row in Table 1.

[0098] Table 1

[0099] Mapping table location Key value Mapping table address 11 Metadata 11-0010 Mapping table address 12 Metadata 12-0101 Mapping table address 13 Metadata 13-0110 Mapping table address 14 Metadata 14-0011 Mapping table address 15 Metadata 15-0001 Mapping table address 16 Metadata 16-0100

[0100] It should be noted that Table 1 is only used as an example to illustrate the first mapping relationship and does not constitute a specific limitation.

[0101] The solution provided in this application directly writes metadata to the disk in an append-only manner, and records the mapping relationship between the metadata and the storage address on the disk in memory. On the one hand, since the performance of sequential disk writes is far superior to that of random writes, directly writing metadata to the disk in an append-only manner ensures efficient metadata writing. By recording the initial mapping relationship in memory, it can be ensured that when reading metadata, the target metadata is accurately read from the disk using the address in the mapping relationship, without needing to read large amounts of data from the disk, thus ensuring efficient metadata reading. On the other hand, storing metadata on the disk ensures that the metadata will not be lost even if the hardware loses power, guaranteeing the reliability of metadata storage.

[0102] Furthermore, to improve access efficiency and prevent the metadata volume in the first region from becoming too large, a second region is configured on the disk. This second region is used for persistent storage of metadata, clearing the first region and allowing it to store the most recently acquired metadata. For example... Figure 3b As shown, the method for storing metadata provided in this application embodiment may further include S204 to S206.

[0103] S204. If the size of the metadata stored in the first area is greater than or equal to the first threshold, the management device will write the metadata stored in the first area to the metadata set in the second area of ​​the disk by appending the metadata.

[0104] The metadata set is a sub-region configured in the second region, and can be viewed as a file.

[0105] The value of the first threshold can be configured according to actual needs, and this application embodiment does not limit it. For example, the first threshold is 512 MHz.

[0106] In one possible implementation, multiple metadata sets are configured in the second region. Each metadata set has a reference information range, and the reference information ranges of different metadata sets do not overlap. Only metadata whose reference information falls within the reference information range can be stored in the corresponding metadata set. The process of appending metadata stored in the first region to the metadata set in the second region on the disk can include: appending third metadata stored in the first region to the first metadata set in the second region, where the reference information of the third metadata falls within the reference information range of the first metadata set.

[0107] For example, the reference information range includes the lexicographical order of the metadata identifiers from smallest to largest. Of course, the reference information can also be other content, which is not limited in this embodiment. By configuring the reference information range of each metadata set, when reading the metadata of the second region later, the metadata set in which the metadata belongs can be determined first, so as to quickly locate the metadata.

[0108] For example, suppose the reference information range of metadata set A in the second region is 0-13, and the reference information range of metadata set B is 14-27. Figure 1 When the metadata stored in the first area is written to the second area in an append-only manner, metadata 11, metadata 12, and metadata 13 are appended to metadata set A, and metadata 14, metadata 15, and metadata 16 are appended to metadata set B.

[0109] In one possible implementation, multiple metadata sets are configured in the second region. The metadata stored in the first region is written to the metadata set in the second region on the disk by appending. Specifically, the data in the first region is written to the metadata set in the second region in the order of storage in the first region. This application does not limit which metadata set the metadata is written to.

[0110] S205, the management device clears the first area.

[0111] Since the metadata stored in the first area has already been migrated to the second area in S204, S205 is executed to clear the first area, ensuring that the first area has sufficient space capacity for the storage system to write the latest metadata. According to the principle of temporal locality, recently written data is more likely to be read. Therefore, executing S205 effectively improves the storage system's efficiency in reading the latest written metadata.

[0112] S206, The management device deletes the first mapping relationship of the second metadata in memory. The second metadata is the metadata written to the second region.

[0113] Specifically, since the second metadata of the first region has been transferred to the second region, the first mapping relationship in memory needs to be updated in a timely manner to avoid data conflicts when new metadata is written to the first region.

[0114] Furthermore, the metadata stored in the second area can be read in the same way as data read from disk. To further improve access efficiency, when reading the metadata from the second area, the metadata set containing the metadata can be quickly retrieved from memory, such as... Figure 3b As shown, the method for storing metadata provided in this application embodiment may further include S207 or S208.

[0115] S207. The management device records a second mapping relationship of the second metadata that meets the conditions in memory. The second mapping relationship includes the identifier of the metadata and the address information of the storage address of the metadata in the second region.

[0116] In one possible implementation, the above condition is that the access popularity is greater than or equal to a third threshold. Here, access popularity refers to the number of times metadata is accessed within a specified time period.

[0117] In one possible implementation, the third threshold can be 0 or any other arbitrary value. It is understood that if the third threshold is 0, then the second mapping relationship of all second metadata is recorded in memory.

[0118] For example, the second mapping could be a hash table.

[0119] It should be noted that the second mapping relationship is similar to the first mapping relationship, except that the address information in the second mapping relationship is the address information in the second region, while the address information in the first mapping relationship is the address information in the first region.

[0120] S208. The management device records reference information of the metadata stored in the metadata set in memory.

[0121] In one possible implementation, each metadata set in the second region is configured with a reference information range. Only metadata whose reference information is within the reference information range can be stored in the corresponding metadata set. In S208, the management device can record the reference information range of the metadata set in memory.

[0122] For example, the reference information range of the metadata set recorded in memory can be as shown in Table 5. When it is necessary to read metadata 30, by querying Table 5, the storage location of metadata 30 and metadata set a can be quickly determined. Then, metadata set a can be read into memory as a whole, and metadata 30 can be found without having to traverse and query all metadata to locate metadata 30.

[0123] Table 2

[0124] Metadata collection identifier Reference information range Metadata collection a Metadata 001-Metadata 100 Metadata collection b Metadata 101-Metadata 200 Metadata collection c Metadata 201-Metadata 300 Metadata collection d Metadata 301-Metadata 400 Metadata collection e Metadata 401-Metadata 500 Metadata set f Metadata 501-Metadata 600 …… ……

[0125] In one possible implementation, when storing metadata in the metadata set in the second region, random storage is used, and in S208, the management device can record reference information of the metadata stored in the metadata set in memory. The reference information of the metadata stored in the metadata set is the same as the reference information of the metadata actually stored in the metadata set.

[0126] For example, metadata set D stores metadata 1, metadata 2, metadata 8, and metadata 20, and can record {metadata set D: metadata 1, metadata 2, metadata 8, metadata 20} in memory.

[0127] In another possible implementation, the management device records reference information of the metadata set stored in the metadata collection in a second area or other disk area capable of persistently storing data.

[0128] Furthermore, to better manage the metadata sets in the second region and prevent excessive metadata storage in any one metadata set from degrading read / write functionality, the management device will monitor the size of each metadata set in the second region. If a metadata set in the second region stores too much metadata, it can be split into metadata subsets. For example... Figure 3b As shown, the method for storing metadata provided in this application embodiment may further include S209.

[0129] S209. If the size of the metadata in the second metadata set in the second region is greater than or equal to the second threshold, the management device splits the second metadata set into multiple metadata subsets.

[0130] The second metadata set is any metadata set in the second region. The union of the reference information ranges of the metadata subsets obtained by splitting the second metadata set is the reference information range of the second metadata set.

[0131] The value of the second threshold can be configured according to actual needs and requirements. For example, the second threshold is 256 MHz.

[0132] As an example, assuming the size of the metadata set 'a' shown in Table 2 exceeds the second threshold, the management device will split the metadata set 'a' into two metadata subsets, a1 and a2. The reference information range of metadata subset a1 is metadata 001-050, and the reference information range of metadata subset a2 is metadata 051-100. The reference information range of the metadata set shown in Table 2, recorded in memory, can be transformed into the reference information range of the metadata set shown in Table 3.

[0133] Table 3

[0134] Metadata collection identifier Reference information range Metadata collection a1 Metadata 001-Metadata 050 Metadata collection a2 Metadata 051-Metadata 100 Metadata collection b Metadata 101-Metadata 200 Metadata collection c Metadata 201-Metadata 300 Metadata collection d Metadata 301-Metadata 400 Metadata collection e Metadata 401-Metadata 500 Metadata set f Metadata 501-Metadata 600 …… ……

[0135] On the other hand, embodiments of this application provide a method for querying metadata. This method for querying metadata can be used with... Figure 2 The illustrated methods for storing metadata can be used in combination or individually. For example... Figure 4 As shown, the method for querying metadata provided in this application embodiment may include:

[0136] S401, The management device obtains a query request, which includes the identifier of the fourth metadata.

[0137] Specifically, the management device may receive the query request input by the user, or obtain the query request through other means. This application does not limit the method by which the management device obtains the query request.

[0138] S402, The management device determines whether the first mapping relationship of the fourth metadata exists in the memory.

[0139] The first mapping relationship has already been described in detail in S203 and will not be repeated here. It should be noted that if the query metadata is independent of... Figure 2The illustrated method for storing metadata includes, in S402, the first mapping relationship of the fourth metadata, which includes the identifier of the fourth metadata and the storage address information of the fourth metadata in the first region of the disk. The metadata in the first region is written in an append-only manner. The append-only write is used to store the metadata after the most recently written metadata.

[0140] If the management device determines that the first mapping relationship of the fourth metadata exists in memory, it executes S403. If the management device determines that the first mapping relationship of the fourth metadata does not exist in memory, it may report a NotFound error.

[0141] S403. In the first mapping relationship of the fourth metadata obtained by the management device, the second address information corresponding to the identifier of the fourth metadata is used to read data at the location indicated by the second address information on the disk as the fourth metadata.

[0142] On the other hand, embodiments of this application provide another method for querying metadata. This method for querying metadata can be combined with... Figure 3b The illustrated methods for storing metadata can be used in combination or individually. For example... Figure 5 As shown, the method for querying metadata provided in this application embodiment may include:

[0143] S501, The management device obtains a query request, which includes the identifier of the fifth metadata.

[0144] The specific implementation process of S501 can be found in S401, and will not be elaborated on here.

[0145] S502, The management device determines whether the first mapping relationship of the fifth metadata exists in the memory.

[0146] If the management device determines that a first mapping relationship for the fifth metadata exists in memory, it executes S503. If the management device determines that a first mapping relationship for the fifth metadata does not exist in memory, it executes S504.

[0147] S503, in the first mapping relationship of the fifth metadata obtained by the management device, the third address information corresponding to the identifier of the fifth metadata is read from the location indicated by the third address information on the disk and used as the fifth metadata.

[0148] For details on the query process, please refer to S502. We will not go into further detail here.

[0149] S504. The management device reads the fifth metadata in the second area based on the identifier of the fifth metadata.

[0150] Specifically, reading the fifth metadata in the second area can include, but is not limited to, the following:

[0151] Case 1: The management device records the second mapping relationship of metadata in memory. In S504, the management device determines whether there is a second mapping relationship of fifth metadata in memory. If there is, the management device obtains the fourth address information corresponding to the identifier of the fifth metadata in the second mapping relationship of the fifth metadata, and reads the data at the location indicated by the fourth address information on the disk as the fifth metadata.

[0152] In case 1, if the second mapping relationship of the fifth metadata does not exist in memory, the management device can report a NotFound error.

[0153] Case 2: The management device records the reference information range of the metadata set of the second region in memory. In S208, the management device determines that the identifier of the fifth metadata is located in the address range of the third metadata set on the disk according to the identifier of the fifth metadata, reads the metadata in the third metadata set in the second region, and obtains the fifth metadata from it.

[0154] In scenario 2, if the reference information range of the metadata set of the second region recorded in memory does not contain the identifier of the fifth metadata, the management device may report a NotFound error.

[0155] Case 3: The management device records reference information of the metadata stored in the metadata set of the second region in memory. In S208, the management device determines that the fifth metadata is stored in the fourth metadata set based on the reference information of the metadata stored in the metadata set of the second region recorded in memory. Then, the management device reads the metadata in the fourth metadata set in the second region according to the address range of the fourth metadata set in the disk and obtains the fifth metadata from it.

[0156] In scenario 3, if the metadata reference information stored in the metadata set of the second region recorded in memory does not contain the identifier of the fifth metadata, the management device may report a NotFound error.

[0157] As can be seen, the above mainly describes the solutions provided by the embodiments of this application from a methodological perspective. To achieve the functions, the embodiments of this application provide corresponding hardware structures and / or software modules for executing each function. Those skilled in the art should readily recognize that, in conjunction with the modules and algorithm steps of the various examples described in the embodiments disclosed herein, the embodiments of this application can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed in hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this invention.

[0158] This application embodiment can divide the management device into functional modules according to the above method example. For example, each function can be divided into its own functional module, or two or more functions can be integrated into one processing module. The integrated module can be implemented in hardware or as a software functional module. Optionally, the module division in this application embodiment is illustrative and only represents one logical functional division; other division methods may be used in actual implementation.

[0159] This application provides a management device for storing metadata, such as... Figure 6 As shown, the management device 60 may include: an acquisition module 601, a writing module 602, and a recording module 603.

[0160] The acquisition module 601 is used to acquire first metadata, which indicates the storage location of a data file stored on a disk. For example, this acquisition module 601 can be used to support the management device 60 in performing [operations / tasks]. Figure 2 The process of S201.

[0161] The write module 602 is used to append first metadata to a first address of a first region on the disk used for temporary storage of metadata. The append write operation stores the metadata after the most recently written metadata, and the first address is the address following the most recently written metadata in the first region. For example, this write module 602 can be used to support the management device 60 in performing... Figure 2 The process of S202.

[0162] The recording module 603 is used to record a first mapping relationship of the first metadata in memory. The first mapping relationship includes the identifier of the first metadata and address information of a first address. This first mapping relationship is used to retrieve the first address when querying the first metadata. For example, the recording module 603 can be used to support the management device 60 in execution. Figure 2 The process of S203.

[0163] Furthermore, such as Figure 7 As shown, the management device 60 may further include: a deletion module 604, a splitting module 605, and a reading module 606.

[0164] In one possible implementation, the write module 602 is further configured to, if the size of the metadata stored in the first region is greater than or equal to a first threshold, write the metadata stored in the first region to the metadata set in the second region of the disk via an append-only write operation. For example, this write module 602 can be used to support the management device 60 in performing... Figure 3b The process of S204.

[0165] The deletion module 604 is used to clear the first region, and also to delete the first mapping relationship of the second metadata in memory, wherein the second metadata is the metadata written to the second region. For example, this deletion module 604 can be used to support the management device 60 in execution. Figure 3b The processes of S205 and S206.

[0166] In one possible implementation, the writing module 602 is specifically used to write the third metadata stored in the first region into the first metadata set in the second region by appending the third metadata, wherein the reference information of the third metadata is located within the reference information range of the first metadata set.

[0167] In one possible implementation, the recording module 603 is further configured to record in memory a second mapping relationship of the second metadata that meets the conditions; the second mapping relationship includes the identifier of the metadata and the address information of the storage address of the metadata in the second region; or, the recording module 603 is further configured to record in memory a reference information range of the metadata set. Exemplarily, the recording module 603 can be used to support the management device 60 in execution. Figure 3b The processes in S207 and S208.

[0168] The splitting module 605 is configured to split the second metadata set into multiple metadata subsets if the size of the metadata in the second metadata set in the second region is greater than or equal to a second threshold. The union of the reference information ranges of the metadata subsets obtained from the splitting of the second metadata set is the reference information range of the second metadata set. For example, this splitting module 605 can be used to support the management device 60 in performing... Figure 3b The process of S209.

[0169] In one possible implementation, the acquisition module 601 is further configured to acquire a query request, the query request including the identifier of fourth metadata. For example, the acquisition module 601 can be used to support the management device 60 in execution. Figure 4 The process of S401.

[0170] The reading module 606 is configured to, if a first mapping relationship for fourth metadata exists in memory, obtain the second address information corresponding to the identifier of the fourth metadata in the first mapping relationship, and read data from the disk at the location indicated by the second address information as the fourth metadata. For example, this reading module 606 can be used to support the management device 60 in performing [operations / tasks]. Figure 4 The process of S403.

[0171] In one possible implementation, the acquisition module 601 is further configured to acquire a query request, which includes the identifier of the fifth metadata. For example, the acquisition module 601 can be used to support the management device 60 in performing [operations / tasks]. Figure 5 The process of S501.

[0172] The reading module 606 is further configured to, if a first mapping relationship for the fifth metadata exists in memory, obtain the third address information corresponding to the identifier of the fifth metadata in the first mapping relationship for the fifth metadata, and read data from the location indicated by the third address information on the disk as the fifth metadata; the reading module 606 is also configured to, if a first mapping relationship for the fifth metadata does not exist in memory, read the fifth metadata in the second area according to the identifier of the fifth metadata. For example, this reading module 606 can be used to support the management device 60 in performing... Figure 5 The process of S503 or S504.

[0173] In one possible implementation, the reading module 606 is further configured to record a second mapping relationship of metadata in memory, and read the fifth metadata in the second area according to the identifier of the fifth metadata; the reading module 606 is further configured to obtain the fourth address information corresponding to the identifier of the fifth metadata in the second mapping relationship of the fifth metadata, and read data at the location indicated by the fourth address information on the disk as the fifth metadata.

[0174] In one possible implementation, the reading module 606 is specifically used to read the fifth metadata based on the address range of the third metadata set on the disk. The reference information range of the metadata set is recorded in memory, and the identifier of the fifth metadata is within the reference information range of the third metadata set. Based on the identifier of the fifth metadata, the fifth metadata is read in the second region.

[0175] Furthermore, embodiments of this application provide a metadata management device, such as... Figure 8 As shown, the management device 80 may include an acquisition module 801 and a reading module 802.

[0176] The acquisition module 801 is used to acquire a query request, which includes an identifier of fourth metadata; the fourth metadata is used to indicate the storage location of a data file stored on a disk. For example, this acquisition module 801 can be used to support the management device 60 in execution. Figure 4 The process of S401.

[0177] The acquisition module 801 is also used to acquire, if there is a first mapping relationship for the fourth metadata in memory, the second address information corresponding to the identifier of the fourth metadata in the first mapping relationship for the fourth metadata.

[0178] The read module 802 is used to read data from the location indicated by the second address information on the disk, as fourth metadata. The first mapping relationship of the fourth metadata includes the identifier of the fourth metadata and the storage address information of the fourth metadata in a first region on the disk, where the metadata is written in an append-only manner. Append-only writing is used to store data after the most recently written metadata. For example, this read module 802 can be used to support the execution of management device 60. Figure 4 The process of S403.

[0179] In one possible implementation, the reading module 802 is further configured to read the fifth metadata in a second region of the disk based on the identifier of the fourth metadata if the first mapping relationship of the fourth metadata does not exist in memory. Wherein, if the size of the metadata stored in the first region is greater than or equal to a first threshold, the metadata stored in the first region is appended to the metadata set in the second region of the disk. Exemplarily, this reading module 802 can be used to support the management device 60 in execution. Figure 5 The process of S504.

[0180] In one possible implementation, a second mapping relationship for metadata that meets the conditions is recorded in memory. The second mapping relationship includes the identifier of the metadata and the address information of the metadata's storage address in the second region. The reading module 802 is further configured to obtain the fourth address information corresponding to the identifier of the fourth metadata in the second mapping relationship of the fourth metadata, and read data from the location indicated by the fourth address information on the disk as the fourth metadata.

[0181] In another possible implementation, multiple metadata sets are configured in the second region. Each metadata set has a reference information range, and the reference information ranges of different metadata sets do not overlap. The reference information ranges of the metadata sets are recorded in memory, and the identifier of the fourth metadata is within the reference information range of the third metadata set. The reading module 802 is further configured to read the fourth metadata based on the address range of the third metadata set on the disk.

[0182] Furthermore, an embodiment of this application provides a schematic diagram of the structure of a server, as shown below. Figure 9 As shown, the server 90 may include a processor 901, and optionally, a memory 902 connected to the processor 901.

[0183] The processor 901 is used to execute the steps of any of the methods for storing metadata or querying metadata provided in the above embodiments.

[0184] Processor 901 can be a CPU, NP, DSP, microprocessor, microcontroller, PLD, or any combination thereof. Processor 901 can also be any other device with processing capabilities, such as a circuit, device, or software module. Processor 901 can also include multiple CPUs, and processor 901 can be a single-core processor or a multi-core processor. Here, "processor" can refer to one or more devices, circuits, or processing cores used to process data (e.g., computer program instructions).

[0185] The memory 902 can be a read-only memory (ROM) or other type of static storage device capable of storing static information and instructions, random access memory (RAM) or other type of dynamic storage device capable of storing information and instructions, or it can be an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.), magnetic disk storage media or other magnetic storage devices, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer. This application embodiment does not impose any limitations on this. The memory 902 can exist independently or be integrated with the processor 901. The memory 902 may contain computer program code. The processor 901 is used to execute the computer program code stored in the memory 902, thereby implementing any method for storing metadata or any method for querying metadata according to this application embodiment.

[0186] This application also provides a computer-readable storage medium, including computer-executable instructions that, when run on a computer, cause the computer to execute any of the methods for storing metadata or any of the methods for querying metadata provided in the above embodiments.

[0187] This application also provides a computer program product containing computer execution instructions, which, when run on a computer, causes the computer to execute any of the methods for storing metadata or any of the methods for querying metadata provided in the above embodiments.

[0188] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented using software programs, implementation can be, in whole or in part, in the form of a computer program product. This computer program product includes one or more computer-executable instructions. When these computer-executable instructions are loaded and executed on a computer, all or part of the flow or function according to the embodiments of this application is generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer-executable instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, computer-executable instructions can be transmitted from one website, computer, server, or data center to another via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium accessible to a computer or a data storage device containing one or more servers, data centers, etc., that can be integrated with the medium. The available media can be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., DVDs), or semiconductor media (e.g., solid-state disks, SSDs).

[0189] Although this application has been described herein in conjunction with various embodiments, those skilled in the art, by reviewing the accompanying drawings, the disclosure, and the appended claims, will understand and implement other variations of the disclosed embodiments in carrying out the claimed application. In the claims, the word "comprising" does not exclude other components or steps, and "a" or "an" does not exclude multiple instances. A single processor or other unit can implement several functions listed in the claims. While different dependent claims may recite certain measures, this does not mean that these measures cannot be combined to produce good results.

[0190] Although this application has been described in conjunction with specific features and embodiments, it is obvious that various modifications and combinations can be made thereto without departing from the spirit and scope of this application. Accordingly, this specification and drawings are merely exemplary illustrations of this application as defined by the appended claims, and are considered to cover any and all modifications, variations, combinations, or equivalents within the scope of this application. Clearly, those skilled in the art can make various alterations and modifications to this application without departing from the spirit and scope of this application. Thus, if such modifications and modifications of this application fall within the scope of the claims of this application and their equivalents, this application is also intended to include such modifications and modifications.

[0191] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for storing metadata, characterized in that, Applied to a server; the method includes: Obtain first metadata, which is used to indicate the storage location of the data file stored on the disk; The first metadata is written to the first address of the first area in the disk used for temporarily storing metadata by appending; the append write is used to store after the most recently written metadata, and the first address is the next address after the most recently written metadata in the first area; A first mapping relationship of the first metadata is recorded in memory. The first mapping relationship of the first metadata includes the identifier of the first metadata and the address information of the first address. The first mapping relationship of the first metadata is used to obtain the first address when querying the first metadata. The first mapping relationship is used by the server in response to a query request carrying an identifier of the first metadata, to obtain the first address information corresponding to the identifier of the first metadata from the first mapping relationship, so as to read data at the location indicated by the first address information on the disk as the first metadata.

2. The method according to claim 1, characterized in that, The method further includes: If the size of the metadata stored in the first area is greater than or equal to the first threshold, the metadata stored in the first area is written to the metadata set in the second area of ​​the disk by appending. Clear the first area; Delete the first mapping relationship of the second metadata in the memory, where the second metadata is the metadata written to the second region.

3. The method according to claim 2, characterized in that, The second region is configured with multiple metadata sets. Each metadata set has a reference information range, and the reference information ranges of different metadata sets do not overlap. The step of appending the metadata stored in the first region to the metadata set in the second region of the disk includes: The third metadata stored in the first region is appended to the first metadata set in the second region, and the reference information of the third metadata is located within the reference information range of the first metadata set.

4. The method according to claim 3, characterized in that, The range of the reference information includes the range of metadata identifiers in lexicographical order from smallest to largest.

5. The method according to any one of claims 2-4, characterized in that, The method further includes: A second mapping relationship for the second metadata that meets the conditions is recorded in the memory; the second mapping relationship includes the identifier of the metadata and the address information of the storage address of the metadata in the second region; or, The reference information range of the metadata set is recorded in the memory.

6. The method according to any one of claims 2-4, characterized in that, The method further includes: If the size of the metadata in the second metadata set in the second region is greater than or equal to the second threshold, the second metadata set is split into multiple metadata subsets, and the union of the reference information ranges of the metadata subsets obtained by splitting the second metadata set is the reference information range of the second metadata set.

7. The method according to claim 5, characterized in that, The method further includes: Obtain a query request, the query request including the identifier of the fifth metadata; If a first mapping relationship for the fifth metadata exists in the memory, obtain the third address information corresponding to the identifier of the fifth metadata in the first mapping relationship for the fifth metadata, and read the data at the location indicated by the third address information on the disk as the fifth metadata; If the first mapping relationship of the fifth metadata does not exist in the memory, the fifth metadata is read in the second region according to the identifier of the fifth metadata.

8. The method according to claim 7, characterized in that, The memory records a second mapping relationship of metadata. Reading the fifth metadata in the second region based on the identifier of the fifth metadata includes: In the second mapping relationship of the fifth metadata, the fourth address information corresponding to the identifier of the fifth metadata is obtained, and data is read from the location indicated by the fourth address information on the disk as the fifth metadata.

9. The method according to claim 7, characterized in that, The memory records the reference information range of the metadata set, and the identifier of the fifth metadata is within the reference information range of the third metadata set. Reading the fifth metadata in the second region based on the identifier of the fifth metadata includes: The fifth metadata is read based on the address range of the third metadata set on the disk.

10. A method for querying metadata, characterized in that, The method includes: Obtain a query request, the query request including an identifier of fourth metadata; the fourth metadata is used to indicate the storage location of the data file stored on the disk; If a first mapping relationship for the fourth metadata exists in memory, obtain the second address information corresponding to the identifier of the fourth metadata in the first mapping relationship for the fourth metadata, and read data at the location indicated by the second address information on the disk as the fourth metadata; the first mapping relationship for the fourth metadata includes the identifier of the fourth metadata and the storage address information of the fourth metadata in a first area on the disk, and the metadata in the first area is written in an append-only manner; the append-only write is used to store after the most recently written metadata.

11. The method according to claim 10, characterized in that, The method further includes: If the first mapping relationship of the fourth metadata does not exist in the memory, the fourth metadata is read in the second area of ​​the disk according to the identifier of the fourth metadata; When the size of the metadata stored in the first area is greater than or equal to the first threshold, the metadata stored in the first area is written to the metadata set in the second area on the disk by appending.

12. The method according to claim 11, characterized in that, The memory records a second mapping relationship of metadata that meets the conditions. The second mapping relationship includes the identifier of the metadata and the address information of the storage address of the metadata in the second region. The step of reading the fourth metadata in the second area of ​​the disk based on the identifier of the fourth metadata includes: Obtain the fourth address information corresponding to the identifier of the fourth metadata in the second mapping relationship of the fourth metadata; Data is read from the location indicated by the fourth address information on the disk and used as the fourth metadata.

13. The method according to claim 11, characterized in that, The second region is configured with multiple metadata sets. Each metadata set has a reference information range. The reference information ranges of different metadata sets do not overlap. The memory records the reference information ranges of the metadata sets. The identifier of the fourth metadata set is within the reference information range of the third metadata set. The step of reading the fourth metadata in the second region based on the identifier of the fourth metadata includes: The fourth metadata is read based on the address range of the third metadata set on the disk.

14. An apparatus for storing metadata, characterized in that, Applied to a server; the device includes: The acquisition module is used to acquire first metadata, which is used to indicate the storage location of the data file stored on the disk; The writing module is used to write the first metadata to a first address of a first area in the disk used for temporarily storing metadata by appending the first metadata; the append write is used to store the metadata after the most recently written metadata, and the first address is the next address after the most recently written metadata in the first area; The recording module records a first mapping relationship of the first metadata in memory. The first mapping relationship of the first metadata includes the identifier of the first metadata and the address information of the first address. The first mapping relationship of the first metadata is used to obtain the first address when querying the first metadata. The first mapping relationship is used by the server to obtain the first address information corresponding to the identifier of the first metadata from the first mapping relationship in response to a query request carrying the identifier of the first metadata, so as to read data at the location indicated by the first address information on the disk as the first metadata.

15. The apparatus according to claim 14, characterized in that, The writing module is further configured to, if the size of the metadata stored in the first region is greater than or equal to a first threshold, write the metadata stored in the first region to the metadata set in the second region of the disk by appending the metadata; the device further includes a deletion module, configured to clear the first region; the deletion module is further configured to delete the first mapping relationship of the second metadata in the memory, wherein the second metadata is the metadata written to the second region; The writing module is specifically used to write the third metadata stored in the first region into the first metadata set in the second region by appending the third metadata, wherein the reference information of the third metadata is located within the reference information range of the first metadata set; The range of the reference information includes the range of metadata identifiers in lexicographical order from smallest to largest; The recording module is further configured to record in the memory a second mapping relationship of the second metadata that meets the conditions; the second mapping relationship includes the identifier of the metadata and the address information of the storage address of the metadata in the second region; or, the recording module is further configured to record in the memory a reference information range of the metadata set; The device further includes a splitting module, used to split the second metadata set into multiple metadata subsets if the size of the metadata in the second metadata set in the second region is greater than or equal to a second threshold, wherein the union of the reference information ranges of the metadata subsets obtained by splitting the second metadata set is the reference information range of the second metadata set. The acquisition module is also used to acquire a query request, the query request including an identifier of fourth metadata; The device further includes a reading module, configured to, if a first mapping relationship of the fourth metadata exists in the memory, obtain second address information corresponding to the identifier of the fourth metadata in the first mapping relationship of the fourth metadata, and read data at the location indicated by the second address information on the disk as the fourth metadata; The acquisition module is further configured to acquire a query request, the query request including the identifier of the fifth metadata; the reading is further configured to, if a first mapping relationship of the fifth metadata exists in the memory, acquire the third address information corresponding to the identifier of the fifth metadata in the first mapping relationship of the fifth metadata, and read data at the location indicated by the third address information on the disk as the fifth metadata; The reading module is further configured to read the fifth metadata in the second region according to the identifier of the fifth metadata if the first mapping relationship of the fifth metadata does not exist in the memory; The reading module is also used to record a second mapping relationship of metadata in the memory, and to read the fifth metadata in the second area according to the identifier of the fifth metadata; The reading module is further configured to obtain the fourth address information corresponding to the identifier of the fifth metadata in the second mapping relationship of the fifth metadata, and read data at the location indicated by the fourth address information on the disk as the fifth metadata; The memory records the reference information range of the metadata set, the identifier of the fifth metadata is within the reference information range of the third metadata set, and the fifth metadata is read in the second region according to the identifier of the fifth metadata; The reading module is specifically used to read the fifth metadata based on the address range of the third metadata set in the disk.

16. An apparatus for querying metadata, characterized in that, The device includes: The acquisition module is used to: acquire a query request, wherein the query request includes an identifier of fourth metadata; the fourth metadata is used to indicate the storage location of a data file stored on a disk; The acquisition module is further configured to: if a first mapping relationship of the fourth metadata exists in memory, acquire the second address information corresponding to the identifier of the fourth metadata in the first mapping relationship of the fourth metadata; The reading module is configured to: read data from the location indicated by the second address information on the disk, as the fourth metadata; the first mapping relationship of the fourth metadata includes the identifier of the fourth metadata and the storage address information of the fourth metadata in a first region on the disk, wherein the metadata in the first region is written in an append-only manner; the append-only write is used to store after the most recently written metadata, and the first address is the next address after the most recently written metadata in the first region.

17. The apparatus according to claim 16, characterized in that, The reading module is further configured to: if the first mapping relationship of the fourth metadata does not exist in the memory, read the fourth metadata in the second area of ​​the disk according to the identifier of the fourth metadata; wherein, when the size of the metadata stored in the first area is greater than or equal to the first threshold, the metadata stored in the first area is written to the metadata set in the second area of ​​the disk by appending; The memory records a second mapping relationship of metadata that meets the conditions. The second mapping relationship includes the identifier of the metadata and the address information of the storage address of the metadata in the second region. The acquisition module is further configured to: acquire the fourth address information corresponding to the identifier of the fourth metadata in the second mapping relationship of the fourth metadata; the reading module is further configured to: read data at the location indicated by the fourth address information in the disk as the fourth metadata. The second region is configured with multiple metadata sets, each metadata set having a reference information range, and the reference information ranges of different metadata sets do not overlap. The memory records the reference information ranges of the metadata sets, and the identifier of the fourth metadata is within the reference information range of the third metadata set. The reading module is further configured to: read the fourth metadata according to the address range of the third metadata set on the disk.

18. A server, characterized in that, Includes one or more processors and one or more memories; The one or more memories are coupled to the one or more processors, the one or more memories being used to store computer program code, the computer program code including computer instructions, and when the one or more processors execute the computer instructions, the server performs the method as described in any one of claims 1-13.

19. A computer storage medium, characterized in that, Includes computer instructions that, when executed on a server, cause the server to perform the method as described in any one of claims 1-13.

20. A computer program product, characterized in that, When the computer program product is run on a computer, it causes the computer to perform the method as described in any one of claims 1-13.

Images