Method and device for storing metadata on disk

Abstract

The present application discloses a method and device for storing metadata on a disk, and relates to the technical field of storage software. Since a virtual address set and an address set identifier with a corresponding relationship are configured, and a virtual address and a physical address with a corresponding relationship are stored in the address set identifier, after obtaining the virtual addresses of multiple target metadata, a corresponding address set identifier is matched for the virtual address set where each target virtual address is located from the virtual address set and the address set identifier with a corresponding relationship, and then the physical address corresponding to each target virtual address is searched in parallel from the target address set indicated by each target set identifier. The physical address corresponding to the virtual address is searched in parallel in each small range instead of the serial search for the physical address in a large range including multiple small ranges, which speeds up the addressing speed of the metadata, solves the technical problem of low efficiency of storing metadata on the disk, and achieves the technical effect of improving the efficiency of storing metadata on the disk.

Description

Technical Field

[0001] The present application relates to the field of storage software technology, and in particular to a method and device for storing metadata on a disk. Background Art

[0002] Metadata, as "data of data", describes the characteristics, structure, location, permissions and other information of data, which can help us better organize, find and understand data. Metadata disk management refers to persisting metadata from memory to external physical storage such as disk. This is crucial to ensure the integrity and persistence of data, especially in the field of data storage and management. It can ensure that even in the event of a system failure, the metadata of the data will not be lost, thereby ensuring the recoverability and consistency of the data. The current storage of metadata depends on the serial search of multiple metadata external storage space addresses. In the case of a large amount of metadata to be stored on disk, this storage method will inevitably consume a lot of time resources and operation resources, and the efficiency of metadata storage is not high. Summary of the invention

[0003] The present application provides a method and device for storing metadata on a disk, so as to at least solve the problem of low efficiency of storing metadata on a disk in the related art.

[0004] The present application provides a method for writing metadata to a disk, comprising: obtaining virtual addresses of multiple target metadata to be written to the disk in a memory to obtain multiple target virtual addresses; matching a corresponding address set identifier for a virtual address set where each target virtual address is located from a virtual address set and an address set identifier having a corresponding relationship to obtain multiple target set identifiers, wherein a physical address set indicated by the address set identifier stores the physical addresses corresponding to each virtual address in the corresponding virtual address set in a target external storage space; searching for the physical address corresponding to each target virtual address from the target address set indicated by each target set identifier in parallel to obtain multiple target physical addresses; and writing the multiple target metadata to the corresponding target physical addresses in the target external storage space.

[0005] The present application also provides a metadata storage device, including: an acquisition module, used to acquire virtual addresses of multiple target metadata to be stored in the memory, and obtain multiple target virtual addresses; a matching module, used to match the corresponding address set identifier for the virtual address set where each target virtual address is located from the virtual address set and the address set identifier with a corresponding relationship, and obtain multiple target set identifiers, wherein the physical address set indicated by the address set identifier stores the physical addresses corresponding to each virtual address in the corresponding virtual address set in the target external storage space; a search module, used to search in parallel for the physical address corresponding to each target virtual address from the target address set indicated by each target set identifier, and obtain multiple target physical addresses; a storage module, used to store multiple target metadata on the disk to the corresponding target physical addresses in the target external storage space.

[0006] The present application also provides an electronic device, including: a memory for storing a computer program; and a processor for implementing the steps of any of the above-mentioned metadata storage methods when executing the computer program.

[0007] The present application also provides a computer-readable storage medium, in which a computer program is stored, wherein when the computer program is executed by a processor, the steps of any of the above-mentioned metadata storage methods are implemented.

[0008] The present application also provides a computer program product, including a computer program, which implements the steps of any of the above-mentioned metadata storage methods when the computer program is executed by a processor.

[0009] Through the present application, since a virtual address set and an address set identifier with a corresponding relationship are configured, and a virtual address and a physical address with a corresponding relationship are stored in the address set identifier, after the virtual addresses of multiple target metadata to be written to the disk in the memory are obtained, the corresponding address set identifier will be matched for the virtual address set where each target virtual address is located from the virtual address set and the address set identifier with a corresponding relationship, and then the physical address corresponding to each target virtual address will be searched in parallel from the target address set indicated by each target set identifier. The method of searching for the physical address corresponding to the virtual address in each small range in parallel replaces the method of searching for the physical address in a large range including multiple small ranges in serial, thereby accelerating the addressing speed of the metadata. Therefore, the technical problem of low efficiency of metadata writing to the disk can be solved, and the technical effect of improving the efficiency of metadata writing to the disk can be achieved. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] In order to more clearly illustrate the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0011] Figure 1 It is a hardware structure block diagram of a server device of a method for storing metadata on a disk according to an embodiment of the present application;

[0012] Figure 2 is a flowchart of a method for storing metadata on a disk according to an embodiment of the present application;

[0013] Figure 3 It is a schematic diagram of a method for storing metadata on a physical disk according to an embodiment of the present application;

[0014] Figure 4 It is a mapping relationship between a physical address set and a memory and a physical disk according to an embodiment of the present application;

[0015] Figure 5 This is a schematic diagram of adjusting the capacity of a physical address set according to an embodiment of the present application. Figure 1 ;

[0016] Figure 6 This is a schematic diagram of adjusting the capacity of a physical address set according to an embodiment of the present application. Figure 2 ;

[0017] Figure 7 It is a structural block diagram of a metadata disk-writing device according to an embodiment of the present application. DETAILED DESCRIPTION

[0018] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0019] It should be noted that, in the description of this application, the terms "include", "comprise" or any other variant thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. The terms "first", "second", etc. in this application are used to distinguish similar objects, and are not used to describe a specific order or sequence.

[0020] In order to enable those skilled in the art to better understand the present application, the present application is further described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0021] In conjunction with the specific application environment architecture or specific hardware architecture on which the execution of the metadata storage method depends, the specific application environment architecture or specific hardware architecture is described herein.

[0022] The method embodiments provided in the embodiments of the present application can be executed in a server device or a similar computing device. Taking running on a server device as an example, Figure 1 1 is a hardware structure block diagram of a server device of a method for storing metadata on a disk according to an embodiment of the present application. Figure 1 As shown, the server device may include one or more ( Figure 1 Only one is shown in the figure) a processor 102 (the processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA) and a memory 104 for storing data, wherein the above-mentioned server device may also include a transmission device 106 and an input / output device 108 for communication functions. It can be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the above server device. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations shown.

[0023] The memory 104 can be used to store computer programs, for example, software programs and modules of application software, such as the computer program corresponding to the startup method of the operating system in the embodiment of the present application. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, that is, to implement the above method. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include a memory remotely arranged relative to the processor 102, and these remote memories can be connected to the server device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof.

[0024] The transmission device 106 is used to receive or send data via a network. The specific example of the above network may include a wireless network provided by a communication provider of the server device. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, referred to as NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 can be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0025] The embodiment of the present application provides a method for writing metadata to a disk, and the method is described in detail in conjunction with the execution flow of the method for writing metadata to a disk.

[0026] The following is an explanation of the professional terms that appear in this application:

[0027] Metadata definition: Metadata is "data about data", which describes the characteristics, structure, content, context, quality, format, storage location, owner, access rights and other information of data. The role of metadata is to help manage and understand data. It can be regarded as a kind of auxiliary information to support data retrieval, access, control, integration and maintenance. In data management and information systems, metadata is a very critical component. It can improve the discoverability, accessibility, understandability and manageability of data, thereby ensuring the effective use of data and the implementation of data governance.

[0028] Definition of disk write: disk write refers to the process of persisting data stored in volatile storage (such as memory) to non-volatile storage media (such as hard disk, SSD, etc.). In computer systems, memory is volatile, that is, after the system is powered off or restarted, the data stored in it will be lost, while external storage devices such as hard disks are persistent storage places for data. Therefore, disk write is a key step to ensure data security and persistence. It writes data from memory to disk, and even if the system fails, the data will not be lost.

[0029] Memory definition: Also known as main memory or RAM (random access memory), it is a volatile memory used to store running programs and data in a computer. Memory communicates directly with the CPU, and its read and write speed is much faster than external storage devices such as hard disks. It is a fast temporary storage space when the computer processes data. Memory has a fast access speed, but the data will disappear after power failure, so it is necessary to cooperate with the disk drop mechanism to persist important data.

[0030] External storage space definition: External storage space refers to the storage space provided by storage devices other than memory in a computer system, such as hard disk drives, solid-state drives (SSDs), USB flash drives, network storage, etc. These storage devices usually provide large-capacity persistent storage, and data is retained after power failure. Due to their slow speed, they are mainly used to store large amounts of data that are not frequently accessed or do not need to be frequently modified.

[0031] In this embodiment, a method for storing metadata on a disk is provided. Figure 2 is a flowchart of a method for storing metadata according to an embodiment of the present application. Figure 2 As shown, the method comprises the following steps:

[0032] Step S202, acquiring virtual addresses of multiple target metadata to be written to the disk in the memory, and obtaining multiple target virtual addresses;

[0033] Step S204, matching the corresponding address set identifier for the virtual address set where each target virtual address is located from the virtual address sets and address set identifiers having a corresponding relationship, and obtaining a plurality of target set identifiers, wherein the physical address set indicated by the address set identifier stores the physical addresses corresponding to each virtual address in the corresponding virtual address set in the target external storage space;

[0034] Step S206, searching the physical address corresponding to each target virtual address from the target address sets indicated by each target set identifier in parallel to obtain multiple target physical addresses;

[0035] Step S208: write the plurality of target metadata to the corresponding target physical addresses in the target external storage space.

[0036] Through the above steps, since a virtual address set and an address set identifier with a corresponding relationship are configured, and a virtual address and a physical address with a corresponding relationship are stored in the address set identifier, after the virtual addresses of multiple target metadata to be written to the disk in the memory are obtained, the corresponding address set identifier will be matched for the virtual address set where each target virtual address is located from the virtual address set and the address set identifier with a corresponding relationship, and then the physical address corresponding to each target virtual address will be searched in parallel from the target address sets indicated by each target set identifier. The method of searching for the physical address corresponding to the virtual address in each small range in parallel replaces the method of searching for the physical address in a large range including multiple small ranges in serial, which speeds up the addressing speed of the metadata. Therefore, the technical problem of low efficiency of metadata writing to the disk can be solved, and the technical effect of improving the efficiency of metadata writing to the disk can be achieved.

[0037] Optionally, in this embodiment, the method of writing metadata to disk includes but is not limited to being executed collaboratively by one or more components including a central processing unit (CPU), a memory management unit (MMU), a direct memory access controller (DMA Controller), an operating system (OS), a file system (File System), a dedicated storage controller, a multi-core processor, or a parallel processing unit.

[0038] Optionally, in this embodiment, including but not limited to designing an intelligent metadata management module (Smart Metadata Management, SMM) that mainly uses ARM and other programmable logic devices on the board, cooperate with the detection module, front-end module, main control module, back-end module and disk array module to jointly execute the metadata disk method in this application. The SMM module can dynamically manage the detection module (used to detect CPU utilization, memory utilization, CPU model, memory model, etc.), the front-end module (used for IO queue management), the main control module (used for metadata management), the back-end module (used for metadata disk management) and the disk array module (used for hard disk management). The front-end module processes IO read and write requests in real time, the detection module obtains the data of the relevant detection object and passes it to the main control module, the main control module processes the current environment configuration data, selects the corresponding metadata processing method, and the back-end module executes the metadata processing method of the main control module to implement the metadata disk write operation. In addition, SMM can also synchronize the current intelligent metadata management strategy in real time through the detection module, front-end module, main control module, back-end module, and disk array module.

[0039] In the embodiment provided in step S202, the metadata storage is the process of persisting the metadata from the memory to the target external storage space. The metadata storage mentioned in this application includes but is not limited to the initial storage of metadata to the target external storage space, or the storage of the modified metadata to the target external storage space after the metadata changes. This application does not make specific limitations on this.

[0040] Optionally, in this embodiment, the virtual address is used to indicate the storage location of the target metadata in the memory.

[0041] Optionally, in this embodiment, obtaining the virtual addresses of multiple target metadata to be written to the disk in the memory includes but is not limited to being implemented through metadata management at the software level. Specifically, in the memory, the mapping relationship of the metadata is maintained through data structures (such as hash tables, trees, linked lists, etc.). When metadata needs to be written to the disk, the software (such as an operating system, a file system, or an application) obtains the virtual address of the metadata in the memory by querying these data structures. For example, a metadata index table can be maintained, which contains an identifier of each metadata and its corresponding memory virtual address. When the software performs a metadata write operation, the address acquisition mechanism can be automatically triggered. For example, when a system call or API function is called to write metadata, metadata search and virtual address acquisition are performed inside these functions. Specifically, this may involve traversing the data structure of metadata management until the record of the target metadata is found and the virtual address is read from it. The specific steps implemented at the software level include: maintaining a hash table with the metadata ID as the key and the virtual address as the value; when there is a request to write metadata to the disk, the software queries the hash table through the metadata ID to obtain its virtual address; using the obtained virtual address, the software reads the metadata from the memory and then writes it to the target external storage space.

[0042] Optionally, in this embodiment, obtaining the virtual addresses of multiple target metadata to be written to the disk in the memory also includes but is not limited to being achieved through acceleration and optimization at the hardware level. Specifically, at the hardware level, the address acquisition process can be optimized by a dedicated hardware acceleration unit. For example, the aforementioned intelligent metadata management module (SMM) can be integrated with a hardware accelerator for quickly finding and obtaining the virtual address of metadata. When the system detects a metadata write operation, the hardware accelerator can directly retrieve the virtual address of the target metadata from the internal cache or dedicated storage without the need for the CPU to perform complex software processing. The specific steps implemented at the hardware level include: pre-loading the metadata management structure inside the SMM module for quickly finding the virtual address of the metadata. When a metadata write signal is received, the SMM module directly searches for the virtual address of the target metadata from the preloaded data structure through internal hardware logic.

[0043] In the embodiment provided in step S204, the virtual address set, i.e., the set of virtual addresses in the memory, may include, but is not limited to, one or more virtual address segments, or multiple non-contiguous virtual addresses, or a combination of the above two, and the present application does not make any specific limitation on this.

[0044] Optionally, in this embodiment, the address set identifier is used to uniquely identify (or uniquely indicate) a physical address set, and a physical address set has its uniquely determined address set identifier. For example, the address set identifier of physical address set 1 is Page1, and the address set identifier of physical address set 2 is Page2.

[0045] Optionally, in this embodiment, each physical address is used to indicate each storage location in the target external storage space.

[0046] Optionally, in this embodiment, the physical address set stores virtual addresses and physical addresses with corresponding relationships, and the virtual addresses and physical addresses with corresponding relationships stored in each physical address set are different.

[0047] Optionally, in this embodiment, the capacity of corresponding virtual addresses and physical addresses that can be stored in each physical address set can be the same or different, including but not limited to setting the same capacity for each physical address set when the access requirements for each metadata are not very different, and setting different capacities for each physical address set when the access requirements for each metadata are very different.

[0048] Optionally, in this embodiment, matching the corresponding address set identifier to the virtual address set where each target virtual address is located from the virtual address sets and address set identifiers having a corresponding relationship includes but is not limited to being implemented in any one of the following three ways:

[0049] Implementation method 1: Use hash table for fast matching:

[0050] When the system is initialized, a hash table is created for each virtual address set. The key of the hash table is the virtual address set identifier, and the value is the corresponding physical address set identifier. When the target virtual address is obtained, the virtual address set identifier to which the address belongs is first determined, and then the hash table is searched to quickly match the corresponding physical address set identifier.

[0051] Implementation method 2: Multi-level index based on tree structure:

[0052] Use a tree structure (such as B-tree, B+ tree or Trie tree) to build an index, where the nodes of the index store the mapping relationship between the virtual address set identifier and the corresponding physical address set identifier, and the leaf nodes store the specific physical address information. For each target virtual address to be stored on disk, according to the virtual address set identifier to which it belongs, start from the root node and recursively traverse the tree structure until the corresponding physical address set identifier is found.

[0053] Implementation method three: using a dedicated hardware accelerator:

[0054] Design a dedicated hardware accelerator to accelerate the addressing and disk-storage process of metadata. During the initialization phase, the mapping relationship between the virtual address set identifier and the physical address set identifier is loaded into the cache of the hardware accelerator. When the target virtual address is received, the hardware accelerator directly performs matching and search operations without CPU intervention at the software level. The accelerator quickly finds the corresponding address set identifier through internal comparison and logical operations, and directly accesses the hardware area that stores the physical address set.

[0055] Each of the above implementations has its own specific application scenarios and advantages. The hash table implementation provides fast matching when data is randomly distributed, the tree structure implementation is suitable for scenarios where data is orderly distributed and requires hierarchical recursive search, and the dedicated hardware accelerator performs well in high-performance and high-concurrency applications. In actual applications, the most appropriate implementation can be selected based on the system's performance requirements, data distribution characteristics, and hardware resources.

[0056] In the embodiment provided in step S206, the physical address corresponding to each target virtual address is searched in parallel from the target address set indicated by each target set identifier, including but not limited to directly recording multiple virtual addresses and physical addresses with corresponding relationships in the target address set. In this case, the physical address corresponding to each target virtual address can be directly searched in parallel from the target address set indicated by each target set identifier. Alternatively, if a multi-layer mapping structure is recorded in the target address set, the address small set of the lowest layer that directly records the virtual addresses and physical addresses with corresponding relationships should be found layer by layer, and then the physical address corresponding to each target virtual address should be searched in the small set.

[0057] Optionally, in this embodiment, step S206 can be executed only when, but not limited to, the address set identifier corresponding to the virtual address set where each target virtual address is located is found to be a plurality of different address set identifiers, or the address set identifier corresponding to each target virtual address can be searched within the target time period, and step S206 is executed after the target time period ends.

[0058] Optionally, in this embodiment, the physical address can be stored in other storage spaces other than the memory, for example, it can be stored in the target external storage space, or it can also be stored in other external storage spaces. It can be stored in the memory with a corresponding virtual address set and address set identifier, and combined with the physical address set indicated by the address set identifier recorded outside the memory, to achieve double-layer mapping of addresses, which can greatly expand the hard disk space manageable by the storage system (i.e., the target external storage space). When the hard disk to be managed (i.e., the target external storage space) is expanded, it is no longer necessary to expand the space occupied by the hard disk management data (i.e., the virtual address set and address set identifier with a corresponding relationship) in the memory, but only to expand the storage space of the physical address set indicated by the address set identifier recorded outside the memory, and then modify the content of the mapping relationship in the memory accordingly, so as to achieve management of a larger range of hard disk space (i.e., the target external storage space).

[0059] Optionally, in this embodiment, searching for the physical address corresponding to each target virtual address from the target address sets indicated by each target set identifier in parallel may be implemented in, but not limited to, the following three ways:

[0060] Implementation method 1: Parallel search based on multi-core processors:

[0061] Using multi-core processors in modern computer systems, an independent search thread is assigned to each target set identifier. Each thread is responsible for finding the mapping of virtual addresses to physical addresses from the corresponding target address set. After all threads complete the search, the collected multiple target physical addresses are aggregated and prepared for the next step of metadata storage.

[0062] Implementation method 2: Use dedicated hardware accelerator for parallel search:

[0063] Develop a dedicated hardware accelerator, such as an FPGA (Field-Programmable Gate Array) or an ASIC (Application-Specific Integrated Circuit), to accelerate the parallel search process from virtual addresses to physical addresses. The hardware accelerator contains multiple search units, each of which can perform search operations independently. Each search unit reads data in parallel from the address set indicated by the target set identifier, and quickly finds the matching physical address through the built-in high-speed comparison logic. Furthermore, the search results can be directly processed by the hardware accelerator without returning to the CPU for further processing. The accelerator directly writes the metadata to the specified physical address, reducing the burden on the CPU and data transmission delay.

[0064] Implementation method 3: Parallel search based on heterogeneous computing platforms:

[0065] Taking advantage of the high parallel computing capability of the GPU, the search tasks are assigned to multiple stream processors (Streaming Multiprocessors) of the GPU. The GPU can handle the search of a large number of target virtual addresses at the same time, significantly improving the search efficiency. The target virtual address set and the corresponding physical address set are mapped to the GPU memory, and the data is sharded. Each stream processor is responsible for searching a part of the data. Once the search is completed, the result is directly transferred from the GPU memory to the main memory through technologies such as DMA (direct memory access), and then uniformly stored in the target external storage space. This method can fully utilize the parallel computing advantages of the GPU and accelerate data processing.

[0066] Through the above methods, the physical address search efficiency during the metadata storage process can be significantly improved, thereby improving the overall storage performance and system response speed. Each implementation method has its own unique advantages. The specific method should be determined based on the hardware configuration, software environment and specific performance requirements of the metadata storage system.

[0067] In the embodiment provided in step S208, multiple target metadata are written to the corresponding target physical addresses in the target external storage space, including but not limited to division according to physical address sets. When metadata stored in virtual addresses corresponding to all physical addresses in a physical address set need to be written to disk, all metadata corresponding to a physical address set are written to disk together; or, when multiple target physical addresses are found in parallel from the target address set indicated by each target set identifier, metadata corresponding to these multiple target physical addresses are written to the target external storage space in parallel.

[0068] Optionally, in this embodiment, the multiple target metadata are stored on the disk at the corresponding target physical addresses in the target external storage space, including but not limited to the following three methods:

[0069] Implementation method 1: Using DMA (Direct Memory Access) technology:

[0070] The DMA controller is configured to identify the target physical address and communicate directly with the external storage space without CPU intervention. Once the target physical address is obtained, the DMA controller starts multiple write channels at the same time to write multiple metadata in parallel to the corresponding physical addresses of the external storage. This method avoids the CPU bottleneck during the data transfer process and improves the write speed. After the write is completed, the DMA controller sends an interrupt or completion signal to the CPU, and the CPU processes these signals as needed, but the data transfer process has minimal impact on the CPU.

[0071] Implementation method 2: Parallel writing based on multi-threading / multi-process:

[0072] At the software level, one or more write threads or processes are created for each target physical address, using multi-threaded or concurrent programming techniques supported by modern operating systems. Each thread or process independently writes metadata to a specified physical address in the external storage space. The system can use the parallel processing capabilities of multi-core processors to perform multiple write tasks simultaneously. The operating system is responsible for managing synchronization between threads or processes to avoid data contention and conflicts and ensure the atomicity and consistency of write operations.

[0073] Implementation method 3: Write acceleration using hardware accelerators (such as FPGA or ASIC):

[0074] Hardware accelerator configuration: Customize or use existing hardware accelerators, such as FPGA or ASIC, to load the mapping information of the target physical address and metadata into the accelerator. The hardware accelerator directly writes the metadata to the specified physical address of the external storage space based on the configuration information, without the need for the CPU at the software level to participate in the data transmission process, which greatly improves the writing speed and reduces latency. After the hardware accelerator completes the writing, it feeds back the writing status to the CPU through a specific interface, and the CPU performs subsequent processing based on these statuses, such as updating the metadata status or releasing memory resources.

[0075] Each of the above implementation methods has its specific advantages and application scenarios: DMA technology is suitable for scenarios with large and continuous data transmission, which can effectively reduce the burden on the CPU; multi-threading / multi-process technology is more suitable for distributed or concurrent processing tasks, and can fully utilize the parallel capabilities of multi-core processors; while hardware accelerators (such as FPGA or ASIC) have extreme performance and efficiency in data processing, but the design and implementation costs are high. In actual application, the most appropriate implementation method should be selected according to the specific needs and resource conditions of the system.

[0076] Figure 3 FIG. 1 is a schematic diagram of a method for storing metadata on a physical disk according to an embodiment of the present application. Figure 3As shown, the metadata in the memory needs to be written to the physical disk. The physical disk here refers to the aforementioned target external storage space. The physical address set can be stored in the middle management layer other than the memory and the physical disk. When the metadata in the memory needs to be stored in the physical disk, the physical address set recorded in the middle management layer can be used to first find the location of the physical address set that can find the physical address corresponding to the virtual address from the virtual address in the memory, and then find the physical address corresponding to the virtual address at the corresponding position of the middle management layer. After determining the accurate physical address, the metadata in the memory can be flushed to the physical disk to achieve persistent storage.

[0077] As an optional implementation, a corresponding address set identifier is matched for the virtual address set where each target virtual address is located from a virtual address set and an address set identifier having a corresponding relationship to obtain multiple target set identifiers, including: searching the virtual address set for multiple disk address sets into which each target virtual address falls; searching the target set identifiers corresponding to the multiple disk address sets from the virtual address set and the address set identifier having a corresponding relationship to obtain multiple target set identifiers.

[0078] Optionally, in this embodiment, after obtaining a virtual address of metadata in the memory, the disk address set into which the virtual address falls can be searched from multiple virtual address sets. After determining the virtual address set into which it falls, the corresponding target set identifier is found based on the one-to-one correspondence between each virtual address set and each address set identifier.

[0079] Through the above steps, by searching the virtual address set for the multiple disk address sets that each target virtual address falls into, and then searching the address set identifiers corresponding to these disk address sets from the virtual address sets and address set identifiers with corresponding relationships, it can be ensured that each target virtual address is accurately mapped to its correct physical address in the external storage space. This precise matching mechanism reduces the possibility of data writing errors and improves the reliability of data management.

[0080] As an optional implementation, the physical address corresponding to each target virtual address is searched in parallel from the target address set indicated by each target set identifier to obtain multiple target physical addresses, including: obtaining the corresponding target address set from multiple target intermediate addresses in the additional external storage space, wherein the corresponding physical address set is stored in the intermediate address in the additional external storage space, the address set identifier includes the intermediate address, and the target set identifier includes the target intermediate address; and searching the target physical address from each target address set in parallel.

[0081] Optionally, in this embodiment, each physical address set may be stored in other storage locations but is not limited to being stored therein, and the address set identifier may be, but is not limited to being, an intermediate address used to indicate a specific storage location of the physical address set.

[0082] As an optional implementation, the physical address corresponding to each target virtual address is searched in parallel from the target address set indicated by each target set identifier, including: repeatedly executing the following steps with each target address set as a candidate address set until the corresponding target physical address is found for each target virtual address corresponding to the candidate address set: traversing the target virtual address corresponding to the candidate address set as a reference virtual address; matching the corresponding physical address for the reference virtual address from the virtual addresses and physical addresses that have a corresponding relationship.

[0083] Optionally, in this embodiment, including but not limited to creating a corresponding thread for each target address set, repeatedly executing the following steps in the thread until the target virtual addresses corresponding to the corresponding target address set all find the corresponding target physical addresses: traversing the target virtual addresses corresponding to the corresponding target physical address set as reference virtual addresses; matching the corresponding physical address for the reference virtual address from the virtual addresses and physical addresses with corresponding relationships. The virtual addresses and physical addresses with corresponding relationships output by each thread are stored in the same storage area, which can trigger, but is not limited to, the operation of writing metadata corresponding to all physical addresses in the storage area to disk when the storage area is filled.

[0084] As an optional implementation, multiple target metadata are written to the corresponding target physical addresses in the target external storage space, including: when the physical addresses corresponding to multiple candidate virtual addresses are found in parallel from each target address set, the candidate metadata corresponding to the multiple candidate virtual addresses are written to the corresponding target physical addresses in the target external storage space in parallel, wherein the multiple target virtual addresses include multiple candidate virtual addresses, the multiple target metadata include multiple candidate metadata, and the target address sets corresponding to each candidate virtual address are different.

[0085] Through the above steps, after multiple physical addresses are found in parallel from each target address set, the metadata corresponding to the aforementioned physical addresses can be written to the target external storage space in parallel according to the physical addresses obtained at the same time. On the basis of parallel addressing, parallel writing of metadata can also be achieved, further improving the efficiency of metadata writing.

[0086] As an optional implementation, the physical address corresponding to each target virtual address is searched in parallel from the target address sets indicated by each target set identifier to obtain multiple target physical addresses, including: in parallel, in each target address set, from virtual address subsets and physical address subset identifiers with corresponding relationships, the corresponding physical address subset identifier is matched for the virtual address subset where each target virtual address is located to obtain multiple target subset identifiers, wherein the virtual address set is divided into multiple virtual address subsets, and the physical address set includes multiple physical address subsets; in parallel, in the target address subsets indicated by each target subset identifier, the physical address is matched for the target virtual address from virtual addresses and physical addresses with corresponding relationships to obtain multiple target physical addresses.

[0087] Optionally, in this embodiment, multiple virtual address subsets and physical address subset identifiers with corresponding relationships can be stored in the physical address set, but are not limited to it, and then virtual addresses and physical addresses with corresponding relationships are stored in the target address subset indicated by each physical address subset identifier.

[0088] Through the above steps, the two-layer mapping is expanded to a three-layer mapping, and the management scope of the target external storage space is further expanded while the storage location of the management data of the target external storage space in the memory remains unchanged.

[0089] Figure 4 It is a mapping relationship between a physical address set and a memory and a physical disk according to an embodiment of the present application. Figure 4 As shown, the mapping ratio between the memory and the middle management layer can be set to 1:16, that is, a virtual address set and address set identifier with a corresponding relationship in the memory corresponds to 16 physical address sets in the middle management layer; the mapping ratio between the middle management layer and the physical disk can also be set to 1:16, that is, a virtual address subset and physical address set identifier with a corresponding relationship in each physical address set in the middle management layer corresponds to 16 physical addresses in the physical disk. Through such a setting, on the one hand, it is possible to realize parallel search in each target address set and perform physical address search in each target address subset in parallel. On the other hand, it is possible to reduce the occupation of memory space and data writing to the memory to a greater extent. According to the above ratio, 256 writes to the physical disk will only involve 16 writes to the management data of the middle management layer and 1 write of the metadata disk management data in the memory. The metadata disk management data mentioned here is the aforementioned virtual address set and address set identifier with a corresponding relationship.

[0090] As an optional implementation, before matching the corresponding address set identifier for the virtual address set where each target virtual address is located from the virtual address sets and address set identifiers with corresponding relationships, the method also includes: establishing a one-to-one correspondence between each virtual address in the memory and each physical address in the target external storage space; storing multiple virtual addresses and physical addresses with corresponding relationships into multiple physical address sets, wherein the physical address sets are located in the additional external storage space; establishing a one-to-one correspondence between each address set identifier and each virtual address set; and storing multiple virtual address sets and address set identifiers with corresponding relationships into the memory.

[0091] Optionally, in this embodiment, the additional external storage space includes but is not limited to being divided from the target external storage space, and may also be an intermediate management layer storage space newly added between the memory and the target external storage space (i.e., it belongs neither to the memory nor to the target external storage space).

[0092] Optionally, in this embodiment, one or more layers of intermediate key value management layers (hereinafter referred to as intermediate management layers) are added between the memory and the target external storage space, including but not limited to. When the intermediate management layer is multiple layers, the mapping relationship is also converted to multiple layers. For example, when the intermediate management layer is 2 layers (including intermediate management layer 1 and intermediate management layer 2), the mapping relationship between the virtual address and the physical address is converted from a single-layer mapping of memory-target external storage space to a three-layer mapping relationship of memory-intermediate management layer 1-intermediate management layer 2-target external storage space. By increasing the number of layers of the intermediate management layer, the management scope of the target external storage space can be further expanded. It is possible, but not limited to, to monitor the system's memory resources and target external storage space resources when starting a thin pool volume, and the SMM management module calculates the optimal number of intermediate management layers M. The aforementioned intermediate management layer can be divided from the target external storage space (generally an SSD disk), and there are multiple page pages (i.e., physical address sets). The number of entries stored in each page page is not the same. For example, 6 gears can be set, gear 1 has 1k entries in a page; gear 2 has 2k entries in a page, gear 3 has 4k entries in a page, gear 4 has 8k entries in a page, gear 5 has 16k entries in a page, and gear 6 has 32k entries in a page. The lower the gear, the fewer the number of entries in a page, and the higher the addressing efficiency and parallel writing efficiency when writing metadata. Here, an entry can correspond to, but is not limited to, a virtual address and a physical address with a corresponding relationship, or can correspond to multiple virtual addresses and physical addresses with a corresponding relationship. In the case where an entry corresponds to multiple virtual addresses and physical addresses with a corresponding relationship, the number of virtual addresses and physical addresses with a corresponding relationship corresponding to each entry can be, but is not limited to, set to be the same, or can also be set to be different. The number of pages in each gear can be flexibly set as needed. For example, each gear in gear 1 to gear 6 can be set to include 100 pages; or gear 1 to gear 3 can be set to include 50 pages, and gear 4 to gear 6 can be set to include 100 pages.

[0093] Optionally, in this embodiment, the following information is stored in the memory, including but not limited to: virtual address 1-1000_page1_middle management layer address 1, virtual address 1001-2000_page2_middle management layer address 2, virtual address 2001-3000_page3_middle management layer address 3, ..., virtual address XX-XX_pageX_middle management layer address X; the following information is stored in a page (i.e., a set of physical addresses) of the middle management layer (i.e., the additional external storage space): entry1_physical address 1, entry2_physical address 2, entry3_physical address 3, ..., entryX_physical address X, where one entry corresponds to one virtual address.

[0094] Through the above steps, multiple virtual addresses and physical addresses with corresponding relationships are stored in the additional external storage space, and only multiple virtual address sets and address set identifiers with corresponding relationships are stored in the memory. When the memory space remains unchanged, the management scope of the target external storage space is expanded through two-layer mapping.

[0095] As an optional implementation, multiple virtual addresses and physical addresses with corresponding relationships are stored in multiple physical address sets, including: determining the number of disk placement positions according to a disk placement capacity parameter, wherein the disk placement capacity parameter is used to indicate the disk placement capacity of metadata, the number of disk placement positions is the number of types of physical address sets divided according to capacity size, and the capacity of the physical address set is used to indicate the number of corresponding physical addresses and virtual addresses allowed to be stored in the corresponding physical address set; dividing the physical addresses in the target external storage space into multiple physical address sets matching the capacity of the physical address sets in order from low to high, wherein the capacity of the multiple physical address sets matches the number of disk placement positions, and the physical addresses in a physical address set with a larger capacity in each physical address set are higher than the physical addresses in a physical address set with a smaller capacity.

[0096] Optionally, in this embodiment, including but not limited to monitoring the system's memory resources and target external storage space resources, after the SMM management module calculates the optimal number of intermediate management layers M, the theoretical number of gears K of the intermediate management layer (i.e., the number of gears to be placed on the disk) is obtained according to the hardware configuration of the CPU, memory, and hard disk and the number of intermediate management layers (because the number of pages owned by each gear is fixed, the more gears there are, the more objects the system needs to process in parallel, the higher the requirements for the system hardware processing performance, and the higher the metadata writing speed).

[0097] Optionally, in this embodiment, including but not limited to, after determining the number of disk positions, lower physical addresses are assigned to physical address sets with lower positions (ie, smaller capacity), and higher physical addresses are assigned to physical address sets with higher positions.

[0098] Optionally, in this embodiment, the capacities of multiple physical address sets match the number of disk positions, that is, the number of types of capacities owned by the multiple physical address sets is the number of disk positions. For example, the capacities of the multiple physical address sets are 1k, 1k, 2k, 2k, 3k, and 3k respectively, and the number of disk positions that match them is 3.

[0099] Optionally, in this embodiment, including but not limited to, the capacity corresponding to the default gear and the number of physical address sets owned by each gear have been configured in the metadata disk system, and the capacity of the physical address set is set by default starting from the lowest gear and can only be set continuously. Therefore, after determining the number of disk gears, the capacity configuration plan for multiple physical address sets can be determined.

[0100] Through the above steps, physical address sets of different capacities are set, so that the addressing efficiency in each physical address set is different, so as to cope with the disk-placing process of metadata with different read and write frequency requirements, and make the disk-placing of metadata more flexible.

[0101] As an optional implementation, a one-to-one correspondence between each address set identifier and each virtual address set is established, including: recording the virtual addresses stored in each physical address set to obtain multiple physical address sets and virtual address sets with corresponding relationships; and establishing a one-to-one correspondence between each address set identifier and each virtual address set based on the corresponding relationships between each physical address set and each virtual address set.

[0102] As an optional implementation, after multiple target metadata are written to the corresponding target physical addresses in the target external storage space, the method also includes: detecting access information of each metadata stored in the target external storage space, wherein the access information is used to indicate the access situation of the corresponding metadata; adjusting the corresponding physical addresses and virtual addresses stored in each physical address set to match the access information to obtain a reorganization adjustment result, wherein the reorganization adjustment result is used to indicate the situation of the virtual addresses stored in each physical address set after adjustment; and modifying the virtual address set corresponding to the address set identifier according to the reorganization adjustment result.

[0103] Optionally, in this embodiment, the metadata access information may be, but is not limited to, used to indicate how frequently the corresponding metadata is accessed.

[0104] Optionally, in this embodiment, the physical addresses and virtual addresses with corresponding relationships stored in each physical address set are adjusted to match the access information, that is, the metadata that is frequently accessed as indicated by the access information is stored in a physical address set that is more convenient to access, and the metadata that is relatively idle as indicated by the access information is stored in a physical address set that is relatively time-consuming to access. Because a physical address set with a small capacity traverses a small range when addressing, and a physical address set with a large capacity traverses a large range when addressing, it is more convenient to access metadata in a physical address set with a small capacity than to access metadata in a physical address set with a large capacity. Therefore, the physical addresses and virtual addresses with corresponding relationships stored in each physical address set are adjusted to match the access information, that is, the metadata that is frequently accessed as indicated by the access information is stored in a physical address set with a small capacity, and the metadata that is relatively idle as indicated by the access information is stored in a physical address set with a large capacity.

[0105] Optionally, in this embodiment, after detecting the access information of each metadata stored in the target external storage space, it is possible but not limited to adjusting the storage position of the metadata in the target external storage space instead of adjusting the content recorded in the physical address set: adjusting the metadata that is accessed more frequently to the physical address corresponding to the physical address space with smaller capacity, and modifying the virtual address of the metadata in the memory accordingly.

[0106] Through the above steps, the storage scheme of the metadata address information is adjusted according to the metadata access situation, so that the storage of the metadata address information is more matched with the metadata access situation, and the frequently accessed metadata can be accessed faster, thereby improving the metadata access efficiency.

[0107] As an optional implementation, the physical addresses and virtual addresses with corresponding relationships stored in each physical address set are adjusted to match the access information, including: sorting each metadata stored in the target external storage space according to the access parameters of the metadata included in the access information to obtain a metadata sequence, wherein the access parameters are used to indicate the frequency of access to the metadata corresponding to the access parameters in the current time period; establishing a reorganized correspondence between each metadata stored in the target external storage space and each physical address set according to the position of each metadata stored in the target external storage space in the metadata sequence and the capacity of each physical address set, wherein the capacity of the physical address set is used to indicate the number of physical addresses and virtual addresses with corresponding relationships stored in the corresponding physical address set, and in the reorganized correspondence, the more frequently the metadata is accessed as indicated by the access parameters, the smaller the capacity of the physical address set corresponding to the metadata; and rewriting the physical addresses and virtual addresses corresponding to each metadata stored in the target external storage space to the corresponding physical address set indicated by the reorganized correspondence.

[0108] Optionally, in this embodiment, the metadata is sorted without deduplication.

[0109] Optionally, in this embodiment, including but not limited to, during initialization, an intermediate management layer from low to high is created according to the order of physical addresses from low to high. When metadata is written, the number of metadata read and write times is recorded. When metadata is first written, the intermediate management layer is gradually written from low to high, and the target external storage space is written from low physical address to high physical address in sequence. After a certain period of time, the intermediate management layer is reorganized according to the number of metadata read and write times, that is, the intermediate management data of metadata with high read and write times (i.e., virtual addresses and physical addresses with corresponding relationships) is written to the low-level page, and the intermediate management data of metadata with low read and write times is written to the high-level page. After reorganization, the intermediate management data of metadata with high frequency of reading and writing is stored in the low-level page. Since the amount of information stored in the low-level page is small, the above adjustment can increase the concurrent access volume and improve the address addressing efficiency within a single page.

[0110] As an optional implementation, the physical addresses and virtual addresses corresponding to each metadata stored in the target external storage space are rewritten to the corresponding physical address set indicated by the reorganization correspondence relationship, including: backing up the physical addresses and virtual addresses with corresponding relationships stored in the reference address set to the backup set, wherein the reference address set is the physical address set to be rewritten; and rewriting the physical addresses and virtual addresses with corresponding relationships corresponding to each metadata corresponding to the reference address set indicated by the reorganization correspondence relationship from the backup set to the reference address set.

[0111] Through the above steps, the physical address set to be rewritten is first stored in the backup set, and then rewritten from the backup set to the reference address set according to the reorganization correspondence, thereby avoiding data loss caused during the rewriting process.

[0112] As an optional implementation, the virtual address set corresponding to the address set identifier is modified according to the reorganization adjustment result, including: determining the virtual addresses stored in each address set according to the reorganization adjustment result to obtain multiple changed address sets corresponding to each physical address set; and modifying the virtual address set corresponding to the address set identifier indicating the physical address set to the corresponding changed address set.

[0113] Through the above steps, after rewriting the corresponding virtual addresses and physical addresses stored in each physical address set, the virtual address set corresponding to the physical address set is also modified accordingly, ensuring the correctness of the correspondence between the addresses and avoiding the use of incorrect mapping relationships that causes the metadata to be unable to be normally stored in the target external storage space.

[0114] As an optional implementation, the method also includes: comparing the relationship between the disk landing parameters and the disk landing parameter threshold, wherein the disk landing parameters are used to indicate the disk landing load of the metadata to be currently stored; when the number of consecutive detections that the disk landing parameters are greater than the disk landing parameter threshold, or the number of consecutive detections that the disk landing parameters are less than the disk landing parameter threshold, is greater than or equal to the number threshold, the capacity of each physical address set is adjusted by adjusting the current disk landing parameters toward the disk landing parameter threshold, wherein the capacity of the physical address set is used to indicate the number of corresponding physical addresses and virtual addresses allowed to be stored in the corresponding physical address set.

[0115] Optionally, in this embodiment, the disk write parameter may include, but is not limited to, one or more of a real-time utilization rate of CPU resources, a real-time utilization rate of memory resources, and a metadata writing speed.

[0116] Optionally, in this embodiment, the disk write parameter threshold may include, but is not limited to, one or more of a theoretical CPU resource usage rate, a theoretical memory resource usage rate, and a theoretical metadata write speed.

[0117] Optionally, in this embodiment, the disk placement parameter threshold can be set manually, or can be determined based on different target external storage space types (such as different hard disk types), target external storage space performance (such as hard disk performance), CPU performance, memory capacity and memory rate, and the number of intermediate management layers, including but not limited to recording the disk placement parameter threshold in a theoretical matrix table after determining the above disk placement parameter threshold, and storing the matrix table in the SMM register corresponding to the SMM management module.

[0118] Through the above steps, the capacity of each physical address set is adjusted according to the disk load of the metadata to be written to the disk, ensuring that the metadata writing system continues to run in a state with large concurrency but will not cause system crashes. While pursuing metadata writing efficiency, the possibility of system crashes is reduced, and the metadata writing efficiency is improved to a certain extent.

[0119] As an optional implementation, the capacity of each physical address set is adjusted by adjusting the current disk placement parameter toward the disk placement parameter threshold, including: when the number of consecutive detections that the disk placement parameter is greater than the disk placement parameter threshold is greater than or equal to the number threshold, searching for a physical address set in multiple physical address sets whose capacity is smaller than the capacity of other address sets, to obtain multiple merged address sets, wherein the capacity is used to indicate the number of physical addresses and virtual addresses with corresponding relationships stored in the corresponding physical address set, and the other address sets are physical address sets in multiple physical address sets except the merged address set; merging the multiple merged address sets to obtain an integrated address set; establishing a correspondence between the virtual address set corresponding to the merged set identifier and the integrated set identifier, wherein the merged set identifier is an address set identifier indicating the merged address set, and the integrated set identifier is an address set identifier indicating the integrated address set; and releasing the correspondence between the merged set identifier and the corresponding virtual address set.

[0120] Optionally, in this embodiment, if the number of times that the disk placement parameter is continuously detected to be greater than the disk placement parameter threshold is greater than or equal to the number threshold, it is considered that the disk placement system is currently operating in a state of excessive concurrency pressure and the concurrency pressure needs to be reduced. Therefore, the method of merging address sets is adopted to reduce the concurrent processing pressure of the system.

[0121] Optionally, in this embodiment, merging multiple merged address sets includes but is not limited to merging multiple merged address sets into one integrated address set, or merging multiple merged address sets into a smaller number of integrated address sets.

[0122] Figure 5 This is a schematic diagram of adjusting the capacity of a physical address set according to an embodiment of the present application. Figure 1 .like Figure 5 As shown, Page n and Page n+1 are the aforementioned merged address sets. By merging Page n and Page n+1, the integrated address set Page n can be obtained. 1 .

[0123] As an optional implementation, the capacity of each physical address set is adjusted by adjusting the current disk placement parameter toward the disk placement parameter threshold, including: when the number of consecutive detections that the disk placement parameter is less than the disk placement parameter threshold is greater than or equal to the number threshold, searching for the physical address set with the largest capacity among multiple physical address sets to obtain a split address set, wherein the capacity is used to indicate the number of corresponding physical addresses and virtual addresses stored in the corresponding physical address set; splitting the split address set into multiple physical address sets to obtain multiple partitioned address sets; establishing a correspondence between the virtual address set corresponding to the split set identifier and the multiple partitioned set identifiers, wherein the split set identifier is an address set identifier indicating the split address set, and the partition set identifier is an address set identifier indicating the partitioned address set; and releasing the correspondence between the split set identifier and the corresponding virtual address set.

[0124] Optionally, in this embodiment, if the number of consecutive detections that the disk placement parameter is less than the disk placement parameter threshold is greater than or equal to the number threshold, it is considered that the disk placement system is currently operating in a state where the concurrency pressure is too small and the concurrency needs to be increased. Therefore, the method of splitting the address set is adopted to improve the concurrent processing capability of the system.

[0125] Optionally, in this embodiment, splitting the split address set into multiple physical address sets includes but is not limited to equally dividing the split address set into multiple physical address sets.

[0126] Figure 6 This is a schematic diagram of adjusting the capacity of a physical address set according to an embodiment of the present application. Figure 2 .like Figure 6 As shown, Page t is the aforementioned split address set. By dividing Page t equally, we can get the partition address set Page t 1 and Page t 1 +1.

[0127] As an optional implementation, comparing the size relationship between the disk drop parameter and the disk drop parameter threshold includes: detecting a first parameter and a second parameter and obtaining the first parameter threshold and the second parameter threshold, wherein the first parameter is used to indicate the current metadata drop to disk usage of operating resources, and the second parameter is used to indicate the current metadata drop to disk usage of memory resources; comparing the size relationship between the first parameter and the first parameter threshold, and comparing the size relationship between the second parameter and the second parameter threshold; when the first parameter is greater than the first parameter threshold and the second parameter is greater than the second parameter threshold, determining that the detected disk drop parameter is greater than the disk drop parameter threshold; when the first parameter is less than the first parameter threshold and the second parameter is less than the second parameter threshold, determining that the detected disk drop parameter is less than the disk drop parameter threshold.

[0128] Optionally, in this embodiment, it includes but is not limited to setting a monitoring module to obtain the real-time metadata write speed, the real-time CPU resource utilization rate, and the real-time memory resource utilization rate, and obtaining the corresponding metadata theoretical write processing speed, CPU resource theoretical utilization rate, and memory resource theoretical utilization rate (preset values ​​established based on historical data) from the SMM register according to the corresponding CPU, memory, and hard disk hardware configuration and the number of intermediate management layers. If the real-time CPU resource utilization rate and the real-time memory resource utilization rate are less than the theoretical CPU resource utilization rate and the theoretical memory resource utilization rate for three consecutive times (three inspection cycles), a page of gear 6 (the highest gear) is divided into two pages and combined into gear 5 for management. This increases the number of low-gear pages and increases the system's concurrent processing efficiency. After the page splitting of gear 6 is completed, the page of gear 5 is split again, and the cycle is repeated. If the real-time CPU resource usage and the real-time memory resource usage are greater than the theoretical CPU resource usage and the theoretical memory resource usage for three consecutive times (three inspection cycles), the two pages in gear 1 are merged into one page for management. That is, two pages in gear 1 are merged into one page in gear 2, which improves the page management object and reduces the system concurrent processing pressure.

[0129] As an optional implementation, obtaining the first parameter threshold and the second parameter threshold includes: detecting whether there is a user threshold set, wherein the user threshold set is a parameter threshold set by the user; when it is detected that there is a user threshold set, extracting the first parameter threshold and the second parameter threshold from the user threshold set; when it is detected that there is no user threshold set, determining the first default threshold as the first parameter threshold and determining the second default threshold as the second parameter threshold.

[0130] Optionally, in this embodiment, the user threshold set is a parameter threshold set by the user, including but not limited to a metadata theoretical write speed, a CPU resource theoretical usage rate, a memory resource theoretical usage rate, etc. set by the user.

[0131] Optionally, in this embodiment, including but not limited to setting a corresponding relationship between a threshold gear and a parameter threshold, the lower the threshold gear, the lower the corresponding metadata theoretical write speed, the theoretical utilization rate of CPU resources and the theoretical utilization rate of memory resources, and the higher the threshold gear, the higher the corresponding metadata theoretical write speed, the theoretical utilization rate of CPU resources and the theoretical utilization rate of memory resources. Users can select different threshold gears according to their needs. When a high threshold gear is selected, it means that the user hopes to obtain a high metadata write speed, that is, the theoretical metadata write speed, the theoretical utilization rate of CPU resources, and the theoretical utilization rate of memory resources are high. When a low threshold gear is selected, it means that the user hopes to give priority to ensuring the security and stability of the entire system, and the metadata write speed has a low priority, that is, the theoretical metadata write speed, the theoretical utilization rate of CPU resources, and the theoretical utilization rate of memory resources are low.

[0132] Through the above steps, a more flexible capacity adjustment triggering scheme is set. Users can set the triggering timing of capacity adjustment based on demand rather than just based on the operation status of the metadata storage system, which improves the flexibility of metadata storage.

[0133] Through the above content, by directly adding an intermediate management layer between the memory and the external storage space, multi-level mapping management is realized, the management scope is expanded, and page mapping management of multiple gears is supported, and the parallel writing of metadata is realized, which can further improve the addressing and parallel writing speed according to the hot and cold data of the business. By comparing with the theoretical utilization rate of the hardware and the theoretical writing rate of metadata, the metadata writing speed of the current system can be detected to fine-tune the space, and the page shifting is realized by merging and disassembling the multi-level pages of the intermediate management layer, realizing intelligent metadata management, and providing basic management capabilities for improving the writing speed of system business.

[0134] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method.

[0135] The embodiment of the present application also provides a device for storing metadata on a disk. Figure 7 is a structural block diagram of a metadata storage device according to an embodiment of the present application, such as Figure 7 As shown, the device comprises:

[0136] The acquisition module 702 is used to acquire virtual addresses of multiple target metadata to be written to the disk in the memory, and obtain multiple target virtual addresses;

[0137] A matching module 704 is used to match the corresponding address set identifier for the virtual address set where each target virtual address is located from the virtual address set and the address set identifier having a corresponding relationship, and obtain multiple target set identifiers, wherein the physical address set indicated by the address set identifier stores the physical addresses corresponding to each virtual address in the corresponding virtual address set in the target external storage space;

[0138] A search module 706 is used to search the physical address corresponding to each target virtual address from the target address set indicated by each target set identifier in parallel to obtain multiple target physical addresses;

[0139] The disk write module 708 is used to write multiple target metadata to the corresponding target physical addresses in the target external storage space.

[0140] Through the above device, since a virtual address set and an address set identifier with a corresponding relationship are configured, and a virtual address and a physical address with a corresponding relationship are stored in the address set identifier, after the virtual addresses of multiple target metadata to be written to the disk in the memory are obtained, the corresponding address set identifier will be matched for the virtual address set where each target virtual address is located from the virtual address set and the address set identifier with a corresponding relationship, and then the physical address corresponding to each target virtual address will be searched in parallel from the target address set indicated by each target set identifier. The method of searching for the physical address corresponding to the virtual address in each small range in parallel replaces the method of searching for the physical address in a large range including multiple small ranges in serial, which speeds up the addressing speed of the metadata. Therefore, the technical problem of low efficiency of metadata writing to the disk can be solved, and the technical effect of improving the efficiency of metadata writing to the disk can be achieved.

[0141] As an optional implementation, the matching module includes: a first search unit, used to search for multiple disk address sets into which each target virtual address falls from a virtual address set; a second search unit, used to search for target set identifiers corresponding to multiple disk address sets from virtual address sets and address set identifiers with corresponding relationships, to obtain multiple target set identifiers.

[0142] As an optional implementation, the search module includes: an acquisition unit, used to acquire a corresponding target address set from multiple target intermediate addresses in an additional external storage space, wherein a corresponding physical address set is stored in the intermediate address in the additional external storage space, the address set identifier includes the intermediate address, and the target set identifier includes the target intermediate address; a third search unit, used to search for the target physical address from each target address set in parallel.

[0143] As an optional implementation, the search module also includes: an execution unit, which is used to repeatedly perform the following steps with each target address set as a candidate address set until the corresponding target physical addresses are found for the target virtual addresses corresponding to the candidate address set: traversing the target virtual addresses corresponding to the candidate address set as reference virtual addresses; matching the corresponding physical address for the reference virtual address from the virtual addresses and physical addresses that have a corresponding relationship.

[0144] As an optional implementation, the disk placement module includes: a disk placement unit, which is used to, when physical addresses corresponding to multiple candidate virtual addresses are found in parallel from each target address set, parallelly place candidate metadata corresponding to multiple candidate virtual addresses on disk to corresponding target physical addresses in a target external storage space, wherein the multiple target virtual addresses include multiple candidate virtual addresses, the multiple target metadata include multiple candidate metadata, and the target address sets corresponding to each candidate virtual address are different.

[0145] As an optional implementation, the search module also includes: a first matching unit, used to match the corresponding physical address subset identifier for the virtual address subset where each target virtual address is located from the virtual address subset and physical address subset identifier with a corresponding relationship in each target address set in parallel, to obtain multiple target subset identifiers, wherein the virtual address set is divided into multiple virtual address subsets, and the physical address set includes multiple physical address subsets; a second matching unit, used to match the physical address for the target virtual address from the virtual address and physical address with a corresponding relationship in the target address subset indicated by each target subset identifier in parallel, to obtain multiple target physical addresses.

[0146] As an optional implementation, the device also includes: a first establishment module, used to establish a one-to-one correspondence between each virtual address in the memory and each physical address in the target external storage space; a first storage module, used to store multiple virtual addresses and physical addresses with corresponding relationships into multiple physical address sets, wherein the physical address sets are located in the additional external storage space; a second establishment module, used to establish a one-to-one correspondence between each address set identifier and each virtual address set; a second storage module, used to store multiple virtual address sets and address set identifiers with corresponding relationships into the memory.

[0147] As an optional implementation, the first storage module includes: a first determination unit, used to determine the number of disk placement positions according to a disk placement capacity parameter, wherein the disk placement capacity parameter is used to indicate the disk placement capacity for metadata, the number of disk placement positions is the number of types of physical address sets divided according to capacity size, and the capacity of the physical address set is used to indicate the number of corresponding physical addresses and virtual addresses allowed to be stored in the corresponding physical address set; a division unit, used to divide the physical addresses in the target external storage space into multiple physical address sets matching the capacity of the physical address sets in order from low to high, wherein the capacity of the multiple physical address sets matches the number of disk placement positions, and the physical addresses in a physical address set with a larger capacity in each physical address set are higher than the physical addresses in a physical address set with a smaller capacity.

[0148] As an optional implementation, the second establishing module includes: a recording unit, used to record the virtual addresses stored in each physical address set to obtain multiple physical address sets and virtual address sets with corresponding relationships; a first establishing unit, used to establish a one-to-one correspondence between each address set identifier and each virtual address set based on the correspondence between each physical address set and each virtual address set.

[0149] As an optional implementation, the device also includes: a detection module, used to detect access information of each metadata stored in the target external storage space, wherein the access information is used to indicate the situation of the corresponding metadata being accessed; a first adjustment module, used to adjust the corresponding physical addresses and virtual addresses stored in each physical address set to match the access information, and obtain a reorganization adjustment result, wherein the reorganization adjustment result is used to indicate the situation of the virtual addresses stored in each physical address set after adjustment; a modification module, used to modify the virtual address set corresponding to the address set identifier according to the reorganization adjustment result.

[0150] As an optional implementation, the first adjustment module includes: a sorting unit, used to sort the metadata stored in the target external storage space according to the access parameters of the metadata included in the access information to obtain a metadata sequence, wherein the access parameters are used to indicate the frequency of access to the metadata corresponding to the access parameters in the current time period; a second establishment unit, used to establish a reorganization correspondence between the metadata stored in the target external storage space and the physical address sets according to the position of the metadata stored in the target external storage space in the metadata sequence and the capacity of the physical address sets, wherein the capacity of the physical address set is used to indicate the number of physical addresses and virtual addresses with corresponding relationships stored in the corresponding physical address set, and in the reorganization correspondence, the more frequently the metadata is accessed as indicated by the access parameters, the smaller the capacity of the physical address set corresponding to the metadata; a rewriting unit, used to rewrite the physical addresses and virtual addresses corresponding to the metadata stored in the target external storage space to the corresponding physical address set indicated by the reorganization correspondence.

[0151] As an optional implementation, the rewrite unit is also used to: back up the corresponding physical addresses and virtual addresses stored in the reference address set to a backup set, wherein the reference address set is the physical address set to be rewritten; and rewrite the corresponding physical addresses and virtual addresses corresponding to each metadata corresponding to the reference address set indicated by the reorganized correspondence from the backup set to the reference address set.

[0152] As an optional implementation, the modification module includes: a second determination unit, used to determine the virtual addresses stored in each address set according to the reorganization adjustment result, and obtain multiple changed address sets corresponding to each physical address set; a modification unit, used to modify the virtual address set corresponding to the address set identifier indicating the physical address set to the corresponding changed address set.

[0153] As an optional implementation, the device also includes: a comparison module, used to compare the size relationship between the disk landing parameter and the disk landing parameter threshold, wherein the disk landing parameter is used to indicate the disk landing load of the metadata currently to be landed; a second adjustment module, used to adjust the capacity of each physical address set by adjusting the current disk landing parameter toward the disk landing parameter threshold when the number of consecutive detections that the disk landing parameter is greater than the disk landing parameter threshold, or the number of consecutive detections that the disk landing parameter is less than the disk landing parameter threshold, is greater than or equal to the number threshold, wherein the capacity of the physical address set is used to indicate the number of corresponding physical addresses and virtual addresses allowed to be stored in the corresponding physical address set.

[0154] As an optional implementation, the second adjustment module includes: a fourth search unit, used to search for a physical address set whose capacity is smaller than that of other address sets in multiple physical address sets when the number of consecutive detections that the disk placement parameter is greater than the disk placement parameter threshold is greater than or equal to the number threshold, so as to obtain multiple merged address sets, wherein the capacity is used to indicate the number of physical addresses and virtual addresses with corresponding relationships stored in the corresponding physical address set, and the other address sets are physical address sets other than the merged address set in the multiple physical address sets; a merging unit, used to merge the multiple merged address sets to obtain an integrated address set; a third establishing unit, used to establish a correspondence between the virtual address set corresponding to the merged set identifier and the integrated set identifier, wherein the merged set identifier is an address set identifier indicating the merged address set, and the integrated set identifier is an address set identifier indicating the integrated address set; a first releasing unit, used to release the correspondence between the merged set identifier and the corresponding virtual address set.

[0155] As an optional implementation, the second adjustment module also includes: a fifth search unit, used to search for the physical address set with the largest capacity in multiple physical address sets when the number of consecutive detections that the disk placement parameter is less than the disk placement parameter threshold is greater than or equal to the number threshold, to obtain a split address set, wherein the capacity is used to indicate the number of physical addresses and virtual addresses with corresponding relationships stored in the corresponding physical address set; a splitting unit, used to split the split address set into multiple physical address sets, to obtain multiple partitioned address sets; a fourth establishing unit, used to establish a correspondence between the virtual address set corresponding to the split set identifier and multiple partitioned set identifiers, wherein the split set identifier is an address set identifier indicating the split address set, and the partition set identifier is an address set identifier indicating the partitioned address set; a second releasing unit, used to release the correspondence between the split set identifier and the corresponding virtual address set.

[0156] As an optional implementation, the comparison module includes: a detection unit, used to detect a first parameter and a second parameter and obtain a first parameter threshold and a second parameter threshold, wherein the first parameter is used to indicate the utilization of operating resources by the current metadata storage, and the second parameter is used to indicate the utilization of memory resources by the current metadata storage; a comparison unit, used to compare the size relationship between the first parameter and the first parameter threshold, and to compare the size relationship between the second parameter and the second parameter threshold; a third determination unit, used to determine that the detected storage parameter is greater than the storage parameter threshold when the first parameter is greater than the first parameter threshold and the second parameter is greater than the second parameter threshold; a fourth determination unit, used to determine that the detected storage parameter is less than the storage parameter threshold when the first parameter is less than the first parameter threshold and the second parameter is less than the second parameter threshold.

[0157] As an optional implementation, the detection unit is also used to: detect whether there is a user threshold set, wherein the user threshold set is a parameter threshold set by the user; when it is detected that there is a user threshold set, extract a first parameter threshold and a second parameter threshold from the user threshold set; when it is detected that there is no user threshold set, determine the first default threshold as the first parameter threshold and determine the second default threshold as the second parameter threshold.

[0158] For the description of the features in the above embodiments corresponding to the metadata storage device, please refer to the relevant description of the embodiments corresponding to the metadata storage method, which will not be repeated here.

[0159] An embodiment of the present application further provides an electronic device, including a memory and a processor, wherein a computer program is stored in the memory, and the processor is configured to run the computer program to execute the steps in any of the above-mentioned metadata storage method embodiments.

[0160] An embodiment of the present application further provides a computer-readable storage medium, in which a computer program is stored, wherein the computer program is configured to execute the steps of any of the above-mentioned metadata storage method embodiments when running.

[0161] In an exemplary embodiment, the computer-readable storage medium may include, but is not limited to, various media that can store computer programs, such as a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk.

[0162] An embodiment of the present application further provides a computer program product, which includes a computer program. When the computer program is executed by a processor, the steps in any of the above-mentioned metadata storage method embodiments are implemented.

[0163] An embodiment of the present application further provides another computer program product, including a non-volatile computer-readable storage medium, wherein the non-volatile computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps in any of the above-mentioned metadata storage method embodiments are implemented.

[0164] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0165] The above is a detailed introduction to the metadata storage provided by the present application. This article uses specific examples to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea. It should be pointed out that for ordinary technicians in this technical field, without departing from the principles of the present application, several improvements and modifications can be made to the present application, and these improvements and modifications also fall within the scope of protection of the claims of the present application.

Claims

1. A method for storing metadata on a disk, characterized in that: Acquire virtual addresses of multiple target metadata to be written to disk in the memory to obtain multiple target virtual addresses; match the corresponding address set identifier for the virtual address set where each of the target virtual addresses is located from the virtual address set and the address set identifier having a corresponding relationship to obtain multiple target set identifiers, wherein the physical address set indicated by the address set identifier stores the physical addresses corresponding to each virtual address in the corresponding virtual address set in the target external storage space; search for the physical address corresponding to each of the target virtual addresses from the target address sets indicated by each of the target set identifiers in parallel to obtain multiple target physical addresses; write the multiple target metadata to the corresponding target physical addresses in the target external storage space; Among them, the matching of the corresponding address set identifier for the virtual address set where each of the target virtual addresses is located from the virtual address set and the address set identifier having a corresponding relationship to obtain multiple target set identifiers includes: searching the virtual address set for the multiple disk address sets into which each of the target virtual addresses falls; searching the target set identifiers corresponding to the multiple disk address sets respectively from the virtual address set and the address set identifier having a corresponding relationship to obtain multiple target set identifiers.

2. The method according to claim 1, characterized in that The step of searching for the physical address corresponding to each of the target virtual addresses from the target address sets indicated by each of the target set identifiers in parallel to obtain a plurality of target physical addresses includes: Acquire the corresponding target address set from a plurality of target intermediate addresses in the additional external storage space, wherein the corresponding physical address set is stored in the intermediate address in the additional external storage space, the address set identifier includes the intermediate address, and the target set identifier includes the target intermediate address; The target physical address is searched from each of the target address sets in parallel.

3. The method according to claim 1, characterized in that The step of searching for the physical address corresponding to each of the target virtual addresses from the target address sets indicated by each of the target set identifiers in parallel includes: The following steps are repeatedly performed using each target address set as a candidate address set until the corresponding target physical address is found for each target virtual address corresponding to the candidate address set: Traversing the target virtual addresses corresponding to the candidate address set as reference virtual addresses; A corresponding physical address is matched for the reference virtual address from virtual addresses and physical addresses having a corresponding relationship.

4. The method according to claim 3, characterized in that The step of writing the plurality of target metadata to the target physical address corresponding to the target external storage space includes: When the physical addresses corresponding to multiple candidate virtual addresses are found in parallel from each of the target address sets, the candidate metadata corresponding to the multiple candidate virtual addresses are written to the disk in parallel to the corresponding target physical addresses in the target external storage space, wherein the multiple target virtual addresses include multiple candidate virtual addresses, the multiple target metadata include multiple candidate metadata, and the target address sets corresponding to each of the candidate virtual addresses are different.

5. The method according to claim 1, characterized in that: The step of searching for the physical address corresponding to each of the target virtual addresses from the target address sets indicated by each of the target set identifiers in parallel to obtain a plurality of target physical addresses includes: In parallel, in each of the target address sets, a corresponding physical address subset identifier is matched for the virtual address subset where each of the target virtual addresses is located from the virtual address subset identifiers and the physical address subset identifiers having a corresponding relationship, to obtain a plurality of target subset identifiers, wherein the virtual address set is divided into a plurality of virtual address subsets, and the physical address set includes a plurality of physical address subsets; In parallel, a physical address is matched for the target virtual address from virtual addresses and physical addresses having a corresponding relationship in the target address subsets indicated by the target subset identifiers to obtain a plurality of target physical addresses.

6. The method according to claim 1, characterized in that Before matching the corresponding address set identifier for the virtual address set where each target virtual address is located from the virtual address sets and address set identifiers having a corresponding relationship, the method further includes: Establishing a one-to-one correspondence between each virtual address in the memory and each physical address in the target external storage space; Storing a plurality of virtual addresses and physical addresses having corresponding relationships into a plurality of physical address sets, wherein the physical address sets are located in an additional external storage space; Establishing a one-to-one correspondence between each of the address set identifiers and each of the virtual address sets; A plurality of corresponding virtual address sets and address set identifiers are stored in the memory.

7. The method according to claim 6, characterized in that The storing of the plurality of virtual addresses and physical addresses having corresponding relationships into the plurality of physical address sets comprises: Determine the number of disk placement positions according to a disk placement capacity parameter, wherein the disk placement capacity parameter is used to indicate the disk placement capacity of metadata, the number of disk placement positions is the number of types of the physical address set divided according to capacity, and the capacity of the physical address set is used to indicate the number of physical addresses and virtual addresses with corresponding relationships that are allowed to be stored in the corresponding physical address set; The physical addresses in the target external storage space are divided into a plurality of physical address sets matching the capacity of the physical address sets in order from low to high, wherein the capacity of the plurality of physical address sets matches the number of disk positions, and the physical addresses in a physical address set with a larger capacity in each of the physical address sets are higher than the physical addresses in a physical address set with a smaller capacity.

8. The method according to claim 6, characterized in that The establishing of a one-to-one correspondence between each of the address set identifiers and each of the virtual address sets includes: Recording the virtual addresses stored in each of the physical address sets to obtain a plurality of the physical address sets and the virtual address sets having a corresponding relationship; A one-to-one correspondence between each of the address set identifiers and each of the virtual address sets is established according to the correspondence between each of the physical address sets and each of the virtual address sets.

9. The method according to claim 1, characterized in that: After writing the plurality of target metadata to the target physical addresses corresponding to the target external storage space, the method further includes: Detecting access information of each metadata stored in the target external storage space, wherein the access information is used to indicate a situation in which the corresponding metadata is accessed; Adjusting the corresponding physical addresses and virtual addresses stored in each of the physical address sets to match the access information, and obtaining a reorganization adjustment result, wherein the reorganization adjustment result is used to indicate the virtual addresses stored in each of the physical address sets after adjustment; The virtual address set corresponding to the address set identifier is modified according to the reorganization adjustment result.

10. The method according to claim 9, characterized in that The adjusting the corresponding physical addresses and virtual addresses stored in each of the physical address sets to match the access information includes: sorting the metadata stored in the target external storage space according to the access parameters of the metadata included in the access information to obtain a metadata sequence, wherein the access parameters are used to indicate the frequency of access to the metadata corresponding to the access parameters in a current time period; Establishing a reorganized correspondence between each metadata stored in the target external storage space and each physical address set according to the position of each metadata stored in the target external storage space in the metadata sequence and the capacity of each physical address set, wherein the capacity of the physical address set is used to indicate the number of physical addresses and virtual addresses having a corresponding relationship stored in the corresponding physical address set, and in the reorganized correspondence, the more frequently accessed metadata indicated by the access parameter is, the smaller the capacity of the physical address set corresponding to the metadata; The physical addresses and virtual addresses corresponding to each metadata stored in the target external storage space are rewritten into the corresponding physical address set indicated by the reorganized correspondence relationship.

11. The method according to claim 10, characterized in that The step of rewriting the physical addresses and virtual addresses corresponding to each metadata stored in the target external storage space to the corresponding physical address set indicated by the reorganized correspondence relationship includes: Backing up the corresponding physical addresses and virtual addresses stored in the reference address set to the backup set, wherein the reference address set is the physical address set to be rewritten; The physical addresses and virtual addresses with corresponding relationships corresponding to the metadata corresponding to the reference address set indicated by the reorganized corresponding relationship are rewritten from the backup set to the reference address set.

12. The method according to claim 9, characterized in that The step of modifying the virtual address set corresponding to the address set identifier according to the reorganization adjustment result includes: Determine the virtual addresses stored in each of the address sets according to the reorganization and adjustment results, and obtain a plurality of changed address sets corresponding to each of the physical address sets; The virtual address set corresponding to the address set identifier indicating the physical address set is modified to the corresponding changed address set.

13. The method according to claim 1, characterized in that The method further comprises: Comparing the relationship between the disk placement parameter and the disk placement parameter threshold, wherein the disk placement parameter is used to indicate the disk placement load of the metadata to be currently placed on disk; When the number of consecutive detections that the disk landing parameter is greater than the disk landing parameter threshold, or the number of consecutive detections that the disk landing parameter is less than the disk landing parameter threshold, is greater than or equal to the number threshold, the capacity of each physical address set is adjusted by adjusting the current disk landing parameter toward the disk landing parameter threshold, wherein the capacity of the physical address set is used to indicate the number of corresponding physical addresses and virtual addresses allowed to be stored in the corresponding physical address set.

14. The method according to claim 13, characterized in that The adjusting the capacity of each of the physical address sets by adjusting the current disk placement parameter toward the disk placement parameter threshold includes: In the case where the number of times that the disk placement parameter is continuously detected to be greater than the disk placement parameter threshold is greater than or equal to the number threshold, searching for the physical address set whose capacity is smaller than the capacity of other address sets in the multiple physical address sets to obtain multiple merged address sets, wherein the capacity is used to indicate the number of physical addresses and virtual addresses having a corresponding relationship stored in the corresponding physical address set, and the other address set is a physical address set in the multiple physical address sets except the merged address set; Merging a plurality of the merged address sets to obtain an integrated address set; Establishing a correspondence between a virtual address set corresponding to a merged set identifier and an integrated set identifier, wherein the merged set identifier is an address set identifier indicating the merged address set, and the integrated set identifier is an address set identifier indicating the integrated address set; The correspondence between the merged set identifier and the corresponding virtual address set is cancelled.

15. The method according to claim 13, characterized in that The adjusting the capacity of each of the physical address sets by adjusting the current disk placement parameter toward the disk placement parameter threshold includes: In the case where the number of times that the disk placement parameter is continuously detected to be less than the disk placement parameter threshold is greater than or equal to the number threshold, searching for the physical address set with the largest capacity among the multiple physical address sets to obtain a split address set, wherein the capacity is used to indicate the number of physical addresses and virtual addresses having a corresponding relationship stored in the corresponding physical address set; Splitting the split address set into multiple physical address sets to obtain multiple partition address sets; Establishing a correspondence between a virtual address set corresponding to a split set identifier and a plurality of partition set identifiers, wherein the split set identifier is the address set identifier indicating the split address set, and the partition set identifier is the address set identifier indicating the partition address set; The correspondence between the split set identifier and the corresponding virtual address set is cancelled.

16. The method according to claim 13, characterized in that The comparing the relationship between the disk placement parameter and the disk placement parameter threshold includes: Detecting a first parameter and a second parameter and obtaining a first parameter threshold and a second parameter threshold, wherein the first parameter is used to indicate the utilization of operation resources by the current metadata storage, and the second parameter is used to indicate the utilization of memory resources by the current metadata storage; Comparing the magnitude relationship between the first parameter and the first parameter threshold, and comparing the magnitude relationship between the second parameter and the second parameter threshold; When the first parameter is greater than the first parameter threshold and the second parameter is greater than the second parameter threshold, it is determined that the plate placement parameter is greater than the plate placement parameter threshold; When the first parameter is less than the first parameter threshold and the second parameter is less than the second parameter threshold, it is determined that the disk placement parameter is detected to be less than the disk placement parameter threshold.

17. The method according to claim 16, characterized in that The obtaining of the first parameter threshold and the second parameter threshold comprises: Detecting whether there is a user threshold set, wherein the user threshold set is a parameter threshold set by a user; In case that the user threshold value set is detected to exist, extracting the first parameter threshold value and the second parameter threshold value from the user threshold value set; In case that the user threshold set is detected to be absent, a first default threshold is determined as the first parameter threshold and a second default threshold is determined as the second parameter threshold.

18. A metadata storage device, characterized in that: include: An acquisition module is used to acquire virtual addresses of multiple target metadata to be written to the disk in the memory, and obtain multiple target virtual addresses; A matching module, used to match the corresponding address set identifier for the virtual address set where each of the target virtual addresses is located from the virtual address sets and address set identifiers having a corresponding relationship, to obtain multiple target set identifiers, wherein the physical address set indicated by the address set identifier stores the physical addresses corresponding to each of the virtual addresses in the corresponding virtual address set in the target external storage space; a search module, used to search in parallel for the physical addresses corresponding to each of the target virtual addresses from the target address sets indicated by each of the target set identifiers, to obtain multiple target physical addresses; a disk placement module, used to place multiple of the target metadata on the disk at the corresponding target physical addresses in the target external storage space; Among them, the matching module includes: a first search unit, used to search for multiple disk address sets into which each target virtual address falls from the virtual address set; a second search unit, used to search for the target set identifiers corresponding to the multiple disk address sets respectively from the virtual address sets and address set identifiers with corresponding relationships, and obtain multiple target set identifiers.

19. An electronic device, characterized in that: include: Memory for storing computer programs; A processor, configured to implement the steps of the method for writing metadata to a disk as claimed in any one of claims 1 to 17 when executing the computer program.

20. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, wherein the computer program, when executed by a processor, implements the steps of the method for storing metadata as claimed in any one of claims 1 to 17.

21. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method for storing metadata on a disk as claimed in any one of claims 1 to 17 are implemented.

Images