Metadata management method and device, electronic equipment and storage medium

Abstract

The invention discloses a metadata management method and device, electronic equipment and a storage medium, and relates to the technical field of storage, and the method comprises the following steps: constructing a first metadata address area; determining a metadata address, and storing the metadata address to the first metadata address area to obtain a second metadata address area; constructing a use record management structure of the data block corresponding to the metadata address, and generating a metadata management structure based on the second metadata address area and the use record management structure; on the basis of the metadata management structure, in response to the data writing request and / or the data reading request, the metadata address area is constructed, so that the metadata and the data can share the space, the space of the hard disk pool is directly used by the metadata and the data, the space use flexibility can be improved, the space utilization rate is effectively increased, and the space utilization rate is increased. And the data reading and writing requirements of multiple scenes can be met, and the overall performance of the storage system is improved.

Description

Technical Field

[0001] This application relates to the field of storage technologies, and in particular, to a metadata management method, apparatus, electronic device, and storage medium. Background Art

[0002] In related technologies, the thin provisioning technology relies on thin metadata. After hard disks are combined into a hard disk pool, according to the length ratio of metadata and data, a data volume and a metadata volume are created on the hard disk pool. The metadata volume is used to store metadata, and the data volume is used to store data. This technology divides the metadata and data spaces in a fixed ratio. However, in different scenarios, the ratio of the metadata and data spaces cannot be fixed. For example, in scenarios with high deduplication and compression ratios, the proportion of metadata in the data increases significantly, and the proportion of metadata reading and writing in the entire reading and writing increases. The reading and writing of a fixed space cannot meet the requirements of all scenarios (performance, space). Summary of the Invention

[0003] This application provides a metadata management method, apparatus, electronic device, and storage medium to at least solve the problem that the reading and writing of a fixed space in related technologies cannot meet the requirements of all scenarios.

[0004] This application provides a metadata management method, including:

[0005] Construct a first metadata address area;

[0006] Determine the metadata address, save the metadata address to the first metadata address area to obtain a second metadata address area, where the metadata address is a regional address describing the mapping relationship between the logical volume address storing data and the hard disk pool space address;

[0007] Construct a usage record management structure for the data block corresponding to the metadata address, and generate a metadata management structure based on the second metadata address area and the usage record management structure;

[0008] Based on the metadata management structure, respond to a data write request and / or a data read request.

[0009] This application also provides a metadata management apparatus, including:

[0010] A first metadata address area generation module, configured to construct a first metadata address area;

[0011] A second metadata address area generation module, configured to determine the metadata address, save the metadata address to the first metadata address area to obtain a second metadata address area, where the metadata address is a regional address describing the mapping relationship between the logical volume address storing data and the hard disk pool space address;

[0012] A metadata management structure generation module, configured to construct a usage record management structure for data blocks corresponding to metadata addresses, and generate a metadata management structure based on a second metadata address area and the usage record management structure;

[0013] A request response module, configured to respond to a data write request and / or a data read request based on the metadata management structure.

[0014] This application also provides an electronic device, including: a memory for storing a computer program; a processor for implementing the following steps of the metadata management method when executing the computer program:

[0015] Construct a first metadata address area;

[0016] Determine a metadata address, save the metadata address to the first metadata address area to obtain a second metadata address area, where the metadata address is an area address used to describe the mapping relationship between the logical volume address of stored data and the hard disk pool space address;

[0017] Construct a usage record management structure for data blocks corresponding to the metadata address, and generate a metadata management structure based on the second metadata address area and the usage record management structure;

[0018] Respond to a data write request and / or a data read request based on the metadata management structure.

[0019] This application also provides a computer-readable storage medium, in which a computer program is stored, and when the computer program is executed by a processor, the following steps of the metadata management method are implemented:

[0020] Construct a first metadata address area;

[0021] Determine a metadata address, save the metadata address to the first metadata address area to obtain a second metadata address area, where the metadata address is an area address used to describe the mapping relationship between the logical volume address of stored data and the hard disk pool space address;

[0022] Construct a usage record management structure for data blocks corresponding to the metadata address, and generate a metadata management structure based on the second metadata address area and the usage record management structure;

[0023] Respond to a data write request and / or a data read request based on the metadata management structure.

[0024] This application also provides a computer program product, including a computer program, and when the computer program is executed by a processor, the following steps of the metadata management method are implemented:

[0025] Construct a first metadata address area;

[0026] Determine the metadata address, save the metadata address to the first metadata address area to obtain the second metadata address area, where the metadata address is the area address used to describe the mapping relationship between the logical volume address storing data and the hard disk pool space address;

[0027] Construct a usage record management structure for the data block corresponding to the metadata address, and generate a metadata management structure based on the second metadata address area and the usage record management structure;

[0028] Based on the metadata management structure, respond to data write requests and / or data read requests.

[0029] In this application, by constructing a metadata address area, the metadata and data can share space, so that the metadata and data can directly use the space of the hard disk pool, thereby improving the flexibility of space use, effectively improving the utilization rate of space, and meeting the data read and write requirements in multiple scenarios, improving the overall performance of the storage system. Description of the Drawings

[0030] To more clearly illustrate the embodiments of this application, the following will briefly introduce the drawings required in the embodiments. Obviously, the drawings in the following description are only some embodiments of this application. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings.

[0031] Figure 1 It is an application environment diagram of a metadata management method provided by an embodiment of this application;

[0032] Figure 2 It is an overall flowchart of a metadata management method provided by an embodiment of this application;

[0033] Figure 3 It is a data write position diagram of a metadata management method provided by an embodiment of this application;

[0034] Figure 4 It is a structural block diagram of a metadata management device provided by an embodiment of this application;

[0035] Figure 5 It is an internal structure diagram of an electronic device in one embodiment. Detailed Embodiments

[0036] The following will clearly and completely describe the technical solutions in the embodiments of this application with reference to the drawings in the embodiments of this application. Obviously, the described embodiments are only some embodiments of this application, rather than all embodiments. Based on the embodiments in this application, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the protection scope of this application.

[0037] It should be noted that in the description of this application, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements not only includes those elements, but also includes other elements not expressly listed, or elements inherent to such process, method, article or device. The terms "first", "second", etc. in this application are used to distinguish similar objects, rather than to describe a specific order or sequence.

[0038] It should be noted that the terms "S1", "S2", etc. are only used for the purpose of describing steps, and do not particularly refer to the meaning of order or sequence, nor are they used to limit this application. They are only used to conveniently describe the method of this application and cannot be understood as indicating the order of steps. In addition, the technical solutions between various embodiments can be combined with each other, but it must be based on the ability of those of ordinary skill in the art to implement. When the combination of technical solutions results in contradictions or cannot be implemented, it should be considered that such a combination of technical solutions does not exist and is not within the scope of protection required by this application.

[0039] In the field of storage technology, the thin provisioning technology is a widely used technology and a basic function of the storage system. Its function is to enable the storage system to allocate space on demand, that is, after creating a volume, the entire volume space will not be immediately allocated, but only as much space as needed will be actually allocated. In this way, the use of space can be effectively saved, thus saving cost investment. At the same time, the thin provisioning technology relies on thin metadata. Currently, the industry mostly uses the form of B+ tree to manage thin metadata, that is, the mapping between the logical address of data and the physical address of data; through the mapping between the logical address and the physical address of data, data aggregation, deduplication, and compression are realized.

[0040] According to the background technology, the read and write of fixed space in the related technology cannot meet the requirements of all scenarios (performance, space). To solve the above technical problems, this application provides a metadata management method, device, electronic device, and storage medium. By constructing a metadata address area, metadata and data can share space, and then metadata and data can directly use the space of the hard disk pool, thereby improving the flexibility of space use, effectively improving the utilization rate of space, and meeting the data read and write requirements of multiple scenarios, improving the overall performance of the storage system.

[0041] To enable those skilled in the art of this technology to better understand the solution of this application, the following further detailed description of this application will be made in conjunction with the accompanying drawings and specific embodiments.

[0042] The metadata management method provided by this application can be applied to, for example Figure 1In the application environment shown. Among them, the terminal 102 communicates with the data processing platform set on the server 104 through the network. Among them, the terminal 102 can be but is not limited to various personal computers, laptop computers, smart phones, tablet computers and portable wearable devices, and the server 104 can be implemented by an independent server or a server cluster composed of multiple servers.

[0043] As Figure 2 shown, an embodiment of the present application provides a metadata management method. Taking the terminal in Figure 1 as an example, the method includes the following steps:

[0044] S1: Construct a first metadata address area.

[0045] It should be noted that the metadata address area refers to the storage space for storing mapping relationship addresses. When creating a logical volume, a unique metadata address area is created. The metadata address area is used to record the metadata addresses used by the logical volume. A logical volume refers to a virtual disk formed by logical disks, also known as a disk partition, which is used to connect to users and provide a virtual space for users to read and write.

[0046] S2: Determine the metadata address, save the metadata address to the first metadata address area to obtain a second metadata address area. The metadata address is the area address used to describe the mapping relationship between the logical volume address for storing data and the hard disk pool space address.

[0047] It should be noted that the logical volume address refers to the logical address (Logical Address), also known as the virtual address (Virtual Address), which is the address provided by the operating system for the program to use. These addresses are used in the program but do not actually exist. The size and number of bits of the logical address depend on the architecture of the processor and the design of the operating system. Usually, it is a fixed-length binary value. When executing an instruction, the CPU (Central Processing Unit) will convert the logical address into a physical address to fetch data; the hard disk pool space address refers to the physical address (Physical Address), which is the actual address in the memory, representing the unique identifier of each storage unit (usually a byte) in the memory module. Therefore, it has uniqueness and is directly associated with the memory. The physical address is usually represented by a hexadecimal number, which determines the actual memory location in the computer. The hard disk pool space includes multiple data blocks, which are used to store data and the mapping relationship between the logical volume address for storing data and the hard disk pool space address. This mapping relationship is recorded in the form of a tree or a table; the metadata address refers to the area address of the data block that stores this mapping relationship.

[0048] S3: Construct a usage record management structure for the data blocks corresponding to the metadata addresses. Based on the second metadata address area and the usage record management structure, generate a metadata management structure.

[0049] It should be noted that the metadata management structure includes the data block addresses in the hard disk pool space and the management structure for whether the data blocks have been used / unused. The management structure for whether the data blocks have been used / unused is used to record whether the space has been used. The manifestation form of the above management structure is a list. By using a variable metadata list, the metadata has a variable-length space. The above management structure, logical volume address, and metadata address area are in a one-to-one correspondence. Specifically, the data structure of a single data block in the list is shown in Table 1. By constructing the forward and backward pointing relationships of the data structure, the space used by the metadata is connected.

[0050] Table 1: Data structure of the data block.

[0051] Data block address

[0052] Bitmap of used data block

[0053] Usage count of data block

[0054] Address of the previous data block structure

[0055] Address of the next data block structure

[0056] S4: Based on the metadata management structure, respond to data write requests and / or data read requests.

[0057] It should be noted that when a data write request is received, respond to the data write request by adding a management structure and / or deleting a management structure and recording the current write position. When a data read request is received, obtain the data block address through the mapping relationship in the metadata management structure to read the data in the data block to respond to the data read request.

[0058] In the above embodiments, by constructing the metadata address area, the metadata and data can share space, so that the metadata and data can directly use the space of the hard disk pool, thereby improving the flexibility of space use, effectively improving the space utilization rate, and meeting the data read and write requirements in multiple scenarios, improving the overall performance of the storage system.

[0059] In some specific embodiments, determining the metadata address and saving the metadata address to the first metadata address area to obtain the second metadata address area includes:

[0060] Based on the data uploaded by the user, obtain the logical volume address and the hard disk pool space address of the data;

[0061] Generate a mapping relationship based on the logical volume address and the hard disk pool space address of the data;

[0062] Save the mapping relationship into a data block, define the address of the data block as the metadata address, and save the metadata address into the first metadata address area to obtain the second metadata address area.

[0063] In the above embodiment, by constructing a mapping relationship, determining the metadata address based on the storage address of the mapping relationship, and saving it into the metadata address area, it is possible to realize the sharing space of metadata and data, and avoid the problem that the read and write of a fixed space cannot meet the requirements of all scenarios.

[0064] In some specific embodiments, construct a usage record management structure for the data block corresponding to the metadata address. Based on the second metadata address area and the usage record management structure, generate a metadata management structure, including:

[0065] Construct a usage record management structure for the data block corresponding to the metadata address based on the used record management structure of the data block corresponding to the metadata address and the unused record management structure of the data block corresponding to the metadata address;

[0066] Obtain the metadata address in the second metadata address area, and generate a metadata management structure based on the metadata address and the usage record management structure. The manifestation form of the metadata management structure is a list.

[0067] In the above embodiment, by constructing a metadata management structure in the form of a list, when a data write or read request is received subsequently, by adjusting the metadata management structure or extracting the data in the metadata management structure, the data write or read request can be quickly responded to, and the current write position can be provided for the subsequent release of data block resources, improving the storage performance of the system.

[0068] In some specific embodiments, based on the metadata management structure, responding to a data write request includes:

[0069] In response to receiving a data write request, parse the data write request to determine the data space size, where the data space size refers to the space required for writing data;

[0070] According to the data space size, apply for a data space, and write the data into the data block in the data space, where the data space is composed of data blocks;

[0071] Obtain the logical volume address and the hard disk pool space address corresponding to the data block in the data space, and form a target mapping relationship;

[0072] Write the target mapping relationship into the data block in the first metadata space, where the first metadata space is composed of data blocks;

[0073] Upon successful writing, obtain the data block address written according to the target mapping relationship;

[0074] Write the data block address into the corresponding list of the metadata management structure and record the writing position;

[0075] Upon the writing position being at a preset position in the list, detect whether there is an idle data block. Here, the preset position refers to the recorded writing position being close to the end of the data list, indicating that the unused space of the metadata reaches the threshold;

[0076] If there is one, construct a metadata management structure based on the idle data block and add it to the list.

[0077] Specifically, when writing data, first apply for space for the data, write the data into the applied space, construct the mapping relationship of the data address, based on the address mapping relationship of the data, read the current writing position, write the address mapping relationship into the position recorded in the metadata space, and update the address of the recorded metadata writing. When the unused space of the metadata reaches the threshold in the above steps (that is, when the recorded writing position is close to the end of the data list), apply for a data block from the hard disk pool space and add it to the end of the list, that is, query the already constructed or real-time constructed data blocks to apply for free memory, construct a data block management structure with the applied data block, and append it to the tail of the data block list.

[0078] In some specific embodiments, the method further includes:

[0079] Obtain the sizes of free memory at different time nodes within multiple preset time periods. Here, the preset time period can be set according to actual needs, such as one month or one year, etc.;

[0080] Upon the size of the free memory corresponding to the target time node within a time period being greater than a first preset threshold, mark the target time node once. Here, the first preset threshold can be set according to actual needs;

[0081] Upon the number of marked target time nodes within multiple preset time periods being greater than a second preset threshold, mark the target time node twice. Here, the second preset threshold can be set according to actual needs;

[0082] Obtain the time nodes for applying for free memory within multiple preset time periods. If the number of occurrences of the same time node is greater than a third preset threshold, mark the time node with an identifier. Here, the third preset threshold can be set according to actual needs;

[0083] Compare the time node corresponding to the identifier with the time node marked twice;

[0084] In response to a successful comparison, when reaching the time node in the next time period, a data block automatically applied according to the memory requirement of the time node is used to construct a data block management structure and appended to the end of the data block list.

[0085] In the above embodiment, by determining the time point at which free memory needs to be applied to directly apply for memory at this time point, the latency of data reading and writing can be reduced, and the performance of the storage system is improved.

[0086] In some specific embodiments, in response to a failed data write, it further includes:

[0087] Obtain a second metadata space, write the target mapping relationship into the second metadata space, record the written address, and read the metadata corresponding to the data with the write failure into the second metadata space;

[0088] In response to the completion of the read, release the first metadata space.

[0089] Specifically, when a data write fails, a new metadata space is applied, the address mapping relationship is written into the metadata space, and the address where the metadata is written is recorded; at the same time, all the data in the space with the write failure is read out and written into the new metadata space, and the old metadata space is released; among them, the write count of the data block in the list is used to count the number of writes and releases of a certain data block. A single data block has a large space, and only a very small part can be used for a single metadata write; when recycling, it is necessary to find a space with less usage, so a count is used for recording.

[0090] In the above embodiment, when responding to a data write request, a write record is added to the list, and when the write position is close to the end position of the data list, by adding a management structure to the list, the shared space can be flexibly utilized to meet the data write requirements and improve the storage performance.

[0091] In some specific embodiments, based on the metadata management structure, responding to a data read request includes:

[0092] In response to receiving a data read request, based on the metadata management structure, determine the data block address corresponding to the data read request;

[0093] Read data based on the data block address to respond to the data read request;

[0094] In response to a failed data write, read the metadata corresponding to the data in the corresponding data block into the third metadata space;

[0095] In response to the completion of the read, release the fourth metadata space corresponding to the metadata.

[0096] Specifically, when reading data, the corresponding relationship between the logical volume address and the data address is obtained by reading metadata, and the data is read according to the obtained data block address and returned to the user; when the data reading fails, all the data in the failed reading space is read out and written into a new metadata space, and the old metadata space is released; if the data at the unreadable position is lost, it is reported to the user.

[0097] In the above embodiment, through the mapping relationship, the data to be returned to the user can be quickly determined, the data reading efficiency is improved, and when the reading fails, the data is migrated, and when the data is lost, a prompt message is sent to the user, improving the reliability of the storage system.

[0098] In some specific embodiments, the method further includes:

[0099] In response to receiving a garbage collection request, the target data is migrated to a new data block, the original data block is recycled, and at the same time, the metadata management structure is adjusted;

[0100] In response to the number of data blocks in the current space meeting a preset standard, the metadata is written into the recycled original data block, where the preset standard refers to the insufficient number of data blocks and it is difficult to meet the huge metadata write volume.

[0101] Specifically, garbage collection finds the data blocks that have been written but have little valid data for recycling through data block write counting. The specific steps are as follows: perform insertion sorting on the used list according to the write count, reconstruct the data block list into an ordered list according to the write count, and select the data blocks with low write counts for recycling; when garbage collecting, the data that is still needed is migrated to a new data block, and the old data blocks are recycled into the data block pool; the management structure of the data block list has the addresses of the previous data block structure and the next data block structure. When a data block is recycled, this management structure is removed, the list addresses of the previous and next management structures are updated, and the memory of this management structure is released.

[0102] Furthermore, when the controller fails, data recovery is performed through the controller data synchronization method. The metadata data block list has a backup controller. When there are multiple controllers, the metadata data blocks are updated on at least two controllers. When the controller fails, this backup relationship is established on a new controller combination; when a pool member disk fails, data recovery is performed according to whether the data block is allocated to the data or metadata space. The reconstructed data block has the same data block address as before, so data and metadata can be accessed without perception; when the metadata write volume is huge and the number of data blocks in the current metadata space is insufficient, the recycled data blocks will be directly used, such as Figure 3As shown, the specific steps are as follows: Find the data block to be recycled as the currently used data block according to the list ordered by write count; Read the old data of the data block into the data block cache, use the used bitmap of the old data block, configure the data block write pointer to its initial position, when the metadata is written to this data block, find the free position of the data block used bitmap backward from the position of the data block write pointer, write the data into the data block cache, update the data block write pointer, and when the data block is full, write the data block into the hard disk pool space; Further, the data block cache is protected by the metadata write log. When writing metadata, the result is not directly updated to the hard disk pool space, but first written into the data block cache. There may be a problem of power loss and data loss in the data block cache. Therefore, the metadata write first has a metadata write log, and the metadata write log is synchronized with the metadata write to the data block cache. When the metadata write log is completed, this step can return successfully.

[0103] In the above embodiment, by merging the data of garbage collection and the data of metadata writing, the performance loss caused by data migration and the metadata writing delay caused by waiting for garbage collection are reduced. By merging scattered metadata writing operations into a full append write method, the influence of scattered small read and write operations on the latency of member disks is avoided, and the storage performance is improved.

[0104] In the above metadata management method, it includes: constructing a first metadata address area; determining the metadata address, saving the metadata address to the first metadata address area to obtain a second metadata address area, where the metadata address is the area address used to describe the mapping relationship between the logical volume address of the stored data and the hard disk pool space address; constructing a usage record management structure for the data block corresponding to the metadata address, and generating a metadata management structure based on the second metadata address area and the usage record management structure; Based on the metadata management structure, in response to a data write request and / or a data read request, in this application, by constructing a metadata address area, the metadata and data can share space, and then the metadata and data can directly use the space of the hard disk pool, so as to improve the flexibility of space use, effectively improve the utilization rate of space, and can meet the data read and write requirements of multiple scenarios, improving the overall performance of the storage system.

[0105] Through the description of the above embodiments, those skilled in the art can clearly understand that the method according to the above embodiments can be implemented by means of software plus a necessary general hardware platform. Of course, it can also be implemented by hardware, but in many cases, the former is a better implementation method.

[0106] It should be understood that although Figure 2The steps in the flowchart are shown in sequence according to the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. Moreover, Figure 2 At least a part of the steps may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed alternately or in turn with at least a part of other steps or sub-steps or stages of other steps.

[0107] An embodiment of the present application also provides a metadata management device, as Figure 4 shown, including a first metadata address area generation module, a second metadata address area generation module, a metadata management structure generation module, and a request response module, where:

[0108] The first metadata address area generation module is used to construct a first metadata address area;

[0109] The second metadata address area generation module is used to determine a metadata address, save the metadata address to the first metadata address area to obtain a second metadata address area, and the metadata address is an area address used to describe the mapping relationship between the logical volume address of the stored data and the hard disk pool space address;

[0110] The metadata management structure generation module is used to construct a usage record management structure for the data block corresponding to the metadata address, and generate a metadata management structure based on the second metadata address area and the usage record management structure;

[0111] The request response module is used to respond to a data write request and / or a data read request based on the metadata management structure.

[0112] As a preferred embodiment, in the embodiment of the present invention, the second metadata address area generation module is specifically used for:

[0113] Based on the data uploaded by the user, obtain the logical volume address and the hard disk pool space address of the data;

[0114] Generate a mapping relationship based on the logical volume address and the hard disk pool space address of the data;

[0115] Save the mapping relationship to the data block, define the address of the data block as the metadata address, and save the metadata address to the first metadata address area to obtain a second metadata address area.

[0116] As a preferred embodiment, in the embodiment of the present invention, the metadata management structure generation module is specifically used for:

[0117] Construct a usage record management structure for data blocks corresponding to metadata addresses based on the used record management structure of data blocks corresponding to metadata addresses and the unused record management structure of data blocks corresponding to metadata addresses.

[0118] Obtain the metadata addresses in the second metadata address area, and generate a metadata management structure based on the metadata addresses and the usage record management structure. The manifestation form of the metadata management structure is a list.

[0119] As a preferred implementation manner, in the embodiment of the present invention, the request response module is specifically configured to:

[0120] In response to receiving a data write request, parse the data write request to determine the data space size.

[0121] Apply for a data space according to the data space size, and write the data into the data blocks in the data space. Wherein, the data space is composed of data blocks.

[0122] Obtain the logical volume address and the hard disk pool space address corresponding to the data blocks in the data space, and form a target mapping relationship.

[0123] Write the target mapping relationship into the data blocks in the first metadata space. Wherein, the first metadata space is composed of data blocks.

[0124] In response to successful writing, obtain the data block address where the target mapping relationship is written.

[0125] Write the data block address into the corresponding list in the metadata management structure, and record the writing position.

[0126] When the writing position is at a preset position in the list, detect whether there are free data blocks.

[0127] If there are, construct a metadata management structure based on the free data blocks and add it to the list.

[0128] As a preferred implementation manner, in the embodiment of the present invention, the request response module is specifically further configured to:

[0129] Obtain the second metadata space, write the target mapping relationship into the second metadata space, record the written address, and read the metadata corresponding to the data with writing failure into the second metadata space.

[0130] In response to the completion of reading, release the first metadata space.

[0131] As a preferred implementation manner, in the embodiment of the present invention, the request response module is specifically further configured to:

[0132] In response to receiving a data reading request, determine the data block address corresponding to the data reading request based on the metadata management structure;

[0133] Read data based on the data block address to respond to the data reading request;

[0134] In response to a data writing failure, read the metadata corresponding to the data in the corresponding data block into the third metadata space;

[0135] In response to the completion of reading, release the fourth metadata space corresponding to the metadata.

[0136] As a preferred implementation manner, in the embodiments of the present invention, the request response module is further specifically configured to:

[0137] In response to receiving a garbage collection request, migrate the target data to a new data block, recycle the original data block, and at the same time, adjust the metadata management structure;

[0138] In response to the number of data blocks in the current space meeting a preset standard, write the metadata into the recycled original data block.

[0139] For the description of the features in the corresponding embodiments of the metadata management device, reference can be made to the relevant descriptions in the corresponding embodiments of the metadata management method, which will not be elaborated here one by one. Each module in the above metadata management device can be implemented in whole or in part by software, hardware, and their combinations. The above modules can be embedded in the processor in the electronic device in the form of hardware or be independent of the processor, or can be stored in the memory in the electronic device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0140] In one embodiment, an electronic device is provided. The electronic device can be a terminal, and its internal structure diagram can be as Figure 5 shown. The electronic device includes a processor, a memory, a network interface, a display screen, and an input device connected through a system bus. Among them, the processor of the electronic device is used to provide computing and control capabilities. The memory of the electronic device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the electronic device is used to communicate with an external terminal through a network connection. The computer program, when executed by the processor, implements a * method. The display screen of the electronic device can be a liquid crystal display screen or an electronic ink display screen, and the input device of the electronic device can be a touch layer covering the display screen, or a button, a trackball, or a touchpad provided on the housing of the electronic device, or an external keyboard, a touchpad, or a mouse, etc.

[0141] Those skilled in the art can understand,Figure 5 The structure shown is only a block diagram of some structures related to the solution of this application, and does not constitute a limitation on the electronic device to which the solution of this application is applied. The specific electronic device may include more or fewer components than those shown in the figure, or combine some components, or have different component arrangements.

[0142] An embodiment of this application provides an electronic device, including a memory and a processor. A computer program is stored in the memory, and the processor is configured to run the computer program to execute the steps in the embodiment of the metadata management method, including:

[0143] S1: Construct a first metadata address area;

[0144] S2: Determine the metadata address, save the metadata address to the first metadata address area to obtain a second metadata address area, and the metadata address is an area address for describing the mapping relationship between the logical volume address of the stored data and the hard disk pool space address;

[0145] S3: Construct a usage record management structure for the data block corresponding to the metadata address, and generate a metadata management structure based on the second metadata address area and the usage record management structure;

[0146] S4: Respond to a data write request and / or a data read request based on the metadata management structure.

[0147] An embodiment of this application also provides a computer-readable storage medium, in which a computer program is stored. The computer program is configured to execute the steps in the embodiment of the metadata management method when running, including:

[0148] S1: Construct a first metadata address area;

[0149] S2: Determine the metadata address, save the metadata address to the first metadata address area to obtain a second metadata address area, and the metadata address is an area address for describing the mapping relationship between the logical volume address of the stored data and the hard disk pool space address;

[0150] S3: Construct a usage record management structure for the data block corresponding to the metadata address, and generate a metadata management structure based on the second metadata address area and the usage record management structure;

[0151] S4: Respond to a data write request and / or a data read request based on the metadata management structure.

[0152] In an exemplary embodiment, the computer-readable storage medium may include, but is not limited to: various media that can store computer programs, such as USB flash drives, read-only memory (ROM for short), random access memory (RAM for short), external hard drives, magnetic disks, or optical discs.

[0153] The embodiments of the present application also provide a computer program product. The computer program product includes a computer program. When the computer program is executed by a processor, the steps in the embodiment of the metadata management method are implemented, including:

[0154] S1: Construct a first metadata address area;

[0155] S2: Determine the metadata address, save the metadata address to the first metadata address area to obtain a second metadata address area. The metadata address is an area address used to describe the mapping relationship between the logical volume address of the stored data and the hard disk pool space address;

[0156] S3: Construct a usage record management structure for the data block corresponding to the metadata address, and generate a metadata management structure based on the second metadata address area and the usage record management structure;

[0157] S4: Respond to data write requests and / or data read requests based on the metadata management structure.

[0158] The embodiments of the present application also provide another computer program product, including a non-volatile computer-readable storage medium. The non-volatile computer-readable storage medium stores a computer program. When the computer program is executed by a processor, the steps in the embodiment of the metadata management method are implemented, including:

[0159] S1: Construct a first metadata address area;

[0160] S2: Determine the metadata address, save the metadata address to the first metadata address area to obtain a second metadata address area. The metadata address is an area address used to describe the mapping relationship between the logical volume address of the stored data and the hard disk pool space address;

[0161] S3: Construct a usage record management structure for the data block corresponding to the metadata address, and generate a metadata management structure based on the second metadata address area and the usage record management structure;

[0162] S4: Respond to data write requests and / or data read requests based on the metadata management structure.

[0163] Those skilled in the art may further realize that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be implemented by electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described according to functions in the above description. Whether these functions are executed in a hardware or software manner depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of this application.

[0164] The above has introduced in detail a metadata management method, apparatus, electronic device, and storage medium provided by this application. Specific examples are used herein to elaborate on the principles and implementation manners of this application. The description of the above embodiments is only used to help understand the method and its core idea of this application. It should be noted that for those of ordinary skill in the art in this technical field, without departing from the principle of this application, several improvements and modifications can be made to this application, and these improvements and modifications also fall within the protection scope of the claims of this application.

Claims

1. A metadata management method, characterized in that, The method includes: Construct a first metadata address area; Determine the metadata address, and save the metadata address to the first metadata address area to obtain a second metadata address area, where the metadata address is an area address used to describe the mapping relationship between the logical volume address of the stored data and the hard disk pool space address; Construct a usage record management structure for the data block corresponding to the metadata address, and generate a metadata management structure based on the second metadata address area and the usage record management structure; Based on the metadata management structure, respond to a data write request and / or a data read request.

2. The metadata management method according to claim 1, wherein Determine the metadata address, and save the metadata address to the first metadata address area to obtain a second metadata address area, which includes: Based on the data uploaded by the user, obtain the logical volume address and the hard disk pool space address of the data; Generate a mapping relationship based on the logical volume address and the hard disk pool space address of the data; Save the mapping relationship to a data block, define the address of the data block as the metadata address, and save the metadata address to the first metadata address area to obtain a second metadata address area.

3. The metadata management method according to claim 2, wherein Construct a usage record management structure for the data block corresponding to the metadata address, and generate a metadata management structure based on the second metadata address area and the usage record management structure, which includes: Based on the used record management structure of the data block corresponding to the metadata address and the unused record management structure of the data block corresponding to the metadata address, construct a usage record management structure for the data block corresponding to the metadata address; Obtain the metadata address in the second metadata address area, and generate a metadata management structure based on the metadata address and the usage record management structure, where the manifestation form of the metadata management structure is a list.

4. The metadata management method according to claim 3, wherein Based on the metadata management structure, responding to a data write request includes: In response to receiving a data write request, parse the data write request to determine the data space size; Apply for a data space according to the data space size, and write the data into the data block in the data space, where the data space is composed of data blocks; Obtain the logical volume address and the hard disk pool space address corresponding to the data block in the data space, and form a target mapping relationship; Write the target mapping relationship into the data block in the first metadata space, where the first metadata space is composed of data blocks; In response to successful writing, obtain the data block address where the target mapping relationship is written; Write the data block address into the list corresponding to the metadata management structure, and record the writing position; When the writing position is at a preset position in the list, detect whether there is an idle data block; If there is, construct a metadata management structure based on the idle data block and add it to the list.

5. The metadata management method according to claim 4, wherein In response to a data write failure, the method further includes: Obtain a second metadata space, write the target mapping relationship into the second metadata space, record the written address, and read the metadata corresponding to the data with a write failure into the second metadata space; In response to the completion of reading, release the first metadata space.

6. The metadata management method according to claim 1, wherein Based on the metadata management structure, the response to a data read request includes: Upon receiving a data read request, determine the data block address corresponding to the data read request based on the metadata management structure; Read data based on the data block address to respond to the data read request; Upon a data write failure, read the metadata corresponding to the data in the corresponding data block into the third metadata space; Upon completion of the read, release the fourth metadata space corresponding to the metadata.

7. The metadata management method according to claim 1, wherein The method further includes: Upon receiving a garbage collection request, migrate the target data to a new data block, recycle the original data block, and at the same time, adjust the metadata management structure; Upon the number of data blocks in the current space meeting a preset standard, write the metadata into the recycled original data block.

8. A metadata management device, characterized in that, It includes: A first metadata address area generation module for constructing a first metadata address area; A second metadata address area generation module for determining a metadata address, saving the metadata address to the first metadata address area to obtain a second metadata address area, where the metadata address is an area address for describing the mapping relationship between the logical volume address storing data and the hard disk pool space address; A metadata management structure generation module for constructing a usage record management structure for the data block corresponding to the metadata address, and generating a metadata management structure based on the second metadata address area and the usage record management structure; A request response module for responding to a data write request and / or a data read request based on the metadata management structure.

9. An electronic device, characterized in that, It includes: A memory for storing a computer program; A processor for implementing the steps of the metadata management method according to any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium, characterized in that, A computer program is stored in the computer-readable storage medium, where the computer program, when executed by a processor, implements the steps of the metadata management method according to any one of claims 1 to 7.

Images