Write method, recovery method and storage host for storing data
By combining large block mapping and data storage linked lists, the problems of changing the order of data writing and limited storage space are solved, thereby improving the data writing efficiency and space utilization of the storage host.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- INSPUR SUZHOU INTELLIGENT TECH CO LTD
- Filing Date
- 2026-04-30
- Publication Date
- 2026-05-29
AI Technical Summary
In existing technologies, the data writing method alters the original order of the data, affecting data writing efficiency, and the limited storage space leads to a decrease in the performance of the storage host.
By employing a large block mapping scheme and a data storage linked list, when querying the target storage block fails, the data is written to a spare storage block. Combined with a garbage collection strategy, storage space utilization is optimized to ensure data ordering and storage performance.
This achieves the goal of avoiding data sequential changes within a limited memory capacity, improving storage space utilization and storage host performance, and increasing data writing efficiency.
Smart Images
Figure CN122111884A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of data storage technology, and in particular to a method for writing and recycling stored data, and a storage host. Background Technology
[0002] Storage hosts are primarily used for managing the hard drives and data stored on them, while also providing data access services to servers. In related technologies, data writing methods that store discrete data not only alter the original order of the data, thus affecting writing efficiency, but also limit storage space, thereby impacting the storage performance of the storage host. Summary of the Invention
[0003] This application provides a method for writing and recycling stored data, as well as a storage host, to at least solve the problems that related technologies not only change the original order of data, thus affecting data writing efficiency, but also have limited storage space, thus affecting the storage performance of the storage host.
[0004] This application provides a method for writing stored data, including: In response to the current data being transmitted to the storage host, the target storage block to which the current data belongs is queried based on multiple storage blocks. Each storage block includes multiple storage pages arranged in sequence. A data storage linked list is generated based on the initialization parameters of multiple storage blocks. The data storage linked list records the link identifiers of multiple spare storage blocks that are in an idle state. The data storage linked list is generated based on the initialization parameters of multiple storage blocks. When querying the target storage block fails, the current data is written sequentially to multiple storage pages of the standby storage block according to the link identifier recorded in the data storage chain, and the current storage status of the standby storage block is updated. When the query for the target storage block is successful, a write operation is performed on the current data based on the current storage state of the target storage page in the target storage block.
[0005] This application also provides a method for recycling stored data, including: By using the data writing method, storage data is written into the storage space of the storage host. The storage space includes: multiple storage blocks and spare storage blocks for the current data. Monitor the current storage capacity of the storage space; When the current storage capacity is less than the preset storage capacity, garbage collection is performed on the garbage data in multiple storage blocks according to the garbage collection strategy.
[0006] This embodiment provides a data writing device, including: The data query module is used to respond to the current data being transmitted to the storage host and query the target storage block to which the current data belongs based on multiple storage blocks. Each storage block includes multiple storage pages arranged in sequence. The data generation module is used to generate a data storage linked list based on the initialization parameters of multiple storage blocks. The data storage linked list records the link identifiers of multiple spare storage blocks that are in an idle state. The data storage linked list is generated based on the initialization parameters of multiple storage blocks. The first write module is used to write the current data sequentially into multiple storage pages of the backup storage block according to the link identifier recorded in the data storage chain when the query of the target storage block fails, and to update the current storage status of the backup storage block. The second write module is used to perform a write operation on the current data based on the current storage state of the target storage page of the target storage block when the query for the target storage block is successful.
[0007] This application also provides a data recycling device, comprising: The data writing module is used to write storage data into the storage space of the storage host through the data writing method. The storage space includes multiple storage blocks and a spare storage block for the current data. The data monitoring module is used to monitor the current storage space remaining. The data recycling module is used to perform garbage collection actions on garbage data in multiple storage blocks according to the garbage collection strategy when the current storage balance is less than the preset storage balance.
[0008] This application also provides a storage host, comprising: a memory for storing a computer program; a processor for implementing the above-described method for writing stored data when executing the computer program; or, a method for reclaiming stored data; or, a method for managing stored data.
[0009] This application also provides a computer-readable storage medium storing a computer program, wherein when the computer program is executed by a processor, it implements the steps of any of the above-mentioned methods for writing stored data; or, a method for reclaiming stored data; or, a method for managing stored data.
[0010] This application also provides a computer program product, including a computer program that, when executed by a processor, implements any of the above-described methods for writing stored data; or, a method for recycling stored data; or, a method for managing stored data.
[0011] By employing a flexible, expandable storage method for data writing within storage pages arranged sequentially across multiple storage blocks, this application resolves the issues of fragmented metadata page mapping affecting the original data order and limited storage space. Simultaneously with continuous data writing, this application utilizes a garbage collection strategy to reclaim and transfer garbage data within storage blocks. This ensures that the stored data size remains within a reasonable range while promoting garbage collection, accelerating the effective utilization of storage space, and further enhancing the continuity of data persistence. Therefore, this application achieves the technical effect of efficiently writing data while simultaneously expanding storage space, thereby enhancing the performance of the storage host. Attached Figure Description
[0012] To more clearly illustrate the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0013] Figure 1 A schematic diagram illustrating the mapping from logical address to physical address provided in an embodiment of this application; Figure 2 This is a schematic diagram illustrating data updates provided in an embodiment of this application. Figure 3 A flowchart illustrating a method for writing stored data provided in an embodiment of this application; Figure 4 A schematic diagram of a large block mapping scheme based on multiple storage blocks provided in an embodiment of this application; Figure 5 A flowchart illustrating another method for writing stored data provided in this application embodiment; Figure 6 A flowchart illustrating another method for writing stored data provided in this application embodiment; Figure 7 A flowchart illustrating another method for writing stored data provided in this application embodiment; Figure 8 A flowchart illustrating a method for reclaiming stored data provided in this application embodiment; Figure 9 A structural block diagram of a data writing device provided in an embodiment of this application; Figure 10 This is a schematic diagram of a hardware device for a storage host provided in an embodiment of this application. Detailed Implementation
[0014] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the protection scope of this application.
[0015] It should be noted that, in the description of this application, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. The terms "first," "second," etc., in this application are used to distinguish similar objects and are not used to describe a specific order or sequence.
[0016] To enable those skilled in the art to better understand the present application, the present application will be further described in detail below with reference to the accompanying drawings and specific embodiments.
[0017] The storage host is primarily used for managing the hard drives and data stored on them, while also providing data access services to the server. The storage unit is the core functional unit of the storage host, employing an "on-demand allocation and dynamic expansion" storage management model. This means that the corresponding capacity is allocated from the storage pool only when data is actually written, significantly reducing the waste of idle resources and overcoming the limitations of "pre-allocated physical space".
[0018] In related technologies, a storage management model of "on-demand allocation and dynamic expansion" is adopted. This involves establishing an association between preset logical addresses and preset physical addresses through a dynamic mapping table. Physical space is allocated only for data that has already been written, and the mapping relationship is recorded. When data is written to an address that has already been written, storage space needs to be re-allocated, the original storage space is invalidated, and the mapping relationship between the preset logical address and the preset physical address is updated. For example... Figure 1 As shown, the data that records the mapping relationship is called metadata. When data is written to an address that has already been written, storage space needs to be re-allocated, the original storage space invalidated, and the mapping relationship between the preset logical address and the preset physical address updated, as follows: Figure 2 As shown.
[0019] Therefore, the data writing method that uses discrete data for data storage not only changes the original order of the data, but also limits the storage space for storing the data, thus affecting the storage performance of the storage host.
[0020] Therefore, embodiments of this application provide a method for writing stored data, which can be used in computer devices such as mobile phones, tablets, desktop computers, laptops, and servers. Figure 3 This is a flowchart of a method for writing stored data according to an embodiment of the present invention, such as... Figure 3 As shown, the process includes the following steps: Step S301: In response to the current data being transmitted to the storage host, query the target storage block to which the current data belongs based on multiple storage blocks. Each storage block includes multiple storage pages arranged in sequence.
[0021] Specifically, the current data refers to the data input by the user to the storage host, which can be represented by I / O. Multiple storage blocks represent multiple physical storage regions, and each storage block can be represented by a block. For example... Figure 4 The diagram illustrates a large-block mapping scheme based on multiple storage blocks, while preserving page mapping in this application. A large-block mapping scheme refers to mapping a contiguous region of a large file to the virtual address space of a process in one go.
[0022] exist Figure 4 In this context, a large block is defined as 32*32=1024 consecutive pages of 4KB size stored sequentially. The large block mapping metadata includes the following members: Struct block_meta{ uint32 lba; / / Logical address uint32 pba; / / Physical address } For a 4MB storage area, metadata is only 8 bytes, resulting in a metadata space:data storage space consumption ratio of 8B:4MB. This means that 1PB of data requires only 2GB of memory. In optimal conditions, 1PB of data requires only 2GB of memory. The storage cluster has a large memory capacity, which can completely store this 2GB in memory. A 32-bit address can support a maximum of 16PB of storage space.
[0023] Within the basic framework of block mapping, page mapping is retained as a flexible supplement. Page mapping metadata includes the following members: Struct page_meta{page data uint64 lba; / / Logical address uint64 pba; / / Physical address } The reason this application uses 64-bit variables instead of 32-bit variables is that 32-bit variables can only support a maximum of 16TB of space, which is insufficient to accommodate the storage capacity available to the storage system. For 4KB of storage space, metadata requires 16B; for 4MB of storage space, metadata requires 16KB. The metadata space:data storage space consumption ratio is 16KB:4MB, which is 2048 times higher than block storage. 1PB of data requires 4TB of memory space, which is clearly beyond the host's memory capacity. If this application uses page mapping for all data storage, a large amount of metadata would need to be stored on the backend disk. When this metadata needs to be read or updated, it would need to be read and written from the backend disk, severely impacting the performance of the storage host. In fact, page mapping may also need to record the current offset address of the physical storage block where the page resides, thus requiring further expansion of the metadata, which in turn reduces the storage capacity of the storage host.
[0024] The target storage block is the storage block that matches the current data I / O, and each storage block consists of multiple storage pages arranged in sequence. This avoids the need for a discrete approach to metadata page mapping, which would alter the original order of the data.
[0025] In some specific implementations, the preset logical address of the target storage block to which the current data belongs is calculated based on the current logical address of the current data and the preset data volume of each storage block.
[0026] Specifically, the preset logical address of the target storage block to which the current data belongs is calculated using the following formula: block_lba = lba / da.
[0027] Where block_lba is the preset logical address of the target storage block to which the previous data belongs, lba is the current logical address of the current data, and da is the preset data size for each storage block, which is usually 4MB.
[0028] For example, after the storage host receives the current data I / O from the front end, it calculates the preset logical address of the target storage block to which the current data belongs using the formula block_lba = lba / da from the preset logical address of the I / O. Therefore, it uses block_lba to query and match the block_meta metadata, and then matches the lba member variable of the metadata.
[0029] Step S302: Generate a data storage linked list based on the initialization parameters of multiple storage blocks. The data storage linked list records the link identifiers of multiple spare storage blocks that are in an idle state. The data storage linked list is generated based on the initialization parameters of multiple storage blocks.
[0030] In some specific implementations, a data storage linked list is generated based on the initialization parameters of multiple storage blocks, including: Step a1: Perform initialization actions on the initialization parameters of multiple storage blocks.
[0031] Step a2: Generate a data storage linked list based on the initialization results of multiple storage blocks.
[0032] In a specific example, the initialization parameters for multiple storage blocks include: the number of invalid pages for each storage block, the preset physical address, the valid page identifier, and the blank page identifier.
[0033] For example, the storage space of the storage host can be divided into multiple storage blocks, or physical blocks, in 4MB granularity, and described by the following pseudocode data structure: Struct physical_block{ uint16 invalid_number; / / (Number of invalid pages, i.e., the number of pages whose data is invalid) uint32 pba; / / Preset physical address uint64 valid_mask
[16] ; / / Valid page identifier, that is, the page contains valid data. uint64 empty_mask
[16] ; / / Blank page identifier, that is, no data is stored in this page, each bit corresponds to one page. } During initialization, the member variables of each physical_block are assigned values as follows, with the meanings described in the description: Struct physical_block{ uint16 invalid_number; / / = 0; No invalid pages uint32 pba; / / = Assigns the physical starting address of the actual block being described; uint64 valid_mask
[16] ; / / = 0; No valid data. uint64 empty_mask
[16] ; / / = 1; all pages are empty} After initialization, the physical_block is added to the data storage linked list free_physical_block_list for subsequent business use.
[0034] Step S303: When querying the target storage block fails, according to the link identifier recorded in the data storage chain, the current data is written sequentially into multiple storage pages of the standby storage block, and the current storage status of the standby storage block is updated.
[0035] If a query for the target storage block fails, a spare storage block that is currently idle is selected from the data storage list and assigned to `page_block_ptr` for use by page mapping. `page_block_ptr` represents the link identifier of the spare storage block.
[0036] If a query for the target storage block fails, the current data is written sequentially to multiple storage pages of a backup storage block. This allows for data storage across multiple storage blocks, avoiding data storage in a discrete manner that could affect the original order of the data. Furthermore, when the storage host has limited memory capacity, the backup storage blocks containing the current data are linked through a data storage list, enabling flexible replenishment of storage space, increasing storage capacity, and improving the performance of the storage host.
[0037] Step S304: When the query for the target storage block is successful, a write operation is performed on the current data based on the current storage state of the target storage page of the target storage block.
[0038] Specifically, the current storage state of the target storage page of the target storage block includes a blank state and a non-blank state. A blank state means that if the current data is written, the current data will be written for the first time. A non-blank state means that the current data may be written for the second, third or Nth time.
[0039] Therefore, this application executes the write state of the current data according to the current storage state of the target storage page of the target storage block, thereby avoiding data storage in a discrete manner and changing the original order of the data, thus affecting the data writing efficiency, under the premise that multiple storage blocks constitute a large block mapping.
[0040] Embodiments of this application provide a method for writing stored data, which can be used in computer devices such as mobile phones, tablets, desktop computers, laptops, and servers. Figure 5 This is a flowchart of a method for writing stored data according to an embodiment of the present invention, such as... Figure 5 As shown, step S303 above, which involves writing the current data sequentially into multiple storage pages of the spare storage block according to the link identifier recorded in the data storage chain, and updating the current storage state of the spare storage block, includes: Step S3031: Select the link identifier of the spare storage block from the data storage chain.
[0041] If the query for the target storage block fails, a link identifier for an idle spare storage block is selected from the data storage list. This link identifier is called page_block_ptr.
[0042] Step S3032: Obtain the backup storage block for the current data based on the link identifier of the backup storage block.
[0043] Specifically, the link identifier links to the backup storage block of the current data to enable flexible replenishment of the current data.
[0044] Step S3033: Calculate the current offset address of the current data based on the current logical address of the current data, the preset logical address of the spare storage block, and the preset data amount of each storage page of the spare storage block.
[0045] For example, a spare storage block `physical_block` and metadata `block_meta` are selected from the data storage list `free_physical_block_list`, where the metadata `block_meta` is assigned the following value: block_meta.lba = block_lba; block_meta.pba = physical_block; Furthermore, the current offset address of the current data is calculated using the following formula.
[0046] page_id = (lba - block_lba) / ya `page_id` is the current offset address of the current data, `lba` is the current logical address of the current data, `block_lba` is the preset logical address of the spare storage block, and `ya` is the preset data size of each storage page in the spare storage block.
[0047] Step S3034: Based on the current offset address of the current data, write the current data sequentially into multiple storage pages of the spare storage block, and update the blank page identifier and valid page identifier of the multiple storage pages of the spare storage block.
[0048] For example, based on the current offset address of the current data, the current data is written sequentially to multiple storage pages of the spare storage block, and the usage status of these multiple storage pages is updated at the same time. That is, the empty page flag empty_mask corresponding to the storage page of the current data is set to 0, and the valid page flag valid_mask corresponding to the storage page of the current data is set to 1.
[0049] By implementing the above-described embodiments, in the event that a query of the target storage block fails, the current data is written sequentially into multiple storage pages of a backup storage block. This achieves the following: on the one hand, data is stored in multiple storage blocks, avoiding data storage in a discrete manner that would affect the original order of the data; on the other hand, when the memory capacity of the storage host is limited, the backup storage blocks of the current data are linked through a data storage list, thereby enabling flexible supplementation of storage space, increasing storage capacity, and improving the performance of the storage host.
[0050] Embodiments of this application provide a method for writing stored data, which can be used in computer devices such as mobile phones, tablets, desktop computers, laptops, and servers. Figure 6 This is a flowchart of a method for writing stored data according to an embodiment of the present invention, such as... Figure 6 As shown, step S304 above, based on the current storage state of the target storage page of the target storage block, performs a write operation on the current data, including: Step S3041: When the current storage state of the target storage page of the target storage block is blank, write the current data into the target storage page of the target storage block, and update the blank page identifier and valid page identifier of the target storage page.
[0051] Specifically, if the current storage state of the target storage page of the target storage block is blank, it means that the current data is about to be written to the target storage page of the target storage block for the first time.
[0052] For example, the target storage block `physical_block` is found by querying the preset physical address `block_meta.pba` of the current data I / O. The current offset address of the current data is calculated using the formula `page_id = (lba - block_lba) / ya`. Based on this current offset address, the initially passed current data is written sequentially into the target storage page of the target storage block, and the empty page flag `empty_mask` of the corresponding target storage page is updated to 0, and the valid page flag `valid` is updated to 1.
[0053] Step S3042: When the current storage state of the target storage page of the target storage block is non-blank, update the valid page identifier and invalid page count of the target storage page, and query the pre-written data stored in the target storage page according to the preset logical address of the target storage page.
[0054] Specifically, if the current storage state of the target storage page of the target storage block is non-blank, it means that the current data is about to be written to the target storage page of the target storage block for the second, third, or Nth time.
[0055] For example, if the current data is not being accessed by the storage host for the first time, the valid page identifier valid_mask of the target storage page is set to 0, and the invalid page count invalid_number is incremented by 1.
[0056] In some specific implementations, the pre-written data stored in the target storage page is queried based on the preset logical address of the target storage page, and calculated using the following formula: page_lba = page_id × ya + block_lba' Where page_lba is the preset logical address of the target storage page, page_id is the current offset address of the current data, ya is the preset data size of the target storage page, which is usually 4KB, and block_lba' is the pre-written data stored in the target storage page.
[0057] By using page_lba to query and match page_meta metadata, and matching the lba member variables of the metadata, the corresponding pre-written data is found based on the preset logical address of the target storage page.
[0058] Step S3043: When the pre-written data query fails, determine the current blank page of the standby storage block according to the blank page identifier of the standby storage block, store the current data through the current blank page, and update the blank page identifier and valid page identifier of the current blank page.
[0059] For example, using `page_lba` to query and match the `page_meta` metadata, the `lba` member variable of the metadata is matched. If the match fails, that is, when the pre-written data query fails, a metadata `page_meta` is requested based on the preset logical address of the target storage page to find the corresponding pre-written data. The data is assigned the following value: `page_meta.lba = page_lba`.
[0060] Starting from the spare storage block pointed to by the link identifier page_block_ptr of the spare storage block for the current data, and using the empty page identifier empty_mask, find the current empty pages in the spare storage block arranged in ascending order, and return the preset logical address block_page_pba of these current empty pages so as to store the current data through the current empty pages. Set the empty_mask of the current empty page to 0 and the valid page identifier valid_mask to 1.
[0061] Step S3044: When the pre-written data query is successful, the current data is written to the target storage page, and the valid page identifier and invalid page count of the target storage page are updated.
[0062] For example, by querying and matching the page_meta metadata using page_lba, the lba member variable of the metadata is matched. If the match is successful, it means that the query was successful when the data was written in advance. The valid page flag physical_block.valid_mask of the physical page corresponding to the metadata on page_meta.pba is set to 0, and the invalid page count invalid_number is incremented by 1.
[0063] Therefore, this application executes the write state of the current data according to the current storage state of the target storage page of the target storage block, thereby avoiding data storage in a discrete manner and changing the original order of the data, thus affecting the data writing efficiency, under the premise that multiple storage blocks constitute a large block mapping.
[0064] In some specific implementations, if the current blank page is used up during the process of storing the current data through a blank page, the next spare storage block for the current data is requested from the data storage chain, and the current data is written sequentially into multiple storage pages of the spare storage block.
[0065] For example, if the space of the spare storage block pointed to by page_block_ptr is exhausted, the next spare storage block is requested from the data storage linked list free_physical_block_list, and simultaneously assigned to the target storage page of the next spare storage block for page mapping to access the space. See steps S3031-S3032 above for details, which will not be repeated here.
[0066] By implementing the above-described embodiments, this application, under the premise that multiple storage blocks successfully query the target storage block, if the current blank page of the spare storage block is exhausted, then another spare storage block is requested for data storage, so as to flexibly expand the storage space and thereby improve the utilization rate of the storage space.
[0067] In some specific implementations, after the current data is written to the storage host, the following is also included: In response to a data read command, perform a data read operation on the current data.
[0068] In some specific implementations, performing a data reading operation on the current data includes: Step b1: Calculate the preset logical address of the target storage block to which the current data belongs, based on the current logical address of the current data and the preset data volume of each storage block.
[0069] Specifically, the preset logical address of the target storage block to which the current data belongs is calculated using the following formula: block_lba = lba / da.
[0070] Where block_lba is the preset logical address of the target storage block to which the previous data belongs, lba is the current logical address of the current data, and da is the preset data size for each storage block, which is usually 4MB.
[0071] For example, after the storage host receives the current data I / O from the front end, it calculates the preset logical address of the target storage block to which the current data belongs using the formula block_lba = lba / da from the preset logical address of the I / O. Therefore, it uses block_lba to query and match the block_meta metadata, and matches the lba member variable of the metadata.
[0072] Step b2: Query the target storage block according to the preset logical address of the target storage block to which the current data belongs.
[0073] By using block_lba to query and match block_meta metadata, matching the lba member variable of the metadata, and finding the corresponding target storage block physical_block through the current physical address of the matched block_meta.pba block data.
[0074] Step b3: Calculate the current offset address of the current data based on the preset logical address of the target storage block, the current logical address of the current data, and the preset logical address of the target storage block.
[0075] The current offset address of the current data is calculated using the following formula: `page_id = (lba - block_lba'') / ya` calculates the current offset address of the current data. It also determines the valid page identifier corresponding to the storage page of the target storage block to which the current data belongs.
[0076] `page_id` is the current offset address of the current data, `lba` is the current logical address of the current data, `block_lba` is the preset logical address of the target storage block, and `ya` is the preset data size of each storage page in the spare storage block.
[0077] Step b4: Based on the current offset address of the current data, query the target storage page of the target storage block.
[0078] Step b5: Read the pre-written data from the target storage page based on the valid page identifier of the target storage page.
[0079] For example, if valid_mask is 1, the data for that page is directly read and returned to the storage host. If the valid page identifier valid_mask corresponding to the target storage page is 0, the preset logical address of the target storage page is calculated using the formula page_lba = page_id × ya + block_lba'. Here, page_lba is the preset logical address of the target storage page, page_id is the current offset address of the current data, ya is the preset data size of the target storage page (usually 4KB), and block_lba' is the pre-written data stored in the target storage page.
[0080] The page_lba matching query is used to match the page_meta metadata page_meta.lba, and the data is read from the matched page_meta.pba and returned to the storage host.
[0081] This application facilitates quick data retrieval for users through the aforementioned steps b1-b5.
[0082] Based on the same concept, this application also provides a method for retrieving stored data, which can be used in computer devices such as mobile phones, tablets, desktop computers, laptops, and servers. Figure 7 This is a flowchart of a method for writing stored data according to an embodiment of the present invention, such as... Figure 7 As shown, the process includes the following steps: Step S701: Write storage data into the storage space of the storage host using the data writing method. The storage space includes multiple storage blocks and a spare storage block for the current data.
[0083] Specifically, the specific data writing method for storing data is detailed in the above implementation method and will not be repeated here.
[0084] Step S702: Monitor the current storage space remaining.
[0085] Step S703: When the current storage balance is less than the preset storage balance, perform garbage collection on the garbage data in multiple storage blocks using a garbage collection strategy.
[0086] Specifically, the preset storage margin can be 20% of the total storage space, and the preset storage margin can be flexibly set according to the actual situation.
[0087] When the current storage space is less than the preset storage space, this application performs garbage collection on garbage data in multiple storage blocks through a garbage collection strategy in order to make full use of storage space and improve the utilization rate of storage space.
[0088] Based on the same concept, this application also provides a method for retrieving stored data, which can be used in computer devices such as mobile phones, tablets, desktop computers, laptops, and servers. Figure 8 This is a flowchart of a method for writing stored data according to an embodiment of the present invention, such as... Figure 8 As shown, step S703 involves performing garbage collection on garbage data in multiple storage blocks based on a garbage collection strategy when the current storage balance is less than the preset storage balance. This includes: Step S7031: Traverse multiple storage blocks and select the storage block with the most invalid pages as the storage block to be reclaimed.
[0089] For example, iterate through all storage blocks (block_meta) and find the storage block (physical_block) through the preset physical address (block_meta.pba) of each storage block.
[0090] Step S7032: Based on the data storage linked list, obtain the transit storage blocks that are in an idle state. The data storage linked list is generated based on the initialization parameters of multiple storage blocks. The initialization parameters of multiple storage blocks include: the number of invalid pages, the preset physical address, the valid page identifier, and the blank page identifier for each storage block.
[0091] For example, by comparing the number of invalid pages in each storage block (hysical_block.invalid_number), the storage block with the most invalid pages is found and used as a transit storage block.
[0092] Step S7033: Based on the valid page identifier of the transfer storage block, transfer the valid pages in the transfer storage block to the transfer storage block, and update the valid page identifier and blank page identifier of the transfer storage block.
[0093] Specifically, by using the valid page identifier `physical_block.valid_mask` of the intermediate storage block, valid pages from the reclaimed storage block are transferred to the intermediate storage block, and the valid page identifiers in the intermediate storage block are... Set valid_mask to 1 and empty_mask, the blank page identifier, to 0.
[0094] Step S7034: Based on the blank page identifier and valid page identifier of the reclaimed storage block, query the failed pages of the reclaimed storage block and obtain the preset logical address of the failed pages.
[0095] Specifically, by using the empty page identifier physical_block.empty_mask and the valid page identifier valid_mask of the reclaimed storage block, the invalid pages in the reclaimed storage block are found, and the preset logical address page_lba of these invalid pages is obtained.
[0096] Step S7035: Determine the failed page data of the reclaimed storage block based on the preset logical address of the failed page.
[0097] Step S7036: Transfer the failed page data of the reclaimed storage block to the storage location corresponding to the intermediate storage block, and update the valid page identifier and blank page identifier of the intermediate storage block.
[0098] Specifically, the default page data `page_meta` in the reclaimed storage block is queried using the preset logical address `page_lba` of the aforementioned default page. The matched `page_meta.pba` is then used to transfer this data to the corresponding location in the intermediate storage block, and the valid page identifier `valid_mask` in the intermediate storage block is set to 1, while the empty page identifier `empty_mask` is set to 0.
[0099] In some specific implementations, after transferring the failed page data of the reclaimed storage block to the storage location corresponding to the intermediate storage block, the method further includes: Step S7037: Update the valid page identifier and blank page identifier of the failed page according to the preset physical address of the failed page of the reclaimed storage block.
[0100] Step S7038: Release the failed page data corresponding to the failed page of the reclaimed storage block.
[0101] Step S7039: Erase all storage data on the reclaimed storage block.
[0102] Step S7040: Perform initialization actions on the initialization parameters of the reclaimed storage block.
[0103] Specifically, the initialization parameters of the reclaimed storage block are initialized, as detailed in step a1 above, and will not be repeated here.
[0104] Step S7041: Update the data storage linked list according to the initialization parameters of the reclaimed storage block.
[0105] For example, based on the preset physical address of the invalid page of the reclaimed storage block, the valid_mask of the page corresponding to page_meta.pba is set to 0, invalid_number is incremented by 1, the metadata page_meta is released, all storage data on the reclaimed storage block is erased, and the reclaimed storage block physical_block is initialized and then added back to the data storage list free_physical_block_list.
[0106] By implementing the above-described embodiments, this application releases the data of the failed pages corresponding to the failed pages in the reclaimed storage block, and further reclaims the storage space of the reclaimed storage block, thereby reducing the size of the metadata, increasing the utilization rate of the storage space, strengthening the continuity of data storage to disk, and reducing the hashing degree of the preset logical address to the preset physical address of the storage block.
[0107] The data writing and recycling methods in this application have the following advantages: (1) This application solves the technical problem of low performance caused by the large size of page-mapped metadata and the discrete mapping in traditional storage units that adopt the storage management mode of "on-demand allocation and dynamic expansion". This application significantly reduces the memory occupation of metadata by using a hybrid storage method with block mapping as the main method and page mapping as the auxiliary method. Combined with a condition-triggered garbage collection strategy, it not only reclaims garbage space to improve storage utilization, but also reduces the amount of metadata and enhances the continuity of data storage, optimizes the efficiency of logical and physical address mapping, and significantly improves the overall performance of the storage unit. Therefore, this application is valuable in solving the problem.
[0108] (2) This application is applicable to high-load storage needs in various scenarios such as enterprise-level, cloud storage, and big data centers, providing key technical support for cost reduction and efficiency improvement in storage systems. Therefore, this application has a potential market.
[0109] (3) This application can significantly reduce metadata memory usage, reduce backend disk storage overhead, improve storage space utilization, and optimize storage performance to reduce hardware expansion needs, thereby significantly reducing the overall cost of the storage system from the resource utilization perspective. Therefore, this application can save costs.
[0110] (4) This application adopts a hybrid metadata management method with block mapping as the main method and page mapping as the auxiliary method. Compared with the traditional method that only uses page mapping mode, the metadata consumption is greatly reduced. In addition, this application also adds the condition of triggering garbage collection based on the remaining metadata memory. The collection process simultaneously realizes the optimization of space, metadata collection and data continuity, solving the pain points of low performance and small storage space of traditional storage from multiple dimensions. Therefore, this application is advanced.
[0111] (5) This application employs a hybrid metadata management system that prioritizes block mapping and supplements it with page mapping, significantly reducing the size of metadata. Furthermore, it incorporates the remaining metadata memory into the garbage collection trigger conditions, enabling simultaneous garbage space release, metadata collection, and improved data storage continuity during the collection process. This overcomes the technical bottlenecks of traditional storage units, which suffer from discrete mapping and limited performance. Therefore, this application is innovative.
[0112] (6) This application adopts a standardized data structure and linked list management, with clear process logic. It can be deployed and implemented using conventional storage hardware without the need for additional special equipment. Therefore, this application is highly feasible.
[0113] This embodiment also provides a data writing device for implementing the above embodiments and preferred embodiments; details already described will not be repeated. As used below, the term "module" can refer to a combination of software and / or hardware that performs a predetermined function. Although the device described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.
[0114] This embodiment provides a data writing device, such as... Figure 9 As shown, it includes: The data query module 901 is used to respond to the current data being transmitted to the storage host and query the target storage block to which the current data belongs based on multiple storage blocks. Each storage block includes multiple storage pages arranged in sequence. The data generation module 902 is used to generate a data storage linked list based on the initialization parameters of multiple storage blocks. The data storage linked list records the link identifiers of multiple spare storage blocks that are in an idle state. The data storage linked list is generated based on the initialization parameters of multiple storage blocks. The first write module 903 is used to write the current data sequentially into multiple storage pages of the backup storage block according to the link identifier recorded in the data storage chain when the query of the target storage block fails, and to update the current storage status of the backup storage block. The second write module 904 is used to perform a write operation on the current data based on the current storage state of the target storage page of the target storage block when the query for the target storage block is successful.
[0115] In some optional implementations, the initialization parameters for multiple storage blocks include: the number of invalid pages for each storage block, a preset physical address, a valid page identifier, and a blank page identifier.
[0116] In some alternative implementations, the first writing module is specifically used for: Select the link identifier of the spare storage block from the data storage list; Based on the link identifier of the spare storage block, obtain the spare storage block for the current data; Calculate the current offset address of the current data based on the current logical address of the current data, the preset logical address of the spare storage block, and the preset data volume of each storage page of the spare storage block; Based on the current offset address of the current data, the current data is written sequentially into multiple storage pages of the spare storage block, and the blank page identifier and valid page identifier of the multiple storage pages of the spare storage block are updated.
[0117] In some specific implementations, the second writing module is specifically used for: When the current storage state of the target storage page of the target storage block is blank, write the current data into the target storage page of the target storage block, and update the blank page identifier and valid page identifier of the target storage page. When the current storage state of the target storage page of the target storage block is not blank, update the valid page identifier and invalid page count of the target storage page, and query the pre-written data stored in the target storage page according to the preset logical address of the target storage page; When the pre-written data query fails, the current blank page of the standby storage block is determined based on the blank page identifier of the standby storage block, and the current data is stored in the current blank page, as well as the blank page identifier and valid page identifier of the current blank page are updated; When the pre-written data query is successful, the current data is written to the target storage page, and the valid page identifier and invalid page count of the target storage page are updated.
[0118] In some alternative implementations, if the current blank page is used up during the process of storing the current data through a blank page, the next spare storage block for the current data is requested from the data storage list, and the current data is written sequentially into multiple storage pages of the spare storage block.
[0119] In some alternative implementations, after the current data is written to the storage host, the following steps are also included: In response to a data read command, perform a data read operation on the current data.
[0120] Based on the same concept, this application also provides a data recycling device, comprising: The data writing module is used to write storage data into the storage space of the storage host through the data writing method. The storage space includes multiple storage blocks and a spare storage block for the current data. The data monitoring module is used to monitor the current storage space remaining. The data recycling module is used to perform garbage collection actions on garbage data in multiple storage blocks according to the garbage collection strategy when the current storage balance is less than the preset storage balance.
[0121] In some alternative implementations, the data recycling module is specifically used for: Traverse multiple storage blocks and select the storage block with the most invalid pages as the storage block to be reclaimed; Based on the data storage linked list, the system obtains the transit storage blocks that are in an idle state. The data storage linked list is generated based on the initialization parameters of multiple storage blocks. The initialization parameters of multiple storage blocks include: the number of invalid pages, the preset physical address, the valid page identifier, and the blank page identifier for each storage block. Based on the valid page identifier of the intermediate storage block, the valid pages in the intermediate storage block are transferred to the intermediate storage block, and the valid page identifier and blank page identifier of the intermediate storage block are updated. Based on the blank page identifier and valid page identifier of the reclaimed storage block, query the invalid pages of the reclaimed storage block and obtain the preset logical address of the invalid pages; Based on the preset logical address of the failed page, determine the failed page data of the reclaimed storage block; The failed page data of the reclaimed storage block is transferred to the storage location corresponding to the intermediate storage block, and the valid page identifier and blank page identifier of the intermediate storage block are updated.
[0122] In some alternative implementations, the data recycling device is specifically used for: Update the valid page identifier and blank page identifier of the failed page based on the preset physical address of the failed page in the reclaimed storage block; Release the failed page data corresponding to the failed page of the reclaimed storage block; Erase all stored data on the reclaimed storage block; Perform initialization actions on the initialization parameters of the reclaimed storage block; Update the data storage linked list based on the initialization parameters of the reclaimed storage block.
[0123] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods according to the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method.
[0124] For a description of the features in the embodiments corresponding to the data writing device, data recycling device, and data management device, please refer to the relevant descriptions in the embodiments corresponding to the data writing device, data recycling device, and data management method; they will not be repeated here.
[0125] Embodiments of this application also provide a storage host, such as Figure 10 As shown, it includes a memory 10 and a processor 20. The memory 10 stores a computer program, and the processor 20 is configured to run the computer program to perform the steps in any of the above-described embodiments of the data storage writing method, recycling method, and management method.
[0126] Embodiments of this application also provide a computer-readable storage medium storing a computer program, wherein the computer program is configured to execute the steps in the embodiments of any of the above-described methods for writing, recycling, and managing stored data when it runs.
[0127] In one exemplary embodiment, the aforementioned computer-readable storage medium may include, but is not limited to, various media capable of storing computer programs, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard disk, magnetic disk, or optical disk.
[0128] The embodiments of this application also provide a computer program product, which includes a computer program. When the computer program is executed by a processor, it implements the steps in any of the above-described embodiments of the data storage writing method, recycling method, and management method.
[0129] Embodiments of this application also provide another computer program product, including a non-volatile computer-readable storage medium storing a computer program. When the computer program is executed by a processor, it implements the steps in any of the above-described embodiments of the data writing method, data recycling method, and data management method.
[0130] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.
[0131] The foregoing has provided a detailed description of the data writing, recycling, and management methods provided in this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the embodiments above are merely for the purpose of helping to understand the methods and core ideas of this application. It should be noted that those skilled in the art can make various improvements and modifications to this application without departing from its principles, and these improvements and modifications also fall within the protection scope of the claims of this application.
Claims
1. A method for writing stored data, characterized in that, include: In response to the current data being transmitted to the storage host, the target storage block to which the current data belongs is queried based on multiple storage blocks, each storage block including multiple storage pages arranged in sequence; A data storage linked list is generated based on the initialization parameters of the plurality of storage blocks. The data storage linked list records the link identifiers of the plurality of spare storage blocks that are in an idle state. The data storage linked list is generated based on the initialization parameters of the plurality of storage blocks. When querying the target storage block fails, the current data is written sequentially into multiple storage pages of the backup storage block according to the link identifier recorded in the data storage chain, and the current storage status of the backup storage block is updated. When the query for the target storage block is successful, a write operation is performed on the current data based on the current storage state of the target storage page of the target storage block.
2. The method for writing stored data according to claim 1, characterized in that, The initialization parameters of the multiple storage blocks include: the number of invalid pages, the preset physical address, the valid page identifier, and the blank page identifier for each storage block.
3. The method for writing stored data according to claim 2, characterized in that, Based on the link identifier recorded in the data storage linked list, the current data is written sequentially into multiple storage pages of the spare storage block, and the current storage state of the spare storage block is updated, including: Select the link identifier of the spare storage block from the data storage list; Based on the link identifier of the backup storage block, obtain the backup storage block for the current data; The current offset address of the current data is calculated based on the current logical address of the current data, the preset logical address of the backup storage block, and the preset data volume of each storage page of the backup storage block. Based on the current offset address of the current data, the current data is written sequentially into multiple storage pages of the spare storage block, and the blank page identifier and the valid page identifier of the multiple storage pages of the spare storage block are updated.
4. The method for writing stored data according to claim 1, characterized in that, Based on the current storage state of the target storage page of the target storage block, a write operation is performed on the current data, including: When the current storage state of the target storage page of the target storage block is blank, the current data is written into the target storage page of the target storage block, and the blank page identifier and valid page identifier of the target storage page are updated. When the current storage state of the target storage page of the target storage block is not blank, update the valid page identifier and invalid page count of the target storage page, and query the pre-written data stored in the target storage page according to the preset logical address of the target storage page; When the pre-written data query fails, the current blank page of the backup storage block is determined according to the blank page identifier of the backup storage block, and the current data is stored through the current blank page, and the blank page identifier and valid page identifier of the current blank page are updated; When the pre-written data query is successful, the current data is written to the target storage page, and the valid page identifier and invalid page count of the target storage page are updated.
5. The method for writing stored data according to claim 4, characterized in that, During the process of storing the current data through the blank page, if the current blank page is used up, the next spare storage block for the current data is requested from the data storage chain, and the current data is written into multiple storage pages of the spare storage block in sequence.
6. The method for writing stored data according to any one of claims 1 to 5, characterized in that, After the current data is written to the storage host, the following is also included: In response to a data read command, perform a data read operation on the current data.
7. A method for reclaiming stored data, characterized in that, include: The storage data writing method according to any one of claims 1 to 6 is used to write storage data into the storage space of the storage host, wherein the storage space includes: a plurality of storage blocks and a spare storage block for the current data; Monitor the current storage capacity of the storage space; When the current storage capacity is less than the preset storage capacity, garbage collection is performed on the garbage data in the multiple storage blocks using a garbage collection strategy.
8. The method for reclaiming stored data according to claim 7, characterized in that, When the current storage balance is less than the preset storage balance, a garbage collection action is performed on the garbage data in the multiple storage blocks according to the garbage collection strategy, including: Traverse the multiple storage blocks and select the storage block with the most invalid pages as the reclaimed storage block; Based on the data storage linked list, a transit storage block in an idle state is obtained. The data storage linked list is generated based on the initialization parameters of the multiple storage blocks. The initialization parameters of the multiple storage blocks include: the number of invalid pages, the preset physical address, the valid page identifier, and the blank page identifier for each storage block. Based on the valid page identifier of the transfer storage block, the valid pages in the transfer storage block are transferred to the transfer storage block, and the valid page identifier and blank page identifier of the transfer storage block are updated; Based on the blank page identifier and valid page identifier of the reclaimed storage block, query the invalid page of the reclaimed storage block and obtain the preset logical address of the invalid page; The failed page data of the reclaimed storage block is determined based on the preset logical address of the failed page; The failed page data of the reclaimed storage block is transferred to the storage location corresponding to the intermediate storage block, and the valid page identifier and blank page identifier of the intermediate storage block are updated.
9. The method for reclaiming stored data according to claim 8, characterized in that, After transferring the failed page data of the reclaimed storage block to the storage location corresponding to the intermediate storage block, the method further includes: Update the valid page identifier and blank page identifier of the failed page according to the preset physical address of the failed page of the reclaimed storage block; Release the failed page data corresponding to the failed page of the reclaimed storage block; Erase all stored data on the reclaimed storage block; Perform initialization actions on the initialization parameters of the reclaimed storage block; The data storage list is updated according to the initialization parameters of the reclaimed storage block.
10. A storage host, characterized in that, include: A memory and a processor, the memory and the processor being communicatively connected to each other, the memory storing computer instructions, the processor executing the computer instructions to perform the steps of the data writing method according to any one of claims 1 to 6; or the data retrieval method according to any one of claims 7 to 9.