A data management method, apparatus, device, and medium based on spatial allocation
By dividing the physical storage space into multiple allocation groups, each with an independent free space management structure, the problem of reduced data storage speed caused by global bitmap lock contention bottleneck is solved, resulting in faster data storage speed.
Patent Information
- Application Number
- CN202611611095.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-10-19
- Publication Date
- 2026-11-17
AI Technical Summary
In existing technologies, the bottleneck of contention for global bitmap locks leads to a decrease in the speed of massive data storage.
A space-allocation-based data management method is adopted, which divides the physical storage space into multiple allocation groups. Each allocation group is configured with an independent free space management structure, allowing multiple CPU cores to search for and store data in parallel, avoiding serial queuing by using a global bitmap lock.
It improves data storage speed by fully utilizing the parallel advantages of multiple central processing unit cores, thereby increasing the speed at which massive amounts of data are stored in physical storage space.
Smart Images

Figure CN123114383A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of data storage management technology, specifically to a data management method, apparatus, device, and medium based on space allocation. Background Technology
[0002] In a secure file system, massive amounts of data are written to disk concurrently using multiple central processing unit (CPU) cores. During the write process, each CPU core must queue sequentially and use a global bitmap lock to search for a free block on the disk. Then, the CPU core writes the data to the free block. Because of this sequential queuing to contend for the global bitmap lock, the speed of storing massive amounts of data to disk is reduced.
[0003] In summary, existing technologies suffer from reduced storage speed for massive amounts of data due to contention bottlenecks in global bitmap locks.
[0004] Therefore, existing technologies still need to be improved and enhanced. Summary of the Invention
[0005] To address the aforementioned technical problems, this invention provides a data management method, apparatus, device, and medium based on space allocation, which solves the problem of reduced data storage speed for massive amounts of data due to contention bottlenecks in global bitmap locks in existing technologies.
[0006] To achieve the above objectives, the present invention adopts the following technical solution: In a first aspect, the present invention provides a data management method based on space allocation, wherein: Identify the target allocation group corresponding to the data to be stored. The allocation group represents a grouping of physical blocks on the physical storage space. The CPU core searches in parallel for the free space management structure of each target allocation group to obtain the target free physical blocks within the target allocation group that are compatible with the data to be stored. Each of the central processing unit cores stores the data to be stored in parallel to its corresponding target free physical block.
[0007] In one implementation, determining the target allocation group corresponding to the data to be stored includes: Obtain the file index node, and determine the first allocation group to which the file belongs based on the number of the file index node; Obtain the available capacity recorded by the free space management structure of each allocation group; Starting from the first allocation group, each allocation group is polled in turn to check whether the available capacity of each allocation group meets the storage capacity required for the data to be stored, so as to filter out the target allocation group.
[0008] In one implementation, the free space management structure includes a CNT tree; the free space management structure of each target allocation group is searched in parallel by the central processing unit core to obtain the target free physical blocks within each target allocation group that are compatible with the data to be stored, including: The CNT tree of each target allocation group is searched in parallel using the central processing unit core; Based on the length of the free area recorded in the CNT tree, a target free area that matches the storage capacity required for the data to be stored is found from the free area. The free area is a physical space region composed of consecutive free physical blocks, and the length of the free area corresponds to the available capacity of the free area. Select target free physical blocks from the target free area that are compatible with the data to be stored.
[0009] In one implementation, the free space management structure further includes a BNO tree; each of the central processing unit cores stores the groups of data to be stored in parallel to their respective target free physical blocks, and then further includes: The remaining free physical blocks formed after the target free physical blocks in the target free area are occupied by the data to be stored; Based on the remaining free physical blocks, a new free area is formed, and the BNO tree is updated based on the starting block number of the new free area, and the CNT tree is updated based on the length of the new free area.
[0010] In one implementation, the free space management structure further includes a BNO tree; each of the central processing unit cores stores the groups of data to be stored in parallel to their respective target free physical blocks, and then further includes: The remaining free physical blocks formed after the target free physical blocks in the target free area are occupied by the data to be stored; Acquire the newly freed physical blocks; Determine whether the starting block number of the free area recorded in the BNO tree, the block number of the new free physical block, and the block number of the remaining free physical blocks are consecutive to obtain a continuity determination result; Based on the continuity judgment result, the free area of the BNO tree is updated, and the updated free area of the BNO tree is mapped to the CNT tree to synchronously update the free area of the CNT tree.
[0011] In one implementation, the CPU core searches in parallel for the free space management structure of each target allocation group to obtain the target free physical block within the target allocation group that matches the data to be stored. This is further followed by: The data to be stored, which needs to be stored in the physical storage space, is temporarily stored in memory to form dirty pages; Based on the memory pressure generated by the dirty pages, determine whether to initiate an operation to write to the physical storage space.
[0012] In one implementation, after an operation to write to the physical storage space is initiated, the space management structure is modified based on the changes to the physical storage space caused by the write operation, thereby creating a delayed modification of the space management structure.
[0013] In one implementation, the free space management structure includes a BNO tree and a CNT tree; the data management method further includes: If no target free physical block matching the data to be stored is found within the target allocation group, check the fragment free area of the target allocation group; Based on the BNO tree, find the free area adjacent to the fragment free area, and merge the fragment free area with the adjacent free area; The BNO tree and the CNT tree are synchronously updated based on the merged free areas.
[0014] Secondly, embodiments of the present invention also provide a data management device based on space allocation, wherein the device comprises the following components: The grouping and filtering module is used to determine the target allocation group corresponding to the data to be stored. The allocation group represents a grouping of physical blocks on the physical storage space. The physical block filtering module is used to search the free space management structure of each target allocation group in parallel through the central processing unit core, so as to obtain the target free physical blocks in the target allocation group that are compatible with the data to be stored; The parallel storage module is used to store each group of data to be stored in parallel to their respective target free physical blocks through each of the central processing unit cores.
[0015] Thirdly, embodiments of the present invention also provide a terminal device, wherein the terminal device includes a memory, a processor, and a space allocation-based data management program stored in the memory and executable on the processor, wherein when the processor executes the space allocation-based data management program, it implements the steps of the space allocation-based data management method described above.
[0016] Fourthly, embodiments of the present invention also provide a computer-readable storage medium storing a space-allocation-based data management program, wherein when the space-allocation-based data management program is executed by a processor, it implements the steps of the space-allocation-based data management method described above.
[0017] Beneficial Effects: This invention is applicable to writing multiple data sets to be stored in parallel to their respective allocation groups. Specifically, this invention configures an independent free space management structure for each target allocation group. This free space management structure manages the free physical blocks within the target allocation group. In other words, the target allocation groups do not share a single free space management structure. Therefore, multiple CPU cores can search for the target free physical blocks of each target allocation group in parallel through the free space management structure. Finally, multiple CPU cores store the multiple data sets to be stored into their respective target free physical blocks in parallel. In summary, because this invention configures an independent free space management structure for each allocation group, multiple CPU cores can simultaneously search for the free space management structure of each allocation group without queuing, thereby accelerating the search for free physical blocks and improving the storage speed of storing data into free physical blocks on the physical storage space. Attached Figure Description
[0018] Figure 1 This is an overall flowchart of the present invention; Figure 2 A structural diagram of the space allocation-based data management device provided by the present invention; Figure 3 This is a block diagram illustrating the internal structure of a terminal device provided in an embodiment of the present invention. Detailed Implementation
[0019] The technical solutions of the present invention will be clearly and completely described below with reference to the embodiments and accompanying drawings. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention.
[0020] Research has revealed that in secure file systems, when massive amounts of data are concurrently written to disk using multiple CPU cores, each CPU core must queue sequentially and use a global bitmap lock to search for free blocks on the disk. Then, the CPU core writes the data to the free block. This sequential queuing for the global bitmap lock slows down the speed of storing massive amounts of data to disk. In other words, current technology suffers from a bottleneck in global bitmap lock contention, resulting in slower storage speeds for massive amounts of data.
[0021] To address the aforementioned technical problems, this invention provides a data management method, apparatus, device, and medium based on space allocation, resolving the issue of reduced data storage speed due to contention bottlenecks caused by global bitmap locks in existing technologies. Specifically, the invention first determines target allocation groups corresponding to the data to be stored; then, the central processing unit (CPU) cores search in parallel for the free space management structure of each target allocation group to obtain target free physical blocks within the target allocation group that are compatible with the data to be stored; finally, each CPU core stores the data to be stored in each group in parallel into its corresponding target free physical block.
[0022] For example, physical storage space (such as a disk) can be divided into several allocation groups, say six allocation groups. Each allocation group contains the same number of physical blocks. Each of the six allocation groups is configured with its own free space management structure, resulting in six free space management structures. Each free space management structure records information about the free physical blocks within its corresponding allocation group. For instance, there are two sets of data to be stored: a first set and a second set. The first set of data is written to file A (software-level file A), and the second set is written to file B (software-level file B). Allocation group one (hardware-level) has already been pre-configured for file A, and allocation group two (hardware-level) for file B. Therefore, allocation group one and allocation group two are the target allocation groups for the first and second sets of data, respectively.
[0023] Two CPU cores simultaneously search for the free space management structures of allocation group one and allocation group two, respectively, to find the target free physical blocks within allocation group one and allocation group two in parallel. Finally, the two CPU cores simultaneously store the first and second groups of data to be stored into their respective target free physical blocks.
[0024] Since each allocation group has its own independent free space management structure, one of the two CPU cores does not have to wait for the other CPU core to run out of free space management structure before it can use the free space management structure to search for free physical blocks. Instead, the two CPU cores are allowed to search for two free space management structures at the same time, thus making full use of the parallel advantages of multiple CPU cores.
[0025] Example 1: The space-allocation-based data management method of this example can be applied to terminal devices, which can be terminal products with video playback capabilities, such as televisions and computers. In this example, as... Figure 1 As shown, the data management method based on space allocation specifically includes the following steps: S110, determine the target allocation group corresponding to the data to be stored, the allocation group represents the grouping of physical blocks on the physical storage space; S120, the CPU core searches in parallel for the free space management structure of each target allocation group to obtain the target free physical block in the target allocation group that matches the data to be stored; S130, each of the central processing unit cores stores each group of data to be stored in parallel to its corresponding target free physical block.
[0026] In this embodiment, the data to be stored consists of several groups. A central processing unit core is responsible for allocating a target free physical block to a group of data to be stored and for storing the group of data to be stored into the target free physical block.
[0027] In this embodiment, the determination of the target allocation group corresponding to the data to be stored in step S110, where the allocation group represents the grouping of physical blocks on the physical storage space, includes the following specific steps S111, S112, and S113: S111, Obtain the file index node, and determine the first allocation group to which the file belongs based on the number of the file index node.
[0028] The file inode is the inode of the file to which the data to be stored needs to be written, and there is a corresponding relationship between the file inode and the allocation group to which the file belongs.
[0029] Physical storage space is a hardware storage space composed of several physical blocks, which can be a disk. The file system divides the physical storage space into allocation groups of the same size, meaning that each allocation group contains the same number of physical blocks.
[0030] S112, obtain the available capacity recorded by the free space management structure of each allocation group.
[0031] S113, using the first allocation group as the scanning starting point, poll each allocation group in a round-robin fashion to check whether the available capacity of each allocation group meets the storage capacity required by the data to be stored, so as to filter out the target allocation group.
[0032] The first allocation group is the pre-selected allocation group selected from the allocation groups.
[0033] A file is a file at the software level that needs to be written. Each file has a corresponding inode (Inode represents a file identifier). In the file system, each inode is permanently fixed in one of the allocation groups AG (AG stands for Allocation Group) when it is created. This allocation group AG is the allocation group AG where the file's inode Inode is located.
[0034] The header of the allocation group records the root of the inode for that allocation group, which is the root of the bitmap tree of inode numbers used for inode slot allocation. This allocation group AG is simply a fixed configuration for the file based on the inode inode. However, the available capacity of this allocation group AG may be insufficient to accommodate the data to be written to the file. Therefore, the allocation group matched based on the inode inode is only a pre-selected allocation group. The subsequent decision on whether to use this pre-selected allocation group as the target allocation group depends on its available capacity; that is, whether to store the data to be stored in this pre-selected allocation group.
[0035] The header of the allocation group AG also stores a free space management structure, which records the number of free physical blocks within the allocation group AG. Knowing the number of free physical blocks allows us to determine the available capacity of the allocation group AG. When the available capacity of the pre-selected allocation group is greater than or equal to the storage capacity required by the data to be stored, the pre-selected allocation group is directly used as the target allocation group for the data to be stored.
[0036] When the available capacity of the free physical blocks contained in the preselected allocation group is insufficient to meet the storage capacity required for the data to be stored, the target allocation group is selected from the remaining allocation groups based on the available capacity of the remaining allocation groups. The remaining allocation groups are the allocation groups after removing the preselected allocation group from all the allocation groups.
[0037] That is, according to the allocation group number, query whether the available capacity of the next allocation group (that is, the remaining allocation group) of the pre-selected allocation group meets the storage capacity required by the data to be stored. If it does not meet the requirements, continue to query the next allocation group until the target allocation group with available capacity that meets the storage capacity required by the data to be stored is found.
[0038] In this embodiment, when the preselected allocation group is not the target allocation group, that is, when cross-allocation group allocation is about to occur, the inode corresponding to the preselected allocation group is not modified. Only the BNO, CNT and free block statistics information in the target allocation group are updated. The mapping relationship between logical block number (the number of physical block in the allocation group) and absolute physical block number is recorded in the interval mapping of the file inode. This is so that the file can be located in the preselected allocation group corresponding to the file based on the inode, and then the target allocation group can be located based on the data block pointer after the preselected allocation group is modified, and then the data stored in the target allocation group can be found.
[0039] For example, the inode of the file test.log (test.log represents the filename) is fixed in allocation group AG1. Allocation group AG1 is the pre-selected allocation group for the file test.log. When more data to be stored than the available capacity of allocation group AG1 needs to be written to the file test.log, since the available capacity (i.e. the remaining space) of allocation group AG1 is insufficient, allocation group AG2 is polled. It is found that the available capacity of allocation group AG2 can accommodate the data to be stored, so allocation group AG2 is selected as the target allocation group. This embodiment does not require modifying any Inode data in the pre-selected allocation group. Instead, it directly records the mapping relationship between the logical block number and the absolute physical block number corresponding to this write in the Inode range mapping table of the file test.log itself. The allocation group where the data is actually stored can be determined by the disk address space to which the absolute physical block number in the mapping record belongs. For example, the write data from the 50MB to the 100MB of the corresponding file is stored in allocation group AG2.
[0040] In this embodiment, step S120, which involves parallel searching of the free space management structure of each target allocation group by the central processing unit core to obtain the target free physical blocks within each target allocation group that are compatible with the data to be stored, includes the following specific steps S121, S122, and S123: S121, the CNT tree of each target allocation group is searched in parallel by the central processing unit core.
[0041] The CNT tree and BNO tree constitute the free space management structure; that is, the free space management structure includes both the CNT tree and the BNO tree. Each allocation group (AG) has a CNT tree and a BNO tree at its header. The CNT tree, or Count Btree, is used to sort the free areas containing free physical blocks according to the length of consecutive free physical blocks. The BNO tree, or Block Number Btree, is used to sort the free areas according to the size of the starting physical block number of the free area.
[0042] For example, the block numbers of the free physical blocks within the target allocation group include 150 to 155, 200 to 208, 250 to 257, 300 to 302, and 400 to 404. That is, there are five free areas within the target allocation group. The five free areas are E1 (150,6), E2 (200,9), E3 (250,8), E4 (300,3), and E5 (400,5). The BNO tree sorts the above five free areas by the starting free physical block number of the free area: E1 (150), E2 (200), E3 (250), E4 (300), and E5 (400). The CNT tree sorts the above five free areas by the length of the free area: E4 (3), E5 (5), E1 (6), E3 (8), and E2 (9).
[0043] S122, based on the length of the free area recorded in the CNT tree, find a target free area from the free area that matches the storage capacity required for the data to be stored. The free area is a physical space region composed of consecutive free physical blocks, and the length of the free area corresponds to the available capacity of the free area.
[0044] The storage capacity of each free physical block is fixed. Therefore, the available capacity of the free area can be determined based on the length of the free area (the length of the free area corresponds to the number of consecutive free physical blocks included in the free area). Thus, the target free area suitable for storing the data to be stored can be selected from the free areas based on the length of the free area.
[0045] Continuing the example above, if the data to be stored requires 4 free physical blocks, then based on the CNT tree, we can find that 5 is greater than 4. Therefore, E5 is chosen as the target free area for the data to be stored, meaning the data can be written to free area E5. Indexing the target free area according to its length ensures that the free area's length matches the data to be stored, rather than using a free area whose length is much greater than the required capacity. Using a free area that is much larger than the required capacity would lead to more fragmentation, resulting in discontinuous free physical blocks. Discontinuous free physical blocks are detrimental to the storage of large amounts of data later on.
[0046] S123, Select target free physical blocks that match the data to be stored from the free physical blocks in the target free area.
[0047] In other words, a certain length of free physical block is extracted from the starting free physical block of the target free area as the target free physical block for the data to be stored, and the length of the extracted block is adapted to the storage amount required for the data to be stored.
[0048] In step S130 of this embodiment, each of the central processing unit (CPU) cores stores each group of data to be stored in parallel to its corresponding target free physical block. In other words, each CPU core simultaneously stores its own data to be stored onto the target free physical block allocated to that data. Each CPU core has the function of writing data to physical blocks in parallel, thus improving the data writing speed.
[0049] After the data to be stored is stored in the target free physical block, the target free physical block is occupied. At this time, it is necessary to modify the CNT tree and simultaneously modify the BNO tree so that the modified CNT tree and BNO tree can correctly record the current free physical block.
[0050] This embodiment modifies and updates the CNT tree and BNO tree, including the following specific steps: determining the remaining free physical blocks formed after the target free physical blocks in the target free area are occupied by the data to be stored; forming a new free area based on the remaining free physical blocks; updating the BNO tree based on the starting block number of the new free area; and updating the CNT tree based on the length of the new free area.
[0051] When the data to be stored does not occupy all the free physical blocks in the target free area, there will be remaining free physical blocks in the target free area. Therefore, the number of free physical blocks in the target free area has changed. Thus, the BNO tree and CNT tree need to be modified so that the BNO tree and CNT tree can correctly record the free physical blocks in the current free area.
[0052] Continuing with the example above, another data to be stored is stored in the free area E5, occupying blocks 400 to 402, leaving only two free physical blocks 403 and 404 in E5. Let e5 represent the modified E5, that is, e5 is the new free area, and the new free area e5: blocks 403 to 404.
[0053] The original BNO tree is modified based on the starting block number of the new free area e5 (which is 403). The free areas recorded in the modified BNO tree are sorted as follows: E1 (150), E2 (200), E3 (250), E4 (300), e5 (403). The original CNT tree is updated based on the length of the new free area e5. The free areas recorded in the updated CNT tree are sorted as follows: e5 (2), E4 (3), E1 (6), E3 (8), E2 (9).
[0054] In this embodiment, after the target free physical block in the target free area of the target allocation group is occupied, the BNO tree and CNT tree inside the target allocation group are modified so that the BNO tree and CNT tree are updated in a timely manner to synchronize the information of the remaining free physical blocks in the target allocation group.
[0055] Example 2: This example provides a method for merging fragmented free areas, including the following steps S210 to S270: S210, determine the target allocation group corresponding to the data to be stored, the allocation group represents the grouping of physical blocks on the physical storage space; S220, the CPU core searches in parallel for the free space management structure of each target allocation group to obtain the target free physical block in the target allocation group that matches the data to be stored; S230, each group of data to be stored is stored in parallel to its corresponding target free physical block through each of the central processing unit cores; S240, determine the remaining free physical block formed after the target free physical block in the target free area is occupied by the data to be stored; S250, acquire the newly released free physical block; S260, determine whether the starting block number of the free area recorded in the BNO tree, the block number of the new free physical block, and the block number of the remaining free physical blocks are consecutive, and obtain the continuity judgment result; S270, based on the continuity judgment result, update the free area of the BNO tree, and map the updated free area of the BNO tree to the CNT tree, so as to synchronously update the free area of the CNT tree.
[0056] S210, S220, and S230 in this embodiment are the same as S110, S120, and S130 in Embodiment 1, respectively, and will not be described again here.
[0057] In this embodiment, the remaining free physical block in S240 is the free physical block remaining after the free physical block in the free area is occupied, and the new free physical block in step S250 is the free physical block that becomes a previously occupied physical block due to the release of the data on it.
[0058] Whether the number of free physical blocks decreases due to being occupied or increases due to the release of new free physical blocks, the distribution and number of free physical blocks in the allocation group are altered. Therefore, the BNO tree and CNT tree need to be updated so that the updated BNO tree and CNT tree can accurately record the information of free physical blocks within the allocation group.
[0059] In S260 of this embodiment, the BNO tree records several free areas. Each free area has a corresponding start number of a free physical block. If the block number of the new free physical block and the block number of the remaining free physical blocks are consecutive with the start block numbers of the several free areas, the new free physical block and the remaining free physical blocks are merged into the free area. The BNO tree is then updated based on the merged free area, and the free areas of the updated BNO tree are mapped to the CNT tree.
[0060] Continuing with the example above, the original free areas of one allocation group are E1(150,6), E2(200,9), E3(250,8), E4(300,3), and E5(400,5). The BNO tree sorts the above five free areas by the starting free physical block number of the free area: E1(150), E2(200), E3(250), E4(300), and E5(400). The CNT tree sorts the above five free areas by the length of the free area: E4(3), E5(5), E1(6), E3(8), and E2(9).
[0061] Free physical blocks 400, 401, and 402 in free space E5 are occupied, leaving only free physical blocks 403 and 404 in free space E5. If the allocation group containing the above five free areas releases physical blocks 405, 406, 407, 408, and 409, resulting in five new free physical blocks in the allocation group, since 403, 404, 405, 406, 407, 408, and 409 are continuous, they can be merged into free area E5 to modify and update free area E5. The updated free area E5 is (403, 7). Based on the updated free area E5, the BNO tree of the allocation group is updated. The updated BNO tree is: E1 (150), E2 (200), E3 (250), E4 (300), E5 (403). The updated CNT tree is: E4 (3), E1 (6), E5 (7), E3 (8), E2 (9).
[0062] In another embodiment, when a physical block is released from an allocation group, the absolute physical block number is first resolved to the allocation group AG number and the offset within the AG (the offset being the position of the released physical block within the allocation group). Then, in the BNO tree, the left (i.e., the side where the starting physical block number of the target segment is located) and right (i.e., the side where the ending physical block number of the target segment are located) adjacent records of the target segment (a segment composed of several released physical blocks) are searched simultaneously. Four cases are handled: if the left side can be merged, the number of consecutive blocks in the left record is updated; if the right side can be merged, the starting block number and the number of consecutive blocks in the right record are updated; if both sides can be merged, they are merged into one large block and one record is deleted; if neither side can be merged, it is inserted as a new record into the BNO tree. After merging, the corresponding record in the CNT tree is updated synchronously, and the free block count is increased. The collaborative work of the two trees ensures that a suitable segment can be found quickly during allocation, and also ensures that adjacent spaces can be placed into the original free area during release to increase the length of the free area, thereby suppressing fragmentation.
[0063] Example 3: This example provides a data management method based on dirty pages, including the following specific steps S310 to S350: S310, determine the target allocation group corresponding to the data to be stored, the allocation group represents the grouping of physical blocks on the physical storage space; S320, the data to be stored that needs to be stored in the physical storage space is temporarily stored in memory to form a dirty page; S330, based on the memory pressure generated by the dirty pages, determine whether to initiate the operation of writing to the physical storage space; When the memory pressure generated by dirty pages reaches the critical threshold for initiating writing to physical storage space, step S340 is executed; otherwise, other data to be stored that needs to be stored in physical storage space continues to be temporarily stored in memory.
[0064] S340, the CPU core searches in parallel for the free space management structure of each target allocation group to obtain the target free physical block in the target allocation group that matches the data to be stored; S350, each of the central processing unit cores stores each group of data to be stored in parallel to its corresponding target free physical block.
[0065] S310, S340, and S350 in this embodiment are the same as S110, S120, and S130 in Embodiment 1, respectively, and will not be described again here.
[0066] In this embodiment, S320 temporarily stores the data to be stored in memory to form dirty pages (dirty pages are memory pages that have been modified but have not yet been flushed back to the physical storage space). Instead of storing each or every piece of data to be stored in memory, the data is temporarily stored in memory and not directly written to the physical storage space. At this time, the BNO tree and CNT tree are not modified. When enough data to be stored is temporarily stored in memory, it is written to the physical storage space together, and the BNO tree and CNT tree are modified.
[0067] This embodiment achieves delayed allocation of free physical blocks for data to be stored through dirty pages, thereby delaying the modification of the BNO tree and CNT tree. The high overhead of "requiring a physical block and immediately modifying the BNO tree and CNT tree" during a single write operation is distributed as "temporarily storing on memory pages first, and then writing to physical storage space": the BNO tree and CNT tree only request large contiguous segments at a time according to the number of adjacent dirty pages during write-back. This not only makes single writes faster, but also ensures that the physical block intervals are likely to be contiguous when written to disk, thereby improving the throughput of subsequent reads.
[0068] Example 4: This example provides a method for updating the BNO tree and CNT tree, including the following specific steps S410, S420, and S430: S410, determine the target allocation group corresponding to the data to be stored, the allocation group represents the grouping of physical blocks on the physical storage space; When no target free physical block matching the data to be stored is found within the target allocation group, the fragmented free area of the target allocation group is checked, and a free area adjacent to the fragmented free area is found based on the BNO tree. The fragmented free area is then merged with the adjacent free area. The BNO tree and the CNT tree are updated synchronously based on the merged free area.
[0069] When no target free physical block matching the data to be stored is found in the target allocation group, it is necessary to check whether there are many fragmented physical blocks in the allocation group, resulting in very short free areas. Consequently, the free areas are insufficient to accommodate the data to be stored, and thus, a sufficient number of free physical blocks cannot be found in each space area to accommodate the data to be stored.
[0070] When this happens, it is necessary to check whether the free area where the fragmented physical block is located can be merged to form a new free area. The length of the new free area will be larger than the length of the fragmented free area. When the length of the new free area can accommodate the data to be stored, the data to be stored is stored in the new free area, and the BNO tree and CNT tree are updated synchronously.
[0071] S420: The central processing unit core searches in parallel the updated CNT tree of each target allocation group to obtain the target free physical block in the target allocation group that matches the data to be stored.
[0072] In this embodiment, only the CNT tree is used when allocating physical blocks. Since the free physical blocks recorded in the BNO tree and CNT tree need to be synchronized, the BNO tree must also be updated when the CNT tree is updated.
[0073] S430, each of the central processing unit cores stores each group of data to be stored in parallel to its corresponding target free physical block.
[0074] In this embodiment, the BNO tree of the allocation group AG is scanned, holding only the AG read lock (i.e., allowing data to be written to the allocation group AG). When two consecutive free areas with different physical addresses are found, the process transitions to the merging phase, which involves upgrading the AG read lock to a write lock (i.e., prohibiting data writing to the allocation group AG, but allowing merging processing). The entire merging process is batch-controlled. Every time a fixed number of fragmented physical blocks are merged or a rescheduling flag is detected, the AG write lock is immediately released, the currently scanned physical block number is recorded as the starting point for the next step, and the rescheduling interface is actively invoked to yield the central processing unit. The merging operation ends when the fragmentation rate falls back to a set range or when the entire allocation group AG has been scanned.
[0075] In Example 5, when the system enables object reuse (a physical block must be zeroed out before it can be reassigned to data to be stored), only zeroed-out clean free blocks are merged. Physical blocks that are not zeroed out are placed in a separate queue and do not enter the BNO tree and CNT tree until the asynchronous erase thread completes the erasure process, thus ensuring that the merging process does not leak old data. When unloading the file system, the synchronous cancellation of delayed work items must be called first to ensure that background worker threads have completely stopped before allowing the release of the BNO tree and CNT tree of the allocation group AG, avoiding dangling reference exceptions in the kernel.
[0076] In summary, this invention achieves simultaneous progress in three objectives—concurrency security, low fragmentation rate, and scalable physical block capacity—through a four-step collaborative approach: allocation groups, dual-tree (BNO tree and CNT tree) free space indexing, delayed allocation, and background block merging. Specifically, it replaces traditional global bitmap locks with grouped read / write semaphores, distributes single-point contention across allocation groups (AGs), postpones metadata modifications on high-frequency write paths to the write-back phase, and moves defragmentation from the foreground to the background for asynchronous execution. This synergy ensures stable write throughput and predictable tail latency in scenarios involving high concurrency, long-term operation, and massive data volumes.
[0077] This embodiment also provides a data management device based on space allocation, such as... Figure 2 As shown, the device comprises the following components: Grouping and filtering module 01 is used to determine the target allocation group corresponding to the data to be stored. The allocation group represents the grouping of physical blocks on the physical storage space. The physical block filtering module 02 is used to search the free space management structure of each target allocation group in parallel through the central processing unit core, so as to obtain the target free physical blocks in the target allocation group that are compatible with the data to be stored; Parallel storage module 03 is used to store each group of data to be stored in parallel to their respective target free physical blocks through each of the central processing unit cores.
[0078] Based on the above embodiments, the present invention also provides a terminal device, the principle block diagram of which can be as follows: Figure 3 As shown, the terminal device includes a processor, memory, network interface, and display screen connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The network interface is used to communicate with external terminals via a network connection. When the computer program is executed by the processor, it implements a space-allocation-based data management method. The display screen can be a liquid crystal display (LCD) or an e-ink display.
[0079] Those skilled in the art will understand that Figure 3 The schematic diagram shown is only a partial structural diagram related to the present invention and does not constitute a limitation on the terminal device to which the present invention is applied. The specific terminal device may include more or fewer components than shown in the figure, or combine certain components, or have different component arrangements.
[0080] In one embodiment, a terminal device is provided, the terminal device including a memory, a processor, and a space-allocation-based data management program stored in the memory and executable on the processor. When the processor executes the space-allocation-based data management program, it implements the following operation instructions: Identify the target allocation group corresponding to the data to be stored. The allocation group represents a grouping of physical blocks on the physical storage space. The CPU core searches in parallel for the free space management structure of each target allocation group to obtain the target free physical blocks within the target allocation group that are compatible with the data to be stored. Each of the central processing unit cores stores the data to be stored in parallel to its corresponding target free physical block.
[0081] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided by this invention can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.
[0082] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims
1. A data management method based on spatial allocation, characterized in that, include: Identify the target allocation group corresponding to the data to be stored. The allocation group represents a grouping of physical blocks on the physical storage space. The free space management structure of each target allocation group is searched in parallel by the central processing unit core to obtain the target free physical blocks in the target allocation group that are compatible with the data to be stored. The free space management structure includes BNO tree and CNT tree. Each of the central processing unit cores stores the data to be stored in parallel to its corresponding target free physical block. The CPU core performs parallel searches of the free space management structure of each target allocation group to obtain target free physical blocks within each target allocation group that are compatible with the data to be stored, including: The CNT tree of each target allocation group is searched in parallel using the central processing unit core; Based on the length of the free area recorded in the CNT tree, a target free area that matches the storage capacity required for the data to be stored is found from the free area. The free area is a physical space region composed of consecutive free physical blocks, and the length of the free area corresponds to the available capacity of the free area. Select target free physical blocks from the target free area that are compatible with the data to be stored.
2. The data management method based on spatial allocation as described in claim 1, characterized in that, Determine the target allocation group corresponding to the data to be stored, including: Obtain the file index node, and determine the first allocation group to which the file belongs based on the number of the file index node; Obtain the available capacity recorded by the free space management structure of each allocation group; Starting from the first allocation group, each allocation group is polled in turn to check whether the available capacity of each allocation group meets the storage capacity required for the data to be stored, so as to filter out the target allocation group.
3. The data management method based on space allocation as described in claim 1, characterized in that, Each of the central processing unit cores stores the data to be stored in parallel to its corresponding target free physical block, and then the process further includes: The remaining free physical blocks formed after the target free physical blocks in the target free area are occupied by the data to be stored; Based on the remaining free physical blocks, a new free area is formed, and the BNO tree is updated based on the starting block number of the new free area, and the CNT tree is updated based on the length of the new free area.
4. The data management method based on space allocation as described in claim 1, characterized in that, Each of the central processing unit cores stores the data to be stored in parallel to its corresponding target free physical block, and then the process further includes: The remaining free physical blocks formed after the target free physical blocks in the target free area are occupied by the data to be stored; Acquire the newly freed physical blocks; Determine whether the starting block number of the free area recorded in the BNO tree, the block number of the new free physical block, and the block number of the remaining free physical blocks are consecutive to obtain a continuity determination result; Based on the continuity judgment result, the free area of the BNO tree is updated, and the updated free area of the BNO tree is mapped to the CNT tree to synchronously update the free area of the CNT tree.
5. The data management method based on spatial allocation as described in claim 1, characterized in that, The CPU core performs parallel searches of the free space management structure of each target allocation group to obtain the target free physical blocks within the target allocation group that are compatible with the data to be stored. This process also includes: The data to be stored, which needs to be stored in the physical storage space, is temporarily stored in memory to form dirty pages; Based on the memory pressure generated by the dirty pages, determine whether to initiate an operation to write to the physical storage space.
6. The data management method based on space allocation as described in claim 5, characterized in that, After the operation of writing to the physical storage space is initiated, the free space management structure is modified based on the changes in the physical storage space caused by the write operation, so as to delay the modification of the space management structure.
7. The data management method based on space allocation as described in claim 1, characterized in that, The data management method based on space allocation also includes: If no target free physical block matching the data to be stored is found within the target allocation group, check the fragment free area of the target allocation group; Based on the BNO tree, find the free area adjacent to the fragment free area, and merge the fragment free area with the adjacent free area; The BNO tree and the CNT tree are synchronously updated based on the merged free areas.
8. A data management device based on spatial allocation, characterized in that, The device comprises the following components: The grouping and filtering module is used to determine the target allocation group corresponding to the data to be stored. The allocation group represents a grouping of physical blocks on the physical storage space. The physical block filtering module is used to search the free space management structure of each target allocation group in parallel through the central processing unit core, so as to obtain the target free physical blocks in the target allocation group that are compatible with the data to be stored. The free space management structure includes a BNO tree and a CNT tree. A parallel storage module is used to store each group of data to be stored in parallel to their respective target free physical blocks through each of the central processing unit cores; The CPU core performs parallel searches of the free space management structure of each target allocation group to obtain target free physical blocks within each target allocation group that are compatible with the data to be stored, including: The CNT tree of each target allocation group is searched in parallel using the central processing unit core; Based on the length of the free area recorded in the CNT tree, a target free area that matches the storage capacity required for the data to be stored is found from the free area. The free area is a physical space region composed of consecutive free physical blocks, and the length of the free area corresponds to the available capacity of the free area. Select target free physical blocks from the target free area that are compatible with the data to be stored.
9. A terminal device, characterized in that, The terminal device includes a memory, a processor, and a space-allocation-based data management program stored in the memory and executable on the processor. When the processor executes the space-allocation-based data management program, it implements the steps of the space-allocation-based data management method as described in any one of claims 1-7.
10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a space-allocation-based data management program, which, when executed by a processor, implements the steps of the space-allocation-based data management method as described in any one of claims 1-7.
