A bitmap processing method and apparatus

Abstract

The embodiment of the application discloses a bitmap processing method and device, the method comprises the following steps: dividing a bitmap into a preset number of bitmap blocks; loading the bitmap blocks hit by program access among all the divided bitmap blocks into a memory. Through the embodiment scheme, the bitmap occupied memory is controlled within a reasonable range while ensuring fine granularity.

Description

Technical Field

[0001] This article relates to data processing techniques, and more particularly to a bitmap processing method and apparatus. Background Technology

[0002] Asynchronous replication is an important data protection technology that periodically replicates changed data from the primary logical volume to secondary logical volumes to continuously protect business data. The primary logical volume hosts user business operations, while the secondary logical volume protects user data. In the event of a disaster in the system hosting the primary logical volume, the system can be manually switched to the secondary logical volume to continue hosting business operations.

[0003] The general implementation of asynchronous replication technology involves recording data modifications to the primary logical volume in its metadata. When the replication cycle arrives, a snapshot technique is used to generate a complete view of the primary logical volume. Based on the records in the metadata, the modified data is copied from the generated snapshot view to the secondary logical volume. This method of recording data differences is not only applied to asynchronous replication technology but is also widely used in other data protection features.

[0004] Generally, bitmaps are used to record data changes on the primary resource. A common implementation of bitmaps (a method for recording dataset differences) involves dividing a logical volume into multiple blocks according to a specified data block size. Bitmap nodes are used to mark whether each data block has been modified. When a write operation occurs, the bitmap node corresponding to the written data block is calculated, and the corresponding bitmap node in memory is marked as modified. For example, if the logical volume size is 1TB and it is divided into 8KB data block sizes, then the logical volume can be divided into 1TB / 8KB = 2^27 data blocks. Each data block corresponds to a bitmap node, so the bitmap for this logical volume has 2^27 bitmap nodes. Each bitmap node describes information about the corresponding data block (such as whether the data block has been modified, whether the data block has been transferred to the slave, etc.). Assuming one bitmap node occupies 1 byte of storage space, the bitmap occupies 128MB of space. When a write operation hits the nth data block, the information in the nth bitmap node is set to modified.

[0005] The granularity of data block partitioning determines the size of the bitmap. The finer the granularity, the larger the bitmap and the more memory it requires. With the rapid increase in disk storage capacity, the memory usage of bitmaps will continue to grow, making memory reduction a critical issue. While coarse-grained partitioning can reduce bitmap memory usage, it requires copying a large amount of unmodified data to the secondary logical volume. This not only lengthens the copying time, impacting the primary logical volume's operations, but also wastes bandwidth resources used for network transmission. Summary of the Invention

[0006] This application provides a bitmap processing method and apparatus that can control the memory occupied by bitmaps within a reasonable range while ensuring fine granularity.

[0007] This application provides a bitmap processing method, which may include:

[0008] Divide the bitmap into a preset number of bitmap blocks;

[0009] Load the bitmap blocks that are accessed and hit by the program from all the divided bitmap blocks into memory.

[0010] In an exemplary embodiment of this application, after loading the bitmap blocks accessed by the program from all the divided bitmap blocks into memory, the method may further include:

[0011] The bitmap block objects loaded into memory are connected using a pre-defined Least Recently Used (LRU) linked list; the LRU linked list has a preset length.

[0012] Whenever a new bitmap block object is inserted at the head of the LRU linked list, the bitmap block objects that were originally inserted in the LRU linked list are moved to the tail of the LRU linked list.

[0013] When the LRU linked list is full, the bitmap block corresponding to the bitmap block object located at the tail of the LRU linked list is evicted from memory, and the evicted bitmap block is written back to the metadata.

[0014] In an exemplary embodiment of this application, the lru linked list may include: an active object linked list;

[0015] The method may further include: when inserting a new bitmap block object at the head of the lru linked list, inserting the new bitmap block object at the head of the active object linked list.

[0016] In an exemplary embodiment of this application, the lru linked list may further include: an inactive object linked list;

[0017] When the active object list is full, the bitmap block object located at the tail of the active object list is removed from the active object list, and the removed bitmap block object is inserted at the head of the inactive object list.

[0018] When the inactive object list is full, the bitmap block corresponding to the bitmap block object located at the tail of the inactive object list is evicted from memory, and the evicted bitmap block is written back to the metadata.

[0019] In an exemplary embodiment of this application, the method may further include:

[0020] Get the sequence number of the first tile that needs to be modified;

[0021] Based on the sequence number, search for the first bitmap object corresponding to the first bitmap in the lru linked list;

[0022] When the first bitmap block object is not found in the LRU linked list, the hash value of the first bitmap block's sequence number is calculated, and the first bitmap block is searched from a preset hash table based on the hash value.

[0023] When the first bitmap is not found in the hash table, the first bitmap is retrieved from the metadata, loaded into memory, and the first bitmap object is inserted into the head of the LRU linked list and the hash table.

[0024] In an exemplary embodiment of this application, the method may further include: defining bitmap block objects within a first preset length at the head of the lru linked list as active bitmap block objects, and searching for the first bitmap block object in the region where the active bitmap block object is located in the lru linked list.

[0025] In an exemplary embodiment of this application, the method may further include:

[0026] When the first bitmap object is found in the LRU linked list, or when the first bitmap is found in the hash table, the first bitmap node to be modified in the first bitmap is modified, and the modified first bitmap object is inserted into the head of the LRU linked list.

[0027] In an exemplary embodiment of this application, the method may further include:

[0028] After retrieving the first bitmap from the metadata, the first bitmap node to be modified in the first bitmap is modified, and the first bitmap object corresponding to the modified first bitmap is added to the head of the LRU linked list and the hash table; or,

[0029] After obtaining the first bitmap from the metadata and adding the first bitmap object corresponding to the first bitmap to the head of the LRU linked list and the hash table, the first bitmap node to be modified in the first bitmap is modified.

[0030] In an exemplary embodiment of this application, calculating the hash value of the sequence number of the first bitmap block includes: calculating the hash value of the sequence number of the first bitmap block using a preset hash calculation formula;

[0031] The hash calculation formula includes: h(x) = (x % (k / 2)) * (y >> 1);

[0032] Where h(x) is the hash value; x is the sequence number of the first bitmap block; k is the number of bitmap blocks in the largest bitmap; and y is the bitmap id.

[0033] In an exemplary embodiment of this application, the method may further include: after obtaining the first bitmap tile from the metadata, calculating the hash value of the sequence number of the first bitmap tile according to the hash calculation formula, and inserting the first bitmap tile object into the hash table according to the calculated hash value, so as to realize hash mapping of the sequence number of the bitmap tile.

[0034] In an exemplary embodiment of this application, the method may further include: establishing a collision chain when the calculated hash values ​​collide.

[0035] This application also provides a bitmap processing apparatus, which may include a processor and a computer-readable storage medium, wherein the computer-readable storage medium stores instructions that, when executed by the processor, implement the bitmap processing method described in any of the above embodiments.

[0036] Compared with related technologies, the embodiments of this application may include: dividing the bitmap into a preset number of bitmap blocks; loading the bitmap blocks that are accessed and hit by the program from all the divided bitmap blocks into memory. Through this embodiment, only the hit bitmap blocks are loaded into memory, thereby controlling the memory occupied by the bitmap within a reasonable range while ensuring fine granularity.

[0037] Other features and advantages of this application will be set forth in the following description, and will be apparent in part from the description, or may be learned by practicing the application. Other advantages of this application can be realized and obtained by means of the solutions described in the description and the accompanying drawings. Attached Figure Description

[0038] The accompanying drawings are used to provide an understanding of the technical solutions of this application and constitute a part of the specification. They are used together with the embodiments of this application to explain the technical solutions of this application and do not constitute a limitation on the technical solutions of this application.

[0039] Figure 1 This is a flowchart of a bitmap processing method according to an embodiment of this application;

[0040] Figure 2 This is a schematic diagram of bitmap block hits in an embodiment of this application;

[0041] Figure 3 This is a schematic representation of the LRU chain in an embodiment of this application;

[0042] Figure 4 This is a schematic diagram of the hash table structure in an embodiment of this application;

[0043] Figure 5The intent of adding LRU linked lists and hash representations to the bitmap block objects in this application embodiment;

[0044] Figure 6 This is a schematic diagram illustrating the process of finding a bitmap block and modifying the bitmap nodes within it, according to an embodiment of this application.

[0045] Figure 7 This is a block diagram of the bitmap processing apparatus according to an embodiment of this application. Detailed Implementation

[0046] This application describes several embodiments, but these descriptions are exemplary and not restrictive, and it will be apparent to those skilled in the art that many more embodiments and implementations are possible within the scope of the embodiments described herein. Although many possible combinations of features are shown in the drawings and discussed in the detailed description, many other combinations of the disclosed features are also possible. Unless specifically limited, any feature or element of any embodiment may be used in combination with, or may replace, any feature or element of any other embodiment.

[0047] This application includes and contemplates combinations of features and elements known to those skilled in the art. The embodiments, features, and elements disclosed in this application may also be combined with any conventional features or elements to form a unique inventive scheme as defined by the claims. Any feature or element of any embodiment may also be combined with features or elements from other inventive schemes to form another unique inventive scheme as defined by the claims. Therefore, it should be understood that any feature shown and / or discussed in this application may be implemented individually or in any suitable combination. Therefore, the embodiments are not limited except by the limitations imposed by the appended claims and their equivalents. Furthermore, various modifications and changes may be made within the scope of the appended claims.

[0048] Furthermore, in describing representative embodiments, the specification may have presented methods and / or processes as a specific sequence of steps. However, the method or process should not be limited to the specific order of steps described herein, to the extent that it does not depend on such a specific order. As will be understood by those skilled in the art, other sequences of steps are also possible. Therefore, the specific order of steps set forth in the specification should not be construed as a limitation of the claims. Moreover, the claims concerning the method and / or process should not be limited to the steps performed in the written order, and those skilled in the art will readily understand that these orders can be varied and still remain within the spirit and scope of the embodiments of this application.

[0049] This application provides a bitmap processing method, such as... Figure 1 As shown, the method may include steps S101-S102:

[0050] S101. Divide the bitmap into a preset number of bitmap blocks;

[0051] S102. Load the bitmap blocks that are accessed and hit by the program from all the divided bitmap blocks into memory.

[0052] In an exemplary embodiment of this application, according to the principle of locality of reference, when a program accesses a storage medium, it is always limited to a certain area. Therefore, the bitmap can be divided into bitmap blocks. In this way, the storage medium area accessed by the program always corresponds to a few bitmap blocks. Therefore, only these bitmap blocks that are hit can be loaded into memory, thus avoiding loading the entire bitmap into memory and causing unnecessary memory occupation.

[0053] In an exemplary embodiment of this application, after loading the bitmap blocks accessed by the program from all the divided bitmap blocks into memory, the method may further include:

[0054] The bitmap block objects loaded into memory are connected using a pre-defined Least Recently Used (LRU) linked list; the LRU linked list has a preset length; and the bitmap block objects are used to describe the corresponding bitmap blocks.

[0055] Whenever a new bitmap block object is inserted at the head of the LRU linked list, the bitmap block objects that were originally inserted in the LRU linked list are moved to the tail of the LRU linked list.

[0056] When the LRU linked list is full, the bitmap block corresponding to the bitmap block object located at the tail of the LRU linked list is evicted from memory, and the evicted bitmap block is written back to the metadata.

[0057] In an exemplary embodiment of this application, a bitmap block size of n bitmap nodes can be defined. All bitmaps in the system are divided according to the bitmap block size. For example, a 1TB logical volume (LV, a logical disk directly provided to users for data storage) can be divided with an 8KB data block size. One bitmap node is 1 byte in size, and one bitmap block contains 2^17 bitmap nodes. Therefore, one bitmap block size is 128KB, and a 128MB bitmap can be divided into 1024 bitmap blocks. One bitmap block corresponds to a 1GB data block. Each bitmap... Figure 1Each bitmap is identified by a unique ID (identity identifier), and for each bitmap, a bitmap object can be maintained to describe the bitmap information, including the bitmap ID, bitmap block size, and the starting address of the bitmap metadata (the corresponding bitmap block can be retrieved based on this starting address). When certain bitmap nodes need to be modified, the corresponding bitmap block is loaded into memory for modification, and a bitmap block object is created to describe that bitmap block. For example, if 1MB of data is written to the beginning of a logical volume (1MB), then according to the above example, bitmap nodes 129 to 256 in the first bitmap block need to be modified. Based on the principle of locality of reference, the memory space accessed by a program during runtime is limited to a certain area; therefore, subsequent bitmap nodes that need to be modified are also highly likely to hit the same bitmap block. Figure 2 As shown. Based on this principle, the bitmap block object corresponding to each modified bitmap block can be placed in a designated area. When the bitmap node needs to be modified again, the corresponding bitmap block object can be queried in this designated area first, thereby improving query efficiency.

[0058] In an exemplary embodiment of this application, to control the memory size occupied by bitmap blocks loaded into memory, a Least Recently Used (LRU) linked list can be used to connect the bitmap block objects loaded into memory together (this LRU linked list can serve as the area defined above for placing the bitmap block objects that have just been modified). The entire LRU linked list has a fixed length. When a bitmap block object is inserted, it is inserted at the head of the LRU linked list. Therefore, the bitmap block object inserted into the LRU linked list first will move towards the tail of the LRU linked list over time, thus evicting the tail node object of the LRU linked list from memory and writing the evicted bitmap block back to the metadata for persistence. During persistence, the corresponding bitmap object can be found based on the bitmap ID recorded in the bitmap object, and the starting address and bitmap block size of the bitmap metadata can be obtained. Then, based on the bitmap block sequence number recorded in the bitmap block object, the offset address of the bitmap block in the metadata can be calculated, i.e., offset address = starting address + bitmap block size * bitmap sequence number. The read / write interface can then be called to write the bitmap block.

[0059] In exemplary embodiments of this application, as Figure 3 As shown, the LRU linked list may include: an active object linked list;

[0060] The method may further include: when inserting a new bitmap block object at the head of the lru linked list, inserting the new bitmap block object at the head of the active object linked list.

[0061] In an exemplary embodiment of this application, in order to accelerate bitmap block lookup, an active object list can be defined. During the lookup, only bitmap block objects in the active object list can be searched, or bitmap block objects of a certain length at the head of the lru list can be defined as active objects, and only these active objects can be searched.

[0062] In exemplary embodiments of this application, as Figure 3 As shown, the LRU linked list may further include: an inactive object linked list;

[0063] When the active object list is full, the bitmap block object located at the tail of the active object list is removed from the active object list, and the removed bitmap block object is inserted at the head of the inactive object list.

[0064] When the inactive object list is full, the bitmap block corresponding to the bitmap block object located at the tail of the inactive object list is evicted from memory, and the evicted bitmap block is written back to the metadata.

[0065] In an exemplary embodiment of this application, the method may further include:

[0066] Get the sequence number of the first tile that needs to be modified;

[0067] Based on the sequence number, search for the first bitmap object corresponding to the first bitmap in the lru linked list;

[0068] When the first bitmap block object is not found in the LRU linked list, the hash value of the first bitmap block's sequence number is calculated, and the first bitmap block is searched from a preset hash table based on the hash value.

[0069] When the first bitmap is not found in the hash table, the first bitmap is retrieved from the metadata, loaded into memory, and the first bitmap object is inserted into the head of the LRU linked list and the hash table.

[0070] In an exemplary embodiment of this application, the method may further include: defining bitmap block objects within a first preset length at the head of the lru linked list as active bitmap block objects, and searching for the first bitmap block object in the region where the active bitmap block object is located in the lru linked list.

[0071] In an exemplary embodiment of this application, the method for obtaining the sequence number of the first bitmap node to be modified can be described by the following embodiment: The bitmap granularity is 8K, that is, the logical volume is divided into 8K granularities. Therefore, if the data is written to the position starting from 1MB of the logical volume, the first bitmap node to be modified is the 1MB / 8KB+1=128+1=129th bitmap node. If the data size is 1MB, the number of bitmap nodes to be modified is 1MB / 8KB=128. Therefore, the last bitmap node to be modified is the 129+128-1=256th bitmap node. Therefore, writing this data requires modifying the 129th to 256th bitmap nodes. According to the example above, a bitmap block contains 2^17 bitmap nodes. Therefore, the 129th to 256th bitmap nodes all hit the first bitmap block, that is, the bitmap block with sequence number 0.

[0072] In an exemplary embodiment of this application, based on the lru linked list established in the aforementioned scheme, when it is necessary to modify a bitmap block, the bitmap object corresponding to the bitmap block can be queried from the active lru linked list first. If the corresponding bitmap block object is not found in the lru linked list, it is searched in the hash table. If it is still not found in the hash table, the bitmap block can be searched in the metadata.

[0073] In an exemplary embodiment of this application, to determine whether the bitmap block corresponding to the bitmap node to be modified has been loaded into memory, and to improve search efficiency, a hash table can be used to organize all bitmap block objects loaded into memory. A hash mapping is performed on the bitmap block's sequence number, and bitmap blocks with conflicting hash values ​​can be connected using a conflict chain (i.e., when calculated hash values ​​conflict, a conflict chain can be established). Based on this hash table, the bitmap block to be modified can be queried from the hash table using the calculated hash value. In an exemplary embodiment of this application, calculating the hash value of the sequence number of the first bitmap block may include: calculating the hash value of the sequence number of the first bitmap block using a preset hash calculation formula;

[0074] The hash calculation formula may include: h(x) = (x % (k / 2)) * (y >> 1);

[0075] Where h(x) is the hash value; x is the sequence number of the first bitmap block; k is the number of bitmap blocks in the largest bitmap; and y is the bitmap id.

[0076] In an exemplary embodiment of this application, it is assumed that the bitmap block number is x, the number of bitmap blocks in the largest size bitmap is k, the bitmap id is y, and the hash calculation formula is: h(x)=(x%(k / 2))*(y>>1). First, for a bitmap, a simple modulo hash operation can be performed on the bitmap block number. k / 2 is taken into account that most bitmaps in the system are not of the largest size, and k is generally an even number, i.e. x%(k / 2). In order to map the bitmap blocks of different bitmaps to different hash value domains as much as possible, and at the same time ensure that the hash table bucket length is not too long, the result of the previous modulo operation is multiplied by (y>>1). For example, if the maximum size of a bitmap is 128MB and the size of a bitmap block is 128KB, then the maximum number of bitmap blocks in a single bitmap is 128MB / 128KB = 1024. According to the hash calculation formula above, the maximum value of the modulo hash operation of the bitmap block number of a single bitmap is 1024 / 2 - 1 = 511. If the system can create a maximum of 128 bitmaps, then the hash bucket length is 511 * (128 >> ​​1) = 32704. When searching for a bitmap block, assuming that the m-th bitmap node of a certain bitmap needs to be modified, since the bitmap block size is n, the index of the hit bitmap block is m / n. Let the number of bitmap blocks in the maximum size bitmap be k, and the bitmap id be y. Then, the search requires traversing the collision chain corresponding to the ((m / n)%(k / 2))*(y >> 1)-th hash table bucket. The hash table structure is as follows: Figure 4 As shown.

[0077] In exemplary embodiments of this application, as Figure 4 As shown, a hash table is actually an array of head nodes, with each head node followed by a linked list of bitmap block objects. The calculated hash value is actually the index of the head node to be inserted. Insertion can be performed in ascending order of bitmap block numbers. In other words, the linked list of bitmap block objects following each head node is arranged in ascending order of bitmap block numbers.

[0078] In the exemplary embodiments of this application, each bitmap tile object is both in an LRU linked list and a hash table. Whether linking a bitmap tile object to an LRU linked list or a hash table, it can be achieved simply by adding a member to the bitmap tile object. Figure 5 As shown.

[0079] In an exemplary embodiment of this application, the method may further include:

[0080] When the first bitmap object is found in the LRU linked list, or when the first bitmap is found in the hash table, the first bitmap node to be modified in the first bitmap is modified, and the modified first bitmap object is inserted into the head of the LRU linked list.

[0081] In an exemplary embodiment of this application, due to the aforementioned principle, the probability of the currently hit bitmap block being hit again is relatively high. Therefore, after modifying the current bitmap block, the bitmap block object corresponding to the modified bitmap block can be inserted into the head of the LRU linked list. Specifically, it can be inserted into the head of the active object linked list, so as to improve the bitmap block search speed when the bitmap block needs to be modified again.

[0082] In an exemplary embodiment of this application, if the bitmap block to be searched is not found in the hash table, it means that the bitmap block has not been loaded into memory and needs to be read into memory from the metadata. When reading, the bitmap object can be found according to the bitmap ID, the starting address of the bitmap metadata and the size of the bitmap block can be obtained, and then the offset address of the bitmap block in the metadata can be calculated according to the bitmap sequence number, that is, offset address = starting address + bitmap block size * bitmap sequence number, and then the read and write interface can be called to read it.

[0083] In an exemplary embodiment of this application, the method may further include:

[0084] After retrieving the first bitmap from the metadata, the first bitmap node to be modified in the first bitmap is modified, and the first bitmap object corresponding to the modified first bitmap is added to the head of the LRU linked list and the hash table; or,

[0085] After obtaining the first bitmap from the metadata and adding the first bitmap object corresponding to the first bitmap to the head of the LRU linked list and the hash table, the first bitmap node to be modified in the first bitmap is modified.

[0086] In an exemplary embodiment of this application, if the bitmap block that needs to be modified does not exist in memory, the bitmap block can be directly obtained from the metadata (usually stored on the disk) and loaded into memory. It should be noted that the bitmap nodes in the bitmap block can be modified after the bitmap block is loaded into memory.

[0087] In an exemplary embodiment of this application, in either case, after the bitmap block is loaded into memory, the bitmap block object corresponding to the bitmap block is inserted at the head of the lru linked list (or active object linked list).

[0088] In an exemplary embodiment of this application, the method may further include: after obtaining the first bitmap tile from the metadata, calculating the hash value of the sequence number of the first bitmap tile according to the hash calculation formula, and inserting the first bitmap tile object into the hash table according to the calculated hash value, so as to realize hash mapping of the sequence number of the bitmap tile.

[0089] In an exemplary embodiment of this application, after a bitmap block is added to memory, in order to facilitate subsequent searching, the hash value corresponding to the bitmap block can be directly calculated based on the sequence number of the bitmap block and added to a preset hash table. If the bitmap block object of the bitmap block to be modified cannot be found in the LRU linked list (or active object linked list), the bitmap block can be searched in the hash table through the hash value of the bitmap block.

[0090] In an exemplary embodiment of this application, using a Least Recently Used (LRU) linked list allows for flexible control of the memory space occupied by the bitmap by adjusting the list length. The bitmap is divided into blocks and dynamically loaded into memory using the principle of program locality (inserted at the head of the LRU linked list). A hash table is used to organize the bitmap blocks, improving the efficiency of bitmap block lookup. The scheme implemented in this application allows for finer granularity of bitmap segmentation, controlling the memory occupied by the bitmap within a reasonable range; reducing the impact of differences in recorded data on the main logical volume's service traffic; and improving transmission efficiency for replication services.

[0091] In an exemplary embodiment of this application, an example of finding a bitmap block to modify bitmap nodes within the bitmap block is given below, such as... Figure 6 As shown, steps S201-S217 may be included:

[0092] S201. Calculate the number of starting bitmap nodes and bitmap nodes hit by IO (here, IO refers to write operations on the storage medium, such as "writing 1MB of data to the beginning of the main logical volume 1MB in the example above, then the IO size is 1MB and the IO starting address is also 1MB).

[0093] S202. Calculate the sequence number of the bitmap block that was hit (multiple bitmap blocks may be hit);

[0094] S203. Search for the bitmap block in the active object list based on its sequence number;

[0095] S204. Check if a bitmap block with the specified number and the bitmap ID of the bitmap block exist. If they exist, proceed to steps S208 and S215 in sequence; if they do not exist, proceed to step S205.

[0096] S205. Calculate the hash value based on the bitmap block number;

[0097] S206. Search for the matching bitmap block in the collision chain of the hash bucket corresponding to the hash value;

[0098] S207. Check if there is a bitmap block with the sequence number and the bitmap ID of the bitmap block in the collision chain list of the hash bucket corresponding to the hash value. If they exist, proceed to step S208 and then to S215; if they do not exist, proceed to step S209.

[0099] S208. Move the bitmap block object corresponding to the bitmap block to the head of the active object list;

[0100] S209. Obtain the bitmap block from the metadata;

[0101] S210. Store the bitmap block in memory;

[0102] S211. Check if the LRU linked list in memory has reached its maximum length. If it has reached the maximum length, proceed to step S212 and then to step S213. If it has not reached the maximum length, proceed to step S213.

[0103] S212. Delete the tail node of the LRU list (i.e., the bitmap block object at the tail), write the corresponding bitmap block back to the metadata, and release the bitmap block object.

[0104] S213. Insert the bitmap block object corresponding to the bitmap block into the head of the active object list;

[0105] S214. Insert the bitmap block object into the hash table;

[0106] S215. Calculate the offset of the starting bitmap node of the IO hit within the bitmap block to determine the position of the bitmap node;

[0107] S216. Modify the corresponding bitmap node within the bitmap block;

[0108] S217, End.

[0109] This application also provides a bitmap processing device 1, such as... Figure 7 As shown, the device may include a processor 11 and a computer-readable storage medium 12, wherein the computer-readable storage medium 12 stores instructions that, when executed by the processor 11, implement the bitmap block processing method described in any of the above-mentioned embodiments.

[0110] In the exemplary embodiments of this application, any of the aforementioned bitmap block processing methods can be applied to this device embodiment, and will not be described in detail here.

[0111] It will be understood by those skilled in the art that all or some of the steps, systems, or apparatuses disclosed above, and their functional modules / units, can be implemented as software, firmware, hardware, or suitable combinations thereof. In hardware implementations, the division between functional modules / units mentioned above does not necessarily correspond to the division of physical components; for example, a physical component may have multiple functions, or a function or step may be performed collaboratively by several physical components. Some or all components may be implemented as software executed by a processor, such as a digital signal processor or microprocessor, or as hardware, or as an integrated circuit, such as an application-specific integrated circuit (ASIC). Such software may be distributed on a computer-readable medium, which may include computer storage media (or non-transitory media) and communication media (or transient media). As is known to those skilled in the art, the term computer storage media includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules, or other data). Computer storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technologies, CD-ROM, digital versatile disc (DVD) or other optical disc storage, magnetic cartridges, magnetic tape, disk storage or other magnetic storage devices, or any other medium that can be used to store desired information and can be accessed by a computer. Furthermore, it is well known to those skilled in the art that communication media typically contain computer-readable instructions, data structures, program modules, or other data in modulated data signals such as carrier waves or other transmission mechanisms, and may include any information delivery medium.

Claims

1. A method of processing a bitmap, characterized by, The method comprises: dividing the bitmap into a preset number of bitmap blocks; loading the bitmap blocks hit by program access among all the divided bitmap blocks into the memory; wherein, after loading the bitmap blocks hit by program access among all the divided bitmap blocks into the memory, the method further comprises: connecting the bitmap block objects of the loaded bitmap blocks in the memory by using a preset least recently used algorithm (lru) linked list; the lru linked list has a preset length; when a new bitmap block object is inserted at the head of the lru linked list, the originally inserted bitmap block objects in the lru linked list are pushed to the tail of the lru linked list; when the lru linked list is full, the bitmap block corresponding to the bitmap block object at the tail of the lru linked list is evicted from the memory, and the evicted bitmap block is written back to the metadata; and wherein the method further comprises: obtaining the serial number of a first bitmap block to be modified; finding the first bitmap block object corresponding to the first bitmap block from the lru linked list according to the serial number; when the first bitmap block object is not found in the lru linked list, calculating the hash value of the serial number of the first bitmap block, and finding the first bitmap block from the preset hash table according to the hash value; when the first bitmap block is not found in the hash table, loading the first bitmap block from the metadata into the memory, and inserting the first bitmap block object into the head of the lru linked list and the hash table.

2. The bitmap processing method of claim 1, wherein, The lru linked list comprises an active object linked list. The method further comprises: when a new bitmap block object is inserted at the head of the lru linked list, inserting the new bitmap block object into the head of the active object linked list.

3. The bitmap processing method of claim 2, wherein, The lru linked list further comprises an inactive object linked list. When the active object linked list is full, the bitmap block object at the tail of the active object linked list is evicted from the active object linked list, and the evicted bitmap block object is inserted into the head of the inactive object linked list. When the inactive object linked list is full, the bitmap block corresponding to the bitmap block object at the tail of the inactive object linked list is evicted from the memory, and the evicted bitmap block is written back to the metadata.

4. The bitmap processing method of claim 1, wherein, The method further comprises: defining the bitmap block objects within a first preset length at the head of the lru linked list as active bitmap block objects, and finding the first bitmap block object from the area of the active bitmap block objects in the lru linked list.

5. The bitmap processing method of claim 1, wherein, The method further comprises: when the first bitmap block object is found in the lru linked list, or when the first bitmap block is found in the hash table, modifying the first bitmap node to be modified in the first bitmap block, and inserting the modified first bitmap block object into the head of the lru linked list.

6. The bitmap processing method of claim 1, wherein, The method further comprises: after obtaining the first bitmap block from the metadata, modifying the first bitmap node to be modified in the first bitmap block, and adding the first bitmap block object corresponding to the modified first bitmap block to the head of the lru linked list and the hash table; or, After the first bitmap block is obtained from the metadata and a first bitmap block object corresponding to the first bitmap block is added to a head of the lru linked list and the hash table, a first bitmap node to be modified in the first bitmap block is modified.

7. The bitmap processing method of claim 1, wherein, The hash value of the sequence number of the first bitmap block is calculated by using a preset hash calculation formula. The hash calculation formula includes: h(x)=(x%(k / 2))*(y>>1). Wherein, h(x) is a hash value, x is the sequence number of the first bitmap block, k is the bitmap block number of the maximum specification bitmap, and y is a bitmap id. The method further includes: after the first bitmap block is obtained from the metadata, calculating the hash value of the sequence number of the first bitmap block according to the hash calculation formula, and inserting the first bitmap block object into the hash table according to the calculated hash value, so as to realize hash mapping of the sequence number of the bitmap block.

8. A bitmap processing apparatus, characterized by comprising: The method includes a processor and a computer readable storage medium, the computer readable storage medium stores instructions, when the instructions are executed by the processor, the bitmap processing method is realized as any one of claims 1-7.

Images