Processing method and device of bitmap
The SuperBlock method for managing block bitmaps utilizes half-full submaps and arrays to quickly determine the bit state, solving the problems of low random access efficiency and large memory consumption in bitmap storage and achieving efficient memory management.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- NEW H3C TECH CO LTD
- Filing Date
- 2022-09-09
- Publication Date
- 2026-07-07
AI Technical Summary
Existing bitmap storage methods are inefficient for random access and consume a lot of memory when there are large-scale routing or peer connections. In particular, the existing array slicing method results in low access efficiency and excessive memory consumption when there are a large number of routing or peer connections.
The SuperBlock structure manages multiple blocks. It utilizes half-full subgraphs and half-full arrays, selects half-full subgraphs by identifying high-order bits, obtains the number and position of set bits, quickly determines the state of the bit required by the user, and optimizes the storage method to save memory.
It improves the efficiency of random access to bitmaps, reduces memory usage, and solves the problems of low random access efficiency and large memory space in existing technologies.
Smart Images
Figure CN116629168B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of communication technology, and in particular to a bitmap processing method and apparatus. Background Technology
[0002] A bitmap is a commonly used data structure that converts indices into offsets corresponding bits. Bitmaps support Boolean storage, where a 1 (set bit) typically represents true and a 0 (not set bit) represents false.
[0003] Bitmaps have a wide range of uses. For example, they can represent free port numbers; a set bit indicates the port number has been allocated, while a non-set bit indicates it hasn't. They can also represent board numbers; a set bit indicates the board is in place, while a non-set bit indicates it hasn't been inserted. In BGP routing, each route corresponds to a Bitmap, and each bit in the Bitmap corresponds to a peer connection; a set bit indicates the route has been synchronized with the corresponding peer, while a non-set bit indicates it hasn't.
[0004] When BGP has a large number of routes (tens of millions) or a large number of peer connections (tens of thousands), or both, the current method of storing Bitmaps is to use array slicing.
[0005] For example, each 1024 bits can be divided into a submap, and each submap can be further subdivided into multiple segments. Each segment consists of 64 bits, and a submap can be further subdivided into a maximum of 16 segments (16 * 64 = 1024 bits in total).
[0006] When a segment is all 0 (hereinafter referred to as "empty") or all 1 (hereinafter referred to as "full"), the FreeFullMap included in the corresponding Submap is set. In this way, the segment can be released. That is, the Submap only retains segments with mixed 0 / 1 (hereinafter referred to as "half full") to save memory space.
[0007] However, the Bitmap storage method described above also exposes the following problems: 1) Low random access efficiency. For example, if a user wants to check whether bit 10000 is set, the network device needs to access 9 (10000 / 1024=9) Submaps first, and then access multiple segments under each Submap, resulting in low access efficiency; 2) Large memory consumption. For example, if only one bit in 10000 is set and the other bits are not set, 9 Submaps are still needed for storage, consuming at least 32*9=288 bytes. In scenarios with a large number of routes (20 million), the memory consumption of Bitmaps reaches the GB level. Summary of the Invention
[0008] In view of this, this application provides a bitmap processing method and apparatus to solve the problems of low random access efficiency and large memory space consumption exposed by the Bitmap storage method.
[0009] In a first aspect, this application provides a bitmap processing method, which is applied to a network device, the network device including a management block SuperBlock and multiple blocks, the method comprising:
[0010] Receive a first operation instruction input by the user, the first operation instruction including an index to be viewed, the index to be viewed can be converted into a string with N bits;
[0011] If the SuperBlock is of type 1, then based on the first number of high-order bits included in the string, select the first bit corresponding to the half-full sub-map PartiaMap included in the SuperBlock;
[0012] If the first bit corresponding to the half-full subgraph is set, then obtain the number of bits that are set below the first bit from the half-full subgraph;
[0013] Based on the number, determine the position of the number in the half-full array included in the management block, and based on the position, determine the second bit in the first block corresponding to the string;
[0014] The operation result is displayed to the user, and the operation result includes the current state of the second bit.
[0015] Secondly, this application provides a bitmap processing apparatus, which is applied to a network device. The network device includes a management block SuperBlock and multiple blocks. The apparatus includes:
[0016] A receiving unit is configured to receive a first operation instruction input by a user, the first operation instruction including an index to be viewed, the index to be viewed being convertible into a string with N bits;
[0017] The first selection unit is used to select the first bit corresponding to the PartiaMap included in the SuperBlock based on the first number of high-order bits included in the string if the type of the SuperBlock is the first type.
[0018] The first acquisition unit is used to acquire the number of bits that are set below the first bit in the half-full subgraph if the first bit corresponding to the half-full subgraph is set.
[0019] The first determining unit is configured to determine the position of the number in the half-full array included in the management block according to the number, and determine the second bit in the first block corresponding to the string according to the position;
[0020] The display unit is used to display the operation result to the user, the operation result including the current state of the second bit.
[0021] Thirdly, this application provides a network device including a processor and a machine-readable storage medium storing machine-executable instructions that can be executed by the processor, which in turn cause the processor to perform the method provided in the first aspect of this application.
[0022] Therefore, by applying the bitmap processing method and apparatus provided in this application, the network device receives a first operation instruction input by a user. The first operation instruction includes an index to be viewed, which can be converted into a string with N bits. If the SuperBlock is of type 1, the network device selects the first bit corresponding to the half-full sub-map (PartiaMap) included in the SuperBlock based on the first number of high-order bits included in the string. If the first bit corresponding to the half-full sub-map is set, the network device obtains the number of bits set below the first bit from the half-full sub-map. Based on the number, the network device determines the position of the number in the half-full array included in the management block, and determines the second bit in the first block corresponding to the string based on the position. The operation result is displayed to the user, which includes the current state of the second bit.
[0023] In this way, the multiple blocks included in the network device are all half-full blocks. By using the half-full submap PartiaMap and the half-full array, the bit required by the user can be quickly determined, which solves the problems of low random access efficiency and large memory space consumption exposed by the Bitmap storage method. Attached Figure Description
[0024] Figure 1 A flowchart illustrating a bitmap processing method provided in an embodiment of this application;
[0025] Figure 2 This is a schematic diagram of the logical storage structure of the Bitmap provided in the embodiments of this application;
[0026] Figure 3 A schematic diagram of a SuperBlock structure provided in an embodiment of this application;
[0027] Figure 4 A schematic diagram of SuperBlock storage provided in an embodiment of this application;
[0028] Figure 5 This is a schematic diagram of another SuperBlock structure provided in an embodiment of this application;
[0029] Figure 6 Another SuperBlock storage diagram provided in this application embodiment;
[0030] Figure 7 A schematic diagram illustrating a half-full block storage method provided in an embodiment of this application;
[0031] Figure 8 This is a schematic diagram of the Block* structure provided in an embodiment of this application;
[0032] Figure 9 A structural diagram of a bitmap processing apparatus provided in an embodiment of this application;
[0033] Figure 10 The network device hardware structure provided in the embodiments of this application. Detailed Implementation
[0034] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.
[0035] The terminology used in this application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. The singular forms “a,” “the,” and “the” used in this application and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any or all possible combinations of one or more of the corresponding listed items.
[0036] It should be understood that although the terms first, second, third, etc., may be used in this application to describe various information, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, without departing from the scope of this application, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to determination."
[0037] The following is a detailed description of a bitmap processing method provided in an embodiment of this application. See also... Figure 1 , Figure 1 This is a flowchart illustrating a bitmap processing method provided in an embodiment of this application. The method is applied to a network device. The bitmap processing method provided in this application may include the following steps.
[0038] Step 110: Receive a first operation instruction input by the user, the first operation instruction including an index to be viewed, the index to be viewed can be converted into a string with N bits;
[0039] Specifically, network devices use bits within a Bitmap to store a specific type of data. For example, whether a port number has been assigned, whether a card has been inserted, and whether BGP routes have been synchronized with the peer, etc.
[0040] If a user wants to access or modify a specific bit within a Bitmap, the user inputs a first operation command into the network device. This first operation command includes the index to be viewed, which can be converted into a string with N bits.
[0041] After obtaining the index, the network device converts it into a binary string. For example, if the index to be viewed is 1636, the network device will convert 1636 into the binary string: 00000110 01100100.
[0042] In this embodiment, the maximum value of the Bitmap is no more than 65535, which is 2 bytes; the value of N can be determined according to the actual situation, including but not limited to 16-bit, 24-bit, 32-bit, etc. In the following examples, a 16-bit string will be used as an example for illustration.
[0043] The network device receives the first operation command and retrieves a 16-bit string from it.
[0044] Step 120: If the type of the SuperBlock is the first type, then select the first bit corresponding to the half-full sub-map PartiaMap included in the SuperBlock according to the first number of high-order bits included in the string;
[0045] Specifically, according to the description in step 110, after the network device obtains the 16-bit string, it first identifies the type of the locally included management block SuperBlock. If the SuperBlock type is the first type, the network device selects the first bit corresponding to the half-full sub-map (PartiaMap) included in the SuperBlock based on the first number of high-order bits included in the 16-bit string. The network device then checks whether the first bit corresponding to the half-full sub-map is set. If the first bit corresponding to the half-full sub-map is set, step 130 is executed.
[0046] The first type specifically includes two subgraphs, a half-full array, and metadata within a SuperBlock; the first quantity specifically includes 8.
[0047] Furthermore, in this embodiment, the network device divides the 65536 bits into groups of 256 bits each, called a Block, for a total of 256 Blocks. Each Block independently selects its own storage method based on the current state of its own bits; the specific selection process will be described in subsequent embodiments. The 256 Blocks are managed by a SuperBlock. Storage methods include, but are not limited to, uncompressed bit storage, range storage, bit offset storage, and embedded storage. The above storage methods can also be replaced by other storage methods; for example, the range storage method can be replaced by WAH encoding.
[0048] like Figure 2 As shown, Figure 2 This is a schematic diagram of the logical storage structure of a Bitmap provided in an embodiment of this application. Figure 2 In SuperBlock, multiple blocks are managed in a unified manner, and each block consists of 256 bits.
[0049] To save memory space in network devices, this embodiment compresses fully full (all bits in the block are set) and fully empty (all bits in the block are not set) blocks, but does not compress half-full blocks. The compression of fully full blocks specifically refers to the representation of the current state of the bits included in the fully full subgraph (described in subsequent embodiments). For example, in the fully full subgraph, the bits corresponding to a fully full block are set, and the bits corresponding to a fully empty block are not set. The lack of compression for half-full blocks specifically means that when selecting a storage method, memory space can be allocated to store the half-full block. However, if the half-full block selects an embedded storage method, memory space may not be allocated for it.
[0050] In the embodiments of this application, SuperBlock includes several types, namely, a first type and a second type. The two types of SuperBlock are described below.
[0051] In one implementation, if SuperBlock is of type 1, then SuperBlock includes two subgraphs: a full subgraph PartialMap and a half-full subgraph FullMap, a half-full array PartialBlockArray, and Metadata. For example... Figure 3 As shown, Figure 3 This is a schematic diagram of a SuperBlock structure provided in an embodiment of this application. Figure 3 In this context, the SuperBlock structure can also be called a BlockArray structure. Meta data occupies 1 byte, and its highest 2 bits can represent the type of SuperBlock; a half-full array occupies 7 bytes; a full subgraph has a variable length and does not exceed 256 bits; a half-full subgraph has a variable length and does not exceed 256 bits.
[0052] Network devices can determine the type of SuperBlock by identifying the two highest bits of the metadata. For example, if the two highest bits are 01, the network device determines the SuperBlock type to be Type 1; if the two highest bits are 00, the network device determines the SuperBlock type to be Type 2. The two highest bits being 10 / 11 are temporarily reserved and unused.
[0053] When a bit in a half-full subgraph is set, it means that the block corresponding to that bit is half-full; when a bit in a full subgraph is set, it means that the block corresponding to that bit is full; when a bit in a full subgraph is not set, it means that the block corresponding to that bit is empty.
[0054] A half-full array contains multiple array elements (a1, a2, ...), each of which can be called a Block*, i.e., a pointer that stores the address of a Block. Multiple Block* arrays are sorted in ascending order according to the bit range represented by the Block. For example... Figure 4 As shown, Figure 4 This is a schematic diagram of a SuperBlock storage provided in an embodiment of this application.
[0055] exist Figure 4 In the array, the elements include a1 and a2, where a1 is Block*1 and a2 is Block*2. Each Block* points to two half-full Blocks, and the bit ranges represented by the two half-full Blocks are: the bit range of Block1 is [0~255] and the bit range of Block2 is [512~767].
[0056] As can be seen from Block1 and Block2 above, the bits included in the two blocks are not consecutive. Thus, it can be determined that the middle bit [256~511] is either completely full or completely empty.
[0057] In another implementation, if the SuperBlock is of type 2, then the SuperBlock includes bit offsets and metadata. For example... Figure 5 As shown, Figure 5 This is a schematic diagram of another SuperBlock structure provided in an embodiment of this application. Figure 5 In this context, the SuperBlock structure can also be called a SortArray structure. Meta data occupies 1 byte, with its highest two bits representing the SuperBlock type; the bit offset position occupies 7 bytes. These 7 bytes can hold a maximum of 31 bit offsets. Specifically, this bit offset position refers to the bit offset position of the Block's built-in bits.
[0058] Understandably, if the number of bits in the Bitmap is small (e.g., no more than 31 set bits), the network device can directly store the offset positions of the set bits in the SuperBlock when storing the Bitmap. In this way, the SuperBlock size is within 64 bytes, which does not exceed the CPU cache line, thus saving memory space. Figure 6 As shown, Figure 6 This is another SuperBlock storage diagram provided for an embodiment of this application.
[0059] exist Figure 6 In the Bitmap, there are 3 bits that are set (the other bits are not set), with offset positions of 100, 30052, and 53331 respectively. The offset positions of the set bits are stored directly in the SuperBlock.
[0060] It should be noted that the SuperBlock type can be switched according to the actual situation. For example, if the number of bits set is small when the user has not made any modifications, the network device can use the second type of SuperBlock; if the number of bits set is large after the user makes modifications, such as more than 31 bits, the network device can use the first type of SuperBlock. The above two types of SuperBlock can be switched in real time according to the actual situation of Bitmap and user modifications.
[0061] In this step, the network device first identifies the two highest bits of the metadata. If the two highest bits are 0 or 1, the network device determines that the SuperBlock type is the first type. Then, following the example in step 110, the network device obtains the high 8 bits (00000110) from the 16-bit string (00000110 01100100) and converts these high 8 bits to a decimal number, which is 5. Based on the decimal number 5, the network device selects the first bit corresponding to the half-full sub-graph, that is, starting from the lowest bit of the half-full sub-graph, it selects the 5th bit as the first bit. The network device checks whether the first bit corresponding to the half-full sub-graph is set. If the first bit corresponding to the half-full sub-graph is set, then step 130 is executed.
[0062] Optionally, while performing this step, the network device also selects the first bit corresponding to the full sub-graph, that is, starting from the least significant bit of the full sub-graph, selects the 5th bit as the first bit. The network device identifies whether the first bit of the full sub-graph is set. If the first bit corresponding to the full sub-graph is not set and the first bit corresponding to the half-full sub-graph is not set, then the network device determines that all the second bits included in the first block indicated by the first bit corresponding to the full sub-graph are not set. That is, the network device determines that the first block indicated by the first bit of the full sub-graph is completely empty, and all the second bits included in the first block are 0, then the network device determines that the bit indicated by the string to be searched by the user is 0.
[0063] The network device generates and displays the operation result to the user, which includes the current state of the second bit being 0.
[0064] Optionally, the network device's identification of whether the first bit of a fully full subgraph is set also includes another case: if the first bit corresponding to the fully full subgraph is set, then the network device determines that all the second bits included in the first block indicated by the first bit of the fully full subgraph are set. That is, if the network device determines that the first block indicated by the first bit of the fully full subgraph is fully full, and all the second bits included in the first block are 1, then the network device determines that the bit indicated by the string to be searched by the user is 1.
[0065] The network device generates and displays the operation result to the user, which includes the current state of the second bit being 1.
[0066] It should be noted that if the first bit corresponding to a full subgraph is set, the network device does not need to check the first bit corresponding to a half-full subgraph; if the first bit of a half-full subgraph is set, the network device does not need to check the first bit corresponding to a full subgraph.
[0067] Optionally, after identifying the two highest bits of the metadata, the network device determines that the SuperBlock is of the second type. The network device directly obtains the offset position of the set third bit in the Bitmap from the SuperBlock; based on the offset position and the second number of low-order bits included in the string, the network device determines the current state of the second bit.
[0068] For example, the offset positions stored in the SuperBlock are specifically 100, 30052, and 53331. That is, the current state of Bits 100, 30052, and 53331 is set to 1. The network device obtains the lower 8 bits (01100100) from the 16-bit string (0000011001100100) and converts these lower 8 bits to a decimal number, 100. Based on the decimal number 100, the network device determines the corresponding second bit within the first block, namely Bit 100. Simultaneously, based on the offset position stored in the SuperBlock being 100, the network device determines that the current state of Bit 100 is set to 1. In other words, when the lower 8 bits correspond to the same offset position as those stored in the SuperBlock, the corresponding lower 8 bits are currently set to 1. If the lower 8 bits do not correspond to the same offset position as those stored in the SuperBlock, then none of the corresponding lower 8 bits are currently set to 0.
[0069] Step 130: If the first bit corresponding to the half-full subgraph is set, then obtain the number of bits that are set below the first bit from the half-full subgraph.
[0070] Specifically, according to the description in step 120, the network device identifies whether the first bit corresponding to the half-full sub-graph is set. If the first bit corresponding to the half-full sub-graph is set, the network device determines that the first block indicated by the first bit corresponding to the half-full sub-graph is a half-full block. The network device obtains the number of bits set below the first bit in the half-full sub-graph. For example, the half-full sub-graph is specifically represented by a binary number, specifically: 0 250 100011. This half-full subgraph consists of 256 bits. The lowest two bits, Bit 0, Bit 1, and Bit 5, are set, while Bits 2-4 are not set, and Bits 6-256 are not set.
[0071] According to the example steps, the first bit is specifically bit 5 in the half-full subgraph. The network device obtains the number of bits set below bit 5 from the half-full subgraph. In this example, the number of bits set below bit 5 is 2, namely bit 0 and bit 1. In practical applications, the network device can obtain the number of bits set below the first bit in the half-full subgraph through the Popcnt hardware instruction (i.e., Popcnt((2<<targetIndex high 8 bits – 1)&half-full subgraph) – 1).
[0072] It should be noted that the number of network devices obtained in this step is also the offset position in the half-full array.
[0073] Step 140: Determine the position of the number in the half-full array included in the management block according to the number, and determine the second bit in the first block corresponding to the string according to the position;
[0074] Specifically, according to the description in step 130, after obtaining the number, the network device determines the position of that number in the half-full array. Based on the position, the network device determines the second bit within the first block corresponding to the string.
[0075] In this embodiment, a half-full array, meaning that different contents can be stored within its elements, allows the half-full array to have multiple types, namely, a first type and a second type. The different contents stored are described below.
[0076] In one implementation, the array element stores the first address of the first block, i.e., the first type. In this step, the network device determines the second bit within the first block corresponding to the string. Specifically, the network device obtains the first address of the first block from the location; based on the first address, the network device accesses the first block, and based on the second number of low-order bits included in the string, determines the corresponding second bit and its current state within the first block.
[0077] The second quantity is specifically 8.
[0078] For example, following the previous example, if the number is 2, the network device determines that the position of the number in the half-full array is 2, which is array element a2. a2 stores the first address of the first block. Based on the first address, the network device locates the first block in memory. The network device retrieves the lower 8 bits (01100100) from the 16-bit string (00000110 01100100) and converts these lower 8 bits to a decimal number, 100. Based on the decimal number 100, the network device determines the corresponding second bit within the first block, namely Bit100, and the current state of Bit100 (e.g., Bit100 is currently set to 1).
[0079] It is understandable that the lower 8 bits are the offset position within the first block.
[0080] In another implementation, the array element stores the offset position of the third bit that has been set in the first block, i.e., the second type. In this step, the network device determines the second bit in the first block corresponding to the string. Specifically, the network device obtains the offset position of the third bit in the first block from the position; based on the offset position and the second number of low-order bits included in the string, the network device determines the current state of the second bit.
[0081] For example, based on the previous example, if the number is 2, the network device determines that the position of the number in the half-full array is 2, which is array element a2. a2 stores the offset position of the third bit that has been set in the first block. For example, a2 stores 28 and 33, meaning that the current state of bits 28 and 33 in the first block is set to 1. The current state of the remaining 256-2 bits is not set to 0.
[0082] The network device retrieves the lower 8 bits (01100100) from the 16-bit string (00000110 01100100) and converts them to a decimal number, 100. Based on the decimal number 100, the network device determines the corresponding second bit, Bit100, within the first block. Simultaneously, based on the values 28 and 33 stored in array a2, the network device determines the current state of Bit100 (e.g., Bit100 is not currently set to 0). In other words, if the bits corresponding to the lower 8 bits do not have the same offset position stored in the array element, then all bits corresponding to the lower 8 bits are not currently set to 0.
[0083] It is understandable that since the number of bits in the first block is relatively small and the first block is only half full, the embedded storage method described above can be used when choosing a storage method for the first block. That is, the offset position of the set bit is directly stored in the array element of the half-full array, so that memory space is no longer needed to store the half-full block.
[0084] like Figure 7 As shown, Figure 7 This is a schematic diagram illustrating a half-full block storage method provided in an embodiment of this application. Figure 7 In this array, SuperBlock is of type 1, as previously explained and will not be repeated here. Each element in the half-full array is a Block*, and each Block* points to a half-full Block. Blocks pointed to by solid lines occupy memory space, while those pointed to by dashed lines do not.
[0085] In this embodiment of the application, Block* includes the block address (BlockAddress) and metadata. For example... Figure 8 As shown, Figure 8 A schematic diagram of the Block* structure provided in the embodiments of this application. Figure 8 In this structure, the meta data occupies 1 byte, and its highest two bits can represent the type of the Block*; the BlockAddress occupies 7 bytes, which can be used to carry the Block address or to carry a bit offset. This bit offset position can specifically refer to the bit offset position of the Block's built-in bits.
[0086] Understandably, if the number of bits in a Block's built-in bits is small (e.g., no more than 7 bits set), the network device can directly store the offset position of the set bits into the Block when storing the Block. This saves memory space.
[0087] Network devices can determine the type of a Block* by identifying the two highest bits of the metadata. For example, if the two highest bits are 01, the network device determines the Block* to be of type 1; if the two highest bits are 00, the network device determines the Block* to be of type 2.
[0088] Of course, the highest two bits are used to indicate the type of the Block*, and also to indicate the storage method of the Block that the Block* points to. For example, when the highest two bits are 00, the network device determines that the type of the Block* is the second type, and at the same time, indicates that the storage method of the Block that the Block* points to is embedded storage; the remaining 6 bits in the metadata can represent the number of offset positions.
[0089] When the highest two bits are 0 or 1, the network device determines that the type of Block* is the first type, and at the same time, indicates that the storage method of the Block pointed to by Block* is uncompressed bit storage method; the remaining 6 bits in the meta data can represent the size of the Block.
[0090] When the highest two bits are 10, the network device determines that the type of Block* is the first type, and at the same time, indicates that the storage method of the Block pointed to by Block* is Range storage method; the remaining 6 bits in the Meta data can represent the number of Ranges.
[0091] When the highest two bits are 11, the network device determines that the type of Block* is the first type, and at the same time, indicates that the storage method of the Block pointed to by Block* is the bit offset storage method; the remaining 6 bits in the meta data can represent the number of offset positions.
[0092] Combination Figure 7 As shown, the Block pointed to by Block*1 is stored in uncompressed Bit storage mode; the Block pointed to by Block*2 is stored in Range storage mode, with the current state of (Bit0~Bit 33) and (Bit 48~Bit 100) set to 1, and the current state of the remaining Bits not set to 0; the Block pointed to by Block*3 is stored in Bit offset storage mode, with the current state of Bit28, Bit34, Bit88, Bit99, Bit128, Bit131, Bit132, Bit241, and Bit245 set to 1, and the current state of the remaining Bits not set to 0; the Block pointed to by Block*4 is stored in embedded storage mode, with the current state of Bit28 and Bit33 set to 1, and the current state of the remaining Bits not set to 0.
[0093] The above representation of the Block* type uses the high 8 bits of the Meta data. In practical applications, the high 16 bits of the Meta data can be used.
[0094] Step 150: Display the operation result to the user, the operation result including the current state of the second bit.
[0095] Specifically, according to the description of step 140, after the network device determines the current state of the second bit, it generates and displays the operation result to the user. The operation result includes whether the current state of the second bit is 0 or 1.
[0096] After receiving the operation result, the user will know the current status of the index to be viewed. Understandably, the user can also modify the index whose current status is known. For example, if the current status of the index is 1, the user can change it to 0; or, if the current status of the index is 0, the user can change it to 1.
[0097] During the modification process, a fully full / empty block may change to a half-full block, and a half-full block may change to a fully full / empty block. Therefore, every time a user modifies a bit, the network device will identify the current state of all bits in the Bitmap and Block and calculate the optimal storage method to save memory space.
[0098] For example, the current state of Bit 200 in the original empty Block is changed from 0 to 1 by the user. The network device determines that the original empty Block has been changed to a new half-full Block. If the current state of other bits has not been modified, the embedded storage method, which does not occupy memory space, is selected from the aforementioned four storage methods. The network device generates a new Block* and directly stores the offset position of Bit 200 into the new Block*. If the SuperBlock is of type 1, the generated new Block*, or new array element, is inserted into the original half-full array (the insertion operation can be performed according to the bit range of the Block where Bit 200 is located). At the same time, the network device updates the bits indicating the original empty Block in the full subgraph and updates the bits indicating the new half-full Block in the half-full subgraph.
[0099] The current state of multiple bits in the original half-full block is changed from 0 to 1 by the user, and the current state of all bits in the modified half-full block is set. The network device determines that the original half-full block has been changed to a new full block and determines that the block no longer needs to be stored in memory space, that is, the block can be stored as a whole in compressed form. The network device deletes the original Block* pointing to the original half-full block from the original half-full array. If SuperBlock is of type 1, then the element that is the same as the original Block* or the original array element is deleted from the original half-full array. At the same time, the network device updates the bits indicating the original half-full block in the half-full subgraph and updates the bits indicating the new full block in the full subgraph.
[0100] It is understandable that the above examples all use SuperBlock as the first type for illustration. When SuperBlock is the second type, the network device will identify the current state of all Bitmaps and Bits in the Block and calculate the optimal storage method every time the user modifies a Bit, which will not be repeated here.
[0101] Therefore, by applying the bitmap processing method provided in this application, the network device receives a first operation instruction input by the user. This first operation instruction includes an index to be viewed, which can be converted into a string with N bits. If the SuperBlock is of type 1, the network device selects the first bit corresponding to the half-full sub-map (PartiaMap) included in the SuperBlock based on the first number of high-order bits included in the string. If the first bit corresponding to the half-full sub-map is set, the network device obtains the number of bits set below the first bit from the half-full sub-map. Based on the number, the network device determines the position of the number in the half-full array included in the management block, and determines the second bit in the first block corresponding to the string based on the position. The operation result is displayed to the user, which includes the current state of the second bit.
[0102] In this way, the multiple blocks included in the network device are all half-full blocks. By using the half-full submap PartiaMap and the half-full array, the bit required by the user can be quickly determined, which solves the problems of low random access efficiency and large memory space consumption exposed by the Bitmap storage method.
[0103] Optionally, before performing step 110, this embodiment of the application further includes a process in which the network device generates different types of SuperBlocks based on the number of bits that have been set in the Bitmap.
[0104] Specifically, the network device obtains the complete bitmap; the network device identifies the number of bits that have been set in the bitmap; if the number is greater than a second threshold, the network device generates a SuperBlock of the first type; if the number is not greater than the second threshold, the network device generates a SuperBlock of the second type.
[0105] The second threshold number can be specifically 31, which is the same as described in the previous embodiment. In the second type of SuperBlock, the remaining 7 bytes after 1 byte of Meta data can carry a maximum of 31 bit offsets.
[0106] Optionally, in this embodiment of the application, after the network device generates a SuperBlock of the first type, it further includes the process of generating a Block, generating a Block*, and selecting the storage method of the Block.
[0107] Specifically, after the network device generates a SuperBlock of the first type, it divides the Bitmap into multiple Blocks according to a preset third number of bits, with each Block containing the third number of bits. Based on the set state of each bit in the Block, the network device selects the storage method with the least memory space from multiple storage methods to store the Block.
[0108] Optionally, the network device selects the storage method with the least memory usage from multiple storage methods based on the bit state of each bit in the block, and stores the block accordingly. The specific process is as follows:
[0109] Network devices identify the number of bits that are set within a block.
[0110] If the number of blocks exceeds the first threshold, the network device generates a half-full array of type 1 and selects the storage method with the least memory usage from multiple storage options to store the block. The network device stores the address of the block in the corresponding position within the half-full array.
[0111] If the number is not greater than the first threshold, the network device generates a half-full array of the second type and stores the offset position of the set bit in the half-full array corresponding to the block.
[0112] Among them, the third quantity can be specifically 256; the first number threshold can be specifically 7; multiple storage methods include uncompressed bit storage method, range storage method and bit offset storage method.
[0113] After the network device receives multiple blocks, let's take the first block as an example. The network device identifies the number of bits that are set in the first block, for example, 5 bits are set. At this time, the network device generates a half-full array of type 2 and stores the offset positions of the 5 set bits in the half-full array corresponding to the block.
[0114] If, for example, 100 bits are already set in the first block, the network device generates a half-full array of type 1. Then, it selects the storage method with the least memory usage from the remaining three storage methods (excluding embedded storage) to store the first block. Finally, the network device stores the address of the block in the corresponding half-full array.
[0115] For example, uncompressed bit storage: the space occupied (in bytes) is: maximum bit offset / 8 + 1;
[0116] Range storage method: The space occupied (in bytes) is: number of Ranges * 2;
[0117] Bit offset storage method: Each block offset is stored separately, and the space occupied (in bytes) is the number of offsets.
[0118] In the above storage methods, the memory space occupied by the uncompressed bit storage method and the bit offset storage method can be calculated simply. However, the range storage method requires multiple calculation steps, as follows:
[0119] For Range-based storage, hardware commands LZCNT and CLSB can be used to accelerate computation. Using the LZCNT command, the network device obtains the number of leading 1s and preceding 0s in a binary value. For example, a binary value of 0... 33 1 30 If the value is 0, the LZCNT instruction returns 33 to the network device. Using the CLSB instruction, the network device obtains the number of times the value is the same as the highest bit. For example, a binary value of 1. 30 01 31 The CLSB instruction then returns 30 to the network device; its binary value is 0. 33 1 30 If 0, the CLSB instruction returns 33 to the network device.
[0120] Using the LZCNT and CLSB commands, network devices can quickly calculate the number of ranges or convert a bitmap into a range.
[0121] For example, a Bitmap with a binary value of 0 8 1 11 0 25 1 10 0 10 .
[0122] Step 1: The LZCNT command returns 8 to the network device, so the starting value of the Range is 8.
[0123] Step 2: Set all the high bits before the start to 1 19 0 25 1 10 0 10 , the CLSB instruction returns 19 to the network device, that is, the range of Range is [8~18]. Clear the high 19 bits to 0, that is, change to 0 44 1 10 0 10 , return to Step 1 to start the calculation of the next Range until all calculations are completed, or when the memory space occupied by the Range storage method exceeds the smaller memory space occupied by any one of the uncompressed Bit storage method and the Bit offset storage method, stop the calculation
[0124] Step 3: Select the method with the smallest occupied memory space to store the Bitmap
[0125] When Range is expanded, the network device can quickly convert it to an uncompressed Bitmap using Bit operations. The specific method is: for each range in turn: (all-1 Bitmap << start of range) & (all-1 Bitmap >> (bitmap length - end of range)), and then perform an OR operation on the results
[0126] Since the Bitmap of each Block does not exceed 256 Bit at most, the network device can very quickly calculate the storage method with the smallest occupied memory space using hardware instructions
[0127] Based on the same inventive concept, the embodiment of the present application also provides a bitmap processing device corresponding to the bitmap processing method. See Figure 9 , Figure 9 is the bitmap processing device provided by the embodiment of the present application. The device is applied to a network device, and the network device includes a management block SuperBlock and multiple blocks Block. The device includes:
[0128] A receiving unit 910, configured to receive a first operation instruction input by a user. The first operation instruction includes a to-be-viewed index, and the to-be-viewed index can be converted into a string with N bit positions Bit
[0129] A first selection unit 920, configured to, if the type of the SuperBlock is the first type, select a first Bit corresponding to a semi-full subgraph PartiaMap included in the SuperBlock according to the first number of high Bit positions included in the string
[0130] The first acquisition unit 930 is used to acquire the number of bits that are set below the first bit in the half-full sub-graph if the first bit corresponding to the half-full sub-graph is set.
[0131] The first determining unit 940 is used to determine the position of the number in the half-full array included in the management block according to the number, and to determine the second bit in the first block corresponding to the string according to the position;
[0132] Display unit 950 is used to display the operation result to the user, the operation result including the current state of the second bit.
[0133] Optionally, the first selection unit 920 is further configured to select the first bit corresponding to the full sub-map (FullMap) included in the SuperBlock;
[0134] The first determining unit 940 is further configured to determine that all second bits included in the first block indicated by the first bit corresponding to the full sub-graph are not set if the first bit corresponding to the full sub-graph is not set and the first bit corresponding to the half-full sub-graph is not set.
[0135] The display unit 950 is further configured to display the operation result to the user, the operation result including the current state of the second bit.
[0136] Optionally, the first determining unit 940 is further configured to, if the first bit corresponding to the full sub-graph is set, determine that all the second bits included in the first block indicated by the first bit corresponding to the full sub-graph are set;
[0137] The display unit 950 is further configured to display the operation result to the user, the operation result including the current state of the second bit.
[0138] Optionally, the location stores the first address of the first block;
[0139] The first determining unit 940 is specifically used to obtain the first address of the first block from the location;
[0140] Based on the first address, access the first block, and based on the second number of low-order bits included in the string, determine the corresponding second bit and the current state of the second bit within the first block.
[0141] Optionally, the location stores the offset position of the third bit that has been set in the first block;
[0142] The first determining unit 940 is specifically used to obtain the offset position of the third bit within the first block from the position;
[0143] The current state of the second bit is determined based on the offset position and the second number of low-order bits included in the string.
[0144] Optionally, the apparatus further includes: a second acquisition unit (not shown in the figure), used to acquire a complete bitmap;
[0145] The identification unit (not shown in the figure) is used to identify the number of bits that have been set in the Bitmap;
[0146] A generation unit (not shown in the figure) is used to generate the SuperBlock of the first type if the number is greater than the second threshold number;
[0147] The generation unit (not shown in the figure) is also used to generate the SuperBlock with the second type if the number is not greater than the second threshold number.
[0148] Optionally, the device further includes: a division unit (not shown in the figure), used to divide the Bitmap into multiple Blocks according to a preset third number of bits;
[0149] The second selection unit (not shown in the figure) is used to select the storage method with the least memory space from a variety of storage methods according to the set state of each bit in the Block, and to store the Block.
[0150] Optionally, the second selection unit (not shown in the figure) is specifically used to identify the number of bits that have been set in the Block;
[0151] If the number is greater than the first threshold, a half-full array of the first type is generated, and a storage method with the least memory space is selected from multiple storage methods to store the Block, and the address of the Block is stored in the position of the half-full array corresponding to the Block.
[0152] If the number is not greater than the first threshold, then a half-full array of the second type is generated, and the offset position of the set bit is stored in the half-full array corresponding to the block;
[0153] The various storage methods include uncompressed bit storage, range storage, and bit offset storage.
[0154] Optionally, the device further includes: a third acquisition unit (not shown in the figure), used to acquire the offset position of the set third bit in the Bitmap from the SuperBlock if the type of the SuperBlock is the second type;
[0155] The second determining unit (not shown in the figure) is used to determine the current state of the second bit based on the offset position and the second number of low-order bits included in the string.
[0156] Therefore, by applying the bitmap processing apparatus provided in this application, the network device receives a first operation instruction input by the user. The first operation instruction includes an index to be viewed, which can be converted into a string with N bits. If the SuperBlock is of type 1, the network device selects the first bit corresponding to the half-full sub-map (PartiaMap) included in the SuperBlock based on the first number of high-order bits included in the string. If the first bit corresponding to the half-full sub-map is set, the network device obtains the number of bits set below the first bit from the half-full sub-map. Based on the number, the network device determines the position of the number in the half-full array included in the management block, and determines the second bit in the first block corresponding to the string based on the position. The operation result is displayed to the user, which includes the current state of the second bit.
[0157] In this way, the multiple blocks included in the network device are all half-full blocks. By using the half-full submap PartiaMap and the half-full array, the bit required by the user can be quickly determined, which solves the problems of low random access efficiency and large memory space consumption exposed by the Bitmap storage method.
[0158] Based on the same inventive concept, embodiments of this application also provide a network device, such as... Figure 10 As shown, the system includes a processor 1010, a transceiver 1020, and a machine-readable storage medium 1030. The machine-readable storage medium 1030 stores machine-executable instructions that can be executed by the processor 1010. The processor 910 is prompted by the machine-executable instructions to execute the bitmap processing method provided in the embodiments of this application. (The foregoing...) Figure 9 The bitmap processing device shown can employ, for example, Figure 10 The hardware structure of the network device shown is implemented.
[0159] The aforementioned computer-readable storage medium 1030 may include random access memory (RAM) or non-volatile memory (NVM), such as at least one disk storage device. Optionally, the computer-readable storage medium 1030 may also be at least one storage device located remotely from the aforementioned processor 1010.
[0160] The processor 1010 mentioned above can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.
[0161] In this embodiment of the application, the processor 1010 reads the machine-executable instructions stored in the machine-readable storage medium 1030, and is prompted by the machine-executable instructions to enable the processor 1010 itself and the transceiver 1020 to execute the bitmap processing method described in the aforementioned embodiment of the application.
[0162] In addition, this application provides a machine-readable storage medium 1030 that stores machine-executable instructions. When called and executed by the processor 1010, the machine-executable instructions cause the processor 1010 itself and the transceiver 1020 to execute the bitmap processing method described in the aforementioned application.
[0163] The specific implementation process of the functions and roles of each unit in the above device can be found in the implementation process of the corresponding steps in the above method, and will not be repeated here.
[0164] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to in the description of the method embodiments. The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this application according to actual needs. Those skilled in the art can understand and implement this without creative effort.
[0165] For the bitmap processing apparatus and machine-readable storage medium embodiments, since the methods involved are basically similar to the aforementioned method embodiments, the description is relatively simple, and relevant details can be found in the descriptions of the method embodiments.
[0166] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application.
Claims
1. A method for processing bitmaps, characterized in that, The method is applied to a network device, which includes a management block SuperBlock and multiple blocks. The method includes: Receive a first operation instruction input by the user, the first operation instruction including an index to be viewed, the index to be viewed can be converted into a string with N bits; If the SuperBlock is of type 1, then based on the first number of high-order bits included in the string, select the first bit corresponding to the half-full sub-map PartiaMap included in the SuperBlock; If the first bit corresponding to the half-full subgraph is set, then obtain the number of bits that are set below the first bit from the half-full subgraph; Based on the number, determine the position of the number in the half-full array included in the management block, and based on the position, determine the second bit in the first block corresponding to the string; The operation result is displayed to the user, and the operation result includes the current state of the second bit; Before receiving the first operation instruction input by the user, the method further includes: Get the complete Bitmap; Identify the number of bits that have been set within the Bitmap; If the number is greater than the second threshold, then the SuperBlock with the first type is generated; If the number is not greater than the second threshold, then the SuperBlock with the second type is generated; After receiving the first operation instruction input by the user, the method further includes: If the SuperBlock is of the second type, then obtain the offset position of the third bit that has been set in the Bitmap from the SuperBlock; The current state of the second bit is determined based on the offset position and the second number of low-order bits included in the string.
2. The method according to claim 1, characterized in that, Before obtaining the number of bits that are set below the first bit in the half-full subgraph if the first bit is set in the half-full subgraph, the method further includes: Select the first bit corresponding to the full sub-map (FullMap) included within the SuperBlock; If the first bit corresponding to the full sub-graph is not set and the first bit corresponding to the half-full sub-graph is not set, then it is determined that all the second bits included in the first block indicated by the first bit corresponding to the full sub-graph are not set. The operation result is displayed to the user, and the operation result includes the current state of the second bit.
3. The method according to claim 2, characterized in that, Before obtaining the number of bits that are set below the first bit in the half-full subgraph if the first bit is set in the half-full subgraph, the method further includes: If the first bit corresponding to the full sub-graph is set, then it is determined that all the second bits included in the first block indicated by the first bit corresponding to the full sub-graph are set; The operation result is displayed to the user, and the operation result includes the current state of the second bit.
4. The method according to claim 1, characterized in that, The location stores the first address of the first block; Determining the second bit within the first block corresponding to the string specifically includes: Obtain the first address of the first block from the location; Based on the first address, access the first block, and based on the second number of low-order bits included in the string, determine the corresponding second bit and the current state of the second bit within the first block.
5. The method according to claim 1, characterized in that, The location stores the offset position of the third bit that has been set in the first block; Determining the second bit within the first block corresponding to the string specifically includes: Obtain the offset position of the third bit within the first block from the stated position; The current state of the second bit is determined based on the offset position and the second number of low-order bits included in the string.
6. The method according to claim 1, characterized in that, After generating the SuperBlock of the first type, the method further includes: Based on a preset third number, the Bitmap is divided into multiple Blocks by Bit, and each Block includes the third number of Bits; Based on the bit state of each bit within the Block, the storage method that occupies the least memory space is selected from multiple storage methods to store the Block.
7. The method according to claim 6, characterized in that, The step of selecting the storage method with the least memory usage from multiple storage methods based on the set state of each bit in the Block and storing the Block specifically includes: Identify the number of bits that have been set within the Block; If the number is greater than the first threshold, a half-full array of the first type is generated, and a storage method with the least memory space is selected from multiple storage methods to store the Block, and the address of the Block is stored in the position of the half-full array corresponding to the Block. If the number is not greater than the first threshold, then a half-full array of the second type is generated, and the offset position of the set bit is stored in the half-full array corresponding to the block; The various storage methods include uncompressed bit storage, range storage, and bit offset storage.
8. A bitmap processing apparatus, characterized in that, The device is applied to a network device, which includes a management block SuperBlock and multiple blocks. The device includes: A receiving unit is configured to receive a first operation instruction input by a user, the first operation instruction including an index to be viewed, the index to be viewed being convertible into a string with N bits; The first selection unit is used to select the first bit corresponding to the PartiaMap included in the SuperBlock based on the first number of high-order bits included in the string if the type of the SuperBlock is the first type. The first acquisition unit is used to acquire the number of bits that are set below the first bit in the half-full subgraph if the first bit corresponding to the half-full subgraph is set. The first determining unit is configured to determine the position of the number in the half-full array included in the management block according to the number, and determine the second bit in the first block corresponding to the string according to the position; The display unit is used to display the operation result to the user, the operation result including the current state of the second bit; The device further includes: a second acquisition unit, used to acquire a complete bitmap; The identification unit is used to identify the number of bits that have been set in the Bitmap; A generation unit is configured to generate the SuperBlock of the first type if the number is greater than the second threshold number. The generation unit is further configured to generate the SuperBlock of the second type if the number is not greater than the second threshold number. The device further includes: a third acquisition unit, configured to acquire the offset position of the set third bit in the Bitmap from the SuperBlock if the type of the SuperBlock is the second type; The second determining unit is used to determine the current state of the second bit based on the offset position and the second number of low-order bits included in the string.