Bit feature efficient identification method based on byte anchoring

Abstract

The invention discloses a bit feature efficient identification method based on byte anchoring, which comprises the following steps: S1, determining the length of a to-be-identified bit feature to be m, and determining the length L of a byte window, m being greater than or equal to 8, L being equal to (m + 7) / 8 + 1, and * representing a function of rounding down; s2, constructing a second byte fast mapping table; s3, reading L byte data in the current window in the data stream; s4, acquiring a possible bit offset set according to the value of the second byte of the current window, and matching to-be-identified bit features based on the possible bit offset set; and S5, adjusting the sliding step of the window according to a matching result, and returning to S3 to continue execution. Through byte-level stepping, second byte anchoring and a fast mapping table, the technical bottleneck of low bit feature matching efficiency in network traffic is specially solved, and the method is suitable for bit features with medium length and more than medium length.

Description

Technical Field

[0001] This invention relates to the field of computer network data processing technology, and specifically to a method for efficient identification and extraction of network traffic based on bit features. Background Technology

[0002] In many scenarios within the field of computer network data processing technology, data characteristics are typically byte-based, conforming to computer memory alignment requirements. The CPU can directly operate on bytes, resulting in high processing efficiency. Unlike byte characteristics, bit characteristics have a finer granularity, accurate to each individual bit, offering greater flexibility and the ability to represent features of arbitrary length. However, because bit characteristics do not meet computer memory alignment requirements, their processing is more complex. Therefore, efficiently identifying bit characteristics in network traffic is a crucial technology.

[0003] Existing bit feature recognition methods typically employ a bit-level sliding window approach. This involves first converting the byte stream into a continuous bit stream, then setting a sliding window of length L, and sliding in 1-bit increments. Each time, L bits are extracted for matching. Current technologies require frequent byte-to-bit conversions in the data stream, with each increment being 1 bit. For a bit feature of length m, this necessitates (N*8-m+1) complete matching operations in a data stream of length N. These frequent bit stream conversions and bit-level operations lead to excessive CPU and memory resource consumption, especially when the feature length is long or the amount of data to be identified is large. This situation is even more pronounced and cannot meet the real-time traffic analysis requirements of high-speed network environments.

[0004] Meanwhile, the existing technologies retrieved indicate that researchers have conducted a series of explorations in bitstream processing, but none of these solutions have effectively addressed the aforementioned technical shortcomings:

[0005] Some solutions (such as patent US12051215B1) focus on the field of video encoding and decoding. Their processing object is highly structured, compressed video bitstreams. Their core goal is to improve compression efficiency or extract structured fields from compressed streams. Their technical means (such as constructing "frames" and using image detection methods) are fundamentally different from the identification of arbitrary bit features in network traffic.

[0006] Other solutions (such as patent CN112115830A) involve bit-domain feature extraction, but their core idea is to quantize the perceived signal (such as image features) into the bit domain to reduce the amount of data transmitted. This falls under the category of data compression and transmission optimization, rather than efficient pattern matching in high-speed data streams.

[0007] In conclusion, with the continuous improvement of network bandwidth and the increasing demand for network security detection, especially when dealing with bit features of medium length or longer (m≥8), there is an urgent need to develop an efficient algorithm for identifying bit features that can reduce computing resource consumption and meet real-time processing requirements. Summary of the Invention

[0008] To address the aforementioned shortcomings of existing technologies, this invention provides a highly efficient bit feature identification method based on byte anchoring, which can effectively solve the technical problems of low bit stepping efficiency and inability to meet the performance requirements of real-time network traffic processing in existing technologies.

[0009] To achieve the above objectives, the present invention provides the following technical solution:

[0010] An efficient bit feature identification method based on byte anchoring includes the following steps:

[0011] S1. Set the bit feature to be identified. Determine the byte window length L based on the length m of the bit feature to be identified, where m≥8, L =⌊(m+7) / 8⌋+1, where ⌊·⌋ represents the floor function.

[0012] S2. Construct a second byte fast mapping table. Construct a second byte fast mapping table based on the bit features to be identified. The second byte fast mapping table is used to record the mapping relationship between the value of the second byte and the possible bit offset k.

[0013] When m=8, the possible bit offset k takes the value of an integer from 1 to 7;

[0014] When m > 8, the possible bit offset k takes the value of an integer from 0 to 7;

[0015] S3. Set a byte window of length L in the data stream, slide the byte window in increments of 1 byte, and read L bytes of data in the current window after each slide;

[0016] S4. Extract the value of the second byte from the L bytes of data in the current window, query the second byte fast mapping table based on the value of the second byte to determine whether there is a possible bit offset k, obtain the possible bit offset set, and based on the possible bit offset set, take the value and match the other bytes in the L bytes of data in the current window, and output the matching result.

[0017] S5. Adjust the window sliding step according to the matching result, and return to S3 to continue execution;

[0018] If a match is successful, the byte window will then slide in increments of (L-1) bytes.

[0019] If a match fails, the byte window is then slid in increments of 1 byte.

[0020] Furthermore, in step S2, the step of constructing the second byte fast mapping table includes:

[0021] For each possible bit offset k, calculate the value or range of the second byte under that value; map the value or range of the second byte to the corresponding possible bit offset k to form a mapping relationship.

[0022] Furthermore, when 8≤m<16, the value range of the second byte is obtained according to each possible bit offset k, and each value in the value range of the second byte is mapped to the corresponding possible bit offset k.

[0023] Furthermore, when m≥16, for each possible bit offset k, the value of the second byte is uniquely determined. The value of the second byte is obtained and mapped to the corresponding possible bit offset k.

[0024] Furthermore, when 8 ≤ m < 16, the step of constructing the second byte fast mapping table in S2 includes:

[0025] The first step is to obtain basic parameters, including the byte window length L and the possible bit offset k;

[0026] The second step is to calculate the value range of the second byte corresponding to each possible bit offset k. First, determine the higher bits of the second byte occupied by the bit feature to be identified. Then, fix the values ​​of the higher bits and set the values ​​of the lower bits to all 0s and all 1s to obtain the value range of the second byte at the possible bit offset k.

[0027] The third step is to create a fast mapping table. Create a fixed-length array of 256. The array index represents the value of the second byte, and the array content represents the set of possible bit offsets k. The default value is empty, indicating that no possible bit offset k that satisfies the characteristics of the bit to be identified was found for that value.

[0028] Furthermore, when m ≥ 16, the step of constructing the second byte fast mapping table in S2 includes:

[0029] The first step is to obtain basic parameters, including the byte window length L and the possible bit offset k;

[0030] The second step is to calculate the value of the second byte corresponding to each possible bit offset k. Since the bit feature length m≥16, the 8 bits of the second byte are completely covered by the bit feature to be identified. For each possible bit offset k, the exact value of the 8 bits of the bit feature to be identified covered on the second byte is directly calculated.

[0031] The third step is to create a fast mapping table. Create an array of fixed length 256. For each possible bit offset k, use the calculated unique second byte value as the array index. The contents of the array represent the set of possible bit offsets k. The default value is empty, indicating that no possible bit offset k that satisfies the bit characteristics to be identified was found for that value.

[0032] Furthermore, when the length of the bit feature to be identified is m=8, in step S4, the first byte of the L bytes of data in the current window is matched first.

[0033] Further, S4 includes:

[0034] The first step is to extract the value of the first byte from the L bytes of data in the current window, check whether the first byte of the current window matches the bit feature to be identified, and if it matches, output the matching result and execute step S5.

[0035] The second step is to extract the value of the second byte from the L bytes of data in the current window and query the fast mapping table of the second byte to obtain the possible set of bit offsets.

[0036] The third step is to take the values ​​of other bytes in the L bytes of data in the current window and match them based on the possible set of bit offsets, and output the matching results.

[0037] Furthermore, the step of retrieving and matching the values ​​of other bytes in the L bytes of data within the current window includes:

[0038] For each possible bit offset k in the set of possible bit offsets, calculate the byte coverage of the bit feature to be identified within the current byte window based on the possible bit offset k;

[0039] Based on the byte coverage range, determine the target byte to be verified and its corresponding bit mask;

[0040] The target byte is subjected to bitwise operations using the bit mask to extract the actual bit value;

[0041] The extracted actual bit value is compared with the expected value of the bit feature to be identified at the corresponding position;

[0042] When all target bytes pass verification, the bit feature match is considered successful.

[0043] Furthermore, the scope of the computed bytes includes:

[0044] Calculate the starting bit for the bit feature: start_bit = k;

[0045] Calculate the end bit of the bit feature: end_bit = k + m - 1;

[0046] Determine the starting byte index: start_byte = start_bit / 8;

[0047] Determine the end byte index: end_byte = end_bit / 8.

[0048] Furthermore, determining the target byte and its bit mask includes:

[0049] Iterate through all bytes from start_byte to end_byte;

[0050] For each byte, calculate the range of bits covered by the bit feature to be identified within that byte;

[0051] Generate the corresponding bit mask based on the range of bits covered;

[0052] Generate the expected value based on the value of the bit feature to be identified at the corresponding position.

[0053] Furthermore, the bitwise operation verification includes at least one of the following cases:

[0054] Verify some of the low-order bits of the first byte: perform an AND operation using the low-order mask and compare it with the expected value;

[0055] Verify the completeness of the intermediate bytes: perform direct byte comparison;

[0056] Verify the high-order bits of the last byte: Perform an AND operation using the high-order bit mask and compare it with the expected value.

[0057] Furthermore, in the verification process for each possible bit offset k, an early termination strategy is adopted, and when the verification of any target byte fails, the subsequent verification of the current bit offset k is immediately terminated.

[0058] On the other hand, the present invention also discloses an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the above-mentioned efficient bit feature identification method based on byte anchoring.

[0059] A computer-readable storage medium storing a computer program is also disclosed, characterized in that the computer program, when executed by a processor, implements the above-described efficient bit feature identification method based on byte anchoring.

[0060] The technical solution provided by this invention has the following advantages compared with the known prior art:

[0061] This invention completely replaces the inefficient bit-level stepping (1 bit) in traditional methods with byte-level sliding stepping (1 byte). Theoretically, this single improvement reduces the frequency of basic window sliding operations by 8 times, significantly reducing the overhead of underlying data access and window movement. The number of matches is reduced by orders of magnitude.

[0062] By introducing a second-byte anchoring mechanism and a fast mapping table, this invention successfully reduces the required number of feature matching operations from the O(n) level (approximately N*8-m+1 times) of the traditional sliding window method to an approximately constant level. In most cases, only 2-3 precise verifications are needed to complete the judgment for a window of data, avoiding a large number of unnecessary full feature comparisons. This is the core reason for the improved algorithm efficiency.

[0063] This invention avoids frequent bitstream conversion operations, reducing a large number of bitwise operations such as shifting and masking. Simultaneously, the drastic reduction in the number of matches directly lowers the CPU's comparison computation burden, allowing CPU resources to be used more effectively for other tasks or to handle higher bandwidth network traffic with equivalent hardware. Attached Figure Description

[0064] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are merely some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without any creative effort.

[0065] Figure 1 This is a flowchart illustrating an efficient bit feature identification method based on byte anchoring.

[0066] Figure 2 This is a schematic diagram of the second byte anchor point for a bit feature length (m=8).

[0067] Figure 3 The schematic diagram of the second byte anchor point for bit feature length (9≤m<16).

[0068] Figure 4 This is a schematic diagram of the second byte anchor point for a bit feature length (m≥16).

[0069] Figure 5 The second byte of the fast mapping table is the bit feature length (8≤m<16).

[0070] Figure 6 This is a fast mapping table for the second byte of the bit feature length (m≥16). Detailed Implementation

[0071] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0072] To address the high computational complexity and low efficiency of direct bit pattern matching in data streams, traditional methods require sliding a window bit by bit for comparison, which is slow. This method introduces a "byte anchoring" strategy, prioritizing the search for a high-probability anchor byte on byte boundaries. This quickly eliminates many impossible matches, transforming most bit-level matching operations into efficient byte lookups and comparisons, significantly improving matching speed.

[0073] like Figure 1 As shown, this embodiment provides an efficient bit feature identification method based on byte anchoring, including the following steps:

[0074] S1. Set the bit feature to be identified. Determine the byte window length L based on the length m of the bit feature to be identified, where m≥8, L =⌊(m+7) / 8⌋+1, where ⌊·⌋ represents the floor function.

[0075] By setting S1, it is ensured that at any bit offset k (0≤k≤7), a byte window of length L is sufficient to completely cover the entire bit feature of length m, providing the basis for the "second byte anchoring" mechanism, ensuring that the second byte always contains feature information, thereby enabling the construction of a fast mapping table and achieving efficient filtering.

[0076] S2. Construct a second-byte fast mapping table. Construct a second-byte fast mapping table based on the characteristics of the bit to be identified. The second-byte fast mapping table is used to record the mapping relationship between the value of the second byte and the possible bit offset k.

[0077] When m=8, the possible bit offset k takes the value of an integer from 1 to 7, and the case of k=0 is directly matched by the first byte.

[0078] When m > 8, the possible bit offset k takes the value of an integer from 0 to 7.

[0079] This step is a preprocessing step, designed to build a mapping table for fast filtering. In this embodiment, the second byte within the current window is selected as the "anchor byte." This is because the first byte has the most possible bit offsets, which is not conducive to filtering, while the second byte can provide good differentiation under different bit offsets.

[0080] S3. Set a byte window of length L in the data stream, slide the byte window in increments of 1 byte, and read L bytes of data in the current window after each slide.

[0081] S4. Extract the value of the second byte from the L bytes of data in the current window. Based on the value of the second byte, query the second byte fast mapping table to determine if there is a possible bit offset k. Obtain the set of possible bit offsets. Based on the set of possible bit offsets, extract and match the values ​​of other bytes in the L bytes of data in the current window, and output the matching result.

[0082] Specifically, in S4, the value of the second byte from the L bytes of data in the current window is extracted. Using this value as the key, the fast mapping table constructed in step S2 is queried. If the value of the second byte is not in the fast mapping table, it means that there is absolutely no possibility of matching the target bit feature at the current window position. The match is immediately determined to be a failure, and the process jumps to S5. If the value of the second byte exists in the fast mapping table, a possible set of bit offsets is obtained. Based on this set, further fine-grained verification is performed to match the bit feature to be identified.

[0083] S5. Adjust the window sliding step based on the matching result, and return to S3 to continue execution.

[0084] If a match is successful, the subsequent sliding byte window advances by (L-1) bytes, since the feature length is m bits, approximately equal to L-1 bytes. After a successful match in the current window, it is impossible for the starting position within the next (L-1) byte window to form a complete match again (unless the feature itself has a special overlapping structure; this strategy prioritizes efficiency). This is an optimization that "skips impossible regions".

[0085] If a match fails, the subsequent byte window slides in 1-byte increments. A failed match only indicates that there is no match under the current alignment, but the starting position of the next byte may be a valid starting point for a match. Therefore, it is necessary to carefully move 1 byte to probe.

[0086] This embodiment proposes an efficient bit feature recognition method based on byte window and second byte anchoring, specifically for bit features with length m≥8. It adopts byte-level stepping instead of traditional bit-level stepping and utilizes the anchoring characteristics of the second byte to quickly determine possible matching positions.

[0087] In a byte window of length L, this embodiment selects the second byte as the anchor point, which greatly improves the screening efficiency for bit features to be identified with a length of not less than 8.

[0088] like Figure 2As shown, for a bit feature of length m (m=8), when the bit offset position of the feature in the window is greater than 0, the 8-bit high-order part of the second byte must contain at least a part of the bit feature.

[0089] like Figure 3 As shown, for a bit feature of length m (9≤m<16), regardless of the bit offset position of the feature in the window, the 8-bit high-order part of the second byte must contain at least a part of the bit feature.

[0090] like Figure 4 As shown, for a bit feature of length m (m≥16), regardless of the bit offset position of the feature in the window, the 8 bits of the second byte must contain at least a part of the bit feature.

[0091] In summary, for features m≥8, regardless of the possible bit offset k, the second byte will always contain a part of the bit feature. Therefore, the second byte can serve as a stable and reliable "anchor" to predict the possible location of the entire feature. By taking the value of the second byte, the possible offset of the bit feature within the byte window can be determined, a fast mapping table can be created, and then the possible offset can be used to further determine whether the values ​​of other bytes satisfy the bit feature, thereby achieving bit feature recognition.

[0092] In this embodiment, the core objective of step S2 is to construct a fast mapping table, which is used to quickly deduce the possible bit offset k from the second byte value of the current window during scanning, thereby significantly narrowing down the candidate range that requires fine comparison.

[0093] In S2, the steps for constructing the second-byte fast mapping table include:

[0094] For each possible bit offset k, calculate the value or range of the second byte under that value; map the value or range of the second byte to the corresponding possible bit offset k to form a mapping relationship.

[0095] When 8≤m<16, the value range of the second byte is obtained according to each possible bit offset k, and each value in the value range of the second byte is mapped to the corresponding possible bit offset k.

[0096] When 8 ≤ m < 16, the step of constructing the second byte fast mapping table in S2 includes:

[0097] The first step is to obtain basic parameters, including the byte window length L and the possible bit offset k. When m=8, the possible bit offset k is an integer from 1 to 7. When m>8, the possible bit offset k is an integer from 0 to 7.

[0098] The second step is to calculate the value range of the second byte corresponding to each possible bit offset k. First, determine the higher-order bits of the second byte occupied by the bit feature to be identified. Then, fix the values ​​of the higher-order bits and set the values ​​of the lower-order bits to all 0s and all 1s. For example, ... Figure 5 As shown, when k=1, it is determined that the bit feature to be identified occupies the high 2 bits of the second byte. Then, the value of the high 2 bits is fixed to obtain the value of the second byte as bHHxxxxxx, where H represents the value of the bit feature to be identified under the possible bit offset k, and x represents any value. Thus, the value range of the second byte under the possible bit offset k is obtained as [bHH000000, bHH111111].

[0099] The third step is to create a fast mapping table. Create a fixed-length array of 256. The array index represents the value of the second byte, and the array content represents the set of possible bit offsets k. The default value is empty, indicating that no possible bit offset k that satisfies the characteristics of the bit to be identified was found for that value.

[0100] When m≥16, for each possible bit offset k, the value of the second byte is uniquely determined. Obtain the value of the second byte and map it to the corresponding possible bit offset k.

[0101] When m≥16, the steps in S2 for constructing the second-byte fast mapping table include:

[0102] The first step is to obtain the basic parameters, including the byte window length L and the possible bit offset k, where k is an integer from 0 to 7;

[0103] The second step is to calculate the value of the second byte corresponding to each possible bit offset k. Since the bit feature length m≥16, the 8 bits of the second byte are completely covered by the bit feature to be identified. For each possible bit offset k, the exact value of the 8 bits of the bit feature to be identified covered on the second byte is directly calculated.

[0104] The third step is to create a fast mapping table. Create an array of fixed length 256. For each possible bit offset k, use the calculated unique second byte value as the array index. The contents of the array represent the set of possible bit offsets k. The default value is empty, indicating that no possible bit offset k that satisfies the bit characteristics to be identified was found for that value.

[0105] Specifically, such as Figure 6As shown, when k=0, the calculated unique second byte value is A; when k=1, the calculated unique second byte value is B; when k=2, the calculated unique second byte value is D; when k=3, the calculated unique second byte value is D; and when k=7, the calculated unique second byte value is D. Based on the above results, a fast mapping table is constructed. The content of the numerical array A is k={0,2}, the content of the numerical array B is k={1}, and the content of the numerical array D is k={3,7}.

[0106] When the length of the bit feature to be identified is m=8, the first byte of the L bytes of data in the current window is matched first in S4. When the feature length is exactly 8 bits, k=0 is a special case of "complete byte matching". The entire 8-bit feature is located on the first byte. By using pure byte matching, the first byte is directly compared with the feature byte.

[0107] Specifically, S4 includes:

[0108] The first step is to extract the value of the first byte from the L bytes of data in the current window, check whether the first byte of the current window matches the bit feature to be identified, and if it matches, output the matching result and execute step S5.

[0109] The second step is to extract the value of the second byte from the L bytes of data in the current window and query the second byte fast mapping table to obtain the possible set of bit offsets.

[0110] The third step is to take the values ​​of other bytes in the L bytes of data in the current window and match them based on the possible set of bit offsets, and output the matching results.

[0111] In this embodiment, the bit feature length m=8, and the content is b10101011. The specific implementation steps are as follows:

[0112] S1. Set the bit feature to be identified as b10101011. Based on the length of the bit feature to be identified, m=8, determine the byte window length L, L=(m+7) / 8+1=2.

[0113] S2. Construct a fast mapping table for the second byte, with a possible bit offset range of k=[1,7], and obtain the value of the second byte at different offsets:

[0114] Offset Second byte HEX The position of the second byte in the bit characteristics 1 [0x80,0xFF] 1010101[1*******] 2 [0xC0,0xFF] 101010[11******] 3 [0x60,0x7F] 10101[011*****] 4 [0xB0,0xBF] 1010[1011****] 5 [0x58,0x5F] 101[01011***] 6 [0xAC,0xAF] 10[101011**] 7 [0x56,0x57] 1[0101011*]

[0115] Create a fast mapping table Relation from the second byte to the offset. The capacity of Relation is 255. Here are some specific values: Relation[0xFF]={1,2}, Relation[0x80]={1}, Relation[0xC0]={2}, Relation[0xAC]={1,6}, Relation[0xAB]={1}, Relation[0xBC]={1,4}.

[0116] S3. Set a byte window of length 2 in the data stream, slide the byte window in increments of 1 byte, and read the 2 bytes of data in the current window as [0xAB, 0xAA] after each slide.

[0117] S4 includes: First, extract the value of the first byte from the two bytes of data in the current window, check whether the first byte of the current window matches the bit feature to be identified, if they match, output the matching result, and execute step S5;

[0118] In this embodiment, the first byte value is extracted as 0xAA, compared with the bit feature, and is not equal to it, so the match fails.

[0119] The second step is to extract the second byte (0xBC) if the first step does not match, and then query the fast mapping table to obtain the possible offset {1,4}.

[0120] The third step is to first verify whether the possible offset 1 satisfies the feature match. At offset 1, the value of the first byte should be bx1010101, where the high 1 bit can be any value, x is any value, and the low 7 bits are 1010101. Through the bitwise operation condition (0xAA & 01010101 == 01010101), it is known that offset 1 may not satisfy the feature match. Next, verify whether the possible offset 4 satisfies the feature match. At offset 4, the value of the first byte should be bxxxx1010, where the high 4 bits can be any value, and the low 4 bits are 1010. Through the bitwise operation condition (0xAA & 00001010 == 00001010), it is known that offset 4 may satisfy the feature match, and the matching result is recorded.

[0121] S5. Continue executing S3 to identify subsequent network data.

[0122] In another embodiment, the bit feature length m is 9 bits (9 ≤ m < 16), and the content is b110011001. The specific implementation steps are as follows:

[0123] S1. Set the byte window length L = (m+7) / 8+1 = 3 bytes;

[0124] S2. Construct the second byte fast mapping table, with a possible bit offset range of k=[0,7]. Figure 3 and Figure 5 Get the value of the second byte at different offsets.

[0125] Offset Second byte HEX The position of the second byte in the bit characteristics 0 [0x80,0xFF] 11001100[1*******] 1 [0x40,0x7F] 1100110[01******] 2 [0x20,0x3F] 110011[001*****] 3 [0x90,0x9F] 11001[1001****] 4 [0xC8,0xCF] 1100[11001***] 5 [0x64,0x67] 110[011001**] 6 [0x32,0x33] 11[0011001*] 7 0x99 1[10011001]

[0126] Create a fast mapping table Relation from the second byte to the offset. The capacity of Relation is 255. The following are some of the specific values: Relation[0x80]={1}, Relation[0xC8]={0,4}, Relation[0x64]={5}, Relation[0x32]={2,6}, Relation[0x99]={0,7}.

[0127] S3. Slide scan in 1-byte increments to read network data [0xAA, 0xBB, 0x32] of the current window size;

[0128] S4. Extract the value of the second byte 0xBB, query the fast mapping table to obtain the possible offset {0}, and verify whether the possible offset 0 satisfies the bit feature to be identified. Under offset 0, the value of byte 1 should be b11001100. Through the bit operation condition (0xAA &11001100==11001100), it is known that the possible offset 0 does not satisfy the feature matching, and the matching fails.

[0129] S5, return to S3. If the match fails, slide scan in 1-byte increments, read network data of window size [0xBB, 0x32, 0x00], and reread three bytes.

[0130] Further extract the second byte value 0x32, and query the fast mapping table to obtain the possible offset {2,6};

[0131] First, verify if offset 2 satisfies the feature match. At offset 2, the first byte should have the value bxx110011, where the high 2 bits can be any value and the low 6 bits are 110011. Using the bitwise operation condition (0xBB & 00110011 == 00110011), we know that offset 2 may not satisfy the feature match. Next, verify if offset 6 satisfies the feature match. At offset 6, the first byte should have the value bxxxxxx11, where the high 6 bits can be any value and the low 2 bits are 11. Using the bitwise operation condition (0xBB & 00000011 == 00000011), we know that offset 6 may satisfy the feature match, and we record the matching result. The window slides L-1 bytes to continue identifying subsequent network data.

[0132] In another embodiment, the bit feature length m is 20 bits (m≥16), and the content is b10101011110011011110. The specific implementation steps are as follows:

[0133] S1. Set the byte window length to the floor, L=(m+7) / 8+1=4 bytes.

[0134] S2. Construct the second byte fast mapping table, with a possible bit offset range of k=[0,7]. Figure 4 and Figure 6 Get the value of the second byte at different offsets:

[0135] Offset Second byte HEX The position of the second byte in the bit characteristics 0 0xCD 10101011[11001101]1110 1 0xE6 1010101[11100110]11110 2 0xF3 101010[11110011]011110 3 0x79 10101[01111001]1011110 4 0xBC 1010[10111100]11011110 5 0x5E 101[01011110]011011110 6 0xAF 10[10101111]0011011110 7 0x57 1[01010111]10011011110

[0136] Create a fast mapping table Relation from the second byte to the offset. The capacity of Relation is 255. The specific values ​​are as follows: Relation[0xCD]={0}, Relation[0xE6]={1}, Relation[0xF3]={2}, Relation[0x79]={3}, Relation[0xBC]={4}, Relation[0x5E]={5}, Relation[0xAF]={6}, Relation[0x57]={7}.

[0137] S3. Slide scan in 1-byte increments to read network data of the window size [0xAA, 0x05, 0x5E, 0x6F];

[0138] S4. Extract the second byte value 0x05. The fast mapping table does not find a possible offset, so the match fails.

[0139] S5. Return to S3. If a match fails, perform a sliding scan in 1-byte increments and read network data of the window size [0x05, 0x5E, 0x6F, 0x00].

[0140] Further extract the second byte value 0x5E, query the fast mapping table to obtain the possible offset {5}, and verify whether the possible offset 5 satisfies the feature match. At offset 5, the value of the first byte should be bxxxxx101, where the high 5 bits are arbitrary and the low 3 bits are 101. Through the bitwise operation condition (0x05 & 00000101 == 00000101), we know that the first byte satisfies the feature match. The value of the third byte should be b01101111. Through the bitwise operation condition (0x6F & 01101111 == 01101111), we know that the third byte satisfies the feature match. The value of the fourth byte should be b0xxxxxx, where the high 1 bit is 0 and the low 7 bits are arbitrary. Through the bitwise operation condition (0x00 & 00000000 == 00000000), we know that the fourth byte satisfies the feature match. This may result in a successful match with a 5-bit offset feature; the matching result is recorded. The window is then slid by L-1 bytes to continue identifying subsequent network data.

[0141] To improve the efficiency and accuracy of retrieving and matching values ​​of other bytes in the L bytes of data within the current window, this embodiment calculates the byte coverage range of the bit feature to be identified within the current byte window based on each possible bit offset k in the possible bit offset set; determines the target byte to be verified and its corresponding bit mask based on the byte coverage range; performs bit operations on the target byte using the bit mask to extract the actual bit value; compares the extracted actual bit value with the expected value of the bit feature to be identified at the corresponding position; and determines that the bit feature matching is successful when all target bytes have passed verification.

[0142] The calculation of the byte coverage includes: calculating the starting bit of the bit feature: start_bit = k; calculating the ending bit of the bit feature: end_bit = k + m - 1; determining the starting byte index: start_byte = start_bit / 8; determining the ending byte index: end_byte = end_bit / 8.

[0143] Determining the target byte and its bit mask includes: traversing all bytes from start_byte to end_byte; for each byte, calculating the range of bits covered by the bit feature to be identified in that byte; generating the corresponding bit mask based on the covered bit range; and generating the expected value based on the value of the bit feature to be identified at the corresponding position.

[0144] Bitwise operation verification includes at least one of the following cases: verifying part of the low-order bits of the first byte: performing an AND operation using a low-order mask and comparing it with the expected value; verifying the complete byte of the middle byte: performing a direct byte comparison; verifying part of the high-order bits of the last byte: performing an AND operation using a high-order mask and comparing it with the expected value.

[0145] In the above embodiment, when m=8, the first byte match fails (0xAA ≠ 0xAB). The second byte 0xBC is extracted, and the possible offset set {1,4} is obtained by querying the mapping table. The matching process in this embodiment is as follows:

[0146] Verify the possible bit offset k=1, and calculate the coverage range: start_bit=1, end_bit=8, start_byte=0, end_byte=1;

[0147] Target byte verification:

[0148] Byte 0: Mask 0x7F, Expected value 0x55;

[0149] Calculation: (0xAA & 0x7F) = 0x2A ≠ 0x55 → Verification failed.

[0150] Verify the possible bit offset k=4, and calculate the coverage range: start_bit=4, end_bit=11, start_byte=0, end_byte=1;

[0151] Target byte verification: Byte 0: Mask 0x0F, Expected value 0x0A;

[0152] Calculation: (0xAA & 0x0F) = 0x0A = 0x0A → Verification passed

[0153] Byte 1: Mask 0xF0, Expected value 0xB0;

[0154] Calculation: (0xBC & 0xF0) = 0xB0 = 0xB0 → Verification passed

[0155] Matching result: A match was found at a possible bit offset of k=4.

[0156] In this embodiment, the position of the feature in the byte window is precisely determined by start_bit / end_bit, and a corresponding bit mask is generated according to different offsets to achieve precise bit operation verification.

[0157] To improve matching efficiency, this implementation adopts an early termination strategy during the verification process of each possible bit offset k. When the verification of any target byte fails, the subsequent verification of the current bit offset k is immediately terminated.

[0158] Another embodiment is an electronic device including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the efficient bit feature identification method based on byte anchoring described in any of the above embodiments.

[0159] A computer-readable storage medium storing a computer program thereon, characterized in that, when the computer program is executed by a processor, it implements the above-described efficient bit feature identification method based on byte anchoring.

[0160] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. The storage medium can be a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM), etc.

[0161] The above descriptions are merely preferred examples of the invention and are not intended to limit the invention. Although the invention has been described in detail with reference to the foregoing examples, those skilled in the art can still modify the technical solutions described in the foregoing examples or make equivalent substitutions for some of the technical features. All modifications and equivalent substitutions made within the spirit and principles of the invention should be included within the scope of protection of the invention.

Claims

1. A method for efficient bit feature identification based on byte anchoring, comprising the following steps: S1. Set the bit feature to be identified. Determine the byte window length L based on the length m of the bit feature to be identified, where m≥8, L =⌊(m+7) / 8⌋+1, where ⌊·⌋ represents the floor function. S2. Construct a second byte fast mapping table. Construct a second byte fast mapping table based on the bit features to be identified. The second byte fast mapping table is used to record the mapping relationship between the value of the second byte and the possible bit offset k. When m=8, the possible bit offset k takes the value of an integer from 1 to 7; When m > 8, the possible bit offset k takes the value of an integer from 0 to 7; S3. Set a byte window of length L in the data stream, slide the byte window in increments of 1 byte, and read L bytes of data in the current window after each slide; S4. Extract the value of the second byte from the L bytes of data in the current window, query the second byte fast mapping table based on the value of the second byte to determine whether there is a possible bit offset k, obtain the possible bit offset set, and based on the possible bit offset set, take the value and match the other bytes in the L bytes of data in the current window, and output the matching result. S5. Adjust the window sliding step according to the matching result, and return to S3 to continue execution; If a match is successful, the byte window will then slide in increments of (L-1) bytes. If a match fails, the byte window is then slid in increments of 1 byte.

2. The efficient bit feature identification method based on byte anchoring according to claim 1, characterized in that, In step S2, the steps for constructing the second byte fast mapping table include: For each possible bit offset k, calculate the value or range of the second byte under that value; map the value or range of the second byte to the corresponding possible bit offset k to form a mapping relationship.

3. The efficient bit feature identification method based on byte anchoring according to claim 2, characterized in that, When 8 ≤ m < 16, the step of constructing the second byte fast mapping table in S2 includes: The first step is to obtain basic parameters, including the byte window length L and the possible bit offset k; The second step is to calculate the value range of the second byte corresponding to each possible bit offset k. First, determine the number of high bits of the second byte occupied by the bit feature to be identified, then fix the values ​​of the high bits and set the values ​​of the low bits to all 0 and all 1, thereby obtaining the value range of the second byte under the possible bit offset k. The third step is to create a fast mapping table. Create an array of fixed length 256. The array index represents the value of the second byte, and the array content represents the set of possible bit offsets k. The default value is empty, which means that no possible bit offset k that satisfies the bit characteristics to be identified was found for that value.

4. The efficient bit feature identification method based on byte anchoring according to claim 2, characterized in that, When m ≥ 16, the step of constructing the second byte fast mapping table in S2 includes: The first step is to obtain basic parameters, including the byte window length L and the possible bit offset k; The second step is to calculate the value of the second byte corresponding to each possible bit offset k. Since the bit feature length m≥16, the 8 bits of the second byte are completely covered by the bit feature to be identified. For each possible bit offset k, the exact value of the 8 bits of the bit feature to be identified covered on the second byte is directly calculated. The third step is to create a fast mapping table. Create an array of fixed length 256. For each possible bit offset k, use the calculated unique second byte value as the array index. The contents of the array represent the set of possible bit offsets k. The default value is empty, indicating that no possible bit offset k that satisfies the bit characteristics to be identified was found for that value.

5. The efficient bit feature identification method based on byte anchoring according to claim 1, characterized in that, When the length of the bit feature to be identified is m=8, in step S4, the first byte of the L bytes of data in the current window is matched first.

6. The efficient bit feature identification method based on byte anchoring according to claim 5, characterized in that, S4 includes: The first step is to extract the value of the first byte from the L bytes of data in the current window, check whether the first byte of the current window matches the bit feature to be identified, and if it matches, output the matching result and execute step S5. The second step is to extract the value of the second byte from the L bytes of data in the current window and query the fast mapping table of the second byte to obtain the possible set of bit offsets. The third step is to take the values ​​of other bytes in the L bytes of data in the current window and match them based on the possible set of bit offsets, and output the matching results.

7. The efficient bit feature identification method based on byte anchoring according to claim 1, characterized in that, In step S4, the steps of retrieving and matching the values ​​of other bytes in the L bytes of data within the current window include: For each possible bit offset k in the set of possible bit offsets, calculate the byte coverage of the bit feature to be identified within the current byte window based on the possible bit offset k; Based on the byte coverage range, determine the target byte to be verified and its corresponding bit mask; The target byte is subjected to bitwise operations using the bit mask to extract the actual bit value; The extracted actual bit value is compared with the expected value of the bit feature to be identified at the corresponding position; When all target bytes pass verification, the bit feature match is considered successful.

8. The efficient bit feature identification method based on byte anchoring according to claim 7, characterized in that, In the verification process for each possible bit offset k, an early termination strategy is adopted. When the verification of any target byte fails, the subsequent verification of the current bit offset k is immediately terminated.

9. An electronic device comprising a memory and a processor, the memory storing a computer program, the processor executing the computer program to implement the method as described in any one of claims 1-8.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1-8.

Images